Compare commits

..
Author SHA1 Message Date
james.maesandCodex 8717d3536e release: promote central CI maintenance verification (#69)
* release: promote central CI maintenance verification

Co-Authored-By: Codex <noreply@openai.com>

* release: include central RC caller maintenance

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 12:01:39 -05:00
james.maesandCodex 8229a66bc5 release: promote policy profiles and CI resilience (#67)
Promote the exact tested develop tree bd5ee3bec5.

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 11:32:55 -05:00
james.maesandCodex 9687feb36c release: promote shared branch policies and source scanning (#64)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 10:00:33 -05:00
james.maesandCodex 0fefe6a064 release: promote stable verification gates and RC completion (#60)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 03:21:14 -05:00
james.maesandCodex 2956e3ffce release: promote Android continuation fix and GitOps validation (#52)
Promote the exact checked develop tree on linear main history.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 22:03:14 -05:00
james.maesandCodex 0afe2a231f release: promote tested Android package authentication (#49)
* release: promote tested Android publisher and All regression coverage

Promote the exact tested develop tree while preserving the existing linear main history after earlier squash promotions.

Co-Authored-By: Codex <noreply@openai.com>

* ci: trigger scans for newly protected release target

The original promotion PR was created before main protection, so default CodeQL did not run. Reopening did not register a scan. Trigger the documented push event without changing the tested source tree.

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:48:10 -05:00
james.maesandCodex 65265e48a4 release: promote public All smoke evidence (#45)
* fix: build marketing site before browser checks (#40)

Run lint and one production build in every shared marketing workflow before Playwright and Lighthouse. Preserve the delegated delivery contracts.

Co-authored-by: Codex <noreply@openai.com>

* fix: use shared Maven credentials for Android publication (#42)

Use the existing organization context and Gradle environment properties. Fail before publishing if either credential is absent and avoid writing secrets to gradle.properties.

Co-authored-by: Codex <noreply@openai.com>

* fix: verify public All smoke evidence without secret export (#44)

Allow unauthenticated Circle API reads for public projects, preserve optional authentication and all fail-closed provenance checks.

Co-authored-by: Codex <noreply@openai.com>

* fix: default All releases to shared Circle smoke evidence (#35)

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:15:15 -05:00
james.maesandCodex 49399deb3d release: promote shared Android publisher (#43)
* fix: build marketing site before browser checks (#40)

Run lint and one production build in every shared marketing workflow before Playwright and Lighthouse. Preserve the delegated delivery contracts.

Co-authored-by: Codex <noreply@openai.com>

* fix: use shared Maven credentials for Android publication (#42)

Use the existing organization context and Gradle environment properties. Fail before publishing if either credential is absent and avoid writing secrets to gradle.properties.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:09:48 -05:00
james.maesandCodex 9fe4257490 fix: build marketing site before browser checks (#40) (#41)
Run lint and one production build in every shared marketing workflow before Playwright and Lighthouse. Preserve the delegated delivery contracts.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 20:58:58 -05:00
james.maes 0965a45653 Merge pull request #39 from QRun-IO/develop
release: gate Voyage publication on artifact security scan
2026-10-20 20:42:00 -05:00
james.maes bfd41c8e0d Merge pull request #37 from QRun-IO/develop
release: ship shared policy continuation fix
2026-10-20 20:08:21 -05:00
james.maes 9c30a30754 Merge pull request #33 from QRun-IO/develop
release: promote Maven reporting and publication fixes
2026-10-20 19:23:39 -05:00
james.maes ee8953bd93 Merge pull request #31 from QRun-IO/develop
fix: release confirmed Central publication checks
2026-10-20 18:48:38 -05:00
james.maes 5bc3167286 Merge pull request #29 from QRun-IO/develop
fix: release resolved Maven publication POMs
2026-10-20 18:39:47 -05:00
james.maes 1a3368e119 Merge pull request #27 from QRun-IO/develop
release: fail closed on GitHub publication errors
2026-10-20 07:39:37 -05:00
james.maes 1ddba883c9 Merge pull request #25 from QRun-IO/develop
release: central QQQ CI and release workers [minor]
2026-10-20 07:17:29 -05:00
james.maes da8044d6f2 Merge pull request #23 from QRun-IO/develop
release: qqq-orb 0.6.9 (snapshot refresh)
2026-10-14 10:04:52 -05:00
james.maes 67ba48da1f Merge pull request #18 from QRun-IO/develop
release: publish orb 0.6.8 with CircleCI npm OIDC correction
2026-09-29 15:17:00 -05:00
james.maes 5196609974 Merge pull request #16 from QRun-IO/develop
release: publish npm Trusted Publishing support in orb 0.6.7
2026-09-29 14:18:44 -05:00
james.maes 49e09a6776 Merge pull request #14 from QRun-IO/develop
release: publish orb log-safety fix as 0.6.6
2026-09-29 13:38:22 -05:00
james.maes 748016f105 Merge pull request #10 from QRun-IO/develop
[minor] feat: add node_app_* job family
2026-02-27 12:59:51 -06:00
james.maes 6514aaa688 Merge pull request #8 from QRun-IO/develop
fix(ci): remove legacy tag-based publish workflow
2026-01-31 18:46:54 -06:00
james.maes 3ac47adc21 Merge pull request #7 from QRun-IO/develop
Release: auto-publish pipelines and security scanning
2026-01-31 18:40:34 -06:00
32 changed files with 2798 additions and 65 deletions

No files matched your search

+4
View File
@@ -49,11 +49,15 @@ jobs:
python3 tests/test_cli_ci_contract.py
python3 tests/test_maven_ci_contract.py
python3 tests/test_standardize_qbit.py
python3 tests/test_qbit_hotfix_profile.py
python3 tests/test_ci_maintenance.py
python3 tests/test_rollout_version.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
python3 tests/test_next_native.py
python3 tests/test_auxiliary_workflows.py
python3 tests/test_qbit_codeql.py
python3 tests/test_repository_policy.py
python3 tests/test_intellij_plugin.py
python3 tests/test_workflow_lint.py
python3 tests/test_business_e2e.py
+58
View File
@@ -0,0 +1,58 @@
name: QBit CodeQL
'on':
workflow_call:
permissions:
contents: read
actions: read
security-events: write
packages: read
jobs:
analyze:
name: Analyze branch source
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- name: Select languages present in this checkout
id: languages
run: |
python3 <<'PY'
import os
from pathlib import PurePosixPath
import subprocess
import sys
files = subprocess.check_output(['git', 'ls-files', '-z']).decode('utf-8').split('\0')
extensions = {PurePosixPath(name).suffix.lower() for name in files if name}
languages = []
if extensions & {'.java', '.kt', '.kts'}:
languages.append('java-kotlin')
if extensions & {'.js', '.jsx', '.mjs', '.cjs', '.ts', '.tsx', '.mts', '.cts'}:
languages.append('javascript-typescript')
if '.py' in extensions:
languages.append('python')
if any((name.startswith('.github/workflows/') and PurePosixPath(name).suffix in {'.yml', '.yaml'})
or PurePosixPath(name).name in {'action.yml', 'action.yaml'} for name in files if name):
languages.append('actions')
if not languages:
sys.exit('No supported source in this branch; classify placeholders before enabling QBit scanning')
build_mode = 'autobuild' if extensions & {'.kt', '.kts'} else 'none'
with open(os.environ['GITHUB_OUTPUT'], 'a', encoding='utf-8') as output:
output.write('languages=' + ','.join(languages) + '\n')
output.write('build-mode=' + build_mode + '\n')
PY
- uses: actions/setup-java@v4
if: steps.languages.outputs.build-mode == 'autobuild'
with:
distribution: temurin
java-version: '21'
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:
languages: ${{ steps.languages.outputs.languages }}
build-mode: ${{ steps.languages.outputs.build-mode }}
- name: Build Kotlin sources
if: steps.languages.outputs.build-mode == 'autobuild'
uses: github/codeql-action/autobuild@v4
- name: Analyze source
uses: github/codeql-action/analyze@v4
+5 -5
View File
@@ -135,7 +135,7 @@ jobs:
echo "merge_sha=$merge_sha"
echo "merged_at=$merged_at"
} >> "$GITHUB_OUTPUT"
echo "Pushed $branch; CircleCI publish_release_candidate cuts the RC." >> "$GITHUB_STEP_SUMMARY"
echo "Pushed $branch; the CircleCI publisher cuts the RC." >> "$GITHUB_STEP_SUMMARY"
echo "Release line: $branch. Waiting without a runner for qbit-rc-published." >> "$GITHUB_STEP_SUMMARY"
- name: Persist trusted release intent
if: steps.merge.outputs.merge_sha != ''
@@ -203,7 +203,7 @@ jobs:
raise ReleaseFailure("Status response does not identify the exact release merge")
# Circle's workflow Check Run is stable even when job statuses receive -1/-2 suffixes.
publishers = [x for x in status.get("check_runs", [])
if x.get("name") == "publish_release_candidate" and x.get("app", {}).get("id") == 18001]
if x.get("name") in ("publish_release_candidate", "verify") and x.get("app", {}).get("id") == 18001]
if not publishers:
return ""
publisher = max(publishers, key=lambda x: x["id"])
@@ -265,10 +265,10 @@ jobs:
job.get('branch'), str(job.get('build_num'))) != (repo, merge, branch, build):
raise ReleaseFailure('Timeout job does not identify the exact cut')
workflow = job.get('workflows', {})
if workflow.get('workflow_name') != 'publish_release_candidate' or workflow.get('job_name') != 'qqq-orb/mvn_publish':
if workflow.get('workflow_name') not in ('publish_release_candidate', 'verify') or workflow.get('job_name') != 'qqq-orb/mvn_publish':
raise ReleaseFailure('Unexpected Circle publication workflow/job')
if (publisher.get('head_sha') != merge or publisher.get('app', {}).get('id') != 18001
or publisher.get('name') != 'publish_release_candidate'
or publisher.get('name') != workflow.get('workflow_name')
or publisher.get('status') != 'completed' or publisher.get('conclusion') != 'failure'):
raise ReleaseFailure('Recovery requires the failed exact Circle publisher')
from urllib.parse import urlsplit
@@ -335,7 +335,7 @@ jobs:
directory = pathlib.Path(directory)
status = json.loads((directory / 'status.json').read_text())
intent = json.loads((directory / 'intent.json').read_text())
publishers = [p for p in status['check_runs'] if p.get('name') == 'publish_release_candidate' and p.get('app', {}).get('id') == 18001]
publishers = [p for p in status['check_runs'] if p.get('name') in ('publish_release_candidate', 'verify') and p.get('app', {}).get('id') == 18001]
if status.get('sha') != merge or not publishers:
raise ReleaseFailure('Exact publisher evidence missing')
class NoRedirect(urllib.request.HTTPRedirectHandler):
+4
View File
@@ -58,11 +58,15 @@ test-pipelines: pack
bash tests/test_cli_ci_behavior.sh
python3 tests/test_cli_ci_contract.py
python3 tests/test_standardize_qbit.py
python3 tests/test_qbit_hotfix_profile.py
python3 tests/test_ci_maintenance.py
python3 tests/test_rollout_version.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
python3 tests/test_next_native.py
python3 tests/test_auxiliary_workflows.py
python3 tests/test_qbit_codeql.py
python3 tests/test_repository_policy.py
python3 tests/test_intellij_plugin.py
python3 tests/test_workflow_lint.py
python3 tests/test_business_e2e.py
+15
View File
@@ -114,6 +114,21 @@ Maven jobs use machine executors with Docker pre-installed. Tests using Testcont
## Orb Publishing
### QBit CI maintenance
The QBit profile runs Maven verification without publishing when a single-parent
`ci: ...` or `ci(scope): ...` commit changes only `.qqq-ci.yml`,
`.circleci/config.yml`, `.github/workflows/codeql.yml`, or
`.github/workflows/release-rc.yml`. This applies on
main, develop, release, and hotfix branches. The normal test job still runs,
including on branches normally excluded from feature tests, and reports through
the required `verify` workflow.
Tags, merge commits (including release automation), application changes, other
files, empty diffs, and unavailable history retain normal publishing behavior.
This lets central configuration updates avoid unnecessary application versions
without introducing per-repository maintenance workflows.
The orb auto-publishes via CircleCI:
| Branch | Action |
+12
View File
@@ -3,6 +3,18 @@
`tools/audit_estate.py` classifies repositories from explicit mappings and observed source.
It never changes consumers, branch rules or publication settings.
Source auditing reads the repository's current default branch unless
`--live --source-branch main` (or an explicit `release/...` or `hotfix/...` branch)
is supplied. Run separate audits for each maintained source branch: a clean
default-branch report does not establish conformity on main or release branches.
The selected branch is resolved to an immutable commit before any files are read.
Reports identify both `source_branch` and the repository's `default_branch`;
the manifest's `default_branch` must still match the repository default.
Unavailable selected branches remain unknown without falling back to default.
Explicit branch selection requires live lookup and cannot relabel an offline SHA.
Required-check policy still controls which branch settings are inspected;
`--source-branch` changes source selection, not that enforcement policy.
- Circle profiles include android, mcp-policy and marketing. Their manifest and canonical
setup caller remain mandatory; a missing configuration is a finding.
- MCP, business-platform and the organization `.github` repository use approved native
+18 -6
View File
@@ -61,11 +61,23 @@ parser is pinned to PyYAML 6.0.3 and uses its safe loader.
| Event | Workflow | Job contract |
| --- | --- | --- |
| develop | publish_snapshot | existing mvn_publish, snapshot |
| release/* | publish_release_candidate | existing mvn_publish, release_candidate |
| vX.Y.Z tag resolving to the checkout | publish_release | existing mvn_publish, release |
| main, feature/*, hotfix/*, other/unknown branch | test_only | existing mvn_test_only |
| Disabled publication or unrecognized/unproven tag | test_only | full verification, no publication |
| develop | verify | existing mvn_publish, snapshot |
| release/* | verify | existing mvn_publish, release_candidate |
| vX.Y.Z tag resolving to the checkout | verify | existing mvn_publish, release |
| main, feature/*, hotfix/*, other/unknown branch | verify | existing mvn_test_only |
| Disabled publication or unrecognized/unproven tag | verify | full verification, no publication |
Generated `qbit`, `core`, `frontend-core`, and `frontend-material` pipelines use
the stable workflow name `verify`. All selected jobs, dependencies and filters
remain intact; this adds no executor or duplicate build. Other profiles retain
their existing workflow names and native checks.
When upgrading from legacy workflow names, upgrade the immutable RC worker pin
alongside the orb so finalization recognizes both names. Confirm the CircleCI
app's successful `verify` check on the exact migration commit before replacing
legacy required contexts. Require `setup` and `verify` on managed branches,
preserving other required checks and controls. Human approval is not required;
automated checks authorize the migration.
The pipeline SHA must match the checkout when Circle supplies it. A tagged release
must resolve to that checkout; missing local tags are fetched individually. A main
@@ -182,7 +194,7 @@ manifest validation already ran in setup. It adds no new security scanner and
makes no security-scanning claim. Keep docs-only disabled if those paths are build
inputs or packaged application content.
The light route keeps workflow `test_only` and names its job
The light route keeps workflow `verify` and names its job
`qqq-orb/mvn_test_only`. Verify actual CircleCI/GitHub status contexts on the canary
before changing any protection rule; workflow/job identity alone is not proof of
the hosted context name. Current protections must not be removed speculatively.
+12
View File
@@ -0,0 +1,12 @@
# Shared QBit source scanning
Generate the thin caller with `python3 src/scripts/generate_auxiliary_workflow.py qbit-codeql --ref <immutable-worker-sha>`.
The reusable worker selects supported languages from tracked files in the actual checked-out branch. Historical Java branches therefore do not request an absent Actions database. Java, JavaScript/TypeScript, Python and Actions share one job; Kotlin selects an actual build instead of silently omitting Kotlin from no-build Java analysis.
The caller scans pull requests, main/develop/release/hotfix pushes and weekly, cancels superseded PR scans, and contains no scan implementation. Set `codeql_worker_sha` in the estate enforcement policy to require the exact canonical caller independently of other native worker pins. Existing default-setup repositories remain unchanged until migrated.
Roll out to both active default and maintained branches. Verify the worker on representative branch contents and preserve existing query coverage before replacing GitHub default setup; do not run duplicate default and advanced scanners permanently. Default setup cannot select languages independently for historical branches. A branch with no supported application or workflow source is a placeholder to classify, not a passing application scan.
Local tests execute the embedded language-selection step against real git indexes, including absent Actions, untracked generated files, mixed source, Kotlin and placeholders. Hosted CodeQL database creation and SARIF upload must also pass before claiming rollout completion.
References: [GitHub setup types](https://docs.github.com/en/code-security/concepts/code-scanning/setup-types) and [CodeQL build modes](https://docs.github.com/en/code-security/reference/code-scanning/codeql/build-options-for-compiled-languages).
+13 -1
View File
@@ -8,11 +8,16 @@ recognized contracts:
|---|---|---|
| `release_on_main` | boolean, default false | Main pushes select the existing release job when release publishing is enabled. |
| `release_tag_policy` | `stable`, `v-prefix`, `none` | Stable `v1.2.3`, the existing broad `/v.*/` contract, or no tags. |
| `test_branch_policy` | `all`, `non-release`, `non-release-no-integration`, `feature`, `non-release-no-main` | Selects one of the five audited test branch exclusions. |
| `test_branch_policy` | `all`, `non-release`, `non-release-no-integration`, `feature`, `non-release-no-main`, `non-release-no-hotfix`, `non-release-no-main-hotfix`, `feature-no-hotfix` | Selects an audited test branch exclusion. |
| `publish.hotfix` | boolean, default false | Runs the existing Maven hotfix publisher for nonempty `hotfix/*` branch events only when explicitly enabled. |
`feature` preserves the original exclusion of develop, main, release branches,
and integration-prefixed branches; it is not a new feature-prefix-only rule.
All policies continue to exclude the RC coordination lock branch.
The three hotfix exclusions preserve the corresponding legacy policies while
the single shared publisher handles hotfix events. Existing manifests do not
enable hotfix publication implicitly; migration enables it only when the source
configuration contains the recognized hotfix publisher and branch filter.
No consumer-supplied regex, command, job, or arbitrary branch-publication rule
is accepted.
@@ -39,3 +44,10 @@ Focused tests cover the immutable inventory, unknown-filter rejection, canonical
caller agreement, main opt-in, broad/stable tag selection, HEAD provenance,
publication opt-out and test exclusions. The packed generator suite also exercises
main, broad RC-shaped tags, and excluded integration branches.
Fifteen additional main-branch fixtures exercise 195 branch/tag cases,
checking both test and publication selection. Source and packed generator tests
also verify hotfix opt-in, its boolean validation, and unchanged main behavior.
CMS's original YAML is retained with its immutable source SHA: its unquoted
stable-tag regex is invalid YAML. The fixture repairs only that quoting. Both
audited spellings of the stable regex map to the same ASCII-digit tag policy;
arbitrary regexes remain rejected.
+16
View File
@@ -0,0 +1,16 @@
# Application-free branch verification
Use `profile: repository-policy` only on a branch containing repository policies,
documentation, workflow configuration and optional unused Maven wrappers. It uses
the same canonical setup caller as other profiles and one small Python verification
job. It checks tracked YAML/JSON syntax and rejects application manifests, source
files, unknown file types and symlinks. It never builds or publishes an application.
The manifest accepts only `profile`, `orb_version` and `default_branch`. The latter
remains the repository's actual default branch. A future application promotion must
select its real build profile in the same change; adding source without doing so fails.
Estate policy can set `branch_profiles: {main: repository-policy}` without changing
the default branch's application profile. Require the CircleCI `setup` and `verify`
checks on the policy branch. This profile does not establish application release
readiness or replace the build/release checks on source-bearing branches.
+12 -1
View File
@@ -26,7 +26,8 @@ https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-t
- Finalization retrieves the unique unexpired intent artifact and verifies its
originating successful `release-rc.yml` run, repository, trigger and attempt.
Callback fields must match that immutable intent.
- The exact merge SHA must have a successful `publish_release_candidate` Check Run
- The exact merge SHA must have a successful `verify` (or legacy
`publish_release_candidate`) Check Run
from CircleCI app ID `18001`; unrelated job statuses cannot authorize release.
Terminal publisher failure stops finalization immediately. Pending/absent publication cannot authorize a tag.
The RC bump must have exactly that one parent and the release branch must still
@@ -107,3 +108,13 @@ is gone, the completed release needs no recovery dispatch. Ordinary callbacks an
scheduled reconciliation retain the requirement for a successful Circle publisher.
Central status API: https://central.sonatype.org/publish/publish-portal-api/
### Auditing independent worker revisions
The estate audit accepts `rc_worker_sha` in a profile or repository policy. For
example, set `profiles.qbit.rc_worker_sha` to the approved immutable RC worker
commit and keep `--worker-sha` at the approved revision for unchanged native
workers. The audit still compares every caller to its exact canonical template
and rejects mutable or mismatched pins. This avoids a fleet-wide native caller
update when only RC release behavior changes. Required-check policy is separate;
omitting it continues to report enforcement as unknown.
+3 -2
View File
@@ -12,8 +12,9 @@ text = source.read_text(encoding="utf-8")
start = text.index("# BEGIN PROFILE RESOURCES")
end = text.index("# END PROFILE RESOURCES") + len("# END PROFILE RESOURCES")
embedded = ["# Generated by scripts/pack_pipeline.py; edit source/resources, then regenerate.", "import base64", "import zlib"]
for variable, name in (("CORE_TEMPLATE", "core"), ("MATERIAL_TEMPLATE", "frontend-material"), ("VOYAGE_TEMPLATE", "voyage"), ("NEXT_TEMPLATE", "frontend-next"), ("ALL_TEMPLATE", "all"), ("STARTER_TEMPLATE", "app-starter"), ("ANDROID_TEMPLATE", "android"), ("POLICY_TEMPLATE", "mcp-policy"), ("MARKETING_TEMPLATE", "marketing")):
payload = base64.b64encode(zlib.compress((source.parent / "templates" / (name + ".json")).read_bytes(), level=9)).decode("ascii")
for variable, name in (("CORE_TEMPLATE", "core"), ("MATERIAL_TEMPLATE", "frontend-material"), ("VOYAGE_TEMPLATE", "voyage"), ("NEXT_TEMPLATE", "frontend-next"), ("ALL_TEMPLATE", "all"), ("STARTER_TEMPLATE", "app-starter"), ("ANDROID_TEMPLATE", "android"), ("POLICY_TEMPLATE", "mcp-policy"), ("REPOSITORY_POLICY_TEMPLATE", "repository-policy"), ("MARKETING_TEMPLATE", "marketing")):
resource = (source.parent / "templates" / (name + ".json")).read_text(encoding="utf-8").encode("utf-8")
payload = base64.b64encode(zlib.compress(resource, level=9)).decode("ascii")
embedded.append(f'{variable} = json.loads(zlib.decompress(base64.b64decode("{payload}")))')
packed = text[:start] + "\n".join(embedded) + text[end:]
if len(packed.encode("utf-8")) > 100_000:
+52 -8
View File
@@ -23,6 +23,7 @@ ALL_TEMPLATE = json.loads((_TEMPLATE_DIR / "all.json").read_text(encoding="utf-8
STARTER_TEMPLATE = json.loads((_TEMPLATE_DIR / "app-starter.json").read_text(encoding="utf-8"))
ANDROID_TEMPLATE = json.loads((_TEMPLATE_DIR / "android.json").read_text(encoding="utf-8"))
POLICY_TEMPLATE = json.loads((_TEMPLATE_DIR / "mcp-policy.json").read_text(encoding="utf-8"))
REPOSITORY_POLICY_TEMPLATE = json.loads((_TEMPLATE_DIR / "repository-policy.json").read_text(encoding="utf-8"))
MARKETING_TEMPLATE = json.loads((_TEMPLATE_DIR / "marketing.json").read_text(encoding="utf-8"))
# END PROFILE RESOURCES
@@ -69,7 +70,7 @@ def load_manifest(path, setup_version):
if not isinstance(data, dict):
raise ManifestError("Manifest must be a mapping")
profile = data.get("profile")
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "marketing"):
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "repository-policy", "marketing"):
raise ManifestError("Unsupported profile")
allowed = ("profile", "orb_version", "default_branch", "docs_only", "publish")
if profile in ("qbit", "core"):
@@ -78,7 +79,7 @@ def load_manifest(path, setup_version):
allowed += ("release_completion", "release_on_main", "release_tag_policy", "test_branch_policy")
elif profile == "cli":
allowed += ("publish_context",)
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "marketing"):
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "repository-policy", "marketing"):
allowed = ("profile", "orb_version", "default_branch")
elif profile == "all":
allowed = ("profile", "orb_version", "default_branch", "publish", "publish_context")
@@ -110,6 +111,8 @@ def load_manifest(path, setup_version):
publish_keys = ("release",)
if profile in ("qbit", "core"):
publish_keys += ("asciidoc",)
if profile == "qbit":
publish_keys += ("hotfix",)
if profile not in ("qbit", "cli", "all", "app-starter", "voyage", "frontend-next"):
publish_keys += ("feature", "hotfix")
publishing = mapping(data.get("publish", {}), publish_keys, "publish")
@@ -126,7 +129,8 @@ def load_manifest(path, setup_version):
if tag_policy not in ("stable", "v-prefix", "none"):
raise ManifestError("Unsupported QBit release tag policy")
test_policy = data.get("test_branch_policy", "all")
if test_policy not in ("all", "non-release", "non-release-no-integration", "feature", "non-release-no-main"):
if test_policy not in ("all", "non-release", "non-release-no-integration", "feature", "non-release-no-main",
"non-release-no-hotfix", "non-release-no-main-hotfix", "feature-no-hotfix"):
raise ManifestError("Unsupported QBit test branch policy")
return {"profile": profile, "orb_version": version, "default_branch": default_branch,
"java_version": java, "node_version": node, "browsers": boolean(data, "browsers", True),
@@ -136,7 +140,8 @@ def load_manifest(path, setup_version):
"release_tag_policy": tag_policy, "test_branch_policy": test_policy,
"docs_only": boolean(data, "docs_only", False),
"publish_context": publish_context,
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all"))
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all", "repository-policy")
and not (profile == "qbit" and key == "hotfix"))
for key in publish_keys}}
@@ -171,9 +176,28 @@ def proven_docs_only(config, branch, tag):
return base.decode().strip() if eligible else False
def proven_ci_maintenance(branch, tag):
"""Keep application tests, but do not publish for isolated CI declarations."""
if tag or not (branch == "main" or branch == "develop" or branch.startswith(("release/", "hotfix/"))):
return False
subject = git("log", "-1", "--format=%s")
if not subject or not re.match(rb"ci(?:\([^\r\n()]+\))?: ", subject):
return False
parents = git("rev-list", "--parents", "-n", "1", "HEAD")
if not parents or not re.fullmatch(rb"[0-9a-f]{40} [0-9a-f]{40}\n", parents):
return False
base = parents.split()[1].decode("ascii")
changed = git("diff", "--no-ext-diff", "--no-renames", "--name-only", "-z", base, "HEAD")
if not changed:
return False
allowed = {b".qqq-ci.yml", b".circleci/config.yml", b".github/workflows/codeql.yml",
b".github/workflows/release-rc.yml"}
return all(path in allowed for path in changed.rstrip(b"\0").split(b"\0"))
def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
if tag:
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag):
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag, flags=re.ASCII):
return "test"
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if not tagged and git("fetch", "--no-tags", "origin", f"refs/tags/{tag}:refs/tags/{tag}") is not None:
@@ -191,21 +215,29 @@ def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
def qbit_template(config, branch, tag):
"""Profile-owned template; add future profiles as reviewed renderers, not raw YAML."""
kind = event_kind(branch, tag, config.get("release_on_main", False), config.get("release_tag_policy", "stable"))
if not tag and re.fullmatch(r"hotfix/.+", branch) and config["publish"].get("hotfix", False):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
result = {"version": 2.1, "orbs": {"qqq-orb": "kingsrook/qqq-orb@" + config["orb_version"]}}
filters = ({"branches": {"ignore": "/.*/"}, "tags": {"only": "/.*/"}} if tag else
{"branches": {"ignore": "automation/qbit-rc-pending"}, "tags": {"ignore": "/.*/"}})
if kind == "test" and not tag:
if kind == "test" and not tag and not maintenance:
ignores = {"all": None, "non-release": "/(develop|release/.*)/",
"non-release-no-integration": "/(develop|release/.*|integration.*)/",
"feature": "/(develop|main|release/.*|integration.*)/",
"non-release-no-main": "/(develop|main|release/.*)/"}
"non-release-no-main": "/(develop|main|release/.*)/",
"non-release-no-hotfix": "/(develop|release/.*|hotfix/.*)/",
"non-release-no-main-hotfix": "/(develop|main|release/.*|hotfix/.*)/",
"feature-no-hotfix": "/(develop|main|release/.*|hotfix/.*|integration.*)/"}
ignored = ignores[config.get("test_branch_policy", "all")]
if ignored:
filters["branches"]["ignore"] = ["automation/qbit-rc-pending", ignored]
docs_base = proven_docs_only(config, branch, tag) if kind == "test" else False
docs_base = proven_docs_only(config, branch, tag) if kind == "test" and not maintenance else False
if docs_base:
# Keep the existing workflow/result identity even when Maven is not needed.
result["jobs"] = {"docs_only": {"resource_class": "small", "docker": [{"image": "cimg/base:2026.03"}],
@@ -407,7 +439,13 @@ def voyage_template(config, branch, tag):
return json.loads(json.dumps(VOYAGE_TEMPLATE).replace("__ORB_VERSION__", config["orb_version"]))
def repository_policy_template(config, branch, tag):
del config, branch, tag
return copy.deepcopy(REPOSITORY_POLICY_TEMPLATE)
PROFILE_TEMPLATES = {"qbit": qbit_template, "core": core_template,
"repository-policy": repository_policy_template,
"frontend-material": frontend_template, "frontend-core": frontend_template,
"cli": cli_template, "voyage": voyage_template, "frontend-next": next_template, "all": all_template, "app-starter": starter_template, "android": android_template, "mcp-policy": policy_template, "marketing": marketing_template}
@@ -441,6 +479,12 @@ def main():
if expected and (not re.fullmatch(r"[0-9a-fA-F]{40}", expected) or git("rev-parse", "HEAD") != (expected + "\n").encode()):
raise ManifestError("Checkout does not match the pipeline commit")
output = PROFILE_TEMPLATES[config["profile"]](config, os.environ.get("CIRCLE_BRANCH", ""), os.environ.get("CIRCLE_TAG", ""))
if config["profile"] in ("qbit", "core", "frontend-core", "frontend-material"):
# One whole-workflow check follows the same commit from feature to release.
# Keep every selected job, dependency and filter; add no runner or duplicate build.
if len(output["workflows"]) != 1:
raise ManifestError("Stable verification requires exactly one selected workflow")
output["workflows"] = {"verify": next(iter(output["workflows"].values()))}
parameters = continuation_parameters(config["profile"])
if config["profile"] == "core":
output["parameters"] = copy.deepcopy(CORE_TEMPLATE["parameters"])
+4
View File
@@ -20,6 +20,10 @@ sudo install -d /etc/needrestart/conf.d
# shellcheck disable=SC2016
printf '%s\n' '$nrconf{restart} = "l";' | sudo tee /etc/needrestart/conf.d/99-qqq-ci.conf >/dev/null
# Retry transient repository downloads within APT, including NodeSource's child process.
# Exhausted downloads retain APT's nonzero exit status and fail this build step.
printf '%s\n' 'Acquire::Retries "3";' | sudo tee /etc/apt/apt.conf.d/80-qqq-download-retries >/dev/null
# Ubuntu archive carries the same signed security suite; avoid the flaky HTTP mirror.
if [[ -f /etc/apt/sources.list ]]; then
sudo sed -i 's|http://security.ubuntu.com/ubuntu|https://archive.ubuntu.com/ubuntu|g' /etc/apt/sources.list
+52 -8
View File
@@ -24,6 +24,7 @@ ALL_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNq1T7EOgjAU3PmKpjNI
STARTER_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNp1UMGKwkAMvc9XlDmKtejR0yJ42MsKCl5EQnUzZbbTiZ0ZqiL9d1OpWqt7y3t5eS/JRUSRrNB5TVZOo8loPGwYcjvP8MI1o7IsY2aYkLm2mXdEedKSXwCL5QzW8+Xqe/EDIHmkvnkcyeXK0LFjFNAHIGvOD4rJP7plbVocPTrd7KSoLHwab2V7Kg7aIOxQkUPgi7RqZCo1Hod9rQ14Ci+ZnbQirdDGDjPtgzvHe4e/aINmI/ki3/ZslTaBH/m2W3N3mn3iuaMzy/s2n01Gg0T2BLX4Dz3re7UVd1SLWlwByepyJQ==")))
ANDROID_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtWG1v2zYQ/p5fQWgF6rSW3CXYMDhJ13YNkm5rkrrugKEKDFqiZCYUyZBUbC/wfvuO1LvtpmmBAcUwIYAp3vHuuXcqdzsIebdEaSq4N0R7wfd9uyPUVMPrHazhDfNYCRrDhhdRFTES0UG592Iv+CF45vULxjSSfsRoh7Hce7Ef7AV7HvCtnIYr0daQc2p8Q7Spt2CTLEiUG6Fae7DLcUasghJABcTPcDSjnJRQHKvBqeWMSYJzZrySsKo4PG2ItCA+NkeiGYmuRW7aYipNCvAJRfxU4ZgRPwKFbXUNxvaZnDu7dMeIAh1s+5HIMuC1MINBIXiOLOU1meap1zqxqterLdg0vn0YMAC0CaXy6XuQ4rQjsBVcpltSCvc0cLPrmCrkS/T3wFlSnhhc2VgOQp5QHqMA+WYpCUoQ0FOyQKEXPBlMc8ri7qngySJjoYd8G3QUSXS32i4YheFByNdxzWeEu6Rgc7zUn3VbxyMuqBOralJZveEgic3Myu9C+ho9WBma4OjhSgq7P6GqXF3utBRXObFRTxIriLOBYu9WlF5CIWQ+zXBK1lB5NnoWkzaK8rTj9rquhl0R+qACsP/sIBUiZWC1pPpg8dOP3gb0phq/kXLfWsbaQNx8kuUMA0Iftv2vquxIcE4iQ2JX3i8L6WMbp7WEXouJd3iImvAFbSp6/vzL8vDb6QDgeJnDAdz44f+G8C80hAyCyv2YSCaW/5UBe28WX+RTRvUMzamZIT3DisTorXUCimBJuKGY3ZfZmhhIvFwgSSXYRlnIh5C4j+5O3oxPP7yajM9/Oz4b/nxCzWk+RRc4urZ9D80VNaSlAlENZXSTU9hZhV7Iv0MvczOzxAgD55QkYC40hUxSBu31ALil0BScsEQ4iojWVgR0DFhhtewjLgySSogEwZ8s9JZqC/4g5FBmFNr1cpJroo4e9eTSzATfhzo5PHx88ef49PzscchpJoUy6EoLXr8IXS9zxRidBhY9BCUEnohhgHMmRgSqHnpYr8sSnI7HFxXtFPzIiNodhty6FvIDLCtIk5K/pwlL+tY9fZTIPjghJn2U6bSPZgTH0OXAWjIHJZUU+yhMNUGjHDyYkWOlhOqFXhkG3LgWLrO1Qlgk4Ik49HatGeCYUp6QhBOFjtZsDVxvmhTUXmPvbmNLSkzP1mYHWXF6U9yotDf0ZsZIPRwMYBgGKWRmPg0g9ANock+RFddvpNmndMPRXejZrBGK/uUsCz2bi68IZLVC7rDQAeG3VAn+EbzRStHQu1wTWj8gFBJGmkIalpKVjhvc8riE99Rlh7fabWS4iip8E9ifXmlmH9mIQF0f7T/bRdjmvZaCazLs6lfE5Iq7tAuYwHGv4it12JwFH1oPh559gbAFxRsTKeUuipaRJq4YqKYchjOPSM9y9xFk/i4SyhGdsHJdyHD1NXgHCeS/OR/c3Nz4RW+saqbRJonKqLafRTr0+jB9akKuZ8C2kZR/YJbXKfnWHuUpquSW1Vmjl3CbMg4xbJCFDQQ6dj+gsBRdiH3vRv3xgpom0+uG02o1EebWzLJrQCWUDa9lGUqUyKCCOQl50QdC7tS7ij8fnUxORi9f/348uRid/3r8y3hSZMG7mxsnalRK+gCobaM9gobY6TW2w32psAtoKnOhYiusm7khb65NPtymuBaMHEmGKUey6O9Okuv1ReqOxclWJaO6q8JF4SFzs/5KBWjXCRPz1qeqG8X2Auj6RHuclp+0rWHXfNbef7+sr+qfvzR8YpZXU7GYNl0Y94DZBqNDvFwbkgllGx8QJWmqoAxnZBvN/keBs6W7NEL8vDXyqvO+2pjL3JCF2WbSZgH77QHfteShgd9Z7fwDUg7law==")))
POLICY_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNqdVU1v2zAMvedXCL442Rq3aYEegubQQwYMWLGeBgxVESg2nWqVJU9S1gZB/vtIWXbsdCuw+WKJH4+PlEjtR4wlv8A6aXQyZ5fZ7IwkP8za4XaPa9ILJQvhYVoL78HqowqVhcmfwaLkIUpYpwt6WYkNoDrJZbU5r3f+yej5VTa7TDqrQ1w9nrWgFpzZ2hxWuRKOwiWuEkolnYHzULtB0CR/gvzZbH1ndELEbnWPdxRqUQVyn7XzGIDd777f3n3pQTTQpqqELsiwljWT0Xg6LaQTa4WFkfU0VnEaeESgxeI6u8iukh7c4Zj2/xD9Fs+CxbNgpVTg3iMcKn7FOL+5Yeny66eUa1nVxnq2E5XqNm7nuC6tqQj5Sck1i4p73HLNNVhrrGML9vDIdYy+ys1We5RdkEVpbMBcESesUnAdp1k6yexGmfU4/ZCRPp3MuSaesmRC78a1sD7DklrvXmT0YARGCsLpQDOSuNadvtxoL/UWKD7tvd31tATHTA163EFMmHCs7NnQhxUVmAUZZU6UsFJGFONy0qJGsmnbAOcp0XLe9nBPIC383EoLBRUslUV6xlIvvQJa5HiAG2N3tC7A5VbWHq9OUGE+oH36OESrpHNSbwisDKUpiUAXA7mVTJtQLMrlxBvVEeCEJH3NuWaixjJhzjzZdzkd5uwuBi4lqMLhtYxAB55MhljDK/FxwWZt9eA1h7q5bhl1xZIi0jFAj877NMiNroODxhCJwJFCDLAMPyzkv2Evh4B4ua3UnszaZivYfpDdYdh90Q2r3ASKkRsYnnCul03vlOhczDvadI6hTwZuR1dkwNh0kCl2aQav0o9nKADl4CTWLY6l9pKyMLeDK7Y913+ZQu3wHbW7MJeSF2OfS2Ve/vAO9Md/fCl6c/jta/E2wugw+g3AlOQ8")))
REPOSITORY_POLICY_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNqdVltv2zYUfs+vIPwiGYjotgGGIa0fis7YAnRO0HgtgigwaOnIZkKRGkkl9gL/951DXSK7aDNML+K56ON3bqSeTxgbPYJ10ujROXvH356S5t6sHIrPuCa7UDIXHpLKKJntegOacpM9gEXNbathvS3YZSnWgOZRJsv1pNr5jdHnZ/ztu1HvtW9Xd6cdqAVnapvBMlPCEY+RK4VSo97BeajcwaajbAPZg6l973RExNZ6wLtValEGchfaedyA+Q2wm49/fmaVsA7DOj10z0xZCp3TF00kZywpWSUrJluAJMmlEyuFqZJV0uY1CdzY1Y6gp9Nf+Bt+Nhog719S8X/If22rE9iLqsISCU/bFhaArazQ2eY/RJKmHz5EVzeLPy7nUaplWRnr2b0zOtWFNRim8BslV6y1XKHYu7l6VVmTgXO9alsqDh4Z8JmCErRfEBvh2GzR++xEqVLEJ2jHpgMYHlK2xHpWtY9vo7X00SmLlEsKqcDROvknuhvzHDKTQxzVvkh+jcbcYfg+jtL0TTR+Ab4ltjElbMwKYxmtsGasscsiKO6QF66Mb9TnqaZUWSEdsOsdtlw52xL23DBvBbZ9zpp5YA0n3I9gaL/n6PPFp9n8ekZE55cLFGj16fLq5mL+e7P8bXb5bT77ck0Sx/jkWhsLnSS8t3JVe4q1IRKeiEMuvbGZ0YVck3P5qJ+6N8/KPNqn2tVFIbctE446ArUupJD7bfMWOLphsStV8xbtgooeFliMCqyXSAJRsWFkAc43sJUp+bb5oMJk4JT332XCrg33prXuEOYeMt9r0Fia/CCuENuqlirnaytyFfIwlPmDD1V34L3UazdwO1IFT6RLdaZC9nVuC4o1JpFLt3QYutQP8Zi1vpwKyJV5AotK/LCP+fyF7HcdUUQfX6aOoFpcZuHvWuJhxgQLsTDMBPUKjjLttqeWGVISblkZJ7e493SK+ceaTp4sjjTYSSkeQSetxO+FjQaUsB0wBTX8FC6MlPM7BZOmfUL9BiizBadJd9h5eh0HCAsiX3rY+hg0Dhrqp92ojV/bPgwZjRPmsRkMPHEaU9OgfaJbp65tf57rv7C9Kzo+DufvPXOgsMteyzUFgw38WnSHoRzxRa7x0dyMB6SVdD4mLXeigKUyuAteDjHt1KUN1A+wQ+HDJA0QSeaE4xoQPNmwRpSN7vDP2bMCHYrmxvujA2qCo17TGdx0aDiu3mPWh7dFmzUahHrVKSkPzZWQ6h/cWN3lfdJJ4Q4bPRn7UGBMw78IsLI4+Hlo/zIGt/jxn8b36Cf7k38B2PmvaA==")))
MARKETING_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtW89v4ygUvuevQD7MSKsSp5lbD6OR9rRa7W1uk8oiGMc0GFzATbNV/vd9+EeaOHaqVpt6knBpbfh4wHufee9z3ZcRQsET04YrGdyh6fj2xrUoPTdw+wLXcJcVkluloSFYqmQ6DeuGH5Pxt/HtbQCoTTlspfQyEWq1MzbXmKaMLl+boPFBleZ/1fdo27M7XShVzKJ5wUUcERlHlhm7Y6QGS5Ixt7AShwGHS9xNC+VMvW4zmE7biJzQJVmwKCMSfpV7lXnWRjnbEVUZoGK3g+DXzMGQLiQSXNpZcINeW8o1NU3PKBdkvdJ8kVrEpbFECERTrTJeZAjjFbcpjllu9m24Ge8a2H6XcKZSVRg2C+7bK6VKWvZs97xcdy1gpmIe7DXft4YnXFhw14G/oWuuiYSQdvU55kix7pi07AsTRmyh2WwWjv8IWytuMKmyCX+uIQeI+1bLZtR393q9uTnOs1jRJdORoUT284tnQAtcYlquKnuiBpcRvWSWywVesbnhlrXhmi24sdo5KVikVI+5Ch8hnJirQ+hjwTUzXUFs8d0Hsw6mYVQza96IZo3qjOdVeYvAefaGqwDS7SfCRaRklHAZA9/L4zAhwrBrdmiZaKj78VgQwe2637EOhRvUifKVP0E+HELwPjj2WPhqxIlCl3G5u4pgcpjhq87IKiUc4qGj8NmCTAFFi15HObHp7vLD5gLXiPGDUbLXTF37lJluGXONcI6aTvTlC2I0VejryyywCgqcWXAHl1AYMVNd5hRKpLvbyWSz2XxF39H7VuHZ/EE2Q7orNBwzb2fFEtZ53Psj6YRBHO2OrUMaxOyJCZV74XRNwmkb9YusjU+6u3OoZU/pAF97DuVyXyv6WnEo9vna7nOdXr8trMqrvDBpv88rKK52XkI/7dVhyoiw6fYJDvv6lXaVTvBtMpn0QOJ5bzI8Fu4jGQb1HojoaPWB+hM4ahP8KL2OlXjg1t+Ym506ASriBbDG64Sr0glN1C9TJ5xyd2ehE07oAK8ThnK51wleJwzFPq8TPtfpXid4nTAYNzt1gmaCEcMw7DvmMbHMKwb/SVZNCv+JzjCuON/vbwbwlhcuZxEfr3K8yjkLqnpJ9BtHyOsnr58ugshVbR/VM/ezuMLhBvcODhzy/8NhuSgd0qmCc63igtrquPby93r+YJYRLv2LvsES1XuKmGFfvJ2KKEXuXrpFNIZckKt+llQ4TGNc4tq+2Q4P/lbJzxWTdv1zpUIqCgP7xvAsqdy0RzH5xLWSGaDdSHcIdhhOlM5ICViCMXjA/2UdqH2StngXvklasNCw0DJJpDXl2LCGh83UxJ3R4zXJxIEFHsEuIwaBKuUb5T+chfEh62tks9Z/qkigP/+63PR6Mvr6Oub/cvR+XeL+e3q0Gf0Hgnpc+w==")))
@@ -69,7 +70,7 @@ def load_manifest(path, setup_version):
if not isinstance(data, dict):
raise ManifestError("Manifest must be a mapping")
profile = data.get("profile")
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "marketing"):
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "repository-policy", "marketing"):
raise ManifestError("Unsupported profile")
allowed = ("profile", "orb_version", "default_branch", "docs_only", "publish")
if profile in ("qbit", "core"):
@@ -78,7 +79,7 @@ def load_manifest(path, setup_version):
allowed += ("release_completion", "release_on_main", "release_tag_policy", "test_branch_policy")
elif profile == "cli":
allowed += ("publish_context",)
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "marketing"):
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "repository-policy", "marketing"):
allowed = ("profile", "orb_version", "default_branch")
elif profile == "all":
allowed = ("profile", "orb_version", "default_branch", "publish", "publish_context")
@@ -110,6 +111,8 @@ def load_manifest(path, setup_version):
publish_keys = ("release",)
if profile in ("qbit", "core"):
publish_keys += ("asciidoc",)
if profile == "qbit":
publish_keys += ("hotfix",)
if profile not in ("qbit", "cli", "all", "app-starter", "voyage", "frontend-next"):
publish_keys += ("feature", "hotfix")
publishing = mapping(data.get("publish", {}), publish_keys, "publish")
@@ -126,7 +129,8 @@ def load_manifest(path, setup_version):
if tag_policy not in ("stable", "v-prefix", "none"):
raise ManifestError("Unsupported QBit release tag policy")
test_policy = data.get("test_branch_policy", "all")
if test_policy not in ("all", "non-release", "non-release-no-integration", "feature", "non-release-no-main"):
if test_policy not in ("all", "non-release", "non-release-no-integration", "feature", "non-release-no-main",
"non-release-no-hotfix", "non-release-no-main-hotfix", "feature-no-hotfix"):
raise ManifestError("Unsupported QBit test branch policy")
return {"profile": profile, "orb_version": version, "default_branch": default_branch,
"java_version": java, "node_version": node, "browsers": boolean(data, "browsers", True),
@@ -136,7 +140,8 @@ def load_manifest(path, setup_version):
"release_tag_policy": tag_policy, "test_branch_policy": test_policy,
"docs_only": boolean(data, "docs_only", False),
"publish_context": publish_context,
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all"))
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all", "repository-policy")
and not (profile == "qbit" and key == "hotfix"))
for key in publish_keys}}
@@ -171,9 +176,28 @@ def proven_docs_only(config, branch, tag):
return base.decode().strip() if eligible else False
def proven_ci_maintenance(branch, tag):
"""Keep application tests, but do not publish for isolated CI declarations."""
if tag or not (branch == "main" or branch == "develop" or branch.startswith(("release/", "hotfix/"))):
return False
subject = git("log", "-1", "--format=%s")
if not subject or not re.match(rb"ci(?:\([^\r\n()]+\))?: ", subject):
return False
parents = git("rev-list", "--parents", "-n", "1", "HEAD")
if not parents or not re.fullmatch(rb"[0-9a-f]{40} [0-9a-f]{40}\n", parents):
return False
base = parents.split()[1].decode("ascii")
changed = git("diff", "--no-ext-diff", "--no-renames", "--name-only", "-z", base, "HEAD")
if not changed:
return False
allowed = {b".qqq-ci.yml", b".circleci/config.yml", b".github/workflows/codeql.yml",
b".github/workflows/release-rc.yml"}
return all(path in allowed for path in changed.rstrip(b"\0").split(b"\0"))
def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
if tag:
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag):
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag, flags=re.ASCII):
return "test"
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if not tagged and git("fetch", "--no-tags", "origin", f"refs/tags/{tag}:refs/tags/{tag}") is not None:
@@ -191,21 +215,29 @@ def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
def qbit_template(config, branch, tag):
"""Profile-owned template; add future profiles as reviewed renderers, not raw YAML."""
kind = event_kind(branch, tag, config.get("release_on_main", False), config.get("release_tag_policy", "stable"))
if not tag and re.fullmatch(r"hotfix/.+", branch) and config["publish"].get("hotfix", False):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
result = {"version": 2.1, "orbs": {"qqq-orb": "kingsrook/qqq-orb@" + config["orb_version"]}}
filters = ({"branches": {"ignore": "/.*/"}, "tags": {"only": "/.*/"}} if tag else
{"branches": {"ignore": "automation/qbit-rc-pending"}, "tags": {"ignore": "/.*/"}})
if kind == "test" and not tag:
if kind == "test" and not tag and not maintenance:
ignores = {"all": None, "non-release": "/(develop|release/.*)/",
"non-release-no-integration": "/(develop|release/.*|integration.*)/",
"feature": "/(develop|main|release/.*|integration.*)/",
"non-release-no-main": "/(develop|main|release/.*)/"}
"non-release-no-main": "/(develop|main|release/.*)/",
"non-release-no-hotfix": "/(develop|release/.*|hotfix/.*)/",
"non-release-no-main-hotfix": "/(develop|main|release/.*|hotfix/.*)/",
"feature-no-hotfix": "/(develop|main|release/.*|hotfix/.*|integration.*)/"}
ignored = ignores[config.get("test_branch_policy", "all")]
if ignored:
filters["branches"]["ignore"] = ["automation/qbit-rc-pending", ignored]
docs_base = proven_docs_only(config, branch, tag) if kind == "test" else False
docs_base = proven_docs_only(config, branch, tag) if kind == "test" and not maintenance else False
if docs_base:
# Keep the existing workflow/result identity even when Maven is not needed.
result["jobs"] = {"docs_only": {"resource_class": "small", "docker": [{"image": "cimg/base:2026.03"}],
@@ -407,7 +439,13 @@ def voyage_template(config, branch, tag):
return json.loads(json.dumps(VOYAGE_TEMPLATE).replace("__ORB_VERSION__", config["orb_version"]))
def repository_policy_template(config, branch, tag):
del config, branch, tag
return copy.deepcopy(REPOSITORY_POLICY_TEMPLATE)
PROFILE_TEMPLATES = {"qbit": qbit_template, "core": core_template,
"repository-policy": repository_policy_template,
"frontend-material": frontend_template, "frontend-core": frontend_template,
"cli": cli_template, "voyage": voyage_template, "frontend-next": next_template, "all": all_template, "app-starter": starter_template, "android": android_template, "mcp-policy": policy_template, "marketing": marketing_template}
@@ -441,6 +479,12 @@ def main():
if expected and (not re.fullmatch(r"[0-9a-fA-F]{40}", expected) or git("rev-parse", "HEAD") != (expected + "\n").encode()):
raise ManifestError("Checkout does not match the pipeline commit")
output = PROFILE_TEMPLATES[config["profile"]](config, os.environ.get("CIRCLE_BRANCH", ""), os.environ.get("CIRCLE_TAG", ""))
if config["profile"] in ("qbit", "core", "frontend-core", "frontend-material"):
# One whole-workflow check follows the same commit from feature to release.
# Keep every selected job, dependency and filter; add no runner or duplicate build.
if len(output["workflows"]) != 1:
raise ManifestError("Stable verification requires exactly one selected workflow")
output["workflows"] = {"verify": next(iter(output["workflows"].values()))}
parameters = continuation_parameters(config["profile"])
if config["profile"] == "core":
output["parameters"] = copy.deepcopy(CORE_TEMPLATE["parameters"])
+1 -1
View File
@@ -116,7 +116,7 @@ def reconcile(api, repo):
validate_run(run, intent, repo, merge, run_id)
checks = api.request(prefix + 'commits/' + merge + '/check-runs?filter=latest&per_page=100')
publishers = [c for c in checks.get('check_runs', [])
if c.get('name') == 'publish_release_candidate' and c.get('app', {}).get('id') == 18001]
if c.get('name') in ('publish_release_candidate', 'verify') and c.get('app', {}).get('id') == 18001]
if not publishers:
return 'publisher-running'
publisher = max(publishers, key=lambda c: c['id'])
+41 -1
View File
@@ -263,7 +263,7 @@
},
"concurrency": {
"group": "e2e-${{ github.ref }}",
"cancel-in-progress": "${{ github.event_name == 'pull_request' }}"
"cancel-in-progress": "${{ github.event_name == 'pull_request' }}"
},
"jobs": {
"validate": {
@@ -427,5 +427,45 @@
}
}
}
},
"qbit-codeql": {
"repository": null,
"profile": "qbit",
"path": ".github/workflows/codeql.yml",
"caller": {
"name": "CodeQL",
"on": {
"push": {
"branches": [
"main",
"develop",
"release/**",
"hotfix/**"
]
},
"pull_request": {},
"schedule": [
{
"cron": "41 16 * * 0"
}
],
"workflow_dispatch": {}
},
"permissions": {
"contents": "read",
"actions": "read",
"security-events": "write",
"packages": "read"
},
"concurrency": {
"group": "codeql-${{ github.event.pull_request.number || github.ref }}",
"cancel-in-progress": "${{ github.event_name == 'pull_request' }}"
},
"jobs": {
"analyze": {
"uses": "QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml@WORKER_SHA"
}
}
}
}
}
@@ -0,0 +1,35 @@
{
"version": 2.1,
"jobs": {
"validate-policy": {
"docker": [
{
"image": "cimg/python:3.12"
}
],
"resource_class": "small",
"steps": [
"checkout",
{
"run": {
"name": "Install the YAML parser",
"command": "python3 -m pip install --disable-pip-version-check PyYAML==6.0.3"
}
},
{
"run": {
"name": "Validate the application-free branch",
"command": "python3 \\<<'PYTHON'\nimport json\nfrom pathlib import Path\nimport subprocess\nimport xml.etree.ElementTree as ET\nimport yaml\n\npaths = subprocess.check_output(['git', 'ls-files', '-z']).decode('utf-8').split('\\0')\npaths = [Path(name) for name in paths if name]\nif not paths:\n raise SystemExit('No tracked policy files')\nnames = {'LICENSE', 'NOTICE', 'COPYING', 'CODEOWNERS', '.gitignore', '.gitattributes',\n '.editorconfig', 'mvnw', 'mvnw.cmd'}\nsuffixes = {'.md', '.rst', '.txt', '.adoc', '.yml', '.yaml', '.json', '.properties'}\nmanifests = {'pom.xml', 'package.json', 'cargo.toml', 'pyproject.toml', 'go.mod',\n 'build.gradle', 'build.gradle.kts', 'settings.gradle', 'settings.gradle.kts'}\nfor path in paths:\n if path.is_symlink() or path.name.lower() in manifests:\n raise SystemExit(f'Application or symlink requires a build profile: {path}')\n if path.as_posix() == '.mvn/wrapper/maven-wrapper.jar':\n continue\n if path.as_posix() == 'checkstyle/config.xml':\n ET.fromstring(path.read_text(encoding='utf-8'))\n continue\n if path.name not in names and path.suffix.lower() not in suffixes:\n raise SystemExit(f'Unsupported policy file; select a build profile: {path}')\n text = path.read_text(encoding='utf-8')\n if path.suffix.lower() in ('.yml', '.yaml'):\n list(yaml.safe_load_all(text))\n elif path.suffix.lower() == '.json':\n json.loads(text)\nprint(f'Validated {len(paths)} tracked policy/documentation files; no application build or publication')\nPYTHON\n"
}
}
]
}
},
"workflows": {
"verify": {
"jobs": [
"validate-policy"
]
}
}
}
+1
View File
@@ -45,6 +45,7 @@ PROFILE_PARAMETERS = {
"app-starter": {},
"android": {},
"mcp-policy": {},
"repository-policy": {},
"marketing": {},
}
File diff suppressed because it is too large. Load diff
+161
View File
@@ -26,6 +26,148 @@ class Snapshot:
class AuditTests(unittest.TestCase):
def test_all_scanner_and_release_requirements_follow_branch_profile(self):
entry = dict(ENTRY, name='qqq-all', nameWithOwner='QRun-IO/qqq-all')
scanner = '.github/workflows/codeql.yml'
release = audit.auxiliary.BRIDGES['all-release']['path']
for profile in ('all', 'repository-policy'):
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({'profile': profile, 'orb_version': VERSION, 'default_branch': 'develop'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller(profile, VERSION))}
policy = dict(POLICY, profile=profile, codeql_worker_sha=SHA)
report = audit.audit(entry, Snapshot(files), VERSION, SHA, policy)
missing = {f.get('path') for f in report['findings'] if f['code'] == 'missing_native_maintenance_bridge'}
self.assertIn(scanner, missing)
self.assertEqual(profile == 'all', release in missing)
files[scanner] = audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-codeql', SHA), sort_keys=False)
if profile == 'all':
files[release] = audit.generator.yaml.safe_dump(audit.auxiliary.canonical('all-release', SHA), sort_keys=False)
self.assertEqual([], audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings'])
files[scanner] = files[scanner].replace(SHA, 'b' * 40)
self.assertIn('native_maintenance_bridge_drift', [f['code'] for f in audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings']])
files[release] = 'jobs: {}'
self.assertTrue(any(f['code'] == 'native_maintenance_bridge_drift' and f.get('path') == release for f in audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings']))
def test_schema_drift_bridge_tracks_branch_source_without_exempting_default(self):
entry = dict(ENTRY, name='qbit-authentik', nameWithOwner='QRun-IO/qbit-authentik')
path = '.github/workflows/schema-drift.yml'
cases = [
('main', {}, False),
('main', {'src/test/java/example/SchemaDriftTest.java': 'class SchemaDriftTest {}'}, True),
('develop', {}, True),
]
for branch, files, required in cases:
with self.subTest(branch=branch, files=files):
snapshot = Snapshot(files)
snapshot.branch, snapshot.default_branch = branch, 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, POLICY)
missing = any(f['code'] == 'missing_native_maintenance_bridge' and f.get('path') == path for f in report['findings'])
self.assertEqual(required, missing)
snapshot = Snapshot({path: 'jobs: {}'})
snapshot.branch, snapshot.default_branch = 'main', 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, POLICY)
self.assertIn('native_maintenance_bridge_drift', [f['code'] for f in report['findings']])
def test_placeholder_can_retain_canonical_release_and_scanner_bridges(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({'profile': 'repository-policy', 'orb_version': VERSION, 'default_branch': 'develop'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('repository-policy', VERSION)),
audit.rc_worker.CALLER_PATH: audit.rc_worker.generate(SHA),
'.github/workflows/codeql.yml': audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-codeql', SHA), sort_keys=False)}
snapshot = Snapshot(files)
snapshot.branch = 'main'
snapshot.default_branch = 'develop'
policy = dict(POLICY, branch_profiles={'main': 'repository-policy'}, rc_worker_sha=SHA, codeql_worker_sha=SHA)
self.assertEqual([], audit.audit(entry, snapshot, VERSION, SHA, policy)['findings'])
files['.github/workflows/codeql.yml'] = files['.github/workflows/codeql.yml'].replace(SHA, 'b' * 40)
self.assertTrue(any(f['code'] == 'native_maintenance_bridge_drift' for f in audit.audit(entry, snapshot, VERSION, SHA, policy)['findings']))
def test_placeholder_profile_is_scoped_to_selected_branch(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({'profile': 'repository-policy', 'orb_version': VERSION, 'default_branch': 'develop'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('repository-policy', VERSION))}
snapshot = Snapshot(files)
snapshot.branch = 'main'
snapshot.default_branch = 'develop'
policy = dict(POLICY, branch_profiles={'main': 'repository-policy'})
self.assertEqual([], audit.audit(entry, snapshot, VERSION, SHA, policy)['findings'])
snapshot.branch = 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, policy)
self.assertEqual('qbit', report['profile'])
self.assertTrue(any(f['code'] == 'profile_mismatch' for f in report['findings']))
def test_qbit_scanner_policy_requires_exact_canonical_worker(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
manifest = {'profile': 'qbit', 'orb_version': VERSION, 'default_branch': 'develop',
'publish': {'snapshot': False, 'release_candidate': False, 'release': False}}
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump(manifest),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('qbit', VERSION))}
policy = dict(POLICY, codeql_worker_sha='b' * 40)
report = audit.audit(entry, Snapshot(files), VERSION, SHA, policy)
self.assertTrue(any(f['code'] == 'missing_native_maintenance_bridge' for f in report['findings']))
files['.github/workflows/codeql.yml'] = audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-codeql', 'b' * 40), sort_keys=False)
self.assertEqual([], audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings'])
files['.github/workflows/codeql.yml'] = files['.github/workflows/codeql.yml'].replace('b' * 40, 'a' * 40)
report = audit.audit(entry, Snapshot(files), VERSION, SHA, policy)
self.assertTrue(any(f['code'] == 'native_maintenance_bridge_drift' for f in report['findings']))
def test_live_snapshot_reads_requested_branch_and_keeps_default_identity(self):
entry = dict(ENTRY, sourceBranch='release/1.2')
replies = [{'default_branch': 'develop', 'archived': False}, {'sha': SHA},
{'truncated': False, 'tree': []}]
with patch.object(audit, 'gh', side_effect=replies) as gh:
snapshot = audit.Snapshot(entry, live=True)
self.assertEqual('develop', snapshot.default_branch)
self.assertEqual('release/1.2', snapshot.branch)
self.assertEqual('repos/QRun-IO/qctl/commits/release%2F1.2', gh.call_args_list[1].args[0])
def test_other_source_branch_does_not_change_manifest_default_branch(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
manifest = {'profile': 'qbit', 'orb_version': VERSION, 'default_branch': 'develop',
'publish': {'snapshot': False, 'release_candidate': False, 'release': False}}
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump(manifest),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('qbit', VERSION))}
snapshot = Snapshot(files)
snapshot.branch = 'main'
snapshot.default_branch = 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, POLICY)
self.assertEqual([], report['findings'])
self.assertEqual('main', report['source_branch'])
self.assertEqual('develop', report['default_branch'])
def test_selected_branch_read_failure_is_unknown_not_default_fallback(self):
captured = []
def factory(entry, live):
captured.append(entry)
raise audit.ReadError('branch unavailable')
report = audit.run([ENTRY], VERSION, SHA, {}, live=True,
snapshot_factory=factory, source_branch='main')
self.assertEqual('main', captured[0]['sourceBranch'])
self.assertEqual('unknown', report['results'][0]['status'])
self.assertEqual('main', report['results'][0]['source_branch'])
self.assertNotIn('sourceBranch', ENTRY)
def test_omitted_source_branch_still_reads_repository_default(self):
replies = [{'default_branch': 'develop', 'archived': False}, {'sha': SHA},
{'truncated': False, 'tree': []}]
with patch.object(audit, 'gh', side_effect=replies) as gh:
snapshot = audit.Snapshot(ENTRY, live=True)
self.assertEqual('develop', snapshot.branch)
self.assertEqual('develop', snapshot.default_branch)
self.assertEqual('repos/QRun-IO/qctl/commits/develop', gh.call_args_list[1].args[0])
def test_explicit_source_branch_cannot_relabel_offline_snapshot(self):
with self.assertRaises(ValueError):
audit.run([ENTRY], VERSION, SHA, {}, source_branch='main')
with self.assertRaises(audit.ReadError):
audit.Snapshot(dict(ENTRY, sourceBranch='main'), live=False)
def test_invalid_source_branch_is_rejected_before_api_calls(self):
for branch in ('', 'main?ref=develop', 'main\n', 123):
with self.subTest(branch=branch), patch.object(audit, 'gh') as gh:
with self.assertRaises(audit.ReadError):
audit.Snapshot(dict(ENTRY, sourceBranch=branch), live=True)
gh.assert_not_called()
def native(self, files=None, checks=None, policy=POLICY):
files = files if files is not None else {'.github/workflows/build.yml': audit.qctl.generate(SHA)}
return audit.audit(ENTRY, Snapshot(files, checks), VERSION, SHA, policy)
@@ -74,6 +216,25 @@ class AuditTests(unittest.TestCase):
report = audit.audit(entry, Snapshot({'.qqq-ci.yml': audit.generator.yaml.safe_dump(manifest)}), VERSION, SHA, POLICY)
self.assertIn('missing_rc_bridge', [f['code'] for f in report['findings']])
def test_rc_pin_policy_does_not_force_unchanged_auxiliary_worker_updates(self):
entry = dict(ENTRY, name='qbit-authentik', nameWithOwner='QRun-IO/qbit-authentik')
rc_sha = 'b' * 40
definition = audit.auxiliary.BRIDGES['qbit-schema-drift']
manifest = {'profile': 'qbit', 'orb_version': VERSION, 'default_branch': 'develop', 'release_completion': True}
files = {audit.rc_worker.CALLER_PATH: audit.rc_worker.generate(rc_sha),
definition['path']: audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-schema-drift', SHA), sort_keys=False),
'.qqq-ci.yml': audit.generator.yaml.safe_dump(manifest),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('qbit', VERSION))}
policy = dict(POLICY, rc_worker_sha=rc_sha)
self.assertEqual('pass', audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['status'])
for pin in (SHA, 'main', ''):
with self.subTest(pin=pin):
report = audit.audit(entry, Snapshot(files), VERSION, SHA, dict(POLICY, rc_worker_sha=pin))
self.assertTrue(report['findings'])
files[definition['path']] = files[definition['path']].replace(SHA, rc_sha)
report = audit.audit(entry, Snapshot(files), VERSION, SHA, policy)
self.assertIn('native_maintenance_bridge_drift', [f['code'] for f in report['findings']])
def test_qbit_rc_bridge_missing_drift_pin_and_duplicate(self):
entry = dict(ENTRY, name='qbit-test', nameWithOwner='QRun-IO/qbit-test')
path = audit.rc_worker.CALLER_PATH
+58
View File
@@ -0,0 +1,58 @@
"""CI maintenance retains Maven verification and never becomes a release request."""
import importlib.util
from pathlib import Path
import unittest
from unittest import mock
import yaml
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('pipeline', ROOT/'src/scripts/generate_pipeline.py')
pipeline = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pipeline)
class MaintenanceTests(unittest.TestCase):
def git_result(self, *args):
return {'log': self.subject, 'rev-list': self.parents, 'diff': self.paths}.get(args[0])
def setUp(self):
self.subject = b'ci: use central pipeline\n'
self.parents = b'a'*40 + b' ' + b'b'*40 + b'\n'
self.paths = b'.qqq-ci.yml\0.circleci/config.yml\0.github/workflows/codeql.yml\0.github/workflows/release-rc.yml\0'
def test_publishing_branches_keep_unfiltered_maven_tests(self):
manifest = {'profile': 'qbit', 'orb_version': '0.7.14', 'default_branch': 'develop',
'release_on_main': True, 'test_branch_policy': 'feature-no-hotfix',
'publish': {'hotfix': True}}
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.7.14')
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
for branch in ('main', 'develop', 'release/0.1', 'hotfix/1.2.3'):
with self.subTest(branch=branch):
result = pipeline.qbit_template(config, branch, '')
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/mvn_test_only']
self.assertEqual(job['filters']['branches'], {'ignore': 'automation/qbit-rc-pending'})
self.assertNotIn('branch_type', job)
def test_application_changes_and_unknown_files_publish_normally(self):
for paths in (b'pom.xml\0', b'.qqq-ci.yml\0src/Main.java\0', b'.github/workflows/custom-release.yml\0', b'', None):
self.paths = paths
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
def test_merge_root_and_unavailable_history_do_not_suppress_release(self):
for parents in (None, b'a'*40+b'\n', self.parents.rstrip()+b' '+b'c'*40+b'\n'):
self.parents = parents
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
def test_tags_and_release_requests_do_not_suppress_release(self):
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('main', 'v1.2.3'))
for subject in (b'Merge develop for next RC\n', b'fix: application\n', None):
self.subject = subject
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
if __name__ == '__main__':
unittest.main()
+66 -13
View File
@@ -192,14 +192,37 @@ class PipelineTests(unittest.TestCase):
return result
def assert_full(self, result, workflow="test_only"):
self.assertEqual(list(result["workflows"]), [workflow])
self.assertEqual(list(result["workflows"]), ["verify"])
self.assertNotIn("docs_only", result.get("jobs", {}))
job, options = next(iter(result["workflows"][workflow]["jobs"][0].items()))
job, options = next(iter(result["workflows"]["verify"]["jobs"][0].items()))
self.assertEqual(job, "qqq-orb/mvn_test_only" if workflow == "test_only" else "qqq-orb/mvn_publish")
if workflow != "test_only":
self.assertEqual(options["branch_type"], workflow.removeprefix("publish_"))
self.assertEqual(options["context"], ["qqq-maven-registry-credentials"])
self.assertFalse(options["compile_before_verify"])
return options
def test_ci_maintenance_source_and_packed_keep_verification(self):
self.write_manifest(test_branch_policy="feature-no-hotfix", publish={"hotfix": True})
workflow = self.repo / ".github/workflows/release-rc.yml"
workflow.parent.mkdir(parents=True)
workflow.write_text("# central RC caller migration\n", encoding="utf-8")
self.git("add", ".qqq-ci.yml", ".github/workflows/release-rc.yml")
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: adopt central pipeline")
for branch in ("develop", "release/0.1", "hotfix/0.1.1"):
result = self.render(CIRCLE_BRANCH=branch)
options = self.assert_full(result)
self.assertEqual(options["filters"]["branches"], {"ignore": "automation/qbit-rc-pending"})
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH=branch))
(self.repo / "App.java").write_text("application change\n", encoding="utf-8")
self.git("add", "App.java")
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: misleading title")
result = self.render(CIRCLE_BRANCH="release/0.1")
self.assert_full(result, "publish_release_candidate")
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="release/0.1"))
def test_defaults_and_options(self):
options = self.assert_full(self.render())
self.assertTrue(options["install_browsers"])
@@ -225,6 +248,33 @@ class PipelineTests(unittest.TestCase):
self.write_manifest(publish={"asciidoc": True})
self.assertTrue(self.assert_full(self.render(CIRCLE_BRANCH="develop"), "publish_snapshot")["publish_asciidoc"])
def test_qbit_hotfix_is_opt_in_in_source_and_packed_generator(self):
for packed in ([False, True] if PACKED else [False]):
self.write_manifest()
self.assert_full(self.render(packed=packed, CIRCLE_BRANCH="hotfix/1.2.3"))
self.write_manifest(publish={"hotfix": True})
options = self.assert_full(self.render(packed=packed, CIRCLE_BRANCH="hotfix/1.2.3"), "publish_hotfix")
self.assertNotIn("notify_rc_published", options)
self.assert_full(self.render(packed=packed, CIRCLE_BRANCH="main"))
self.assert_full(self.render(packed=packed, CIRCLE_BRANCH="hotfix/"))
self.write_manifest(publish={"hotfix": "true"})
self.render(packed=packed, CIRCLE_BRANCH="hotfix/1.2.3", ok=False)
def test_repository_policy_source_and_packed_never_select_publication(self):
self.write_manifest(profile="repository-policy")
for packed in ([False, True] if PACKED else [False]):
for branch, tag in (("main", ""), ("develop", ""), ("release/1", ""), ("", "v1.0.0")):
result = self.render(packed=packed, CIRCLE_BRANCH=branch, CIRCLE_TAG=tag)
self.assertEqual({"verify": {"jobs": ["validate-policy"]}}, result["workflows"])
self.assertEqual({"validate-policy"}, set(result["jobs"]))
self.assertNotIn("orbs", result)
def test_stable_tags_require_ascii_digits_in_source_and_packed_generator(self):
tag = "v\u0661.\u0662.\u0663"
self.git("tag", tag)
for packed in ([False, True] if PACKED else [False]):
self.assert_full(self.render(packed=packed, CIRCLE_TAG=tag))
def test_qbit_rc_completion_is_explicit_and_rc_only(self):
options = self.assert_full(self.render(CIRCLE_BRANCH="release/1"), "publish_release_candidate")
self.assertFalse(options["notify_rc_published"])
@@ -298,8 +348,8 @@ class PipelineTests(unittest.TestCase):
self.change_docs()
for packed in ([False, True] if PACKED else [False]):
result = self.render(packed=packed)
self.assertEqual(list(result["workflows"]), ["test_only"])
options = result["workflows"]["test_only"]["jobs"][0]["docs_only"]
self.assertEqual(list(result["workflows"]), ["verify"])
options = result["workflows"]["verify"]["jobs"][0]["docs_only"]
self.assertEqual(options["name"], "qqq-orb/mvn_test_only")
self.assertIn("git diff --check", result["jobs"]["docs_only"]["steps"][1]["run"]["command"])
(self.repo / "App.java").write_text("changed\n", encoding="utf-8")
@@ -360,8 +410,8 @@ class PipelineTests(unittest.TestCase):
for branch, tag, workflow in events:
with self.subTest(workflow=workflow):
result = self.render(CIRCLE_BRANCH=branch, CIRCLE_TAG=tag)
self.assertEqual(list(result["workflows"]), [workflow])
actual = result["workflows"][workflow]["jobs"]
self.assertEqual(list(result["workflows"]), ["verify"])
actual = result["workflows"]["verify"]["jobs"]
expected = baseline["workflows"][workflow]["jobs"]
self.assertEqual(len(actual), len(expected))
for before, after in zip(expected, actual):
@@ -422,8 +472,8 @@ class PipelineTests(unittest.TestCase):
with self.subTest(branch=branch, enabled=enabled):
result = self.render(CIRCLE_BRANCH=branch, QQQ_CI_RUN_PLAYWRIGHT=str(enabled).lower())
workflow = base + ("_with_playwright" if enabled else "")
self.assertEqual(list(result["workflows"]), [workflow])
jobs = result["workflows"][workflow]["jobs"]
self.assertEqual(list(result["workflows"]), ["verify"])
jobs = result["workflows"]["verify"]["jobs"]
self.assertEqual(len(jobs), 2 if enabled else 1)
maven = [j for j in jobs if next(iter(j)).startswith("qqq-orb/mvn_frontend_")]
self.assertEqual(len(maven), 1)
@@ -454,8 +504,10 @@ class PipelineTests(unittest.TestCase):
("hotfix/fix", "", "publish_hotfix_release"), ("", "publish-fixture", "publish_feature"),
("", "v1.0.0", "publish_release")]:
result = self.render(CIRCLE_BRANCH=branch, CIRCLE_TAG=tag)
self.assertEqual(list(result["workflows"]), [workflow])
options = next(iter(result["workflows"][workflow]["jobs"][0].values()))
self.assertEqual(list(result["workflows"]), ["verify"])
options = next(iter(result["workflows"]["verify"]["jobs"][0].values()))
expected_kind = {"publish_hotfix_release": "hotfix"}.get(workflow, workflow.removeprefix("publish_"))
self.assertEqual(options["branch_type"], expected_kind)
if profile == "frontend-core":
self.assertTrue(options["npm_trusted_publishing"])
self.assertEqual(options["node_version"], "24.21")
@@ -463,9 +515,9 @@ class PipelineTests(unittest.TestCase):
else:
self.assertEqual(options["context"], ["qqq-maven-registry-credentials"])
main = self.render(CIRCLE_BRANCH="main")
self.assertEqual(list(main["workflows"]), ["publish_release" if profile == "frontend-core" else "test_only"])
self.assertEqual(list(main["workflows"]), ["verify"])
rc_tag = self.render(CIRCLE_BRANCH="", CIRCLE_TAG="v1.0.0-RC.1")
self.assertEqual(list(rc_tag["workflows"]), ["publish_release" if profile == "frontend-core" else "test_only"])
self.assertEqual(list(rc_tag["workflows"]), ["verify"])
def test_profile_specific_options_rejected(self):
for profile, options in [("qbit", {"node_version": "22"}), ("frontend-core", {"browsers": False}),
@@ -532,7 +584,8 @@ class PipelineTests(unittest.TestCase):
{"CIRCLE_BRANCH": "integration-example"}):
result = self.render(**environment)
if environment.get("CIRCLE_BRANCH") != "integration-example":
self.assertIn("publish_release", result["workflows"])
self.assertEqual(list(result["workflows"]), ["verify"])
self.assertEqual(result["workflows"]["verify"]["jobs"][0]["qqq-orb/mvn_publish"]["branch_type"], "release")
if PACKED:
self.assertEqual(result, self.render(packed=True, **environment))
+12
View File
@@ -6,6 +6,7 @@ scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
mkdir "$scratch/bin"
export INSTALL_LOG="$scratch/install.log"
export APT_CONFIG_CAPTURE="$scratch/apt.conf"
export BASH_ENV="$scratch/bash-env"
export QQQ_MAVEN_BIN_DIR="$scratch/maven-bin"
: > "$BASH_ENV"
@@ -14,6 +15,11 @@ cat > "$scratch/bin/sudo" <<'STUB'
printf '%s\n' "$*" >> "$INSTALL_LOG"
if [[ "$*" == *apt-get* ]]; then
[[ "$1" == env && "$2" == DEBIAN_FRONTEND=noninteractive && "$3" == NEEDRESTART_MODE=l ]] || exit 90
[[ -f "$APT_CONFIG_CAPTURE" ]] && grep -qx 'Acquire::Retries "3";' "$APT_CONFIG_CAPTURE" || exit 91
[[ ${APT_FAIL:-0} == 0 ]] || exit "$APT_FAIL"
fi
if [[ "$1" == tee && "$2" == /etc/apt/apt.conf.d/80-qqq-download-retries ]]; then
cat > "$APT_CONFIG_CAPTURE"; exit 0
fi
if [[ "$1" == tee || "$1" == gpg || "$1" == -E ]]; then cat >/dev/null; fi
STUB
@@ -32,6 +38,7 @@ grep -q 'tee /etc/needrestart/conf.d/99-qqq-ci.conf' "$INSTALL_LOG"
grep -q 'export JAVA_HOME=' "$BASH_ENV"
: > "$INSTALL_LOG"
: > "$BASH_ENV"
rm "$APT_CONFIG_CAPTURE"
PARAM_INSTALL_NODE=true bash "$script_dir/install_java.sh" >/dev/null 2>&1
[[ $(grep -c 'apt-get' "$INSTALL_LOG") -eq 5 ]]
grep -q 'apt-get install -y -qq nodejs' "$INSTALL_LOG"
@@ -82,4 +89,9 @@ if PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1;
echo 'invalid native Maven accepted' >&2; exit 1
fi
unset -f mvn
if APT_FAIL=100 PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1; then
echo 'persistent apt failure accepted' >&2; exit 1
else
[[ $? -eq 100 ]]
fi
printf 'PASS: unattended installation and native/executable/nonexecutable wrapper selection\n'
+58
View File
@@ -0,0 +1,58 @@
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
import yaml
ROOT = Path(__file__).parents[1]
class BranchLanguages(unittest.TestCase):
def select(self, tracked, untracked=()):
worker = yaml.safe_load((ROOT / '.github/workflows/qbit-codeql.yml').read_text())
script = next(s['run'] for s in worker['jobs']['analyze']['steps'] if s.get('id') == 'languages')
script = script.split("python3 <<'PY'\n", 1)[1].rsplit('\nPY', 1)[0]
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
subprocess.run(['git', 'init', '-q', directory], check=True)
for name in (*tracked, *untracked):
path = root / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text('fixture')
if tracked:
subprocess.run(['git', 'add', '--', *tracked], cwd=root, check=True)
output = root / 'outputs'
result = subprocess.run([sys.executable, '-c', script], cwd=root,
env={**os.environ, 'GITHUB_OUTPUT': str(output)}, capture_output=True, text=True)
return result, output.read_text() if output.exists() else ''
def test_historical_java_branch_does_not_scan_absent_actions(self):
result, output = self.select(['src/Example.java'])
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('languages=java-kotlin\n', output)
self.assertIn('build-mode=none\n', output)
def test_current_branch_preserves_all_present_supported_languages(self):
result, output = self.select(['src/Example.java', '.github/workflows/ci.yaml', 'web/app.tsx', 'tools/check.py'])
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('languages=java-kotlin,javascript-typescript,python,actions\n', output)
def test_kotlin_requires_a_build(self):
result, output = self.select(['src/Example.kt', '.github/workflows/ci.yml'])
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('build-mode=autobuild\n', output)
def test_untracked_files_do_not_select_languages(self):
result, output = self.select(['src/Example.java'], ['web/generated.js', '.github/workflows/untracked.yml'])
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('languages=java-kotlin\n', output)
def test_placeholder_fails_explicitly(self):
result, output = self.select(['README.md'])
self.assertNotEqual(0, result.returncode)
self.assertIn('No supported source', result.stderr)
if __name__ == '__main__':
unittest.main()
+73
View File
@@ -0,0 +1,73 @@
"""Preserve the audited main-branch hotfix contracts in the shared profile."""
import importlib.util
import json
from pathlib import Path
import re
import unittest
from unittest import mock
import yaml
ROOT = Path(__file__).resolve().parents[1]
def load(name, path):
spec = importlib.util.spec_from_file_location(name, ROOT / path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
migration = load('migration', 'tools/standardize_qbit.py')
pipeline = load('pipeline', 'src/scripts/generate_pipeline.py')
def selects(filters, branch, tag):
selector = filters.get('tags' if tag else 'branches', {})
if tag and not selector:
return False
def matches(patterns):
patterns = patterns if isinstance(patterns, list) else [patterns]
return any(re.fullmatch(p[1:-1], tag or branch, flags=re.ASCII) is not None if p.startswith('/') and p.endswith('/') else p == (tag or branch) for p in patterns)
return ('only' not in selector or matches(selector['only'])) and ('ignore' not in selector or not matches(selector['ignore']))
class HotfixMigrationTests(unittest.TestCase):
def test_audited_main_job_selection_is_preserved(self):
fixtures = json.loads((ROOT/'tests/fixtures/qbit-main-hotfix-policies.json').read_text())
for fixture in fixtures:
manifest = migration.infer_manifest(fixture['config'], '0.7.13', 'develop')
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.7.13')
cases = [(branch, '') for branch in ('main','develop','feature/a','release/1','hotfix/1.2.3','integration-test','staging')]
cases += [('', tag) for tag in ('v1.2.3','v1.2.3-RC.1','vanything','publish1','invalid','v\u0661.\u0662.\u0663')]
for branch, tag in cases:
with self.subTest(repository=fixture['repository'], branch=branch, tag=tag):
expected = sorted((job, options.get('branch_type')) for workflow in fixture['config']['workflows'].values() for entry in workflow['jobs'] for job, options in entry.items() if selects(options.get('filters', {}), branch, tag))
with mock.patch.object(pipeline, 'git', return_value=b'verified-head\n'):
generated = pipeline.qbit_template(config, branch, tag)
actual = sorted((job, options.get('branch_type')) for workflow in generated['workflows'].values() for entry in workflow['jobs'] for job, options in entry.items() if selects(options.get('filters', {}), branch, tag))
# The canonical setup's tag filter prevents non-release tag runs.
caller = migration.canonical_caller('0.7.13', manifest['release_tag_policy'])
setup = caller['workflows']['setup']['jobs'][0]['qqq-orb/generate_pipeline']
if not selects(setup['filters'], branch, tag): actual = []
self.assertEqual(actual, expected)
def test_cms_repair_only_quotes_the_existing_tag_pattern(self):
fixtures = json.loads((ROOT/'tests/fixtures/qbit-main-hotfix-policies.json').read_text())
fixture = next(row for row in fixtures if row['repository'] == 'QRun-IO/qbit-cms')
with self.assertRaises(yaml.YAMLError):
yaml.safe_load(fixture['original_yaml'])
repaired = fixture['original_yaml'].replace(r'only: [/^v[0-9]+\.[0-9]+\.[0-9]+$/]',
r"only: ['/^v[0-9]+\.[0-9]+\.[0-9]+$/']")
self.assertEqual(yaml.safe_load(repaired), fixture['config'])
def test_stable_tag_selection_uses_ascii_digits(self):
with mock.patch.object(pipeline, 'git', return_value=b'verified-head\n'):
self.assertEqual(pipeline.event_kind('', 'v\u0661.\u0662.\u0663'), 'test')
self.assertEqual(pipeline.event_kind('', 'v1.2.3'), 'release')
def test_existing_manifest_does_not_enable_hotfix_publication(self):
raw = 'profile: qbit\norb_version: 0.7.13\ndefault_branch: develop\n'
with mock.patch.object(Path, 'read_text', return_value=raw):
config = pipeline.load_manifest('unused','0.7.13')
self.assertFalse(config['publish'].get('hotfix', False))
result = pipeline.qbit_template(config, 'hotfix/1.2.3', '')
self.assertIn('qqq-orb/mvn_test_only', result['workflows']['test_only']['jobs'][0])
if __name__ == '__main__': unittest.main()
+11
View File
@@ -58,6 +58,17 @@ class FixtureAPI:
class SweepTests(unittest.TestCase):
def test_stable_gate_and_mixed_name_latest_failure(self):
api = FixtureAPI()
api.status['check_runs'][0]['name'] = 'verify'
self.assertEqual('dispatched', sweep.reconcile(api, REPO))
api.sent.clear()
api.status['check_runs'].append(dict(api.status['check_runs'][0],
id=10, name='publish_release_candidate', conclusion='failure'))
with self.assertRaises(sweep.InvalidIntent):
sweep.reconcile(api, REPO)
self.assertEqual([], api.sent)
def test_idle_never_dispatches_or_reads_artifacts(self):
api = FixtureAPI(); api.lock = None
self.assertEqual('idle', sweep.reconcile(api, REPO))
+11
View File
@@ -23,6 +23,17 @@ def bump(parent=SHA, version='1.2.3-RC.4'):
class ReleaseGuardTests(unittest.TestCase):
def test_stable_workflow_preserves_exact_source_and_latest_attempt_checks(self):
result = status()
result['check_runs'][0]['name'] = 'verify'
self.assertEqual('1.2.3-RC.4', evaluate(SHA, result, bump()))
result['check_runs'].append(dict(result['check_runs'][0], id=10, conclusion='failure'))
with self.assertRaisesRegex(Failure, 'CircleCI failed'):
evaluate(SHA, result, bump())
result['check_runs'] = [dict(result['check_runs'][0], head_sha=OTHER)]
with self.assertRaises(Failure):
evaluate(SHA, result, bump())
def test_success_exact_built_source(self):
self.assertEqual('1.2.3-RC.4', evaluate(SHA, status(), bump()))
+9
View File
@@ -30,6 +30,15 @@ def validate(job, publisher):
class TimeoutRecoveryTests(unittest.TestCase):
def test_stable_gate_must_match_the_actual_publishing_workflow(self):
job, publisher = evidence()
job['workflows']['workflow_name'] = 'verify'
publisher['name'] = 'verify'
validate(job, publisher)
publisher['name'] = 'publish_release_candidate'
with self.assertRaises(Failure):
validate(job, publisher)
def test_exact_upload_timeout_after_successful_verify(self):
self.assertEqual((124, 0), validate(*evidence()))
+89
View File
@@ -0,0 +1,89 @@
"""Execute the policy job against real git indexes, including application promotion."""
import importlib.util
import json
import shutil
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
import yaml
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('pipeline', ROOT / 'src/scripts/generate_pipeline.py')
pipeline = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pipeline)
TEMPLATE = json.loads((ROOT / 'src/scripts/templates/repository-policy.json').read_text())
SCRIPT = TEMPLATE['jobs']['validate-policy']['steps'][-1]['run']['command'].split("<<'PYTHON'\n", 1)[1].rsplit('PYTHON', 1)[0]
class RepositoryPolicyTests(unittest.TestCase):
def test_packing_is_independent_of_checkout_line_endings(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
shutil.copytree(ROOT / 'src/scripts', root / 'src/scripts')
(root / 'scripts').mkdir()
shutil.copy(ROOT / 'scripts/pack_pipeline.py', root / 'scripts/pack_pipeline.py')
resources = list((root / 'src/scripts/templates').glob('*.json'))
for resource in resources:
resource.write_bytes(resource.read_bytes().replace(b'\r\n', b'\n'))
subprocess.run([sys.executable, str(root / 'scripts/pack_pipeline.py')], check=True)
packed = (root / 'src/scripts/packed_generate_pipeline.py').read_bytes()
for resource in resources:
resource.write_bytes(resource.read_bytes().replace(b'\n', b'\r\n'))
subprocess.run([sys.executable, str(root / 'scripts/pack_pipeline.py'), '--check'], check=True)
self.assertEqual(packed, (root / 'src/scripts/packed_generate_pipeline.py').read_bytes())
def execute(self, files, untracked=None):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
subprocess.run(['git', 'init', '-q'], cwd=root, check=True)
for name, content in files.items():
file = root / name
file.parent.mkdir(parents=True, exist_ok=True)
file.write_text(content, encoding='utf-8')
subprocess.run(['git', 'add', '.'], cwd=root, check=True)
for name, content in (untracked or {}).items():
(root / name).write_text(content, encoding='utf-8')
return subprocess.run([sys.executable, '-c', SCRIPT], cwd=root, capture_output=True, text=True)
def test_placeholder_with_wrappers_and_workflows(self):
result = self.execute({'LICENSE': 'Apache-2.0', 'docs/DESIGN.md': '# Planned application',
'.github/workflows/release-rc.yml': 'name: Release\non: workflow_dispatch\n',
'.mvn/wrapper/maven-wrapper.properties': 'distributionUrl=https://example.invalid/maven.zip',
'mvnw': '#!/bin/sh\n', 'mvnw.cmd': '@echo off\n'})
self.assertEqual(0, result.returncode, result.stderr)
def test_new_application_requires_real_build_profile(self):
for name in ('pom.xml', 'package.json', 'src/App.java', 'main.py', 'main.go', 'Dockerfile'):
with self.subTest(name=name):
self.assertNotEqual(0, self.execute({'LICENSE': 'license', name: '{}'}).returncode)
def test_malformed_configuration_fails(self):
for name in ('.github/dependabot.yml', 'settings.json'):
with self.subTest(name=name):
self.assertNotEqual(0, self.execute({name: '[unterminated'}).returncode)
def test_unused_checkstyle_policy_is_validated_without_allowing_application_xml(self):
self.assertEqual(0, self.execute({'checkstyle/config.xml': '<module name="Checker"/>'}).returncode)
self.assertNotEqual(0, self.execute({'checkstyle/config.xml': '<module>'}).returncode)
self.assertNotEqual(0, self.execute({'application.xml': '<application/>'}).returncode)
self.assertNotEqual(0, self.execute({'pom.xml': '<project/>'}).returncode)
def test_only_tracked_files_are_policy_inputs(self):
self.assertEqual(0, self.execute({'README.md': '# Docs'}, {'main.py': 'temporary tool'}).returncode)
def test_manifest_cannot_enable_publication(self):
manifest = {'profile': 'repository-policy', 'orb_version': '0.7.14', 'default_branch': 'develop'}
with patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.7.14')
self.assertFalse(any(config['publish'].values()))
self.assertEqual({'verify': {'jobs': ['validate-policy']}}, pipeline.repository_policy_template(config, 'main', '')['workflows'])
manifest['publish'] = {'release': True}
with patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)), self.assertRaises(pipeline.ManifestError):
pipeline.load_manifest('unused', '0.7.14')
if __name__ == '__main__':
unittest.main()
+48 -15
View File
@@ -6,6 +6,10 @@ Example: python tools/audit_estate.py --inventory inventory.json --orb-version 0
Policy: {"profiles":{"qbit":{"required_checks":{"main":["observed context"],
"develop":["observed context"]}}},"repositories":{"QRun-IO/example":{...}}}.
Only use contexts verified on hosted canaries. Absent policy is unknown, not pass.
Use --live --source-branch main (or release/...) to inspect that branch's source;
omitting it retains default-branch auditing. Unavailable branches remain unknown.
A profile/repository policy may set rc_worker_sha independently of --worker-sha
so an RC-only worker change does not require updating unchanged native workers.
"""
import argparse
import base64
@@ -69,11 +73,17 @@ class Snapshot:
raise ReadError('invalid repository name')
self.live = live
self.path = entry.get('localPath', '')
self.branch = (entry.get('defaultBranchRef') or {}).get('name')
requested = entry.get('sourceBranch')
if requested is not None and (not live or not isinstance(requested, str)
or not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9._/-]*', requested)):
raise ReadError('explicit source branch requires a valid branch name and live lookup')
self.default_branch = (entry.get('defaultBranchRef') or {}).get('name')
self.branch = requested or self.default_branch
self.sha = entry.get('sha', '')
if live:
info = gh('repos/' + self.repo)
self.branch = info['default_branch']
self.default_branch = info['default_branch']
self.branch = requested or self.default_branch
if info['archived']:
raise ReadError('repository is now archived; refresh inventory')
self.sha = gh('repos/' + self.repo + '/commits/' + quote(self.branch, safe=''))['sha']
@@ -165,9 +175,10 @@ def owner_gate_errors(snapshot):
def audit(entry, snapshot, version, worker_sha, policy):
name = entry['name']
profile = policy.get('profile') or ('qbit' if name.startswith('qbit-') else PROFILES.get(name))
profile = policy.get('branch_profiles', {}).get(snapshot.branch) or policy.get('profile') or ('qbit' if name.startswith('qbit-') else PROFILES.get(name))
result = {'repository': entry['nameWithOwner'], 'sha': snapshot.sha,
'default_branch': snapshot.branch, 'profile': profile, 'findings': [], 'required_checks': {}}
'default_branch': getattr(snapshot, 'default_branch', snapshot.branch),
'source_branch': snapshot.branch, 'profile': profile, 'findings': [], 'required_checks': {}}
findings = result['findings']
def finding(code, path=None, detail=None):
findings.append({k: v for k, v in {'code': code, 'path': path, 'detail': detail}.items() if v is not None})
@@ -196,7 +207,7 @@ def audit(entry, snapshot, version, worker_sha, policy):
manifest = Path(directory) / 'manifest.yml'
manifest.write_text(raw, encoding='utf-8')
generator.load_manifest(manifest, pin)
if data.get('default_branch') != snapshot.branch:
if data.get('default_branch') != result['default_branch']:
finding('default_branch_drift', '.qqq-ci.yml')
if caller is not None:
standard.validate(caller, raw, pin)
@@ -206,14 +217,15 @@ def audit(entry, snapshot, version, worker_sha, policy):
native_paths = set()
rc_source = snapshot.read(rc_worker.CALLER_PATH)
existing_rc = rc_source is not None and 'qbit-release-rc.yml@' in rc_source
if profile == 'qbit' and (rc_enabled or existing_rc):
if profile in ('qbit', 'repository-policy') and (rc_enabled or existing_rc):
path = rc_worker.CALLER_PATH
raw = rc_source
rc_sha = policy.get("rc_worker_sha", worker_sha)
if raw is None:
finding('missing_rc_bridge', path)
elif not re.fullmatch(r'[0-9a-f]{40}', worker_sha or ''):
elif not re.fullmatch(r'[0-9a-f]{40}', rc_sha or ''):
finding('native_pin_policy_missing', path)
elif raw != rc_worker.generate(worker_sha):
elif raw != rc_worker.generate(rc_sha):
finding('rc_bridge_drift', path)
else:
native_paths.add(path)
@@ -221,17 +233,30 @@ def audit(entry, snapshot, version, worker_sha, policy):
if other != path and 'qbit-release-rc.yml@' in snapshot.read(other):
finding('duplicate_rc_bridge', other)
for kind, definition in auxiliary.BRIDGES.items():
if definition['repository'] != entry['nameWithOwner']:
bridge_sha = worker_sha
if kind == 'qbit-codeql':
if profile not in ('qbit', 'repository-policy', 'all') or 'codeql_worker_sha' not in policy:
continue
bridge_sha = policy['codeql_worker_sha']
elif definition['repository'] != entry['nameWithOwner']:
continue
path = definition['path']
raw = snapshot.read(path)
if kind == 'all-release' and profile == 'repository-policy' and raw is None:
continue
# Older release branches can predate Authentik's schema contract test.
# Keep the default branch required, and validate every existing bridge.
if (kind == 'qbit-schema-drift' and raw is None
and snapshot.branch != result['default_branch']
and not any(path.endswith('/SchemaDriftTest.java') for path in snapshot.paths)):
continue
if raw is None:
finding('missing_native_maintenance_bridge', path)
continue
if not re.fullmatch(r'[0-9a-f]{40}', worker_sha or ''):
if not re.fullmatch(r'[0-9a-f]{40}', bridge_sha or ''):
finding('native_pin_policy_missing', path)
continue
expected = generator.yaml.safe_dump(auxiliary.canonical(kind, worker_sha), sort_keys=False)
expected = generator.yaml.safe_dump(auxiliary.canonical(kind, bridge_sha), sort_keys=False)
if raw != expected:
finding('native_maintenance_bridge_drift', path)
else:
@@ -321,9 +346,13 @@ def audit(entry, snapshot, version, worker_sha, policy):
return result
def run(entries, version, worker_sha, policies, live=False, snapshot_factory=Snapshot):
def run(entries, version, worker_sha, policies, live=False, snapshot_factory=Snapshot, source_branch=None):
if source_branch is not None and not live:
raise ValueError('Explicit source branch audits require --live')
results = []
for entry in entries:
if source_branch is not None:
entry = dict(entry, sourceBranch=source_branch)
if entry.get('isArchived'):
results.append({'repository': entry['nameWithOwner'], 'status': 'archived_skip'})
continue
@@ -334,10 +363,11 @@ def run(entries, version, worker_sha, policies, live=False, snapshot_factory=Sna
try:
results.append(audit(entry, snapshot_factory(entry, live), version, worker_sha, policy))
except (ReadError, OSError, ValueError, KeyError, subprocess.TimeoutExpired):
results.append({'repository': entry['nameWithOwner'], 'status': 'unknown', 'reason': 'source/API/policy read failed'})
results.append({'repository': entry['nameWithOwner'], 'source_branch': source_branch,
'status': 'unknown', 'reason': 'source/API/policy read failed'})
return {'inventory_count': len(entries), 'active_count': sum(not e.get('isArchived', False) for e in entries),
'covered_count': len(results), 'unique_repository_count': len({e['nameWithOwner'] for e in entries}),
'live': live, 'results': results}
'live': live, 'source_branch_selector': source_branch or 'default', 'results': results}
def main():
@@ -347,12 +377,15 @@ def main():
parser.add_argument('--worker-sha')
parser.add_argument('--policy', type=Path)
parser.add_argument('--live', action='store_true', help='Read current source and branch rules using existing gh authentication')
parser.add_argument('--source-branch', help='Inspect this branch instead of the repository default; requires --live')
parser.add_argument('--expected-active', type=int, default=68)
parser.add_argument('--output', type=Path)
args = parser.parse_args()
if args.source_branch is not None and not args.live:
parser.error('--source-branch requires --live')
entries = json.loads(args.inventory.read_text(encoding='utf-8'))
policies = json.loads(args.policy.read_text(encoding='utf-8')) if args.policy else {}
report = run(entries, args.orb_version, args.worker_sha, policies, args.live)
report = run(entries, args.orb_version, args.worker_sha, policies, args.live, source_branch=args.source_branch)
report['expected_active_count'] = args.expected_active
report['coverage_matches_expected'] = (report['active_count'] == args.expected_active
and report['unique_repository_count'] == report['inventory_count'])
+9 -4
View File
@@ -43,8 +43,8 @@ def publication_filters(kind, filters):
raise ValueError("Unknown publication filters")
branches = filters.get("branches")
tags = filters.get("tags")
if kind in ("snapshot", "release_candidate"):
expected = "develop" if kind == "snapshot" else r"/release\/.*/"
if kind in ("snapshot", "release_candidate", "hotfix"):
expected = {"snapshot": "develop", "release_candidate": r"/release\/.*/", "hotfix": r"/hotfix\/.*/"}[kind]
if not isinstance(branches, dict) or set(branches) != {"only"} or single_filter(branches["only"]) != expected or tags not in (None, {"only": []}):
raise ValueError("Unknown snapshot/RC filters")
return False, "none"
@@ -58,7 +58,8 @@ def publication_filters(kind, filters):
tag_filter = single_filter(tags["only"])
if not isinstance(tag_filter, str):
raise ValueError("Unknown release tag filter")
policy = {r"/^v\d+\.\d+\.\d+$/": "stable", "/v.*/": "v-prefix"}.get(tag_filter)
policy = {r"/^v\d+\.\d+\.\d+$/": "stable", r"/^v[0-9]+\.[0-9]+\.[0-9]+$/": "stable",
"/v.*/": "v-prefix"}.get(tag_filter)
if policy is None:
raise ValueError("Unknown release tag filter")
return main, policy
@@ -80,6 +81,9 @@ def test_filters(filters):
"/(develop|release/.*|integration.*)/": "non-release-no-integration",
"/(develop|main|release/.*|integration.*)/": "feature",
"/(develop|main|release/.*)/": "non-release-no-main",
"/(develop|release/.*|hotfix/.*)/": "non-release-no-hotfix",
"/(develop|main|release/.*|hotfix/.*)/": "non-release-no-main-hotfix",
"/(develop|main|release/.*|hotfix/.*|integration.*)/": "feature-no-hotfix",
}.get(pattern.replace(r"\/", "/"))
if policy is None:
raise ValueError("Unknown test branch filters")
@@ -119,7 +123,7 @@ def infer_manifest(config, version, default_branch):
raise ValueError("Custom pre-steps require review")
if job.endswith("/mvn_publish"):
kind = options.get("branch_type")
if kind not in ("snapshot", "release_candidate", "release"):
if kind not in ("snapshot", "release_candidate", "release", "hotfix"):
raise ValueError("Custom publication contract requires review")
if kind in publications:
raise ValueError("Duplicate publication jobs require review")
@@ -143,6 +147,7 @@ def infer_manifest(config, version, default_branch):
"test_branch_policy": test_policy or "all",
"publish": {**{kind: kind in publications
for kind in ("snapshot", "release_candidate", "release")},
**({"hotfix": True} if "hotfix" in publications else {}),
"asciidoc": False},
}