Compare commits

..
Author SHA1 Message Date
james.maesandCodex 17c94592d9 ci: centralize branch-aware QBit source scanning
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 09:29:04 -05:00
james.maesandCodex 9906c8f120 fix: audit explicitly selected repository branches (#62)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 09:15:33 -05:00
james.maesandCodex 4dcf2c1891 fix: preserve legacy qbit hotfix publication policies (#61)
* fix: preserve legacy qbit hotfix publication policies

Co-Authored-By: Codex <noreply@openai.com>

* fix: regenerate packed profiles with Linux zlib

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 09:00:43 -05:00
james.maesandCodex cc30f599e6 fix(ci): provide a stable verification gate across branches (#59)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 03:12:12 -05:00
james.maesandCodex 05a2a5217a fix: audit rc worker pins independently of native workers (#58)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:23:51 -05:00
james.maesandCodex 7d73ac90b6 fix: distinguish missing RC refs from GitHub error bodies (#57)
* fix: distinguish missing rc refs from github error bodies

Co-Authored-By: Codex <noreply@openai.com>

* fix: reject empty ref responses across jq versions

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:15:32 -05:00
james.maesandCodex edc1365ac9 fix: recover verified rc uploads after publisher timeout (#56)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:02:14 -05:00
james.maesandCodex 0f33ddadf8 ci: run business E2E on changes and preserve main validation (#55)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 01:19:08 -05:00
james.maesandCodex 5e9bbb54ec ci: add batch version rollout for canonical consumers (#54)
Validate every checkout before updating its orb pin. Preserve publication policy, filters, credentials contexts and forwarded parameters; reject custom callers and mismatched declarations. Cover all 12 profiles, dry runs, no-op repeats, invalid batches and duplicate input with six new tests in the existing contract job. Existing migration and standard tests pass; Linux packed generator check passes.

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 00:45:50 -05:00
james.maesandCodex 5bcec90e33 ci: report the shared database gate on every pull request (#53)
Preserve migration verification, avoid duplicate feature push builds, and cancel superseded pull request runs so the gate can be required safely.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 22:25:38 -05:00
james.maesandCodex a50b07d927 fix: escape Android heredoc in continued Circle configuration (#51)
Circle accepts the continuation request before reporting an unescaped heredoc as a pipeline configuration error. Escape the template marker and emulate Circle unescaping in executable publisher tests. Validate the actual continued config with CircleCI.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 22:00:56 -05:00
james.maesandCodex 6318d90e96 feat: centralize lightweight GitOps rendering validation (#50)
Add a read-only native worker and canonical consumer bridge. Verify the pinned Kustomize binary, render every configuration without logging resources, and audit adoption. Behavioral and audit tests pass; all five marketing GitOps configurations render successfully.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:54:25 -05:00
james.maesandCodex a6fea661ff fix: authenticate Android publishing with GitHub package token (#47)
* fix: authenticate Android publishing with GitHub package token

Use the shared GitHub Packages token and verify its identity and registry repository access before Gradle starts. Preserve unit and emulator gates; package write success still requires hosted publication.

Co-Authored-By: Codex <noreply@openai.com>

* fix: keep embedded JSON resources consistent across checkouts

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:40:01 -05:00
james.maesandCodex 2894bae29a test: retain All release shell behavior in shared owner (#46)
Move the consumer publication regression to the reusable worker; existing test discovery executes it. Verify exact RC1 prerelease/non-latest flags, GA behavior and rejection of other RC versions.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:19:09 -05:00
james.maesandCodex a14e8f9b24 fix: default All releases to shared Circle smoke evidence (#35)
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:12:58 -05:00
james.maesandCodex 83611872a3 fix: verify public All smoke evidence without secret export (#44)
Allow unauthenticated Circle API reads for public projects, preserve optional authentication and all fail-closed provenance checks.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:12:00 -05:00
james.maesandCodex 30c4e0e85c fix: use shared Maven credentials for Android publication (#42)
Use the existing organization context and Gradle environment properties. Fail before publishing if either credential is absent and avoid writing secrets to gradle.properties.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:07:21 -05:00
james.maesandCodex f429d9f3c4 fix: build marketing site before browser checks (#40)
Run lint and one production build in every shared marketing workflow before Playwright and Lighthouse. Preserve the delegated delivery contracts.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 20:56:32 -05:00
16 changed files with 15 additions and 415 deletions

No files matched your search

-2
View File
@@ -50,14 +50,12 @@ jobs:
python3 tests/test_maven_ci_contract.py
python3 tests/test_standardize_qbit.py
python3 tests/test_qbit_hotfix_profile.py
python3 tests/test_ci_maintenance.py
python3 tests/test_rollout_version.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
python3 tests/test_next_native.py
python3 tests/test_auxiliary_workflows.py
python3 tests/test_qbit_codeql.py
python3 tests/test_repository_policy.py
python3 tests/test_intellij_plugin.py
python3 tests/test_workflow_lint.py
python3 tests/test_business_e2e.py
-2
View File
@@ -59,14 +59,12 @@ test-pipelines: pack
python3 tests/test_cli_ci_contract.py
python3 tests/test_standardize_qbit.py
python3 tests/test_qbit_hotfix_profile.py
python3 tests/test_ci_maintenance.py
python3 tests/test_rollout_version.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
python3 tests/test_next_native.py
python3 tests/test_auxiliary_workflows.py
python3 tests/test_qbit_codeql.py
python3 tests/test_repository_policy.py
python3 tests/test_intellij_plugin.py
python3 tests/test_workflow_lint.py
python3 tests/test_business_e2e.py
-15
View File
@@ -114,21 +114,6 @@ Maven jobs use machine executors with Docker pre-installed. Tests using Testcont
## Orb Publishing
### QBit CI maintenance
The QBit profile runs Maven verification without publishing when a single-parent
`ci: ...` or `ci(scope): ...` commit changes only `.qqq-ci.yml`,
`.circleci/config.yml`, `.github/workflows/codeql.yml`, or
`.github/workflows/release-rc.yml`. This applies on
main, develop, release, and hotfix branches. The normal test job still runs,
including on branches normally excluded from feature tests, and reports through
the required `verify` workflow.
Tags, merge commits (including release automation), application changes, other
files, empty diffs, and unavailable history retain normal publishing behavior.
This lets central configuration updates avoid unnecessary application versions
without introducing per-repository maintenance workflows.
The orb auto-publishes via CircleCI:
| Branch | Action |
-16
View File
@@ -1,16 +0,0 @@
# Application-free branch verification
Use `profile: repository-policy` only on a branch containing repository policies,
documentation, workflow configuration and optional unused Maven wrappers. It uses
the same canonical setup caller as other profiles and one small Python verification
job. It checks tracked YAML/JSON syntax and rejects application manifests, source
files, unknown file types and symlinks. It never builds or publishes an application.
The manifest accepts only `profile`, `orb_version` and `default_branch`. The latter
remains the repository's actual default branch. A future application promotion must
select its real build profile in the same change; adding source without doing so fails.
Estate policy can set `branch_profiles: {main: repository-policy}` without changing
the default branch's application profile. Require the CircleCI `setup` and `verify`
checks on the policy branch. This profile does not establish application release
readiness or replace the build/release checks on source-bearing branches.
+2 -3
View File
@@ -12,9 +12,8 @@ text = source.read_text(encoding="utf-8")
start = text.index("# BEGIN PROFILE RESOURCES")
end = text.index("# END PROFILE RESOURCES") + len("# END PROFILE RESOURCES")
embedded = ["# Generated by scripts/pack_pipeline.py; edit source/resources, then regenerate.", "import base64", "import zlib"]
for variable, name in (("CORE_TEMPLATE", "core"), ("MATERIAL_TEMPLATE", "frontend-material"), ("VOYAGE_TEMPLATE", "voyage"), ("NEXT_TEMPLATE", "frontend-next"), ("ALL_TEMPLATE", "all"), ("STARTER_TEMPLATE", "app-starter"), ("ANDROID_TEMPLATE", "android"), ("POLICY_TEMPLATE", "mcp-policy"), ("REPOSITORY_POLICY_TEMPLATE", "repository-policy"), ("MARKETING_TEMPLATE", "marketing")):
resource = (source.parent / "templates" / (name + ".json")).read_text(encoding="utf-8").encode("utf-8")
payload = base64.b64encode(zlib.compress(resource, level=9)).decode("ascii")
for variable, name in (("CORE_TEMPLATE", "core"), ("MATERIAL_TEMPLATE", "frontend-material"), ("VOYAGE_TEMPLATE", "voyage"), ("NEXT_TEMPLATE", "frontend-next"), ("ALL_TEMPLATE", "all"), ("STARTER_TEMPLATE", "app-starter"), ("ANDROID_TEMPLATE", "android"), ("POLICY_TEMPLATE", "mcp-policy"), ("MARKETING_TEMPLATE", "marketing")):
payload = base64.b64encode(zlib.compress((source.parent / "templates" / (name + ".json")).read_bytes(), level=9)).decode("ascii")
embedded.append(f'{variable} = json.loads(zlib.decompress(base64.b64decode("{payload}")))')
packed = text[:start] + "\n".join(embedded) + text[end:]
if len(packed.encode("utf-8")) > 100_000:
+5 -34
View File
@@ -23,7 +23,6 @@ ALL_TEMPLATE = json.loads((_TEMPLATE_DIR / "all.json").read_text(encoding="utf-8
STARTER_TEMPLATE = json.loads((_TEMPLATE_DIR / "app-starter.json").read_text(encoding="utf-8"))
ANDROID_TEMPLATE = json.loads((_TEMPLATE_DIR / "android.json").read_text(encoding="utf-8"))
POLICY_TEMPLATE = json.loads((_TEMPLATE_DIR / "mcp-policy.json").read_text(encoding="utf-8"))
REPOSITORY_POLICY_TEMPLATE = json.loads((_TEMPLATE_DIR / "repository-policy.json").read_text(encoding="utf-8"))
MARKETING_TEMPLATE = json.loads((_TEMPLATE_DIR / "marketing.json").read_text(encoding="utf-8"))
# END PROFILE RESOURCES
@@ -70,7 +69,7 @@ def load_manifest(path, setup_version):
if not isinstance(data, dict):
raise ManifestError("Manifest must be a mapping")
profile = data.get("profile")
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "repository-policy", "marketing"):
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "marketing"):
raise ManifestError("Unsupported profile")
allowed = ("profile", "orb_version", "default_branch", "docs_only", "publish")
if profile in ("qbit", "core"):
@@ -79,7 +78,7 @@ def load_manifest(path, setup_version):
allowed += ("release_completion", "release_on_main", "release_tag_policy", "test_branch_policy")
elif profile == "cli":
allowed += ("publish_context",)
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "repository-policy", "marketing"):
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "marketing"):
allowed = ("profile", "orb_version", "default_branch")
elif profile == "all":
allowed = ("profile", "orb_version", "default_branch", "publish", "publish_context")
@@ -140,7 +139,7 @@ def load_manifest(path, setup_version):
"release_tag_policy": tag_policy, "test_branch_policy": test_policy,
"docs_only": boolean(data, "docs_only", False),
"publish_context": publish_context,
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all", "repository-policy")
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all")
and not (profile == "qbit" and key == "hotfix"))
for key in publish_keys}}
@@ -176,25 +175,6 @@ def proven_docs_only(config, branch, tag):
return base.decode().strip() if eligible else False
def proven_ci_maintenance(branch, tag):
"""Keep application tests, but do not publish for isolated CI declarations."""
if tag or not (branch == "main" or branch == "develop" or branch.startswith(("release/", "hotfix/"))):
return False
subject = git("log", "-1", "--format=%s")
if not subject or not re.match(rb"ci(?:\([^\r\n()]+\))?: ", subject):
return False
parents = git("rev-list", "--parents", "-n", "1", "HEAD")
if not parents or not re.fullmatch(rb"[0-9a-f]{40} [0-9a-f]{40}\n", parents):
return False
base = parents.split()[1].decode("ascii")
changed = git("diff", "--no-ext-diff", "--no-renames", "--name-only", "-z", base, "HEAD")
if not changed:
return False
allowed = {b".qqq-ci.yml", b".circleci/config.yml", b".github/workflows/codeql.yml",
b".github/workflows/release-rc.yml"}
return all(path in allowed for path in changed.rstrip(b"\0").split(b"\0"))
def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
if tag:
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag, flags=re.ASCII):
@@ -219,14 +199,11 @@ def qbit_template(config, branch, tag):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
result = {"version": 2.1, "orbs": {"qqq-orb": "kingsrook/qqq-orb@" + config["orb_version"]}}
filters = ({"branches": {"ignore": "/.*/"}, "tags": {"only": "/.*/"}} if tag else
{"branches": {"ignore": "automation/qbit-rc-pending"}, "tags": {"ignore": "/.*/"}})
if kind == "test" and not tag and not maintenance:
if kind == "test" and not tag:
ignores = {"all": None, "non-release": "/(develop|release/.*)/",
"non-release-no-integration": "/(develop|release/.*|integration.*)/",
"feature": "/(develop|main|release/.*|integration.*)/",
@@ -237,7 +214,7 @@ def qbit_template(config, branch, tag):
ignored = ignores[config.get("test_branch_policy", "all")]
if ignored:
filters["branches"]["ignore"] = ["automation/qbit-rc-pending", ignored]
docs_base = proven_docs_only(config, branch, tag) if kind == "test" and not maintenance else False
docs_base = proven_docs_only(config, branch, tag) if kind == "test" else False
if docs_base:
# Keep the existing workflow/result identity even when Maven is not needed.
result["jobs"] = {"docs_only": {"resource_class": "small", "docker": [{"image": "cimg/base:2026.03"}],
@@ -439,13 +416,7 @@ def voyage_template(config, branch, tag):
return json.loads(json.dumps(VOYAGE_TEMPLATE).replace("__ORB_VERSION__", config["orb_version"]))
def repository_policy_template(config, branch, tag):
del config, branch, tag
return copy.deepcopy(REPOSITORY_POLICY_TEMPLATE)
PROFILE_TEMPLATES = {"qbit": qbit_template, "core": core_template,
"repository-policy": repository_policy_template,
"frontend-material": frontend_template, "frontend-core": frontend_template,
"cli": cli_template, "voyage": voyage_template, "frontend-next": next_template, "all": all_template, "app-starter": starter_template, "android": android_template, "mcp-policy": policy_template, "marketing": marketing_template}
-4
View File
@@ -20,10 +20,6 @@ sudo install -d /etc/needrestart/conf.d
# shellcheck disable=SC2016
printf '%s\n' '$nrconf{restart} = "l";' | sudo tee /etc/needrestart/conf.d/99-qqq-ci.conf >/dev/null
# Retry transient repository downloads within APT, including NodeSource's child process.
# Exhausted downloads retain APT's nonzero exit status and fail this build step.
printf '%s\n' 'Acquire::Retries "3";' | sudo tee /etc/apt/apt.conf.d/80-qqq-download-retries >/dev/null
# Ubuntu archive carries the same signed security suite; avoid the flaky HTTP mirror.
if [[ -f /etc/apt/sources.list ]]; then
sudo sed -i 's|http://security.ubuntu.com/ubuntu|https://archive.ubuntu.com/ubuntu|g' /etc/apt/sources.list
+5 -34
View File
@@ -24,7 +24,6 @@ ALL_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNq1T7EOgjAU3PmKpjNI
STARTER_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNp1UMGKwkAMvc9XlDmKtejR0yJ42MsKCl5EQnUzZbbTiZ0ZqiL9d1OpWqt7y3t5eS/JRUSRrNB5TVZOo8loPGwYcjvP8MI1o7IsY2aYkLm2mXdEedKSXwCL5QzW8+Xqe/EDIHmkvnkcyeXK0LFjFNAHIGvOD4rJP7plbVocPTrd7KSoLHwab2V7Kg7aIOxQkUPgi7RqZCo1Hod9rQ14Ci+ZnbQirdDGDjPtgzvHe4e/aINmI/ki3/ZslTaBH/m2W3N3mn3iuaMzy/s2n01Gg0T2BLX4Dz3re7UVd1SLWlwByepyJQ==")))
ANDROID_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtWG1v2zYQ/p5fQWgF6rSW3CXYMDhJ13YNkm5rkrrugKEKDFqiZCYUyZBUbC/wfvuO1LvtpmmBAcUwIYAp3vHuuXcqdzsIebdEaSq4N0R7wfd9uyPUVMPrHazhDfNYCRrDhhdRFTES0UG592Iv+CF45vULxjSSfsRoh7Hce7Ef7AV7HvCtnIYr0daQc2p8Q7Spt2CTLEiUG6Fae7DLcUasghJABcTPcDSjnJRQHKvBqeWMSYJzZrySsKo4PG2ItCA+NkeiGYmuRW7aYipNCvAJRfxU4ZgRPwKFbXUNxvaZnDu7dMeIAh1s+5HIMuC1MINBIXiOLOU1meap1zqxqterLdg0vn0YMAC0CaXy6XuQ4rQjsBVcpltSCvc0cLPrmCrkS/T3wFlSnhhc2VgOQp5QHqMA+WYpCUoQ0FOyQKEXPBlMc8ri7qngySJjoYd8G3QUSXS32i4YheFByNdxzWeEu6Rgc7zUn3VbxyMuqBOralJZveEgic3Myu9C+ho9WBma4OjhSgq7P6GqXF3utBRXObFRTxIriLOBYu9WlF5CIWQ+zXBK1lB5NnoWkzaK8rTj9rquhl0R+qACsP/sIBUiZWC1pPpg8dOP3gb0phq/kXLfWsbaQNx8kuUMA0Iftv2vquxIcE4iQ2JX3i8L6WMbp7WEXouJd3iImvAFbSp6/vzL8vDb6QDgeJnDAdz44f+G8C80hAyCyv2YSCaW/5UBe28WX+RTRvUMzamZIT3DisTorXUCimBJuKGY3ZfZmhhIvFwgSSXYRlnIh5C4j+5O3oxPP7yajM9/Oz4b/nxCzWk+RRc4urZ9D80VNaSlAlENZXSTU9hZhV7Iv0MvczOzxAgD55QkYC40hUxSBu31ALil0BScsEQ4iojWVgR0DFhhtewjLgySSogEwZ8s9JZqC/4g5FBmFNr1cpJroo4e9eTSzATfhzo5PHx88ef49PzscchpJoUy6EoLXr8IXS9zxRidBhY9BCUEnohhgHMmRgSqHnpYr8sSnI7HFxXtFPzIiNodhty6FvIDLCtIk5K/pwlL+tY9fZTIPjghJn2U6bSPZgTH0OXAWjIHJZUU+yhMNUGjHDyYkWOlhOqFXhkG3LgWLrO1Qlgk4Ik49HatGeCYUp6QhBOFjtZsDVxvmhTUXmPvbmNLSkzP1mYHWXF6U9yotDf0ZsZIPRwMYBgGKWRmPg0g9ANock+RFddvpNmndMPRXejZrBGK/uUsCz2bi68IZLVC7rDQAeG3VAn+EbzRStHQu1wTWj8gFBJGmkIalpKVjhvc8riE99Rlh7fabWS4iip8E9ifXmlmH9mIQF0f7T/bRdjmvZaCazLs6lfE5Iq7tAuYwHGv4it12JwFH1oPh559gbAFxRsTKeUuipaRJq4YqKYchjOPSM9y9xFk/i4SyhGdsHJdyHD1NXgHCeS/OR/c3Nz4RW+saqbRJonKqLafRTr0+jB9akKuZ8C2kZR/YJbXKfnWHuUpquSW1Vmjl3CbMg4xbJCFDQQ6dj+gsBRdiH3vRv3xgpom0+uG02o1EebWzLJrQCWUDa9lGUqUyKCCOQl50QdC7tS7ij8fnUxORi9f/348uRid/3r8y3hSZMG7mxsnalRK+gCobaM9gobY6TW2w32psAtoKnOhYiusm7khb65NPtymuBaMHEmGKUey6O9Okuv1ReqOxclWJaO6q8JF4SFzs/5KBWjXCRPz1qeqG8X2Auj6RHuclp+0rWHXfNbef7+sr+qfvzR8YpZXU7GYNl0Y94DZBqNDvFwbkgllGx8QJWmqoAxnZBvN/keBs6W7NEL8vDXyqvO+2pjL3JCF2WbSZgH77QHfteShgd9Z7fwDUg7law==")))
POLICY_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNqdVU1v2zAMvedXCL442Rq3aYEegubQQwYMWLGeBgxVESg2nWqVJU9S1gZB/vtIWXbsdCuw+WKJH4+PlEjtR4wlv8A6aXQyZ5fZ7IwkP8za4XaPa9ILJQvhYVoL78HqowqVhcmfwaLkIUpYpwt6WYkNoDrJZbU5r3f+yej5VTa7TDqrQ1w9nrWgFpzZ2hxWuRKOwiWuEkolnYHzULtB0CR/gvzZbH1ndELEbnWPdxRqUQVyn7XzGIDd777f3n3pQTTQpqqELsiwljWT0Xg6LaQTa4WFkfU0VnEaeESgxeI6u8iukh7c4Zj2/xD9Fs+CxbNgpVTg3iMcKn7FOL+5Yeny66eUa1nVxnq2E5XqNm7nuC6tqQj5Sck1i4p73HLNNVhrrGML9vDIdYy+ys1We5RdkEVpbMBcESesUnAdp1k6yexGmfU4/ZCRPp3MuSaesmRC78a1sD7DklrvXmT0YARGCsLpQDOSuNadvtxoL/UWKD7tvd31tATHTA163EFMmHCs7NnQhxUVmAUZZU6UsFJGFONy0qJGsmnbAOcp0XLe9nBPIC383EoLBRUslUV6xlIvvQJa5HiAG2N3tC7A5VbWHq9OUGE+oH36OESrpHNSbwisDKUpiUAXA7mVTJtQLMrlxBvVEeCEJH3NuWaixjJhzjzZdzkd5uwuBi4lqMLhtYxAB55MhljDK/FxwWZt9eA1h7q5bhl1xZIi0jFAj877NMiNroODxhCJwJFCDLAMPyzkv2Evh4B4ua3UnszaZivYfpDdYdh90Q2r3ASKkRsYnnCul03vlOhczDvadI6hTwZuR1dkwNh0kCl2aQav0o9nKADl4CTWLY6l9pKyMLeDK7Y913+ZQu3wHbW7MJeSF2OfS2Ve/vAO9Md/fCl6c/jta/E2wugw+g3AlOQ8")))
REPOSITORY_POLICY_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNqdVltv2zYUfs+vIPwiGYjotgGGIa0fis7YAnRO0HgtgigwaOnIZkKRGkkl9gL/951DXSK7aDNML+K56ON3bqSeTxgbPYJ10ujROXvH356S5t6sHIrPuCa7UDIXHpLKKJntegOacpM9gEXNbathvS3YZSnWgOZRJsv1pNr5jdHnZ/ztu1HvtW9Xd6cdqAVnapvBMlPCEY+RK4VSo97BeajcwaajbAPZg6l973RExNZ6wLtValEGchfaedyA+Q2wm49/fmaVsA7DOj10z0xZCp3TF00kZywpWSUrJluAJMmlEyuFqZJV0uY1CdzY1Y6gp9Nf+Bt+Nhog719S8X/If22rE9iLqsISCU/bFhaArazQ2eY/RJKmHz5EVzeLPy7nUaplWRnr2b0zOtWFNRim8BslV6y1XKHYu7l6VVmTgXO9alsqDh4Z8JmCErRfEBvh2GzR++xEqVLEJ2jHpgMYHlK2xHpWtY9vo7X00SmLlEsKqcDROvknuhvzHDKTQxzVvkh+jcbcYfg+jtL0TTR+Ab4ltjElbMwKYxmtsGasscsiKO6QF66Mb9TnqaZUWSEdsOsdtlw52xL23DBvBbZ9zpp5YA0n3I9gaL/n6PPFp9n8ekZE55cLFGj16fLq5mL+e7P8bXb5bT77ck0Sx/jkWhsLnSS8t3JVe4q1IRKeiEMuvbGZ0YVck3P5qJ+6N8/KPNqn2tVFIbctE446ArUupJD7bfMWOLphsStV8xbtgooeFliMCqyXSAJRsWFkAc43sJUp+bb5oMJk4JT332XCrg33prXuEOYeMt9r0Fia/CCuENuqlirnaytyFfIwlPmDD1V34L3UazdwO1IFT6RLdaZC9nVuC4o1JpFLt3QYutQP8Zi1vpwKyJV5AotK/LCP+fyF7HcdUUQfX6aOoFpcZuHvWuJhxgQLsTDMBPUKjjLttqeWGVISblkZJ7e493SK+ceaTp4sjjTYSSkeQSetxO+FjQaUsB0wBTX8FC6MlPM7BZOmfUL9BiizBadJd9h5eh0HCAsiX3rY+hg0Dhrqp92ojV/bPgwZjRPmsRkMPHEaU9OgfaJbp65tf57rv7C9Kzo+DufvPXOgsMteyzUFgw38WnSHoRzxRa7x0dyMB6SVdD4mLXeigKUyuAteDjHt1KUN1A+wQ+HDJA0QSeaE4xoQPNmwRpSN7vDP2bMCHYrmxvujA2qCo17TGdx0aDiu3mPWh7dFmzUahHrVKSkPzZWQ6h/cWN3lfdJJ4Q4bPRn7UGBMw78IsLI4+Hlo/zIGt/jxn8b36Cf7k38B2PmvaA==")))
MARKETING_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtW89v4ygUvuevQD7MSKsSp5lbD6OR9rRa7W1uk8oiGMc0GFzATbNV/vd9+EeaOHaqVpt6knBpbfh4wHufee9z3ZcRQsET04YrGdyh6fj2xrUoPTdw+wLXcJcVkluloSFYqmQ6DeuGH5Pxt/HtbQCoTTlspfQyEWq1MzbXmKaMLl+boPFBleZ/1fdo27M7XShVzKJ5wUUcERlHlhm7Y6QGS5Ixt7AShwGHS9xNC+VMvW4zmE7biJzQJVmwKCMSfpV7lXnWRjnbEVUZoGK3g+DXzMGQLiQSXNpZcINeW8o1NU3PKBdkvdJ8kVrEpbFECERTrTJeZAjjFbcpjllu9m24Ge8a2H6XcKZSVRg2C+7bK6VKWvZs97xcdy1gpmIe7DXft4YnXFhw14G/oWuuiYSQdvU55kix7pi07AsTRmyh2WwWjv8IWytuMKmyCX+uIQeI+1bLZtR393q9uTnOs1jRJdORoUT284tnQAtcYlquKnuiBpcRvWSWywVesbnhlrXhmi24sdo5KVikVI+5Ch8hnJirQ+hjwTUzXUFs8d0Hsw6mYVQza96IZo3qjOdVeYvAefaGqwDS7SfCRaRklHAZA9/L4zAhwrBrdmiZaKj78VgQwe2637EOhRvUifKVP0E+HELwPjj2WPhqxIlCl3G5u4pgcpjhq87IKiUc4qGj8NmCTAFFi15HObHp7vLD5gLXiPGDUbLXTF37lJluGXONcI6aTvTlC2I0VejryyywCgqcWXAHl1AYMVNd5hRKpLvbyWSz2XxF39H7VuHZ/EE2Q7orNBwzb2fFEtZ53Psj6YRBHO2OrUMaxOyJCZV74XRNwmkb9YusjU+6u3OoZU/pAF97DuVyXyv6WnEo9vna7nOdXr8trMqrvDBpv88rKK52XkI/7dVhyoiw6fYJDvv6lXaVTvBtMpn0QOJ5bzI8Fu4jGQb1HojoaPWB+hM4ahP8KL2OlXjg1t+Ym506ASriBbDG64Sr0glN1C9TJ5xyd2ehE07oAK8ThnK51wleJwzFPq8TPtfpXid4nTAYNzt1gmaCEcMw7DvmMbHMKwb/SVZNCv+JzjCuON/vbwbwlhcuZxEfr3K8yjkLqnpJ9BtHyOsnr58ugshVbR/VM/ezuMLhBvcODhzy/8NhuSgd0qmCc63igtrquPby93r+YJYRLv2LvsES1XuKmGFfvJ2KKEXuXrpFNIZckKt+llQ4TGNc4tq+2Q4P/lbJzxWTdv1zpUIqCgP7xvAsqdy0RzH5xLWSGaDdSHcIdhhOlM5ICViCMXjA/2UdqH2StngXvklasNCw0DJJpDXl2LCGh83UxJ3R4zXJxIEFHsEuIwaBKuUb5T+chfEh62tks9Z/qkigP/+63PR6Mvr6Oub/cvR+XeL+e3q0Gf0Hgnpc+w==")))
@@ -70,7 +69,7 @@ def load_manifest(path, setup_version):
if not isinstance(data, dict):
raise ManifestError("Manifest must be a mapping")
profile = data.get("profile")
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "repository-policy", "marketing"):
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "marketing"):
raise ManifestError("Unsupported profile")
allowed = ("profile", "orb_version", "default_branch", "docs_only", "publish")
if profile in ("qbit", "core"):
@@ -79,7 +78,7 @@ def load_manifest(path, setup_version):
allowed += ("release_completion", "release_on_main", "release_tag_policy", "test_branch_policy")
elif profile == "cli":
allowed += ("publish_context",)
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "repository-policy", "marketing"):
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "marketing"):
allowed = ("profile", "orb_version", "default_branch")
elif profile == "all":
allowed = ("profile", "orb_version", "default_branch", "publish", "publish_context")
@@ -140,7 +139,7 @@ def load_manifest(path, setup_version):
"release_tag_policy": tag_policy, "test_branch_policy": test_policy,
"docs_only": boolean(data, "docs_only", False),
"publish_context": publish_context,
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all", "repository-policy")
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all")
and not (profile == "qbit" and key == "hotfix"))
for key in publish_keys}}
@@ -176,25 +175,6 @@ def proven_docs_only(config, branch, tag):
return base.decode().strip() if eligible else False
def proven_ci_maintenance(branch, tag):
"""Keep application tests, but do not publish for isolated CI declarations."""
if tag or not (branch == "main" or branch == "develop" or branch.startswith(("release/", "hotfix/"))):
return False
subject = git("log", "-1", "--format=%s")
if not subject or not re.match(rb"ci(?:\([^\r\n()]+\))?: ", subject):
return False
parents = git("rev-list", "--parents", "-n", "1", "HEAD")
if not parents or not re.fullmatch(rb"[0-9a-f]{40} [0-9a-f]{40}\n", parents):
return False
base = parents.split()[1].decode("ascii")
changed = git("diff", "--no-ext-diff", "--no-renames", "--name-only", "-z", base, "HEAD")
if not changed:
return False
allowed = {b".qqq-ci.yml", b".circleci/config.yml", b".github/workflows/codeql.yml",
b".github/workflows/release-rc.yml"}
return all(path in allowed for path in changed.rstrip(b"\0").split(b"\0"))
def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
if tag:
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag, flags=re.ASCII):
@@ -219,14 +199,11 @@ def qbit_template(config, branch, tag):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
result = {"version": 2.1, "orbs": {"qqq-orb": "kingsrook/qqq-orb@" + config["orb_version"]}}
filters = ({"branches": {"ignore": "/.*/"}, "tags": {"only": "/.*/"}} if tag else
{"branches": {"ignore": "automation/qbit-rc-pending"}, "tags": {"ignore": "/.*/"}})
if kind == "test" and not tag and not maintenance:
if kind == "test" and not tag:
ignores = {"all": None, "non-release": "/(develop|release/.*)/",
"non-release-no-integration": "/(develop|release/.*|integration.*)/",
"feature": "/(develop|main|release/.*|integration.*)/",
@@ -237,7 +214,7 @@ def qbit_template(config, branch, tag):
ignored = ignores[config.get("test_branch_policy", "all")]
if ignored:
filters["branches"]["ignore"] = ["automation/qbit-rc-pending", ignored]
docs_base = proven_docs_only(config, branch, tag) if kind == "test" and not maintenance else False
docs_base = proven_docs_only(config, branch, tag) if kind == "test" else False
if docs_base:
# Keep the existing workflow/result identity even when Maven is not needed.
result["jobs"] = {"docs_only": {"resource_class": "small", "docker": [{"image": "cimg/base:2026.03"}],
@@ -439,13 +416,7 @@ def voyage_template(config, branch, tag):
return json.loads(json.dumps(VOYAGE_TEMPLATE).replace("__ORB_VERSION__", config["orb_version"]))
def repository_policy_template(config, branch, tag):
del config, branch, tag
return copy.deepcopy(REPOSITORY_POLICY_TEMPLATE)
PROFILE_TEMPLATES = {"qbit": qbit_template, "core": core_template,
"repository-policy": repository_policy_template,
"frontend-material": frontend_template, "frontend-core": frontend_template,
"cli": cli_template, "voyage": voyage_template, "frontend-next": next_template, "all": all_template, "app-starter": starter_template, "android": android_template, "mcp-policy": policy_template, "marketing": marketing_template}
@@ -1,35 +0,0 @@
{
"version": 2.1,
"jobs": {
"validate-policy": {
"docker": [
{
"image": "cimg/python:3.12"
}
],
"resource_class": "small",
"steps": [
"checkout",
{
"run": {
"name": "Install the YAML parser",
"command": "python3 -m pip install --disable-pip-version-check PyYAML==6.0.3"
}
},
{
"run": {
"name": "Validate the application-free branch",
"command": "python3 \\<<'PYTHON'\nimport json\nfrom pathlib import Path\nimport subprocess\nimport xml.etree.ElementTree as ET\nimport yaml\n\npaths = subprocess.check_output(['git', 'ls-files', '-z']).decode('utf-8').split('\\0')\npaths = [Path(name) for name in paths if name]\nif not paths:\n raise SystemExit('No tracked policy files')\nnames = {'LICENSE', 'NOTICE', 'COPYING', 'CODEOWNERS', '.gitignore', '.gitattributes',\n '.editorconfig', 'mvnw', 'mvnw.cmd'}\nsuffixes = {'.md', '.rst', '.txt', '.adoc', '.yml', '.yaml', '.json', '.properties'}\nmanifests = {'pom.xml', 'package.json', 'cargo.toml', 'pyproject.toml', 'go.mod',\n 'build.gradle', 'build.gradle.kts', 'settings.gradle', 'settings.gradle.kts'}\nfor path in paths:\n if path.is_symlink() or path.name.lower() in manifests:\n raise SystemExit(f'Application or symlink requires a build profile: {path}')\n if path.as_posix() == '.mvn/wrapper/maven-wrapper.jar':\n continue\n if path.as_posix() == 'checkstyle/config.xml':\n ET.fromstring(path.read_text(encoding='utf-8'))\n continue\n if path.name not in names and path.suffix.lower() not in suffixes:\n raise SystemExit(f'Unsupported policy file; select a build profile: {path}')\n text = path.read_text(encoding='utf-8')\n if path.suffix.lower() in ('.yml', '.yaml'):\n list(yaml.safe_load_all(text))\n elif path.suffix.lower() == '.json':\n json.loads(text)\nprint(f'Validated {len(paths)} tracked policy/documentation files; no application build or publication')\nPYTHON\n"
}
}
]
}
},
"workflows": {
"verify": {
"jobs": [
"validate-policy"
]
}
}
}
-1
View File
@@ -45,7 +45,6 @@ PROFILE_PARAMETERS = {
"app-starter": {},
"android": {},
"mcp-policy": {},
"repository-policy": {},
"marketing": {},
}
-69
View File
@@ -26,75 +26,6 @@ class Snapshot:
class AuditTests(unittest.TestCase):
def test_all_scanner_and_release_requirements_follow_branch_profile(self):
entry = dict(ENTRY, name='qqq-all', nameWithOwner='QRun-IO/qqq-all')
scanner = '.github/workflows/codeql.yml'
release = audit.auxiliary.BRIDGES['all-release']['path']
for profile in ('all', 'repository-policy'):
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({'profile': profile, 'orb_version': VERSION, 'default_branch': 'develop'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller(profile, VERSION))}
policy = dict(POLICY, profile=profile, codeql_worker_sha=SHA)
report = audit.audit(entry, Snapshot(files), VERSION, SHA, policy)
missing = {f.get('path') for f in report['findings'] if f['code'] == 'missing_native_maintenance_bridge'}
self.assertIn(scanner, missing)
self.assertEqual(profile == 'all', release in missing)
files[scanner] = audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-codeql', SHA), sort_keys=False)
if profile == 'all':
files[release] = audit.generator.yaml.safe_dump(audit.auxiliary.canonical('all-release', SHA), sort_keys=False)
self.assertEqual([], audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings'])
files[scanner] = files[scanner].replace(SHA, 'b' * 40)
self.assertIn('native_maintenance_bridge_drift', [f['code'] for f in audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings']])
files[release] = 'jobs: {}'
self.assertTrue(any(f['code'] == 'native_maintenance_bridge_drift' and f.get('path') == release for f in audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings']))
def test_schema_drift_bridge_tracks_branch_source_without_exempting_default(self):
entry = dict(ENTRY, name='qbit-authentik', nameWithOwner='QRun-IO/qbit-authentik')
path = '.github/workflows/schema-drift.yml'
cases = [
('main', {}, False),
('main', {'src/test/java/example/SchemaDriftTest.java': 'class SchemaDriftTest {}'}, True),
('develop', {}, True),
]
for branch, files, required in cases:
with self.subTest(branch=branch, files=files):
snapshot = Snapshot(files)
snapshot.branch, snapshot.default_branch = branch, 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, POLICY)
missing = any(f['code'] == 'missing_native_maintenance_bridge' and f.get('path') == path for f in report['findings'])
self.assertEqual(required, missing)
snapshot = Snapshot({path: 'jobs: {}'})
snapshot.branch, snapshot.default_branch = 'main', 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, POLICY)
self.assertIn('native_maintenance_bridge_drift', [f['code'] for f in report['findings']])
def test_placeholder_can_retain_canonical_release_and_scanner_bridges(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({'profile': 'repository-policy', 'orb_version': VERSION, 'default_branch': 'develop'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('repository-policy', VERSION)),
audit.rc_worker.CALLER_PATH: audit.rc_worker.generate(SHA),
'.github/workflows/codeql.yml': audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-codeql', SHA), sort_keys=False)}
snapshot = Snapshot(files)
snapshot.branch = 'main'
snapshot.default_branch = 'develop'
policy = dict(POLICY, branch_profiles={'main': 'repository-policy'}, rc_worker_sha=SHA, codeql_worker_sha=SHA)
self.assertEqual([], audit.audit(entry, snapshot, VERSION, SHA, policy)['findings'])
files['.github/workflows/codeql.yml'] = files['.github/workflows/codeql.yml'].replace(SHA, 'b' * 40)
self.assertTrue(any(f['code'] == 'native_maintenance_bridge_drift' for f in audit.audit(entry, snapshot, VERSION, SHA, policy)['findings']))
def test_placeholder_profile_is_scoped_to_selected_branch(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({'profile': 'repository-policy', 'orb_version': VERSION, 'default_branch': 'develop'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('repository-policy', VERSION))}
snapshot = Snapshot(files)
snapshot.branch = 'main'
snapshot.default_branch = 'develop'
policy = dict(POLICY, branch_profiles={'main': 'repository-policy'})
self.assertEqual([], audit.audit(entry, snapshot, VERSION, SHA, policy)['findings'])
snapshot.branch = 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, policy)
self.assertEqual('qbit', report['profile'])
self.assertTrue(any(f['code'] == 'profile_mismatch' for f in report['findings']))
def test_qbit_scanner_policy_requires_exact_canonical_worker(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
manifest = {'profile': 'qbit', 'orb_version': VERSION, 'default_branch': 'develop',
-58
View File
@@ -1,58 +0,0 @@
"""CI maintenance retains Maven verification and never becomes a release request."""
import importlib.util
from pathlib import Path
import unittest
from unittest import mock
import yaml
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('pipeline', ROOT/'src/scripts/generate_pipeline.py')
pipeline = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pipeline)
class MaintenanceTests(unittest.TestCase):
def git_result(self, *args):
return {'log': self.subject, 'rev-list': self.parents, 'diff': self.paths}.get(args[0])
def setUp(self):
self.subject = b'ci: use central pipeline\n'
self.parents = b'a'*40 + b' ' + b'b'*40 + b'\n'
self.paths = b'.qqq-ci.yml\0.circleci/config.yml\0.github/workflows/codeql.yml\0.github/workflows/release-rc.yml\0'
def test_publishing_branches_keep_unfiltered_maven_tests(self):
manifest = {'profile': 'qbit', 'orb_version': '0.7.14', 'default_branch': 'develop',
'release_on_main': True, 'test_branch_policy': 'feature-no-hotfix',
'publish': {'hotfix': True}}
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.7.14')
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
for branch in ('main', 'develop', 'release/0.1', 'hotfix/1.2.3'):
with self.subTest(branch=branch):
result = pipeline.qbit_template(config, branch, '')
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/mvn_test_only']
self.assertEqual(job['filters']['branches'], {'ignore': 'automation/qbit-rc-pending'})
self.assertNotIn('branch_type', job)
def test_application_changes_and_unknown_files_publish_normally(self):
for paths in (b'pom.xml\0', b'.qqq-ci.yml\0src/Main.java\0', b'.github/workflows/custom-release.yml\0', b'', None):
self.paths = paths
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
def test_merge_root_and_unavailable_history_do_not_suppress_release(self):
for parents in (None, b'a'*40+b'\n', self.parents.rstrip()+b' '+b'c'*40+b'\n'):
self.parents = parents
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
def test_tags_and_release_requests_do_not_suppress_release(self):
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('main', 'v1.2.3'))
for subject in (b'Merge develop for next RC\n', b'fix: application\n', None):
self.subject = subject
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
if __name__ == '__main__':
unittest.main()
-30
View File
@@ -202,27 +202,6 @@ class PipelineTests(unittest.TestCase):
self.assertFalse(options["compile_before_verify"])
return options
def test_ci_maintenance_source_and_packed_keep_verification(self):
self.write_manifest(test_branch_policy="feature-no-hotfix", publish={"hotfix": True})
workflow = self.repo / ".github/workflows/release-rc.yml"
workflow.parent.mkdir(parents=True)
workflow.write_text("# central RC caller migration\n", encoding="utf-8")
self.git("add", ".qqq-ci.yml", ".github/workflows/release-rc.yml")
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: adopt central pipeline")
for branch in ("develop", "release/0.1", "hotfix/0.1.1"):
result = self.render(CIRCLE_BRANCH=branch)
options = self.assert_full(result)
self.assertEqual(options["filters"]["branches"], {"ignore": "automation/qbit-rc-pending"})
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH=branch))
(self.repo / "App.java").write_text("application change\n", encoding="utf-8")
self.git("add", "App.java")
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: misleading title")
result = self.render(CIRCLE_BRANCH="release/0.1")
self.assert_full(result, "publish_release_candidate")
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="release/0.1"))
def test_defaults_and_options(self):
options = self.assert_full(self.render())
self.assertTrue(options["install_browsers"])
@@ -260,15 +239,6 @@ class PipelineTests(unittest.TestCase):
self.write_manifest(publish={"hotfix": "true"})
self.render(packed=packed, CIRCLE_BRANCH="hotfix/1.2.3", ok=False)
def test_repository_policy_source_and_packed_never_select_publication(self):
self.write_manifest(profile="repository-policy")
for packed in ([False, True] if PACKED else [False]):
for branch, tag in (("main", ""), ("develop", ""), ("release/1", ""), ("", "v1.0.0")):
result = self.render(packed=packed, CIRCLE_BRANCH=branch, CIRCLE_TAG=tag)
self.assertEqual({"verify": {"jobs": ["validate-policy"]}}, result["workflows"])
self.assertEqual({"validate-policy"}, set(result["jobs"]))
self.assertNotIn("orbs", result)
def test_stable_tags_require_ascii_digits_in_source_and_packed_generator(self):
tag = "v\u0661.\u0662.\u0663"
self.git("tag", tag)
-12
View File
@@ -6,7 +6,6 @@ scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
mkdir "$scratch/bin"
export INSTALL_LOG="$scratch/install.log"
export APT_CONFIG_CAPTURE="$scratch/apt.conf"
export BASH_ENV="$scratch/bash-env"
export QQQ_MAVEN_BIN_DIR="$scratch/maven-bin"
: > "$BASH_ENV"
@@ -15,11 +14,6 @@ cat > "$scratch/bin/sudo" <<'STUB'
printf '%s\n' "$*" >> "$INSTALL_LOG"
if [[ "$*" == *apt-get* ]]; then
[[ "$1" == env && "$2" == DEBIAN_FRONTEND=noninteractive && "$3" == NEEDRESTART_MODE=l ]] || exit 90
[[ -f "$APT_CONFIG_CAPTURE" ]] && grep -qx 'Acquire::Retries "3";' "$APT_CONFIG_CAPTURE" || exit 91
[[ ${APT_FAIL:-0} == 0 ]] || exit "$APT_FAIL"
fi
if [[ "$1" == tee && "$2" == /etc/apt/apt.conf.d/80-qqq-download-retries ]]; then
cat > "$APT_CONFIG_CAPTURE"; exit 0
fi
if [[ "$1" == tee || "$1" == gpg || "$1" == -E ]]; then cat >/dev/null; fi
STUB
@@ -38,7 +32,6 @@ grep -q 'tee /etc/needrestart/conf.d/99-qqq-ci.conf' "$INSTALL_LOG"
grep -q 'export JAVA_HOME=' "$BASH_ENV"
: > "$INSTALL_LOG"
: > "$BASH_ENV"
rm "$APT_CONFIG_CAPTURE"
PARAM_INSTALL_NODE=true bash "$script_dir/install_java.sh" >/dev/null 2>&1
[[ $(grep -c 'apt-get' "$INSTALL_LOG") -eq 5 ]]
grep -q 'apt-get install -y -qq nodejs' "$INSTALL_LOG"
@@ -89,9 +82,4 @@ if PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1;
echo 'invalid native Maven accepted' >&2; exit 1
fi
unset -f mvn
if APT_FAIL=100 PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1; then
echo 'persistent apt failure accepted' >&2; exit 1
else
[[ $? -eq 100 ]]
fi
printf 'PASS: unattended installation and native/executable/nonexecutable wrapper selection\n'
-89
View File
@@ -1,89 +0,0 @@
"""Execute the policy job against real git indexes, including application promotion."""
import importlib.util
import json
import shutil
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
import yaml
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('pipeline', ROOT / 'src/scripts/generate_pipeline.py')
pipeline = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pipeline)
TEMPLATE = json.loads((ROOT / 'src/scripts/templates/repository-policy.json').read_text())
SCRIPT = TEMPLATE['jobs']['validate-policy']['steps'][-1]['run']['command'].split("<<'PYTHON'\n", 1)[1].rsplit('PYTHON', 1)[0]
class RepositoryPolicyTests(unittest.TestCase):
def test_packing_is_independent_of_checkout_line_endings(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
shutil.copytree(ROOT / 'src/scripts', root / 'src/scripts')
(root / 'scripts').mkdir()
shutil.copy(ROOT / 'scripts/pack_pipeline.py', root / 'scripts/pack_pipeline.py')
resources = list((root / 'src/scripts/templates').glob('*.json'))
for resource in resources:
resource.write_bytes(resource.read_bytes().replace(b'\r\n', b'\n'))
subprocess.run([sys.executable, str(root / 'scripts/pack_pipeline.py')], check=True)
packed = (root / 'src/scripts/packed_generate_pipeline.py').read_bytes()
for resource in resources:
resource.write_bytes(resource.read_bytes().replace(b'\n', b'\r\n'))
subprocess.run([sys.executable, str(root / 'scripts/pack_pipeline.py'), '--check'], check=True)
self.assertEqual(packed, (root / 'src/scripts/packed_generate_pipeline.py').read_bytes())
def execute(self, files, untracked=None):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
subprocess.run(['git', 'init', '-q'], cwd=root, check=True)
for name, content in files.items():
file = root / name
file.parent.mkdir(parents=True, exist_ok=True)
file.write_text(content, encoding='utf-8')
subprocess.run(['git', 'add', '.'], cwd=root, check=True)
for name, content in (untracked or {}).items():
(root / name).write_text(content, encoding='utf-8')
return subprocess.run([sys.executable, '-c', SCRIPT], cwd=root, capture_output=True, text=True)
def test_placeholder_with_wrappers_and_workflows(self):
result = self.execute({'LICENSE': 'Apache-2.0', 'docs/DESIGN.md': '# Planned application',
'.github/workflows/release-rc.yml': 'name: Release\non: workflow_dispatch\n',
'.mvn/wrapper/maven-wrapper.properties': 'distributionUrl=https://example.invalid/maven.zip',
'mvnw': '#!/bin/sh\n', 'mvnw.cmd': '@echo off\n'})
self.assertEqual(0, result.returncode, result.stderr)
def test_new_application_requires_real_build_profile(self):
for name in ('pom.xml', 'package.json', 'src/App.java', 'main.py', 'main.go', 'Dockerfile'):
with self.subTest(name=name):
self.assertNotEqual(0, self.execute({'LICENSE': 'license', name: '{}'}).returncode)
def test_malformed_configuration_fails(self):
for name in ('.github/dependabot.yml', 'settings.json'):
with self.subTest(name=name):
self.assertNotEqual(0, self.execute({name: '[unterminated'}).returncode)
def test_unused_checkstyle_policy_is_validated_without_allowing_application_xml(self):
self.assertEqual(0, self.execute({'checkstyle/config.xml': '<module name="Checker"/>'}).returncode)
self.assertNotEqual(0, self.execute({'checkstyle/config.xml': '<module>'}).returncode)
self.assertNotEqual(0, self.execute({'application.xml': '<application/>'}).returncode)
self.assertNotEqual(0, self.execute({'pom.xml': '<project/>'}).returncode)
def test_only_tracked_files_are_policy_inputs(self):
self.assertEqual(0, self.execute({'README.md': '# Docs'}, {'main.py': 'temporary tool'}).returncode)
def test_manifest_cannot_enable_publication(self):
manifest = {'profile': 'repository-policy', 'orb_version': '0.7.14', 'default_branch': 'develop'}
with patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.7.14')
self.assertFalse(any(config['publish'].values()))
self.assertEqual({'verify': {'jobs': ['validate-policy']}}, pipeline.repository_policy_template(config, 'main', '')['workflows'])
manifest['publish'] = {'release': True}
with patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)), self.assertRaises(pipeline.ManifestError):
pipeline.load_manifest('unused', '0.7.14')
if __name__ == '__main__':
unittest.main()
+3 -11
View File
@@ -175,7 +175,7 @@ def owner_gate_errors(snapshot):
def audit(entry, snapshot, version, worker_sha, policy):
name = entry['name']
profile = policy.get('branch_profiles', {}).get(snapshot.branch) or policy.get('profile') or ('qbit' if name.startswith('qbit-') else PROFILES.get(name))
profile = policy.get('profile') or ('qbit' if name.startswith('qbit-') else PROFILES.get(name))
result = {'repository': entry['nameWithOwner'], 'sha': snapshot.sha,
'default_branch': getattr(snapshot, 'default_branch', snapshot.branch),
'source_branch': snapshot.branch, 'profile': profile, 'findings': [], 'required_checks': {}}
@@ -217,7 +217,7 @@ def audit(entry, snapshot, version, worker_sha, policy):
native_paths = set()
rc_source = snapshot.read(rc_worker.CALLER_PATH)
existing_rc = rc_source is not None and 'qbit-release-rc.yml@' in rc_source
if profile in ('qbit', 'repository-policy') and (rc_enabled or existing_rc):
if profile == 'qbit' and (rc_enabled or existing_rc):
path = rc_worker.CALLER_PATH
raw = rc_source
rc_sha = policy.get("rc_worker_sha", worker_sha)
@@ -235,21 +235,13 @@ def audit(entry, snapshot, version, worker_sha, policy):
for kind, definition in auxiliary.BRIDGES.items():
bridge_sha = worker_sha
if kind == 'qbit-codeql':
if profile not in ('qbit', 'repository-policy', 'all') or 'codeql_worker_sha' not in policy:
if profile != 'qbit' or 'codeql_worker_sha' not in policy:
continue
bridge_sha = policy['codeql_worker_sha']
elif definition['repository'] != entry['nameWithOwner']:
continue
path = definition['path']
raw = snapshot.read(path)
if kind == 'all-release' and profile == 'repository-policy' and raw is None:
continue
# Older release branches can predate Authentik's schema contract test.
# Keep the default branch required, and validate every existing bridge.
if (kind == 'qbit-schema-drift' and raw is None
and snapshot.branch != result['default_branch']
and not any(path.endswith('/SchemaDriftTest.java') for path in snapshot.paths)):
continue
if raw is None:
finding('missing_native_maintenance_bridge', path)
continue