Compare commits

..
Author SHA1 Message Date
james.maesandCodex 92255e6699 fix: isolate Ubuntu bootstrap from image PPAs
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 14:24:15 -05:00
james.maesandCodex f76b9f7a7c fix: refresh only the newly added Temurin repository (#76)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 13:51:50 -05:00
james.maesandCodex fdf62a2f5a fix: align effective Maven versions before publication (#75)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 13:15:09 -05:00
james.maesandCodex 94341ef481 fix(ci): retain verification for schema caller migrations (#73)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 12:57:36 -05:00
james.maesandCodex 2debeada5d feat(ci): centralize opt-in QBit feature tag publication (#72)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 12:52:18 -05:00
james.maesandCodex 6febeff590 fix(audit): verify release and hotfix branch enforcement (#71)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 12:30:59 -05:00
james.maesandCodex 8b58a45f35 ci: include shared RC caller updates in maintenance verification (#70)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 11:59:07 -05:00
james.maesandCodex fbe36d683d ci: verify maintenance changes without publishing applications (#68)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 11:53:20 -05:00
james.maesandCodex bd5ee3bec5 ci: centralize policy-only branches and shared scanner enforcement (#65)
Add the repository-policy profile, validate actual legacy policy trees, normalize packed resource line endings, and scope maintenance requirements to branch source. Preserve canonical scanner enforcement for qqq-all and policy-only main branches.

Validation: 32 audit tests, 7 policy tests, all four target source trees, and Linux packed-generator check pass. Replaces this PR's unrecognized-email signatures using the established verified identity; signature enforcement remains enabled.

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 11:21:15 -05:00
james.maesandCodex d4d6e97df0 fix: retry transient package downloads during CI bootstrap (#66)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 10:56:34 -05:00
james.maesandCodex c288a8864f ci: centralize branch-aware QBit source scanning (#63)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 09:36:07 -05:00
james.maesandCodex 9906c8f120 fix: audit explicitly selected repository branches (#62)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 09:15:33 -05:00
james.maesandCodex 4dcf2c1891 fix: preserve legacy qbit hotfix publication policies (#61)
* fix: preserve legacy qbit hotfix publication policies

Co-Authored-By: Codex <noreply@openai.com>

* fix: regenerate packed profiles with Linux zlib

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 09:00:43 -05:00
james.maesandCodex cc30f599e6 fix(ci): provide a stable verification gate across branches (#59)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 03:12:12 -05:00
james.maesandCodex 05a2a5217a fix: audit rc worker pins independently of native workers (#58)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:23:51 -05:00
james.maesandCodex 7d73ac90b6 fix: distinguish missing RC refs from GitHub error bodies (#57)
* fix: distinguish missing rc refs from github error bodies

Co-Authored-By: Codex <noreply@openai.com>

* fix: reject empty ref responses across jq versions

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:15:32 -05:00
james.maesandCodex edc1365ac9 fix: recover verified rc uploads after publisher timeout (#56)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:02:14 -05:00
james.maesandCodex 0f33ddadf8 ci: run business E2E on changes and preserve main validation (#55)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 01:19:08 -05:00
james.maesandCodex 5e9bbb54ec ci: add batch version rollout for canonical consumers (#54)
Validate every checkout before updating its orb pin. Preserve publication policy, filters, credentials contexts and forwarded parameters; reject custom callers and mismatched declarations. Cover all 12 profiles, dry runs, no-op repeats, invalid batches and duplicate input with six new tests in the existing contract job. Existing migration and standard tests pass; Linux packed generator check passes.

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 00:45:50 -05:00
james.maesandCodex 5bcec90e33 ci: report the shared database gate on every pull request (#53)
Preserve migration verification, avoid duplicate feature push builds, and cancel superseded pull request runs so the gate can be required safely.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 22:25:38 -05:00
james.maesandCodex a50b07d927 fix: escape Android heredoc in continued Circle configuration (#51)
Circle accepts the continuation request before reporting an unescaped heredoc as a pipeline configuration error. Escape the template marker and emulate Circle unescaping in executable publisher tests. Validate the actual continued config with CircleCI.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 22:00:56 -05:00
james.maesandCodex 6318d90e96 feat: centralize lightweight GitOps rendering validation (#50)
Add a read-only native worker and canonical consumer bridge. Verify the pinned Kustomize binary, render every configuration without logging resources, and audit adoption. Behavioral and audit tests pass; all five marketing GitOps configurations render successfully.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:54:25 -05:00
james.maesandCodex a6fea661ff fix: authenticate Android publishing with GitHub package token (#47)
* fix: authenticate Android publishing with GitHub package token

Use the shared GitHub Packages token and verify its identity and registry repository access before Gradle starts. Preserve unit and emulator gates; package write success still requires hosted publication.

Co-Authored-By: Codex <noreply@openai.com>

* fix: keep embedded JSON resources consistent across checkouts

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:40:01 -05:00
james.maesandCodex 2894bae29a test: retain All release shell behavior in shared owner (#46)
Move the consumer publication regression to the reusable worker; existing test discovery executes it. Verify exact RC1 prerelease/non-latest flags, GA behavior and rejection of other RC versions.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:19:09 -05:00
james.maesandCodex a14e8f9b24 fix: default All releases to shared Circle smoke evidence (#35)
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:12:58 -05:00
james.maesandCodex 83611872a3 fix: verify public All smoke evidence without secret export (#44)
Allow unauthenticated Circle API reads for public projects, preserve optional authentication and all fail-closed provenance checks.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:12:00 -05:00
james.maesandCodex 30c4e0e85c fix: use shared Maven credentials for Android publication (#42)
Use the existing organization context and Gradle environment properties. Fail before publishing if either credential is absent and avoid writing secrets to gradle.properties.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:07:21 -05:00
james.maesandCodex f429d9f3c4 fix: build marketing site before browser checks (#40)
Run lint and one production build in every shared marketing workflow before Playwright and Lighthouse. Preserve the delegated delivery contracts.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 20:56:32 -05:00
26 changed files with 27 additions and 450 deletions

No files matched your search

-2
View File
@@ -85,8 +85,6 @@ jobs:
bash tests/test_maven_ci_cache.sh
bash tests/test_maven_test_results.sh
bash tests/test_maven_ci_install.sh
bash tests/test_browser_install.sh
bash tests/test_frontend_build_reuse.sh
bash tests/test_snapshot_refresh.sh
bash tests/test_maven_settings_output.sh
# Create jobs to test the commands of your orbs.
-2
View File
@@ -4,8 +4,6 @@ CircleCI orb for QQQ framework projects.
For centralized pipeline ownership, see [standardized manifest-driven pipelines](docs/GENERATED-PIPELINES.md).
The initial QBit profile uses a small `.qqq-ci.yml` and a setup entry point; existing direct job callers remain supported.
For centrally managed patch adoption and exact-version rollback, see
[managed patch updates](docs/managed-patch-updates.md).
**For:** Teams using CircleCI to build, test, and publish QQQ applications
**Status:** Stable (v0.6.x)
-28
View File
@@ -80,14 +80,6 @@ required-check unknown states, and the lint image/tool/command/caller contract.
## Batch orb-version rollout
For a standard QBit without a manifest, start directly on the managed patch
track using `python tools/standardize_qbit.py /checkout --orb-version 0.8
--default-branch main`. Inspect the preview, then repeat with `--write`.
This preserves the inferred publication and branch-filter policy while writing
the same track to the manifest and setup caller. Use `rollout_version.py` below
for repositories that already have a manifest. Neither command configures
repository protection or automatically enrolls newly created repositories.
Upgrade existing canonical CircleCI consumers with one command:
```sh
@@ -111,23 +103,3 @@ verification and artifact evidence. The requested version must already be publis
```sh
python3 -m unittest discover -s tests -p test_rollout_version.py
```
## Live organization discovery
Use `--organization QRun-IO --live` instead of `--inventory inventory.json` to
include newly created repositories on each audit. Discovery uses the existing
`gh` authentication and all API pages, retaining archived entries for explicit
skip reporting. Unknown repositories receive no automatic exemption. Duplicate,
empty, malformed or failed discovery aborts instead of reusing a stale inventory.
```sh
python tools/audit_estate.py --organization QRun-IO --live --orb-version 0.8 \
--worker-sha <40SHA> --policy observed-checks.json --output audit.json
```
The organization mode has no fixed repository-count ceiling. Supply
`--expected-active N` for an independent count check when needed; saved-inventory
mode preserves its default of 68. Reports identify the inventory source and
credential visibility limit: inaccessible repositories cannot be certified.
This is discovery and audit, not automatic enrollment or branch-rule mutation.
Scheduling and authenticated onboarding still need to be configured centrally.
-37
View File
@@ -1,37 +0,0 @@
# Managed patch updates
The canonical caller and manifest accept either an exact stable version such as
`0.8.0` or a quoted minor track such as `"0.8"`. CircleCI resolves a minor track
to its latest published patch. A major-only version, `latest`, `volatile`, and
wildcards are rejected. The caller import, setup parameter, and manifest must
still agree exactly.
Use a minor track after qualifying that release line. Future patch fixes then
arrive on the repository's next pipeline without a version-bump PR or an extra
fleet-wide build. Existing exact pins remain supported and are not changed by
publishing this capability. YAML minor versions must be quoted to remain strings.
The central batch tool applies the same update to all supplied checkouts:
```sh
python3 tools/rollout_version.py --orb-version 0.8 --write /path/to/repo1 /path/to/repo2
```
This changes only the two canonical CI declarations. Publication policies,
contexts, branch/tag filters, and profile options stay unchanged. Normal source
and publication checks still run when a pipeline is triggered; this setting does
not skip tests. GitHub reusable-worker commit pins are separate and unchanged.
Automatic patch adoption trades exact-version reproducibility for centrally
managed fixes. Keep patch releases backward compatible, qualify central changes
before production publication, and use an exact stable pin for a canary or
rollback. For example, the same tool can move a consumer from `0.8` back to
`0.8.0`. Historical pipeline reruns can resolve a newer patch; choose an exact
pin when reproducing an earlier run. Setup and continuation each resolve their
orb imports, so patch compatibility also matters when publication overlaps a run.
Changing minor tracks is an explicit migration. Do not use this feature to push
breaking changes into a patch release or to bypass repository checks.
Reference: [CircleCI orb semantic versioning](https://circleci.com/docs/orbs/use/orb-concepts/).
+1 -8
View File
@@ -10,11 +10,6 @@ description: >
- Collects JUnit XML test results
- Stores test results and screenshots
- Saves Maven cache for future builds
parameters:
build_before_test:
type: boolean
default: true
description: "Build the React app first; combined jobs already build it with frontend_build"
steps:
# Ensure no stale Chrome/Driver is shadowing the PATH (your base image had Chrome 109)
- run:
@@ -22,7 +17,7 @@ steps:
command: <<include(scripts/frontend_test_cleanup.sh)>>
# Install current Chrome; DO NOT install chromedriver (let Selenium 4 auto-manage it)
- install_chrome
- browser-tools/install-chrome
# System libs Chrome needs in headless CI
- run:
@@ -57,8 +52,6 @@ steps:
# GPG signing will happen later in mvn_jar_deploy when deploying to Maven Central.
- run:
name: Build app, start server, run tests
environment:
PARAM_BUILD_BEFORE_TEST: << parameters.build_before_test >>
command: <<include(scripts/frontend_test_run.sh)>>
# Always collect and publish test results & screenshots
-5
View File
@@ -1,5 +0,0 @@
description: Install stable Chrome using only its signed repository and existing Ubuntu dependency indexes.
steps:
- run:
name: Install Google Chrome
command: <<include(scripts/install_chrome.sh)>>
+1 -1
View File
@@ -21,7 +21,7 @@ steps:
- when:
condition: << parameters.install_browsers >>
steps:
- install_chrome
- browser-tools/install-chrome
- browser-tools/install-chromedriver
- when:
condition: << parameters.compile_before_verify >>
+1 -2
View File
@@ -27,8 +27,7 @@ steps:
- git_full_checkout
- mvn_manage_version
- frontend_build
- frontend_test:
build_before_test: false
- frontend_test
- frontend_package
- mvn_jar_deploy
- git_operations:
+1 -2
View File
@@ -23,5 +23,4 @@ steps:
node_version: << parameters.node_version >>
- git_full_checkout
- frontend_build
- frontend_test:
build_before_test: false
- frontend_test
+2 -12
View File
@@ -11,20 +11,10 @@
## Output: Installs required system libraries
############################################################################
set -euo pipefail
set -e
echo "Installing browser dependencies"
export DEBIAN_FRONTEND=noninteractive
ubuntu_sources=/etc/apt/sources.list
if [[ ! -s "$ubuntu_sources" ]]; then
ubuntu_sources=/etc/apt/sources.list.d/ubuntu.sources
fi
if [[ ! -s "$ubuntu_sources" ]]; then
echo 'Cannot locate Ubuntu package sources' >&2
exit 1
fi
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update \
-o "Dir::Etc::sourcelist=$ubuntu_sources" \
-o Dir::Etc::sourceparts=- -o APT::Get::List-Cleanup=0
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \
libnss3-dev libgdk-pixbuf2.0-dev libgtk-3-dev libxss-dev
+1 -3
View File
@@ -15,9 +15,7 @@ set -e
echo "Building app, starting server, running tests"
echo "HTTPS=true" >> ./.env
if [[ "${PARAM_BUILD_BEFORE_TEST:-true}" == true ]]; then
npm run build
fi
npm run build
export REACT_APP_PROXY_LOCALHOST_PORT=8001
export PORT=3001
npm run start &
+3 -3
View File
@@ -89,8 +89,8 @@ def load_manifest(path, setup_version):
allowed += ("java_version",)
mapping(data, allowed, "manifest")
version = data.get("orb_version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be an explicit stable or development version")
if version != setup_version:
raise ManifestError("Manifest orb_version must match the setup job orb_version")
default_branch = data.get("default_branch")
@@ -225,7 +225,7 @@ def qbit_template(config, branch, tag):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = proven_ci_maintenance(branch, tag)
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
-25
View File
@@ -1,25 +0,0 @@
#!/bin/bash
set -euo pipefail
# Machine executors are Ubuntu amd64. Preserve an existing browser for Maven
# consumers; frontend_test deliberately removes its stale browser beforehand.
if command -v google-chrome >/dev/null 2>&1; then
google-chrome --version
exit 0
fi
architecture=$(dpkg --print-architecture)
if [[ "$architecture" != amd64 ]]; then
echo "Google Chrome requires an amd64 machine executor" >&2
exit 1
fi
curl --fail --silent --show-error --location https://dl.google.com/linux/linux_signing_key.pub \
| sudo gpg --batch --yes --dearmor -o /usr/share/keyrings/qqq-google-chrome.gpg
printf '%s\n' 'deb [arch=amd64 signed-by=/usr/share/keyrings/qqq-google-chrome.gpg] https://dl.google.com/linux/chrome/deb/ stable main' \
| sudo tee /etc/apt/sources.list.d/qqq-google-chrome.list >/dev/null
# Ubuntu dependency indexes were refreshed by install_java. Only the newly added
# signed Google repository needs refreshing; image PPAs are unrelated to Chrome.
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update \
-o Dir::Etc::sourcelist=/etc/apt/sources.list.d/qqq-google-chrome.list \
-o Dir::Etc::sourceparts=- -o APT::Get::List-Cleanup=0
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y google-chrome-stable
google-chrome --version
+4 -4
View File
@@ -16,7 +16,7 @@ import yaml
# Generated by scripts/pack_pipeline.py; edit source/resources, then regenerate.
import base64
import zlib
CORE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtPWt3m0iy3+dX9NVk13ZWgN+PJJ5zE9tz4908HWd39kS5Oi1oJMYIMI3s6CTe375V3Q1CCBDIsvyIM19k6K6urndVVzPffyGkccFC7vhe4xlZ19ea+CSgIe2zCJ7Dw+/wBJ71fa/rtzuOZ7Uth3Y9n0eOOXoPI6JhwODvRsf3XUa9RjN+YTGbDtwI3tnU5Uw8vpJvGx77FrWpabIgop7J2p2BZ7msPQjdPNA8Ch2vmwe50agClvfo+tb2XCGHzGROEF0L9C8KfMMPOymSn5+fa/AEB54BBB76/pmhHv7vqr6t7zWSmabf71PPSs3mEQ0jTbBNQ7ZpUUhNlkaQRyzACV/UAwTTY+aZP4gSbEkyXLwPB14KgnrogbAgkh9CBpLDiH/pMYu8xZUJ980zFpG0yDTHZyvMEYBjEz6wfEKDSOvCrEFg0YgR7fyc/PWv468cD/bnukQbitdcbA5GtTyAqUAS7YJ4nHkgyITzBEQyFQb0yc7WFjGifiAIm6JWCuMYrJitVtIYET/2cTDR/GkwjCBktut0e5Hu+l2YO2DPSdRjHkIOQEAimyyxb4EfRuTjx4/tt+/f/d/79unJy4Oj9snnd/t/OW+1vCXSajw5OD45eHPU/tf7k3/8/ub9v9p/f/+qfXzYapDffsPXr15+et0+evfPVqPlMdA2hM/Mnk+WJEMQaxAmkuBDBh69oI5LOy6gZPpe5HgDYGLodB2PuuQtvWAeEYLB9aWWZzstr5Fi4VXy+2oWoXnf4Sy8gPU8dwgEcTj50+8scQL4MNdN5Mh2vkWDkE1IT4eaZ93QHwgBQqoWS1fEeEQ0D4n0PYfEz7QrIOKPH4R9cyKy2vIEuzUNCCUw1Jh3sS+Jf3DczOdCMwcuCYZRz/c2iG46oeky0zGE5KQERQ+GhTRVv76OGSEe+UFN1a7Okd8dz+G9mlp8CaKMk6l7SYdlOj4TFyJ/YPamqhhSRUwV4n7gD1yLeH5EuNMVgiy2YvogVWbkh0KW/ZDQKGL9AC0KebLM2TlZI+urK8+J5aPqCHT/h2j21NVBTMCq639y30MkOiGjZwiBu4wFZK3lWb7H1H7MXt8Hq3FC6N/CP6aan5wt9ekZI2jERuMImHvHpmbECSxtoTrjFsGmSpLX2sHINpk0qjMx3+iksFT+kvQdzsEOPY9lC8Ut5D0ngBF9CjYa7FIQ+mB7ZjM6IAwhayc0mRT3gEY9FMgpm8uKsgXEBLsYyYipUTCphi6DxbWdLtg2LQBbRrtMUw5qXvr8D5Q/BTt2fgJ/tPwg/J7FrBKF/ZWc9sACks7AAeHrU7PneIwTdPSWw00awnRCbXSyIDNovfWW9yv5DFM+Q9gVDZZwMMwCI993wtAPm+Sy54A+QzAIqkDD0AF4OBcVFYBxZg5CJxqCw3Uipiud4fBGc8gS/9GLouCZYcTD9IFYRgecDflTjOAwRK2bM6K7RAwWmQbEEwb3B6HJuO46PFKLpWIEMcxjzAIpx5DKQIbpVsuLvfZfuHDNS0+8EF99V+OuyD4YObfVeL5EfsjYIWKsCJyxt6ehEJqOjg/Ib4bFLgxv4Lq1PQNIEqf9wGWa1Es+L0E6kKaT/P2zB7ZZWBWI+CBimd0b9M8sJyRaQHDvCmvQ+j9hGQOoA5Ge8ecAVou3Ik22XBUtdvE0DhplOyESQaBonB59OtWe6t/6bsksGwIhTu28WbFHAIP6Ba0pODA5CPzV15G9BIsZjL2ruDMDJ6OZk25iFmOHHGlP8jxj7yricxsICIVgoUaDwHVMYaM0gdQNmf5iTDqhf8kRkyTRWzgKYFwDlw61S8eCP/nC15frapgQhL67+PVtRjHiBwQuWIhuMRVlLBwZzPpTwqCBpHZZGSq5xjnASguP4jXYhWMxbzxwzym9yD30wNFmMM9WGCZWTwgyF7MvsSYx1gQtMXrtmEGkC8n69T2BiMcyFDJevCAjyuiCGJDvSm8A80RAP0SHkDX7JGvRSaGRIZNKTzJKSDJKMe4TLLD7xSL0JMEzx2FAKlB5bg0CXcupKGltR3770g/PeDCeY8ai4/tREkZnBbs5qWfjUqhe5GPfGBv2tYqiJfUwCENTtTDGO23g7hkL2xhd+WGfZvROxbXjSuf0QaxxczJo1NbXVzefQdAZMi9xSSMdA/skAsm26VKOizdc5F6jWVZvKwz+R6SL64CGetX+k17Q9Pucol0yp3/htUXkPlNF758sdOwhOcQULSQOBPJxpUhovyQpSZO0WMEtBQTGzrF8dDLwyEvQ5z4i51nkhHY6TvT2Yz3cgEpEe0U0LpUrKdTAc42zKMLqq4gctUBGjyBQRKN9oh3CLz21iM7PnGBf1LmJiWVwciFoOM8ts/MBmIN4hyqPSO+UQG73xvEG30q2HFelNNJqvXix9OHfmGWHfp+gkrpOhzh9EWR/gD/VGyCAzqIQkhj17shlfVCFU3jUAlsWW9h9MWlZGjSgT1Fw3WqsSAuudgLmGyYpbrYaTTCJMTth6DNpLBE9WCFeyyB2qyGCdNibnpTJdVhaz7Lmu1zn6mD07PgU2Yq1UiIKvJBvAvDUvnSwS5wt46orOmwCzd3yihzepxGoHu73i0m5VAnxQ/ghTF1tx7NAY2FXGBrjO9gIegv8ieDgDbIVn+5jughW3vc8MPX8JWbUrwTCJ5LDrcZXuS5YFwbUd5m3rFAQs9eaSIt4k8+y8iHyNVV1iTesICkoX1a/CowBKZTigFkCW07egfeoAPySgneNJ7a8D/++6TSmRLZq5czCa7U7wFqs7gdtacfT7oF9Y+YAsItXRtM6ZoYXadxrhG44mSQbIzDYA66hoXQZRDLIwC6QGs//eAVjkRehILdGpBvFTqKoPW2iCvLbcQxZVgmf7UgKABPJzxQdhDkD24yF32F8wgERAJuNBgXbJ5rME+L1NQ3jKAxzk5pjMlNTMjd/N6HilUkiFEVp8yzejPaXpA2VKz1xoaWQWEaK2qLsnoqqzaDGvCr4Z+bcyBpJNC8L3uhP8LAQ3EnxOk/xPDGnNgVPR3mGqknJZ7UweZ5KH+5y6W3Wqlt6fyUaLoM4LVuTyFXyUCpePFjlzXpC3JvNwwo3XjkZK5T5SvnY3a5QXq8oOKEe96EoVhbypMqbiwh2xqiRG/hMkCR+6/kWUyf802h+g/nwK3EGZmOlAlWRWCzA4zPPxBOsuB4maTuXbPMwBEnD4Gx/VYf/YqFIFai0Q4y4T0GvuMo3Y7LPL5KQUSQQVGwvcDzRGEAhP3GoSyzKex0fq5PFO+46kJ+6YGRJfDIHT3qDjjiR+wg5vHb8Xrg3CMw8PJHU+gq8loDXEUi6yjQ5pOXhGO1gyjASCwgYa4tFEPeRtZ3dTbbJdq31PXPL3GP2rrm5ysxNe33bpFumZW/tdtb2dtZrB6dMKr0MLicofMaGuSY4wdkL+trFmlYVPe37d9WpM+iDOywngxErquubZzKoaZCrq+tY+QrKUypChHbx6D+SjUCJnpVIlmlNlYlfCST8QBBwo6IzQgJXagJofHj/llyCOMYCLszDEo+7yDTXAbkFOwx+OAI910eBgmaS0qpJAC+SYgj8IWsNKy0Q8W8RvMFHiFEb/8aqgg8EwjLACyxfqL7M3zZR9V8Y6UeYXqvcHaeCGg28aBlmx4WAVuODpPGItB8/fiRqPsgI9boyS0ccLkMnYhIJAQ6iKBeCDATYzGJTYIiy+K3AboA2IL/EdEDPXNal5lALGAtB5wIuG6ts+Mu3bUFhTRNKQv5j6DCr5R0c76O1JwhCdrBomEFhqeKl66oSG0RbA+/YewV8Qka/xbeCiRIzIAAwzhqYot1BMPY5dhZhC0dXRWwgMwG5pKGHZlc1nyR/6wIPuRYiAqtJMC0vyX54aBo4yYgLwNzIE0NZ5ReTDb3aJAP3dCqlFd5hE5OgROJ2hmRMkkUPHWxUCPhYvwbso4bXSbGyxPFU8E3doCsLooKTS9ohor//FN3V0ri7ql0qgpy9xKoKN3ubRrRqsC2l/cZsLpbH1bmdRSPaJK9PTz+I+o867CKpELBCsfgVtsxiEC7l0OEiEEWhmoxZUTeWRMT/FPKkpUSWP74CJ43l+lgBhdiqBs9mSpxB6P2xllDfBA0QTK8nz7WiKMgvU3tBqmnnHScylP3Oiu180aimeTgqjpT0nEhpKgQ7L8eNN6h9SB10I/94M6FCGkr6eGOuRTsU2je037FiCakmpEBtorGBD+FFwLCaMGfmqKOfvmNZLgMHATovkGzi00HkuE40VI94hjgVKmXZoolwGLnrlZdZxo9B5Fo0CPRP4pck66HDA+FHk92h6E+pz0wsl2CY2ft18Pt0zj9LaNfFbao85rfKzal8UZlpN3tAUpEzM9WjSrpYMj0rqtoArA/ZDEWKPuzTwfx/0eWKWzx4vyOFhoT0C6kziGAFcIKFxZrqopQ82BeoyOfLr1+uVHMJ8yRATZ9JWi0sWauQV1k3sbl3AO5Qbe1UJDaHsULq6LaO7Xf+pwDQtB1mCYrHyc6hE+kpcG8ROYR2fArvkpp0j7rRe+93gARQc1oR5uur42YEP+qhlLr+EI/OSR/Q4gTitx4E6pg2LbYVQRq68U6EXNeR9CBIU9gkfcdz+hD4YzMCHoTP4sRy2SyT/ybZXmnKw/e8f7Bi/jlI+YopSYiXWVtbiVsmitsaFJkMtfuJDocQhGYf4/Bl2c8gOxaEfGOKDxzLvhs1DqyMtRggpN/2Y+rKZgK56BXIFry9IlJt4GcTUnSQf8zev6sJV5mOheyytpR2idX+PlkVNiM7iuHtg9GYptp2xVaF/IChzHOZqu706LjGHVdMF/6AiuSxIVQ5bZd5YI1RhFW/qRDIVM9pyoGJ6iNx8YLM2+Gnj2+yVPH8NvAqGETtyOkzZBust7Har5cLJcKICr7/ZLl/hvfexA2X+HA5aY0d+bI/xDLine truncated
CORE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtPdl22ziy7/0VuOrM2M6IpPclifvcxHbfeCar48z0nChXByJBiW1uJig7Oonn26cKACmKIiVSluUlTr/IJFAo1F6FAvv7L4Q0LljEncBvPCPr+loTn4Q0oh6L4Tk8/A5P4JkX+N2g3XF8q205tOsHPHbM4XsYEQ9CBn83OkHgMuo3mskLi9m078bwzqYuZ+LxlXzb8Nm3uE1Nk4Ux9U3W7vR9y2XtfuQWgeZx5PjdIsiNRhWwvEfXt7bnCjliJnPC+Fqgf1HgG0HUyZD8/Pxcgyc48Awg8CgIzgz18H9X9W19r5HONAPPo76Vmc1jGsWaYJuGbNPiiJosiyCPWYgTvqgHCKbHzLOgH6fYknS4eB/1/QwE9dAHYUEkP0QMJIeR4NJnFnmLKxMemGcsJlmRaY7OVpgjAMcmvG8FhIax1oVZ/dCiMSPa+Tn5619HXzk+7M91iTYQr7nYHIxq+QBTgSTaBfE580GQCecpiHQqDPDIztYWMWIvFITNUCuDcQJWzFYraYyIH/s4mGjBNBhGGDHbdbq9WHeDLszts+ck7jEfIYcgILFNlti3MIhi8vHjx/bb9+/+73379OTlwVH75PO7/b+ct1r+Emk1nhwcnxy8OWr/6/3JP35/8/5f7b+/f9U+Pmw1yG+/4etXLz+9bh+9+2er0fIZaBvCZ2YvIEuSIYg1CBNJ8SF9n15Qx6UdF1AyAz92/D4wMXK6jk9d8pZeMJ8IweD6Usu3nZbfyLDwKv19NYvQvO9wFl3Aer47AII4nPwZdJY4AXyY66ZyZDvf4n7ExqSnQ82zbhT0hQAhVculK2Y8JpqPRPpeQOJn2hUQ8ccPwr45MVlt+YLdmgaEEhhqzL/Yl8Q/OG4Wc6FZAJeEg7gX+BtEN53IdJnpGEJyMoKih4NSmqpfX0eMEI+DsKZqV+fI747v8F5NLb4EUcbJ1L2kg0k6PhMX4qBv9qaqGFJFTBXifhD0XYv4QUy40xWCLLZiBiBVZhxEQpaDiNA4Zl6IFoU8WebsnKyR9dWV58QKUHUEuv9DNHvq6iAmYNX1P3ngIxKdiNEzhMBdxkKy1vKtwGdqP2bPC8BqnBD6t+iPqeanYEsePWMEjdhwHAFz79jUjDmBpS1UZ9wi2FRJ8lo7GNomk8Z1JhYbnQyWyl8Sz+Ec7NDzRLZQ3CLec0IY4VGw0WCXwigA2zOb0QFhiFg7pcm4uIc07qFATtlcXpQtICbYxVhGTI2SSTV0GSyu7XTBtmkh2DLaZZpyUPPS53+g/CnYifMT+KPlB+H3LWZNUNhfyWkPLCDp9B0QPo+aPcdnnKCjtxxu0gimE2qjkwWZQeutt/xfyWeY8hnCrri/hINhFhh5z4miIGqSy54D+gzBIKgCjSIH4OFcVFQAxpnZj5x4AA7XiZmudIbDG80hS/xHL47DZ4aRDNP7YhkdcDbkTzGCwxC1bsGI7hIxWGwaEE8YPOhHJuO66/BYLZaJEcQwnzELpBxDKgMZplstP/Haf+HCNS898SN89V2NuyL7YOTcVuP5EvkhY4eYsTJwxt6ehkJoOjo+IL8ZFrsw/L7r1vYMIEmceqHLNKmXfF6CdCBNJ/n7Zx9ss7AqEPFBxDK7N/DOLCciWkhw7wpr0Po/YRkDqAORnvFnH1ZLtiJNtlwVLXb5NA4aZTsREkGgaJwefTrVnurfPHfCLBsCIU7tolmJRwCD+gWtKTgwOQj81dehvQSLGY68q7gzAyejmZNuYhZjhxxpj/M8Z+8q4nMbCAiFYJFGw9B1TGGjNIHUDZn+ckw6UXDJEZM00Vs4CmBcQ5cOtEvHgj/5wteX62qYEESBu/j1bUYx4gcELliEbjETZSwcGcz6M8KggaR22SRUCo1ziJUWHidrsAvHYv5o4F5QepF76IGjzWGerzCMrZ4SZC5mX2JNEqwJWmL02gmDSBeS9et7AhGP5ShkvHhBhpTRBTEg35XeAOaJgH6ADiFv9kneopNSI0PGlZ7klJDklGLUJ1hg98tF6EmKZ4HDgFSg8twaBLqWU1HS2o6D9mUQnfFwNMdMRCcI4jSMzgt2c1zPRqVQvSjGvjEy7GsVRUvrYRCGZmphjHfawN0zFrUxugoij+b0TsW1o0rneCDWuDkZNGrr66ubzyDojJifuqShjoF9EoFk23Qpx8UbLnKv0ZxUbysN/oekS+qAhnrV/pNe0Oz7gqJdOse78Nsicp+povdPFjn2gBxiihYRBwL5pFIktF+SlGRJWq7glgICY+dYPjrp++Ql6LOHyPkWOaGdjhO//VgPN6AS0V4RjUvlSgs18FzjLI6x+ioiRy2U0SMIFNGoR7RD+KVnFtH5mRPuizo3MbEMTi4EDee5ZXbeB3OQ7FDlEdmdEsjt3jh+/9uELSdVKY20Wi9eLH34N2bZUeARVFLX6RDHE0H2B/hTvQEC6CyOIIlR745c5oEqnMKjFtiyxMLui0nL0qABfcqC61ZjRVpwtRMw3zBJcbPVaIJJTNgJQ59JY4nowQrJWgaxWw0RpMPe9LRMrsPSep413+U6VwfDZ8enyFaslRJR4IV8E4Bn9qWDXeJsGVdd0WETaO6WV+Rwj8agerjfLyblUiXED+GHMHW1Hd8CjYVdYWiM72Aj6C3wJ4KDN8hWfLqP6SJY+cD3wdTzl5hRvxIIn0gOtxpf5bpgXRhQ32X+skJBzF5rIi2STT7Ly4fI11TVJdmwgqSgfFn9KjAGpFCKQ2YJbDl5B96jAvBLCt41mdjyP/z7ptOYCbJVK2cWXqvdAdZidT9sSzuedQ/sGzP7gF2yMprWETO8SONeI3TDySTdGIHBPnANDaXLIJJBBnaB1Hj+xysYi6IIBbk1JN0wdhJF7WkTVZDfTmLISZXw2Y6kADCR/MzQQZgzsM1Y+B0kJxwQAbDZaFCyfaLJPCFZX9MwjsIwN605pjM1JXPzdxMqXhknQlmUNs/izXB/adpQudKTFFpKiWVkqC3K7pmo2gxrzKuCf27OjayRRvOy4I3+BA8LwZ2Ur/MUzxMLalPwdJhnqJqUfFYLk+eZ9OEul95mrbpl9zdBw2UQp+VrEoVKHknFSwarvFlPiXuzeVjpxisnY6UyXykfu9sVyusVBcfU4z4UxSaFPJny5iKCnRFqFAY+YyRJ3vqBxdQJ/zSa32A+/EqcgdlYqUBVJBYL8fjMN/EEK6mHSdrOJds8jEDSMDjbX9Xhv0QoMgUq7RAj7lPQK67yzYTs84skZBQJBBXbCx1fNAZQyE8c6hKL8l4nwOpk+Y67DuSnLhhZkpzMwZNevyNO5D5CDq8dvxfuDQIzH08kNU+B11LwOgLJVpnGh7R8HKMdTBlGEgEBY22xGOI+srazu8k22a61vmdumXvM3jU3V5m5aa9vm3TLtOyt3c7a3s567eCUSaWXweUYhc/YoNAEpzj7oaddrGlV0dO+f1edOn0P3OFkMhiJorqBeSaDmga5urqOla+gPBNFiNAuHv3HshEo1bMJkmVaU2XiVwIJPxAE3KjojJDAlZoAGh/evyWXII6JgAvzsMSTLjLNdUBuwQ6DH45Bz/VhoKCZZGLVJIQXaTEE/pC1hpUWiPi3GN7gI8SojX9jVSEAAmEZ4AWWL1Rf5m+bqPovjOwjTK9V7o5TQY36frwMs5NCQKvxQdJ4SNqPHz8SNR9khPpdmaUjDpeREzOJhAAHUZQLQQYCbOaxKTFEefxWYDdAG5BfYjqgZy7rUnOghYxFoHMhl41VNvwV2LagsKYJJSH/MXSY1fIPjvfR2hMEITtYNMygsFTx0nVViQ2irb5/7L8CPiGj3+JbwUSJGRAAGGf1TdHuIBj7HDuLsIWjqyI2kJmQXNLIR7Ormk/Sv3WBh1wLEYHVJJiWn2Y/PDINnGQkBWBuFImhrPKLyYZebZKBezqV0grvsIlJUCJ1OwMyIsmihw42KgR8pF8D9lHD62RYOcHxVPBN3bArC6KCk0vaIaK//xTd1dKou6pdKoKcfYJVFW72No1o1WBbSvuN2Vwsj6tzO4vGtElen55+EPUfddhFMiFghWLxK2yZxSBcyqHDRSCKQjUes6JuLImI/ynkSUupLH98BU4ay/WJAgqxVQ2ezYw4g9AHIy2hgQkaIJheT55rRVGQX2b2glTTzjtObCj7nRfb+aJRTfNwVBIp6QWR0lQIdlGOm2xQ+5A56Eb+8WZKhSyU7PHGXIt2KLRvqNexEgmpJqRAbaKxfgDhRciwmjBn5qijH8+xLJeBgwCdF0g28Wk/dlwnHqhHPEecCpWyfNFEOIzC9SaXWUaPQeRaNAz1T+KXJOuhw0PhR9PdoehPqc+MLZdimNv7dfD7dM4/S2jXxW2qPBa3ys2pfFGZaTd7QFKRMzPVoyZ0seR6VlS1AVgfsRmKFB7s08H8f9Hlils8eL8jhYaU9AupM4hgBXCChcWa6qKUPNgXqMjny69frlRzCfMkQE2fSVotLFmrkFdZN7G5dwDuUG3tVCQ2h4lC6ui2ju13wacQ0LQdZgmKJ8nOoRPrGXBvETmEdnwK79KadI+68Xv/d4AEUAtaEebrq5NmhCDuoZS6wQCPzokHaHEC8VsPAnVMmxbbiiAN3WgnQqHrSHsQpClsEs/xHQ8Cf2xGwIPwWZxYIZtl8t8k2ytNefhe9A9WLD4HmbxiRhKSZdbWVpKWifK2BkUmQ+1+rMMhAqHZxzh8WfYzyI4FId+Y4gPH8u+GjQMrIy0GCOm3/YS6splALnoFsgVvr4hUG/jZhBQd5B+z9+9qwlWuYyG/rC2lXWK1v09Whc3Ij2J4+2A4pqm2XbFVoThgmOS5TFV3enRco44roQt/QEXyxBCqnLbLfLDGKMKq31QIZKbnNOPARPWRuHhB5u3g08c3ear4QRt4Ffbjdux4DNkG622sevVyoVQYUcH3nyx7Z3jvTdxwSQ6X09bYoS/7Q/zDkiLine truncated
MATERIAL_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNq1lMtugzAQRfd8heV1TNRKlar8QXdVt02EXDMhDthGfqRBiH/vQAmPvKRU6g7PjH3PXI+pI0IoHEEEb6yjK1JjAENlwatvK7OdH2IYTY3IwWLks4+QIdflpeIZYJoqYWMlhTXObH0sjFqOB64OT/HLa/zM9lypig77m/5rE51WzaKl25uvq2CJB+fdFO/UR0swaWBxyjsPpZvRU7EDkZswFp11ZFHDWEgEx8qJVp/OoZof2Md1qViKYqyuSSfhgiJrWnKRo0OsQB/jvTN6TUnTTLTPt9NZZjNZNaNvt9CDvgTWXHUX9Kad50VBUAV0ClpIcGccFK9NcZ225QhEhCSMFZBxUbESwHaE9CEkxw/3rOyV/uwcVvnd9fswKSTKpKEA92+efgRN3oexIwfpAi+IhQyHyEmjye/A3nP5SMa57cof9LebVW693HIxexsTg+avg1kojfXLc6wUxaXmHrmvbvgLWNtQgmaE4h5bW8X6quUNmctfRdREPykCLqM=")))
VOYAGE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtW+tz2zYS/56/AqPJXJKeSUqqHzndqBNbdm1NGzsnObneVDkNREISLBKgAVCWYuv+9lvwpQepV2K1TpoPsUhggV38drEPgLl/hlBhSISknBUqqGyW9nTLDe9IeL2HZ3jrBNR1DMwcQxGp0nbo8bDdp4zMNEEj9XBPNxWCTsBUYJTLxf2KHQhBmCrEZJO9ZApBJA+ETdq2i6VmWnCxgPEpAWFDKjjz9Og5Pm+PP5xdtq/eXTf1KOM3bwSMXntZFlIRX8/8+3Ss3Sf2gAcqZYNmZob+29tbg4uORZlU2HXbN3iI57iHVLq1PUWvUC4VZggm6fNkyiWd2Ruytg3wkTYgoLggy0QRActyZtgLIf5ABO2OEfZ9l9pYgRx7iDJFeiJ8QVphcg/dBtilCuiYg2wOEuPeLL8IEu550K9nBdmQ7RLM0DCa3zhBxvU+MozAd7AihmTYl32uJDQxbiiBmewSYfiC92A58nNgkHi4FIMQoLZejEYrcJXMQuJj1dfCK20+ypKBIF0qiCGIz4VaL9KXMuxi6krc/SKGWCjaxfYG3IZ8DDo0b7BY1KMDQlMWql/Tg2mEVDuVRlJF1sgRm2DoRlIT3EooX280UIji7TsuBtLHNslKJjjXfqJgLsqjRZ73AnFHvIbQ/xC5MGzar43BWEeUsbplhEv1h1aBOu28wTa3uUlGxM7SnHJ7QESXujnjfe6ZI8/NdkiRM5Npc9alvRxqznC432+IrUz4Be0oSuasHqGP+Qp+Ntsbq7sgiS2Ikoa08azDKzjhYuYUN2cWabixqdezOliSxWAzw3FF2JEeOPrClwSNlZ66HkUSdE4VeNaBXOF+70DByLhFfaV8WbGsHlX9oAOq8PRjOHr6IAj8gklaDr9jLseONXxtlktmOSVpR+9tl7Jg1B4d7ptgQ2bvU4vBLzJGn7poPand97iD/j5KSVtsu927EpxGwDYBxkzXhByiwPAg/ERqRCY8RraKzITIVNxzoR2cTYfLZd4m3xgxbPbPtkSw40AwBRtU+JYkHsREv+LiMHvayiY94tDA251RNiPZUPO4eY2aernLwY/XgTQqqNVqMTSFHAeKL7b5lk6Psq38DkvfUBz+EZbtBjcQiDBQBA5VaT8Rgov07QbcD/zAsv1AoVgyI47UZtgbkZro4QEpEZAW26X8Jir/9LcSAlaEzIijQ4CpRmoqxI7CcB4Ca+Jx7pBdS5cAsrFs6YDtAklkQYv7dyfVyvotOgcaVgqkWJXA4Dh92c67RjVElEpn5xyQcW7qE2UhkL/T4dgYFo8Oi8ZGAfyzfI2d1BOUOLMVC/hynzCHMJtmMqsC4+1ol7cV9Yj2czBV6cBb6aggLpCAI5/6RGflEOgAHmRIlEm+WswbOBSioI9ahecXV2/PLDPE0AoxaRWSIalRxYldi0XRAMGawTsIL3EUQz3Ru3+fWhleFUsb5GyD4MBgybhFhhUr4Tw/JCMzEEIKnDSl1BgScJgzpimaR4dm8Y3s4/LBYaW8Xzp0St2j/eL+QefgqHy073RL+Mf9IiHksHNQKsJcztGP9uEhPrTtzmsHF/9R7pRKR6Rz5JTLZaQ5dmXqK/U6GWTraBi4Gh1JtN6h/ryon1/s1Rr163rt+Fft1UcUcmruEFRKB7uQ4xuQKRn+oKfryy4XHlZo3uUnYFizthM5/hDlrT0tFJ8rNo82q5y9cn+PQAy7jyaTjauN/80p60t229ZF2oJFrXHCKbLgSOORW3nhsEB4Ii54bZa0I7f8tRwjNbWyai4PHHTMsDuGOnvN8ZBxgkINV8K/yeY9Dd/MuCz8hYyrt8DXoLwd7Z4FOi56mNFPockllAskfS6VGQi3mhRDYbOtRTUzxIoPCKs+v29eXR432tdXv5xdThZIkqMHMy6goRRuhNvhnd6w1ZnSOy6xrSnhwlTS9kyHStxxiVNdmd7lbpAofhjh4epfeIustMsTDQ6KjjNQ3Vt9ahkH5BBQZCgUmVzl+X2t3qj9etZuXhyXKsXK0QSZafRWuLeCLu6JKrdHLXivtfNHNQ4lOOharKu9rhv1D/9pfzhrNOtXl9VWoWgeQKXeKqzIQqwhFhY0WxGFKeGnkteYmx88v59jOUGhAU4DfBLQk1i+pyP7tFjLCeu0x8A7GQFUUSPI/6LmpdA/KtjvAtlHUOCdU3URdNA7bA+A6yoPR+w+18nV/flFLXEkkEk9oBhul/coQ72+LbQTMoIp7fvmWePy+O2ZJjcMH7YfbBjHkMqhbEOri6e1YodorQApO19kq8vmSCw5HudrYLZnt2pYwoF20e8hKvEsJ43jy9qFRqUKzQ7Yj8t9ePuoa+MVlB7sD032T6T6hGmbeRQMEybT+TbCYmZYbCLatsCcNx0IO7ZLW2y7wVMNFLYLMbLDvc8/SuWQ/904g0q5ZBb/gKPUR05pzgn4VTBG1Dy5eote1saQMzBy+turNYkN5CSmHRE7o0r6ZHhQITDDdwPY+RUPD8hxrydIDzic8NTtnkblyc9hyVKdOYdKei5BuqpWS9J+srMzoSSVAWYbnQeldFsYWHwdaDthTfEkT+3nwSMq8MGsPK5IOxVxAcCZTNwhXRy46hFj0fsQMVQ7RRoxdEdVHzFyF8dXh/b0GfEfH5harP72+PysfVo/P2teV5+/jOcDvrSrAwpUML4+cN/YV03P3VH5JyjThxYLXBckvblFhkAvzFMibUF9MGczWjayLEQ8X41fJOelr7RgYSQxPoUrmRVykhsWfM1ki5CGUP7Ck/Um5w/VF/f3FMrhETJ1vXAaSixRcTJ5sQUmD8gOYE7nDTK65VdRLEijQX3GBCoos9gtsIgnPGs0rhoVyDIDSIgZV+GlifAg4ZwzNz01jIGcDZUSkXrwEno+AKAjMLP7SEfi9EoKjKp+ffH+JDbBNzNXVP9qAAr1K+tDdF5SOzXeUZ+4wFVfzLSY7aBsl+YpIRQaFOSWDwkI8fmU+cPDElgeYOG6onPxWFpQdjqBrR2aNQjAT3pxaWmOsefG8OmFObTbhXXdBpSoDUdnsL3kyO5jBkLpzFLvUJjYiBFF2BUEO2MU+LpX7/gZkItzIEfXCoGEbFz7CJi6RqG+JbW6HrJIQkALUIeC13yjrQ2sLqHCjrMxEtGskcienq0PMeXldNCrCoo8e5xhxeYCS8nPSfR8Yf7EBdX+J0zapobdDGybSNmFrTmOJ3ZWe8BtE534EhTCs/75/DjEoGaplItmqfSk7o719ejCEpeHCOaPFohRtDuNuDUtyZLjkrmtnPZq7Rj65CXZ0PNHOMlshhr7pBpmR2mXH1VYhpa/Gg1rsRy50lfxJ8m0kY09i+2soI8+ui6/m/k0Tocb4HobLHwYF39At8Toln5QF/d3qQuOOpvb6aGhMyZ5fdqawyI759g5PlmiLPMlR5wLhvVYfqf133m1GYb5g7WGtNWyNFGG5uNCy+TZsrcNMuH8D0a+Y7jVvUf2O4fvAG5nhLmXzWmogCxDkNyboAUvMH8TtPddGSuUkX+gHov857thztxxVD5GEj19p7dLiXfgYnYL8JPe0Ltcet6VbZpgMkVGKm/Z0xu51Uv+pqGbP2ldD1qUkv91AVty+7kBcH1bPBps2c+aZ91rtjd1ZXkDZ9zG01XHqvL56UTOLZOVb6Hs+BpWvrtI/sQV/jVkBE8awr9GZvGkVfAtZyhPGfjvmc43qdalV98b76y9x1X4nJ3tNgENVfCIWC69vtm9l/oaUFtyHcHF7H/Sj7+e0YwGlPWk4HxgxY1v2u2rxknyMV+7raWYPJs8+z8FFeia")))
NEXT_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrNVE1v2zAMvedXCFqAtlscb70tQLGtXTEUGFYgHXqJM0OxmUSJLCmS7DYr8t9HubaXOOnHYYfdLPLp6fGR9EOHEFqAsVxJOiCn/Q89H1FmYvH4gN94Wq1WAUYwQJdczqxRahlWwc9xfD08j28vhzdX1z/imOKVTcmxUNscOp8Ibudxyux8ophJ4wUzTRoBcA9J7pSP1Q+GC1Yw2qsR1oH2jKMqgKFkDslS5a4BkYZwW3nIpXVMiLgkHOxgEFVnpUoBs87k0NtF+Ez81yZ6ekq3AJvme/OEDpPL/Wcly/xz9BYMn66JmwPhWZY7NhFADAhgFkj9aEtQorKMydRfr5C1vLOIdh8uroYX3y/jn1++vSk2EY3k1CjpQKYNqnvsayKBJhEyrHJu4PioH2qWLNkM+gur5NFJv0JH9CSSGStAbt3PCkmCcxKsyByEHkDBRM4cUn6Fe23AljBt1AISVxNhcqpMAjcuxa4hqQPrUEG3rS+i5MzHW8X5UuorO3qewf/DTp3nXKRlo3BgHE8IVqqMe6Y5iTLgPSUgfVsjqaXOSDVwJAiw7t8gA6GS5ZQ3ebfWUI52dRZcujqF1VefE69m8CjhFWU2y4CN88sXp6CFWtf3xp365uP63imznAp1d2CHK5O3t7da9tHza1iuEdM6LqXv+4wWOGzcXgJTE8MkWnIo5zd4JtFob3jYfxvSFmDTa7M5NnuCSUmx3ilkKxcWB8hr5w653z69PH4v/SabCcM1uHcHhJZel6sRGJhx68w6SAykIB1nwu6qH/f+V/9p+KsYvQ8+jt9FUb/1cRwML5rDyafu/nuvacD+2Hc2nT/snNGl")))
@@ -89,8 +89,8 @@ def load_manifest(path, setup_version):
allowed += ("java_version",)
mapping(data, allowed, "manifest")
version = data.get("orb_version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be an explicit stable or development version")
if version != setup_version:
raise ManifestError("Manifest orb_version must match the setup job orb_version")
default_branch = data.get("default_branch")
@@ -225,7 +225,7 @@ def qbit_template(config, branch, tag):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = proven_ci_maintenance(branch, tag)
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
+1 -3
View File
@@ -649,9 +649,7 @@
"qqq-orb/mvn_publish": {
"requires": [
"sample_bootstrap_source",
"snapshot_esb_broker_conformance",
"snapshot_sample_coverage_gate",
"qqq-orb/static_analysis"
"snapshot_esb_broker_conformance"
],
"pre-steps": [
"configure-package-install"
+2 -2
View File
@@ -109,8 +109,8 @@ def validate(config_raw, manifest_raw, setup_version):
if not isinstance(manifest, dict):
raise StandardError("Manifest must be a mapping")
version = manifest.get("orb_version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise StandardError("Manifest must select a quoted minor track, exact stable pin, or development version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise StandardError("Manifest must pin an explicit orb version")
if version != setup_version:
raise StandardError("Manifest and running setup orb versions differ")
publishing = manifest.get("publish", {})
-50
View File
@@ -1,6 +1,4 @@
import importlib.util
import json
import io
import pathlib
import unittest
from unittest.mock import patch
@@ -27,54 +25,6 @@ class Snapshot:
return self.required
class InventoryTests(unittest.TestCase):
def repo(self, name, archived=False):
return {'name': name, 'full_name': 'QRun-IO/' + name,
'archived': archived, 'default_branch': 'main'}
def test_all_pages_include_new_private_and_archived_repositories(self):
new = dict(self.repo('new-product'), private=True)
pages = [[self.repo('qbit-existing')], [new, self.repo('retired', True)]]
with patch.object(audit, 'command', return_value=json.dumps(pages)) as command:
entries = audit.discover_inventory('QRun-IO')
self.assertEqual([e['name'] for e in entries], ['new-product', 'qbit-existing', 'retired'])
self.assertTrue(entries[-1]['isArchived'])
self.assertEqual(command.call_args.args[0], ['gh', 'api', '--method', 'GET',
'--paginate', '--slurp', 'orgs/QRun-IO/repos?type=all&per_page=100'])
def test_empty_duplicate_foreign_and_malformed_inventories_fail_closed(self):
repo = self.repo('qbit-example')
for pages in ([], [[]], {}, [repo], [[repo], [repo]],
[[dict(repo, full_name='Other/qbit-example')]],
[[dict(repo, archived='false')]], [[dict(repo, default_branch=None)]]):
with self.subTest(pages=pages), patch.object(audit, 'command', return_value=json.dumps(pages)):
with self.assertRaises(audit.ReadError):
audit.discover_inventory('QRun-IO')
with patch.object(audit, 'command') as command:
with self.assertRaises(audit.ReadError):
audit.discover_inventory('QRun-IO/../../other')
command.assert_not_called()
def test_live_cli_count_is_dynamic_but_explicit_expectation_is_enforced(self):
for expected, status in ((None, 0), ('1', 0), ('68', 1)):
args = ['audit', '--organization', 'QRun-IO', '--live', '--orb-version', '0.8']
if expected:
args += ['--expected-active', expected]
report = {'active_count': 1, 'unique_repository_count': 1,
'inventory_count': 1, 'results': [{'status': 'pass'}]}
output = io.StringIO()
with patch.object(audit.sys, 'argv', args), patch.object(audit.sys, 'stdout', output), \
patch.object(audit, 'discover_inventory', return_value=[ENTRY]), \
patch.object(audit, 'run', return_value=report):
self.assertEqual(audit.main(), status)
self.assertEqual(json.loads(output.getvalue())['inventory_source'], 'organization-api')
def test_discovery_api_failure_never_falls_back_to_old_inventory(self):
with patch.object(audit, 'command', side_effect=audit.ReadError('unavailable')):
with self.assertRaises(audit.ReadError):
audit.discover_inventory('QRun-IO')
class AuditTests(unittest.TestCase):
def test_all_scanner_and_release_requirements_follow_branch_profile(self):
entry = dict(ENTRY, name='qqq-all', nameWithOwner='QRun-IO/qqq-all')
-69
View File
@@ -1,69 +0,0 @@
#!/bin/bash
# Exercise real browser scripts without sudo, package installation or network.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
mkdir "$scratch/bin"
export INSTALL_LOG="$scratch/install.log"
cat > "$scratch/bin/sudo" <<'STUB'
#!/bin/bash
printf '%s\n' "$*" >> "$INSTALL_LOG"
if [[ "$*" == *'apt-get update'* ]]; then
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || { echo 'unrelated image PPA: 503' >&2; exit 100; }
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 91
[[ ${REQUIRED_REPO_FAIL:-0} == 0 ]] || exit 100
fi
if [[ "$1" == tee ]]; then cat >> "$INSTALL_LOG"; fi
if [[ "$1" == gpg ]]; then cat >/dev/null; fi
STUB
chmod +x "$scratch/bin/sudo"
export PATH="$scratch/bin:$PATH"
bash "$root/src/scripts/frontend_test_dependencies.sh"
echo 'PASS: browser dependencies ignore unavailable image PPAs'
# Model Chrome presence without depending on the host's installed browsers.
export CHROME_MARKER="$scratch/chrome-installed"
export BASH_ENV="$scratch/shell-env"
cat > "$BASH_ENV" <<'STUB'
command() {
if [[ "$*" == '-v google-chrome' ]]; then [[ -f "$CHROME_MARKER" ]]; else builtin command "$@"; fi
}
google-chrome() { [[ -f "$CHROME_MARKER" ]] && echo 'Google Chrome test'; }
STUB
cat > "$scratch/bin/curl" <<'STUB'
#!/bin/bash
printf '%s\n' "$*" >> "$INSTALL_LOG"
[[ ${KEY_FAIL:-0} == 0 ]] || exit 22
printf 'signing-key\n'
STUB
cat > "$scratch/bin/dpkg" <<'STUB'
#!/bin/bash
echo amd64
STUB
# Successful installation makes Chrome available to the final version check.
# shellcheck disable=SC2016
sed -i '/if \[\[ "$1" == tee/i if [[ "$*" == *"apt-get install -y google-chrome-stable"* ]]; then touch "$CHROME_MARKER"; fi' "$scratch/bin/sudo"
chmod +x "$scratch/bin/"*
: > "$INSTALL_LOG"
bash "$root/src/scripts/install_chrome.sh" >/dev/null
grep -q 'Dir::Etc::sourcelist=/etc/apt/sources.list.d/qqq-google-chrome.list' "$INSTALL_LOG"
grep -q 'gpg --batch --yes --dearmor -o /usr/share/keyrings/qqq-google-chrome.gpg' "$INSTALL_LOG"
grep -q 'signed-by=/usr/share/keyrings/qqq-google-chrome.gpg] https://dl.google.com/linux/chrome/deb/' "$INSTALL_LOG"
grep -q 'apt-get install -y google-chrome-stable' "$INSTALL_LOG"
: > "$INSTALL_LOG"
bash "$root/src/scripts/install_chrome.sh" >/dev/null
[[ ! -s "$INSTALL_LOG" ]] # Existing Chrome needs no network/package work.
rm "$CHROME_MARKER"
for script in install_chrome frontend_test_dependencies; do
: > "$INSTALL_LOG"
if REQUIRED_REPO_FAIL=1 bash "$root/src/scripts/$script.sh" >/dev/null 2>&1; then
echo "Required repository outage accepted: $script" >&2; exit 1
fi
if grep -q 'apt-get install' "$INSTALL_LOG"; then exit 1; fi
done
: > "$INSTALL_LOG"
if KEY_FAIL=1 bash "$root/src/scripts/install_chrome.sh" >/dev/null 2>&1; then
echo 'Signing key failure accepted' >&2; exit 1
fi
if grep -q 'apt-get' "$INSTALL_LOG"; then exit 1; fi
echo 'PASS: Chrome ignores unrelated PPAs; required repositories and signing key fail closed; existing browser reused'
-22
View File
@@ -34,28 +34,6 @@ class MaintenanceTests(unittest.TestCase):
self.assertEqual(job['filters']['branches'], {'ignore': 'automation/qbit-rc-pending'})
self.assertNotIn('branch_type', job)
def test_nonpublishing_branches_verify_maintenance_but_keep_application_policy(self):
manifest = {'profile': 'qbit', 'orb_version': '0.8', 'default_branch': 'main',
'release_on_main': False, 'test_branch_policy': 'feature-no-hotfix',
'publish': {'snapshot': False, 'release_candidate': False, 'hotfix': False}}
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.8')
maintenance_paths = self.paths
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
for branch in ('main', 'develop', 'release/0.1', 'hotfix/1.2.3'):
for maintenance in (True, False):
with self.subTest(branch=branch, maintenance=maintenance):
self.paths = maintenance_paths if maintenance else b'src/Main.java\0'
result = pipeline.qbit_template(config, branch, '')
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/mvn_test_only']
ignored = job['filters']['branches']['ignore']
if maintenance:
self.assertEqual(ignored, 'automation/qbit-rc-pending')
else:
self.assertEqual(ignored, ['automation/qbit-rc-pending',
'/(develop|main|release/.*|hotfix/.*|integration.*)/'])
self.assertNotIn('branch_type', job)
def test_application_changes_and_unknown_files_publish_normally(self):
for paths in (b'pom.xml\0', b'.qqq-ci.yml\0src/Main.java\0', b'.github/workflows/custom-release.yml\0', b'', None):
self.paths = paths
-37
View File
@@ -1,37 +0,0 @@
#!/bin/bash
# Exercise the actual test runner with no server, browser, network or Maven work.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
mkdir "$scratch/bin"
export RUN_LOG="$scratch/calls"
cat > "$scratch/bin/npm" <<'STUB'
#!/bin/bash
printf 'npm %s\n' "$*" >> "$RUN_LOG"
if [[ "$*" == 'run build' && ${BUILD_FAIL:-0} == 1 ]]; then exit 12; fi
STUB
for tool in mvn dockerize; do
# shellcheck disable=SC2016
printf '#!/bin/bash\nprintf "%s %%s\\n" "$*" >> "$RUN_LOG"\n' "$tool" > "$scratch/bin/$tool"
done
chmod +x "$scratch/bin/"*
export PATH="$scratch/bin:$PATH"
cd "$scratch"
PARAM_BUILD_BEFORE_TEST=false bash "$root/src/scripts/frontend_test_run.sh" >/dev/null
if grep -qx 'npm run build' "$RUN_LOG"; then
echo 'FAIL: combined job rebuilt the already-built React app' >&2; exit 1
fi
grep -q '^mvn .* -B test$' "$RUN_LOG"
echo 'PASS: combined job retains tests without a duplicate build'
: > "$RUN_LOG"
bash "$root/src/scripts/frontend_test_run.sh" >/dev/null
grep -qx 'npm run build' "$RUN_LOG"
grep -q '^mvn .* -B test$' "$RUN_LOG"
: > "$RUN_LOG"
if BUILD_FAIL=1 bash "$root/src/scripts/frontend_test_run.sh" >/dev/null 2>&1; then
echo 'FAIL: required standalone build failure ignored' >&2; exit 1
fi
if grep -q '^mvn ' "$RUN_LOG"; then exit 1; fi
echo 'PASS: standalone command still builds and fails before tests when its build fails'
-39
View File
@@ -241,15 +241,6 @@ class PipelineTests(unittest.TestCase):
self.assertTrue(options["check_middleware_api_versions"])
self.assertEqual(options["java_version"], "17")
def test_minor_track_reaches_continuation_without_changing_jobs(self):
expected = self.render()
expected['orbs']['qqq-orb'] = 'kingsrook/qqq-orb@0.7'
self.write_manifest(orb_version='0.7')
self.assertEqual(expected, self.render(QQQ_CI_ORB_VERSION='0.7'))
if PACKED:
self.assertEqual(expected, self.render(packed=True, QQQ_CI_ORB_VERSION='0.7'))
self.render(ok=False, QQQ_CI_ORB_VERSION='0.8')
def test_branch_matrix(self):
for branch, workflow in [("develop", "publish_snapshot"), ("release/1.0", "publish_release_candidate"),
("main", "test_only"), ("hotfix/fix", "test_only"),
@@ -429,27 +420,9 @@ class PipelineTests(unittest.TestCase):
self.change_docs()
self.assertIn("docs_only", self.render()["jobs"])
def test_core_snapshot_publisher_waits_for_all_acceptance_and_analysis(self):
self.write_manifest(profile="core")
result = self.render(CIRCLE_BRANCH="develop")
jobs = {options.get("name", kind): options
for entry in result["workflows"]["verify"]["jobs"]
for kind, options in entry.items()}
def ancestors(name, path=()):
self.assertNotIn(name, path, "Workflow dependencies must be acyclic")
required = jobs[name].get("requires", [])
return set(required).union(*(ancestors(parent, path + (name,)) for parent in required))
self.assertEqual(ancestors("qqq-orb/mvn_publish"), set(jobs) - {"qqq-orb/mvn_publish"})
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="develop"))
def test_core_workflow_graphs_match_baseline(self):
self.write_manifest(profile="core")
baseline = yaml.safe_load((ROOT / "tests/fixtures/core_pipeline_baseline.yml").read_text(encoding="utf-8"))
# Snapshot publication must now wait for the existing sample and analysis gates.
for entry in baseline["workflows"]["publish_snapshot"]["jobs"]:
if "qqq-orb/mvn_publish" in entry:
entry["qqq-orb/mvn_publish"]["requires"] += ["snapshot_sample_coverage_gate", "qqq-orb/static_analysis"]
self.git("tag", "v1.0.0")
self.git("tag", "publish-fixture")
events = [("feature/test", "", "test_only"), ("develop", "", "publish_snapshot"),
@@ -623,18 +596,6 @@ class PipelineTests(unittest.TestCase):
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="main"))
def test_nonpublishing_main_ci_change_runs_verification_with_packed_parity(self):
self.write_manifest(default_branch="main", release_on_main=False,
test_branch_policy="feature")
self.git("add", ".qqq-ci.yml")
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: use central pipeline")
result = self.render(CIRCLE_BRANCH="main")
job = result["workflows"]["verify"]["jobs"][0]["qqq-orb/mvn_test_only"]
self.assertEqual(job["filters"]["branches"], {"ignore": "automation/qbit-rc-pending"})
self.assertNotIn("branch_type", job)
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="main"))
def test_qbit_publication_policy_packed_parity(self):
self.write_manifest(release_on_main=True, release_tag_policy="v-prefix",
test_branch_policy="feature")
+1 -25
View File
@@ -68,30 +68,6 @@ class RolloutTests(unittest.TestCase):
self.assertFalse(rollout.rollout([root], '0.7.11', write=True)[0]['changed'])
self.assertEqual(applied, self.contents(root))
def test_managed_patch_track_and_exact_pin_rollback_preserve_every_profile(self):
roots = [self.checkout(profile, profile=profile, default_branch='main')
for profile in rollout.standard.PROFILE_PARAMETERS]
originals = [yaml.safe_load((root / '.qqq-ci.yml').read_text()) for root in roots]
rollout.rollout(roots, '0.7', write=True)
for root, original in zip(roots, originals):
self.assertEqual({**original, 'orb_version': '0.7'},
yaml.safe_load((root / '.qqq-ci.yml').read_text()))
rollout.standard.validate(*(data.decode() for data in self.contents(root)), '0.7')
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.7')
with self.assertRaises(ValueError):
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.8')
self.assertTrue(all(not r['changed'] for r in rollout.rollout(roots, '0.7', write=True)))
rollout.rollout(roots, '0.7.4', write=True)
for root, original in zip(roots, originals):
self.assertEqual(original, yaml.safe_load((root / '.qqq-ci.yml').read_text()))
def test_unquoted_track_is_rejected_as_numeric_yaml(self):
root = self.checkout('numeric')
manifest = root / '.qqq-ci.yml'
manifest.write_text(manifest.read_text().replace('0.7.4', '0.7'))
with self.assertRaises(ValueError):
rollout.pipeline.load_manifest(manifest, '0.7')
def test_one_custom_caller_rejects_entire_batch_before_any_write(self):
good = self.checkout('good')
bad = self.checkout('bad')
@@ -121,7 +97,7 @@ class RolloutTests(unittest.TestCase):
def test_invalid_target_and_duplicate_checkout_are_rejected(self):
root = self.checkout('qbit')
before = self.contents(root)
for version in ['latest', 'volatile', 'dev:test', '0', '0.7.*', '0.7.11\n']:
for version in ['latest', 'dev:test', '0.7', '0.7.11\n']:
with self.assertRaises(ValueError):
rollout.rollout([root], version, write=True)
with self.assertRaises(ValueError):
-18
View File
@@ -76,24 +76,6 @@ class MigrationTests(unittest.TestCase):
manifest = migration.infer_manifest(self.config(), "0.7.0", "main")
self.assertFalse(any(manifest["publish"].values()))
def test_managed_track_preserves_policy_and_caller_pins(self):
fixtures = json.loads((Path(__file__).parent / "fixtures/qbit-publication-policies.json").read_text(encoding="utf-8"))
for fixture in fixtures:
with self.subTest(repository=fixture["name"]):
expected = migration.infer_manifest(fixture["config"], "0.8.1", fixture["default_branch"])
expected["orb_version"] = "0.8"
actual = migration.infer_manifest(fixture["config"], "0.8", fixture["default_branch"])
self.assertEqual(actual, expected)
caller = migration.canonical_caller("0.8", actual["release_tag_policy"], actual["publish"].get("feature", False))
self.assertEqual(caller["orbs"]["qqq-orb"], "kingsrook/qqq-orb@0.8")
self.assertEqual(caller["workflows"]["setup"]["jobs"][0]["qqq-orb/generate_pipeline"]["orb_version"], "0.8")
self.assertEqual(yaml.safe_load(yaml.safe_dump(actual))["orb_version"], "0.8")
def test_reject_ambiguous_or_invalid_versions(self):
for version in (0.8, None, "0", "latest", "volatile", "0.8.*", "0.8.1.2", ""):
with self.subTest(version=version), self.assertRaises(ValueError):
migration.infer_manifest(self.config(), version, "main")
def test_snapshot_does_not_enable_rc_or_release(self):
manifest = migration.infer_manifest(
self.config("qqq-orb/mvn_publish", branch_type="snapshot",
+5 -47
View File
@@ -66,38 +66,6 @@ def gh(path):
return json.loads(command(['gh', 'api', '--method', 'GET', path]))
def discover_inventory(organization):
"""Discover every repository visible to existing gh authentication, across all pages."""
if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9-]*', organization):
raise ReadError('invalid organization name')
pages = json.loads(command(['gh', 'api', '--method', 'GET', '--paginate', '--slurp',
f'orgs/{organization}/repos?type=all&per_page=100']))
if not isinstance(pages, list) or any(not isinstance(page, list) for page in pages):
raise ReadError('invalid organization inventory response')
entries, seen = [], set()
for page in pages:
for repo in page:
if not isinstance(repo, dict):
raise ReadError('invalid organization repository')
name = repo.get('name')
full_name = repo.get('full_name')
archived = repo.get('archived')
branch = repo.get('default_branch')
if (not isinstance(name, str) or not re.fullmatch(r'[A-Za-z0-9_.-]+', name)
or not isinstance(full_name, str)
or full_name.lower() != f'{organization}/{name}'.lower()
or type(archived) is not bool
or not isinstance(branch, str) or not branch
or full_name.lower() in seen):
raise ReadError('invalid or duplicate organization repository')
seen.add(full_name.lower())
entries.append({'name': name, 'nameWithOwner': full_name,
'isArchived': archived, 'defaultBranchRef': {'name': branch}})
if not entries:
raise ReadError('empty organization inventory; coverage is unproven')
return sorted(entries, key=lambda entry: entry['nameWithOwner'].lower())
class Snapshot:
def __init__(self, entry, live=False):
self.repo = entry['nameWithOwner']
@@ -406,32 +374,22 @@ def run(entries, version, worker_sha, policies, live=False, snapshot_factory=Sna
def main():
parser = argparse.ArgumentParser(description=__doc__)
inventory = parser.add_mutually_exclusive_group(required=True)
inventory.add_argument('--inventory', type=Path)
inventory.add_argument('--organization', help='Discover current repositories using existing gh authentication; requires --live')
parser.add_argument('--inventory', type=Path, required=True)
parser.add_argument('--orb-version', required=True)
parser.add_argument('--worker-sha')
parser.add_argument('--policy', type=Path)
parser.add_argument('--live', action='store_true', help='Read current source and branch rules using existing gh authentication')
parser.add_argument('--source-branch', help='Inspect this branch instead of the repository default; requires --live')
parser.add_argument('--expected-active', type=int, help='Optional independent coverage check; saved inventories default to 68')
parser.add_argument('--expected-active', type=int, default=68)
parser.add_argument('--output', type=Path)
args = parser.parse_args()
if args.source_branch is not None and not args.live:
parser.error('--source-branch requires --live')
if args.organization and not args.live:
parser.error('--organization requires --live')
entries = (discover_inventory(args.organization) if args.organization else
json.loads(args.inventory.read_text(encoding='utf-8')))
expected_active = args.expected_active
if expected_active is None and not args.organization:
expected_active = 68
entries = json.loads(args.inventory.read_text(encoding='utf-8'))
policies = json.loads(args.policy.read_text(encoding='utf-8')) if args.policy else {}
report = run(entries, args.orb_version, args.worker_sha, policies, args.live, source_branch=args.source_branch)
report['inventory_source'] = ('organization-api' if args.organization else 'saved-inventory')
report['inventory_visibility'] = 'authenticated repositories; inaccessible repositories cannot be certified'
report['expected_active_count'] = expected_active
report['coverage_matches_expected'] = ((expected_active is None or report['active_count'] == expected_active)
report['expected_active_count'] = args.expected_active
report['coverage_matches_expected'] = (report['active_count'] == args.expected_active
and report['unique_repository_count'] == report['inventory_count'])
text = json.dumps(report, indent=2) + '\n'
if args.output:
+2 -2
View File
@@ -56,8 +56,8 @@ def plan(checkout, version):
def rollout(checkouts, version, write=False):
if not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', version):
raise ValueError('Fleet upgrades require a stable minor track or exact patch version')
if not re.fullmatch(r'\d+\.\d+\.\d+', version):
raise ValueError('Fleet upgrades require an explicit stable orb version')
# Validate the entire batch before writing any repository.
plans = [plan(path, version) for path in checkouts]
if len({item['checkout'] for item in plans}) != len(plans):
+2 -2
View File
@@ -100,8 +100,8 @@ def test_filters(filters):
def infer_manifest(config, version, default_branch):
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise ValueError("A minor track, exact patch or explicit development orb version is required")
if not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise ValueError("An explicit published orb version is required")
if default_branch not in ("main", "develop"):
raise ValueError("Unsupported default branch")
if set(config) - {"version", "orbs", "commands", "workflows"}: