Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2106955040
|
||
|
|
02f0b93e41 | ||
|
|
5e433ec629 |
No files matched your search
@@ -4,6 +4,8 @@ CircleCI orb for QQQ framework projects.
|
||||
|
||||
For centralized pipeline ownership, see [standardized manifest-driven pipelines](docs/GENERATED-PIPELINES.md).
|
||||
The initial QBit profile uses a small `.qqq-ci.yml` and a setup entry point; existing direct job callers remain supported.
|
||||
For centrally managed patch adoption and exact-version rollback, see
|
||||
[managed patch updates](docs/managed-patch-updates.md).
|
||||
|
||||
**For:** Teams using CircleCI to build, test, and publish QQQ applications
|
||||
**Status:** Stable (v0.6.x)
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# Managed patch updates
|
||||
|
||||
The canonical caller and manifest accept either an exact stable version such as
|
||||
`0.8.0` or a quoted minor track such as `"0.8"`. CircleCI resolves a minor track
|
||||
to its latest published patch. A major-only version, `latest`, `volatile`, and
|
||||
wildcards are rejected. The caller import, setup parameter, and manifest must
|
||||
still agree exactly.
|
||||
|
||||
Use a minor track after qualifying that release line. Future patch fixes then
|
||||
arrive on the repository's next pipeline without a version-bump PR or an extra
|
||||
fleet-wide build. Existing exact pins remain supported and are not changed by
|
||||
publishing this capability. YAML minor versions must be quoted to remain strings.
|
||||
|
||||
The central batch tool applies the same update to all supplied checkouts:
|
||||
|
||||
```sh
|
||||
python3 tools/rollout_version.py --orb-version 0.8 --write /path/to/repo1 /path/to/repo2
|
||||
```
|
||||
|
||||
This changes only the two canonical CI declarations. Publication policies,
|
||||
contexts, branch/tag filters, and profile options stay unchanged. Normal source
|
||||
and publication checks still run when a pipeline is triggered; this setting does
|
||||
not skip tests. GitHub reusable-worker commit pins are separate and unchanged.
|
||||
|
||||
Automatic patch adoption trades exact-version reproducibility for centrally
|
||||
managed fixes. Keep patch releases backward compatible, qualify central changes
|
||||
before production publication, and use an exact stable pin for a canary or
|
||||
rollback. For example, the same tool can move a consumer from `0.8` back to
|
||||
`0.8.0`. Historical pipeline reruns can resolve a newer patch; choose an exact
|
||||
pin when reproducing an earlier run. Setup and continuation each resolve their
|
||||
orb imports, so patch compatibility also matters when publication overlaps a run.
|
||||
|
||||
Changing minor tracks is an explicit migration. Do not use this feature to push
|
||||
breaking changes into a patch release or to bypass repository checks.
|
||||
|
||||
Reference: [CircleCI orb semantic versioning](https://circleci.com/docs/orbs/use/orb-concepts/).
|
||||
|
||||
@@ -89,8 +89,8 @@ def load_manifest(path, setup_version):
|
||||
allowed += ("java_version",)
|
||||
mapping(data, allowed, "manifest")
|
||||
version = data.get("orb_version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ManifestError("orb_version must be an explicit stable or development version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
|
||||
if version != setup_version:
|
||||
raise ManifestError("Manifest orb_version must match the setup job orb_version")
|
||||
default_branch = data.get("default_branch")
|
||||
|
||||
@@ -29,8 +29,21 @@ if [[ -f /etc/apt/sources.list ]]; then
|
||||
sudo sed -i 's|http://security.ubuntu.com/ubuntu|https://archive.ubuntu.com/ubuntu|g' /etc/apt/sources.list
|
||||
fi
|
||||
|
||||
# Add Eclipse Temurin repository
|
||||
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq
|
||||
# Base tools come from Ubuntu, not the unrelated PPAs bundled in machine images.
|
||||
# Support both the 22.04 list and the newer deb822 Ubuntu source file. Preserve
|
||||
# other indexes for later jobs without contacting their repositories here.
|
||||
ubuntu_sources=/etc/apt/sources.list
|
||||
if [[ ! -s "$ubuntu_sources" ]]; then
|
||||
ubuntu_sources=/etc/apt/sources.list.d/ubuntu.sources
|
||||
fi
|
||||
if [[ ! -s "$ubuntu_sources" ]]; then
|
||||
echo 'Cannot locate Ubuntu package sources' >&2
|
||||
exit 1
|
||||
fi
|
||||
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq \
|
||||
-o "Dir::Etc::sourcelist=$ubuntu_sources" \
|
||||
-o Dir::Etc::sourceparts=- \
|
||||
-o APT::Get::List-Cleanup=0
|
||||
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y -qq wget apt-transport-https gpg maven curl
|
||||
|
||||
# Add Temurin GPG key
|
||||
@@ -39,8 +52,13 @@ wget -qO - https://packages.adoptium.net/artifactory/api/gpg/key/public | sudo g
|
||||
# Add repository
|
||||
echo "deb [signed-by=/usr/share/keyrings/adoptium.gpg] https://packages.adoptium.net/artifactory/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/adoptium.list
|
||||
|
||||
# Install Java
|
||||
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq
|
||||
# Refresh only the repository just added. Re-fetching unrelated image PPAs here
|
||||
# caused otherwise successful setup to fail on Launchpad 503 responses. Keep
|
||||
# Ubuntu indexes from the first refresh so Java dependencies remain resolvable.
|
||||
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq \
|
||||
-o Dir::Etc::sourcelist=/etc/apt/sources.list.d/adoptium.list \
|
||||
-o Dir::Etc::sourceparts=- \
|
||||
-o APT::Get::List-Cleanup=0
|
||||
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y -qq "temurin-${JAVA_VERSION}-jdk"
|
||||
|
||||
# Set JAVA_HOME
|
||||
|
||||
@@ -89,8 +89,8 @@ def load_manifest(path, setup_version):
|
||||
allowed += ("java_version",)
|
||||
mapping(data, allowed, "manifest")
|
||||
version = data.get("orb_version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ManifestError("orb_version must be an explicit stable or development version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
|
||||
if version != setup_version:
|
||||
raise ManifestError("Manifest orb_version must match the setup job orb_version")
|
||||
default_branch = data.get("default_branch")
|
||||
|
||||
@@ -109,8 +109,8 @@ def validate(config_raw, manifest_raw, setup_version):
|
||||
if not isinstance(manifest, dict):
|
||||
raise StandardError("Manifest must be a mapping")
|
||||
version = manifest.get("orb_version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise StandardError("Manifest must pin an explicit orb version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise StandardError("Manifest must select a quoted minor track, exact stable pin, or development version")
|
||||
if version != setup_version:
|
||||
raise StandardError("Manifest and running setup orb versions differ")
|
||||
publishing = manifest.get("publish", {})
|
||||
|
||||
@@ -241,6 +241,15 @@ class PipelineTests(unittest.TestCase):
|
||||
self.assertTrue(options["check_middleware_api_versions"])
|
||||
self.assertEqual(options["java_version"], "17")
|
||||
|
||||
def test_minor_track_reaches_continuation_without_changing_jobs(self):
|
||||
expected = self.render()
|
||||
expected['orbs']['qqq-orb'] = 'kingsrook/qqq-orb@0.7'
|
||||
self.write_manifest(orb_version='0.7')
|
||||
self.assertEqual(expected, self.render(QQQ_CI_ORB_VERSION='0.7'))
|
||||
if PACKED:
|
||||
self.assertEqual(expected, self.render(packed=True, QQQ_CI_ORB_VERSION='0.7'))
|
||||
self.render(ok=False, QQQ_CI_ORB_VERSION='0.8')
|
||||
|
||||
def test_branch_matrix(self):
|
||||
for branch, workflow in [("develop", "publish_snapshot"), ("release/1.0", "publish_release_candidate"),
|
||||
("main", "test_only"), ("hotfix/fix", "test_only"),
|
||||
|
||||
@@ -17,6 +17,24 @@ if [[ "$*" == *apt-get* ]]; then
|
||||
[[ "$1" == env && "$2" == DEBIAN_FRONTEND=noninteractive && "$3" == NEEDRESTART_MODE=l ]] || exit 90
|
||||
[[ -f "$APT_CONFIG_CAPTURE" ]] && grep -qx 'Acquire::Retries "3";' "$APT_CONFIG_CAPTURE" || exit 91
|
||||
[[ ${APT_FAIL:-0} == 0 ]] || exit "$APT_FAIL"
|
||||
if [[ "$*" == *'apt-get update'* ]]; then
|
||||
updates=$(grep -c 'apt-get update' "$INSTALL_LOG")
|
||||
if [[ $updates -eq 1 ]]; then
|
||||
# An unavailable image PPA must not participate in base-tool setup.
|
||||
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || exit 100
|
||||
[[ "$*" == *'Dir::Etc::sourcelist=/etc/apt/sources.list'* || "$*" == *'Dir::Etc::sourcelist=/etc/apt/sources.list.d/ubuntu.sources'* ]] || exit 93
|
||||
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 92
|
||||
[[ ${UBUNTU_UPDATE_FAIL:-0} == 0 ]] || exit "$UBUNTU_UPDATE_FAIL"
|
||||
fi
|
||||
if [[ $updates -eq 2 ]]; then
|
||||
# Model a PPA outage after the initial base-package installation. Only
|
||||
# the new signed repository may be contacted; existing indexes survive.
|
||||
[[ "$*" == *'Dir::Etc::sourcelist=/etc/apt/sources.list.d/adoptium.list'* ]] || exit 100
|
||||
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || exit 100
|
||||
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 92
|
||||
[[ ${TEMURIN_UPDATE_FAIL:-0} == 0 ]] || exit "$TEMURIN_UPDATE_FAIL"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
if [[ "$1" == tee && "$2" == /etc/apt/apt.conf.d/80-qqq-download-retries ]]; then
|
||||
cat > "$APT_CONFIG_CAPTURE"; exit 0
|
||||
@@ -42,6 +60,25 @@ rm "$APT_CONFIG_CAPTURE"
|
||||
PARAM_INSTALL_NODE=true bash "$script_dir/install_java.sh" >/dev/null 2>&1
|
||||
[[ $(grep -c 'apt-get' "$INSTALL_LOG") -eq 5 ]]
|
||||
grep -q 'apt-get install -y -qq nodejs' "$INSTALL_LOG"
|
||||
# An outage of the required repository must still stop before Java installation.
|
||||
: > "$INSTALL_LOG"
|
||||
if UBUNTU_UPDATE_FAIL=100 PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1; then
|
||||
echo 'required Ubuntu repository failure accepted' >&2; exit 1
|
||||
else
|
||||
[[ $? -eq 100 ]]
|
||||
fi
|
||||
if grep -q 'apt-get install' "$INSTALL_LOG"; then
|
||||
echo 'package installation continued after Ubuntu repository failure' >&2; exit 1
|
||||
fi
|
||||
: > "$INSTALL_LOG"
|
||||
if TEMURIN_UPDATE_FAIL=100 PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1; then
|
||||
echo 'required Temurin repository failure accepted' >&2; exit 1
|
||||
else
|
||||
[[ $? -eq 100 ]]
|
||||
fi
|
||||
if grep -q 'apt-get install -y -qq temurin-' "$INSTALL_LOG"; then
|
||||
echo 'Java installation continued after repository failure' >&2; exit 1
|
||||
fi
|
||||
# Execute the actual generated selector. Arguments containing spaces stay intact.
|
||||
export SELECTOR_LOG="$scratch/selector.log"
|
||||
cat > "$scratch/bin/mvn" <<'NATIVE'
|
||||
|
||||
@@ -68,6 +68,30 @@ class RolloutTests(unittest.TestCase):
|
||||
self.assertFalse(rollout.rollout([root], '0.7.11', write=True)[0]['changed'])
|
||||
self.assertEqual(applied, self.contents(root))
|
||||
|
||||
def test_managed_patch_track_and_exact_pin_rollback_preserve_every_profile(self):
|
||||
roots = [self.checkout(profile, profile=profile, default_branch='main')
|
||||
for profile in rollout.standard.PROFILE_PARAMETERS]
|
||||
originals = [yaml.safe_load((root / '.qqq-ci.yml').read_text()) for root in roots]
|
||||
rollout.rollout(roots, '0.7', write=True)
|
||||
for root, original in zip(roots, originals):
|
||||
self.assertEqual({**original, 'orb_version': '0.7'},
|
||||
yaml.safe_load((root / '.qqq-ci.yml').read_text()))
|
||||
rollout.standard.validate(*(data.decode() for data in self.contents(root)), '0.7')
|
||||
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.7')
|
||||
with self.assertRaises(ValueError):
|
||||
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.8')
|
||||
self.assertTrue(all(not r['changed'] for r in rollout.rollout(roots, '0.7', write=True)))
|
||||
rollout.rollout(roots, '0.7.4', write=True)
|
||||
for root, original in zip(roots, originals):
|
||||
self.assertEqual(original, yaml.safe_load((root / '.qqq-ci.yml').read_text()))
|
||||
|
||||
def test_unquoted_track_is_rejected_as_numeric_yaml(self):
|
||||
root = self.checkout('numeric')
|
||||
manifest = root / '.qqq-ci.yml'
|
||||
manifest.write_text(manifest.read_text().replace('0.7.4', '0.7'))
|
||||
with self.assertRaises(ValueError):
|
||||
rollout.pipeline.load_manifest(manifest, '0.7')
|
||||
|
||||
def test_one_custom_caller_rejects_entire_batch_before_any_write(self):
|
||||
good = self.checkout('good')
|
||||
bad = self.checkout('bad')
|
||||
@@ -97,7 +121,7 @@ class RolloutTests(unittest.TestCase):
|
||||
def test_invalid_target_and_duplicate_checkout_are_rejected(self):
|
||||
root = self.checkout('qbit')
|
||||
before = self.contents(root)
|
||||
for version in ['latest', 'dev:test', '0.7', '0.7.11\n']:
|
||||
for version in ['latest', 'volatile', 'dev:test', '0', '0.7.*', '0.7.11\n']:
|
||||
with self.assertRaises(ValueError):
|
||||
rollout.rollout([root], version, write=True)
|
||||
with self.assertRaises(ValueError):
|
||||
|
||||
@@ -56,8 +56,8 @@ def plan(checkout, version):
|
||||
|
||||
|
||||
def rollout(checkouts, version, write=False):
|
||||
if not re.fullmatch(r'\d+\.\d+\.\d+', version):
|
||||
raise ValueError('Fleet upgrades require an explicit stable orb version')
|
||||
if not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', version):
|
||||
raise ValueError('Fleet upgrades require a stable minor track or exact patch version')
|
||||
# Validate the entire batch before writing any repository.
|
||||
plans = [plan(path, version) for path in checkouts]
|
||||
if len({item['checkout'] for item in plans}) != len(plans):
|
||||
|
||||
Reference in new issue
Block a user