Compare commits

...
Author SHA1 Message Date
james.maesandCodex 3afce45e69 ci: submit enrollment with GitHub-signed API commits
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 20:15:30 -05:00
james.maes a92d6d13bb Merge pull request #102 from QRun-IO/feat/preserve-restricted-branch-access
feat: enforce CI without changing branch access
2026-10-21 19:55:58 -05:00
james.maesandCodex 9772090684 feat: enforce CI without changing branch access
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 19:52:42 -05:00
james.maes ab666bb785 Merge pull request #101 from QRun-IO/feat/verify-existing-rulesets
feat: verify existing orb gate rulesets
2026-10-21 19:50:26 -05:00
james.maesandCodex fc39293b9d feat: verify existing orb gate rulesets
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 19:47:45 -05:00
james.maes e83b8481b9 Merge pull request #100 from QRun-IO/feat/finish-consumer-enrollment
feat: qualify and finish shared enrollment
2026-10-21 19:45:49 -05:00
james.maesandCodex e4ecae6294 feat: qualify and finish shared enrollment
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 19:40:48 -05:00
james.maes 37b91db07b Merge pull request #99 from QRun-IO/feat/discover-enrollment-targets
feat: discover current consumer enrollment targets
2026-10-21 19:34:22 -05:00
james.maesandCodex c30d821f7e feat: discover current consumer enrollment targets
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 19:30:56 -05:00
james.maes 6ae688dcf4 Merge pull request #98 from QRun-IO/feat/remote-consumer-enrollment
feat: reconcile signed remote enrollment submissions
2026-10-21 19:30:20 -05:00
james.maesandCodex 78207c16be feat: reconcile signed remote enrollment submissions
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 19:25:53 -05:00
james.maes efd50aee2f Merge pull request #97 from QRun-IO/feat/batch-consumer-enrollment
feat: enroll consumers with one validated batch
2026-10-21 19:19:12 -05:00
12 changed files with 1024 additions and 43 deletions

No files matched your search

+3
View File
@@ -58,6 +58,9 @@ jobs:
python3 tests/test_rollout_version.py
python3 tests/test_enforce_orb_checks.py
python3 tests/test_enroll_consumers.py
python3 tests/test_reconcile_enrollment.py
python3 tests/test_discover_enrollment.py
python3 tests/test_finish_enrollment.py
python3 tests/test_static_analysis_exit.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
+3
View File
@@ -63,6 +63,9 @@ test-pipelines: pack
python3 tests/test_rollout_version.py
python3 tests/test_enforce_orb_checks.py
python3 tests/test_enroll_consumers.py
python3 tests/test_reconcile_enrollment.py
python3 tests/test_discover_enrollment.py
python3 tests/test_finish_enrollment.py
python3 tests/test_static_analysis_exit.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
+19 -2
View File
@@ -22,9 +22,9 @@ python tools/enforce_orb_checks.py --targets targets.json --orb-version 0.8 --ou
Every branch must already contain the exact canonical caller for its manifest and
have successful `setup` and `verify` checks from CircleCI app 18001 at its current
commit. Supported profiles are QBit, Core, frontend-core and frontend-material.
commit. Supported profiles are QBit, Core, frontend-core, frontend-material and repository-policy.
The complete batch is inspected before any mutation. Source/head/protection drift,
unavailable APIs, unknown profiles, and ruleset-managed branches fail closed.
unavailable APIs and unknown profiles fail closed. Existing repository rulesets are verified as described below.
The tool appends the app-bound checks, enables strict checks and administrator
enforcement, and reads the result back. Existing required checks remain required.
@@ -47,3 +47,20 @@ GitHub API reference: [protected branches](https://docs.github.com/en/rest/branc
```sh
python -m unittest discover -s tests -p test_enforce_orb_checks.py -v
```
## Existing rulesets
When effective branch rules come from repository rulesets, the tool can verify
existing enforcement without adding classic protection. Each contributing ruleset
must be active for branches, have no bypass actors and require both setup and
verify from CircleCI app18001. The current branch must also have successful gate
runs. The tool rechecks the branch, effective rules and full ruleset records before
reporting verified. Rule drift or an unbound/missing gate stops verification.
This path writes nothing. It preserves scanner requirements, conditions and the
existing up-to-date-branch policy, including a false strict setting that avoids
unnecessary rebuilds. Reports identify protection_mode=rulesets and the observed
strict value. Unsupported organization-level rulesets, bypass actors and missing
gates remain unresolved rather than being rewritten or ignored.
For classic protection with an empty push allowlist and admin enforcement disabled, the tool leaves the entire classic policy unchanged. After current-head gates pass, it creates a narrowly scoped active repository ruleset requiring app-bound setup and verify with no bypass actors. This enforces CI even for administrators without changing their existing branch access or locking all actors out. The ruleset retains the fleet freshness policy (strict=false); it does not weaken any existing classic policy. A same-named ruleset causes rejection rather than overwrite or duplication. Readback verifies the effective gates and unchanged classic protection; subsequent runs recognize the existing ruleset without writes.
+128 -37
View File
@@ -1,37 +1,128 @@
# Batch consumer enrollment
Use one target file for canonical consumers of every supported Circle profile and
legacy QBits whose jobs and publication filters the shared converter recognizes:
```json
[
{"checkout": "/work/qbit-example", "default_branch": "main"},
{"checkout": "/work/qqq", "default_branch": "develop"}
]
```
The branch values must come from repository inventory. Run on clean isolated
checkouts of the branches being migrated.
```sh
python tools/enroll_consumers.py targets.json --orb-version 0.8
python tools/enroll_consumers.py targets.json --orb-version 0.8 --write
```
The command validates the entire batch before writing. Existing manifests keep
all publication and profile options; only the managed orb version changes.
Recognized legacy QBits receive the canonical caller and inferred publication
policy. An unknown job, filter, profile option, duplicate target, symlink, or
concurrent declaration edit rejects the batch. Repeating a completed enrollment
makes no changes. Filesystem errors during the write phase can leave a partial
batch; rerun after correcting the error. This is not a filesystem transaction.
Only `.circleci/config.yml` and `.qqq-ci.yml` are changed. The tool does not delete
other workflows, infer publication from repository names, or enroll unknown
custom pipelines. Their behavior must first be represented by a central profile.
After committing and qualifying the exact source, apply the shared
[branch enforcement tool](branch-enforcement.md) to supported gate profiles.
This command provides batch source conversion. Remote discovery, signed PR
submission, merge reconciliation and scheduled enrollment are separate remaining
work; local conversion alone does not prove fleet deployment or release success.
# Batch consumer enrollment
Use one target file for canonical consumers of every supported Circle profile and
legacy QBits whose jobs and publication filters the shared converter recognizes:
```json
[
{"checkout": "/work/qbit-example", "default_branch": "main"},
{"checkout": "/work/qqq", "default_branch": "develop"}
]
```
The branch values must come from repository inventory. Run on clean isolated
checkouts of the branches being migrated.
```sh
python tools/enroll_consumers.py targets.json --orb-version 0.8
python tools/enroll_consumers.py targets.json --orb-version 0.8 --write
```
The command validates the entire batch before writing. Existing manifests keep
all publication and profile options; only the managed orb version changes.
Recognized legacy QBits receive the canonical caller and inferred publication
policy. An unknown job, filter, profile option, duplicate target, symlink, or
concurrent declaration edit rejects the batch. Repeating a completed enrollment
makes no changes. Filesystem errors during the write phase can leave a partial
batch; rerun after correcting the error. This is not a filesystem transaction.
Only `.circleci/config.yml` and `.qqq-ci.yml` are changed. The tool does not delete
other workflows, infer publication from repository names, or enroll unknown
custom pipelines. Their behavior must first be represented by a central profile.
After committing and qualifying the exact source, apply the shared
[branch enforcement tool](branch-enforcement.md) to supported gate profiles.
This command provides batch source conversion. Use the coordinator below for remote signed submissions. Local conversion alone does not prove fleet deployment or release success.
## Remote signed submissions
Supply observed current branches, without cloning each repository yourself:
```json
[
{"repository": "QRun-IO/qbit-example", "branch": "main"},
{"repository": "QRun-IO/qqq", "branch": "develop"}
]
```
```sh
python tools/reconcile_enrollment.py --targets remote-targets.json --orb-version 0.8 --output plan.json
python tools/reconcile_enrollment.py --targets remote-targets.json --orb-version 0.8 --output result.json --apply
```
Use existing `gh` authentication with repository write access. The coordinator
never installs credentials or changes authentication. Every target is inspected
at an immutable head and validated before any remote mutation. No repository
clone, SSH key or local signing key is required. A moved target stops submission;
refresh the plan instead of overwriting it.
An automation branch derives from the source commit and proposed declarations.
The coordinator creates the branch at the observed base and uses GitHub's
`createCommitOnBranch` API with `expectedHeadOid` for an atomic append. GitHub
signs the commit when supported by the authenticated identity; the coordinator
requires remote signature verification, exact parent, changed paths and
declaration bytes before opening a PR. It never force-updates a branch.
An interrupted branch creation, commit or PR creation can be resumed:
an existing branch still at the planned base receives the missing commit;
matching signed branches and PRs are reused. A mismatched branch is rejected.
Closed existing PRs are reported as existing; they are not silently reopened.
Reports include repository, branch, immutable head and submission URL, without
embedding declaration contents or credentials. Keep the report if a later target
fails; earlier valid submissions remain intact.
API permission, signature or commit-conflict failures stop the run. A timeout
can occur after GitHub accepted a mutation, so inspect remote state or repeat
the same invocation; do not create a different automation branch to retry.
See [GitHub commit API documentation](https://docs.github.com/en/graphql/reference/commits#createcommitonbranch).
This handles remote target lists and signed submissions. Organization discovery is described below. Qualification-aware merging and classic branch enforcement are described below; scheduling and ruleset reconciliation remain separate work. A submitted PR is not a completed migration. Unknown profiles and
native workflows require their own central conversion, not deletion by this tool.
## Organization discovery
The coordinator can select observed current branches directly from GitHub:
```sh
python tools/reconcile_enrollment.py --organization QRun-IO --orb-version 0.8 --output organization-plan.json
python tools/reconcile_enrollment.py --organization QRun-IO --orb-version 0.8 --output organization-result.json --apply
```
Repository and branch inventories are paginated. Only existing main/develop
branches are considered; historical release branches are excluded. Archived
repositories and the orb's own bootstrap repository are reported separately.
Each supported branch receives the same immutable preflight as explicit targets.
Unknown workflows, missing entrypoints, unsupported branch layouts and unavailable
APIs stay `unresolved` in the output. They are never counted as conforming or
silently converted. No-CI and native-only repositories may therefore appear as
unresolved here; use the estate auditor for their separate policy classification.
Discovery finishes before any submission. `--apply` processes qualified plans
and retains unresolved entries for follow-up; one unsupported repository does
not prevent independent valid submissions. An explicit `--targets` batch keeps
its stricter behavior: any preflight failure stops the whole batch. By default neither mode merges PRs or changes branch protections. Use the explicit completion flags below.
## Qualification and enforcement
```sh
python tools/reconcile_enrollment.py --targets remote-targets.json --orb-version 0.8 --output result.json --apply --merge-ready --enforce
```
Both flags require `--apply` and are optional. `--merge-ready` revalidates the
signed submission, exact parent/files/content and current base, requires successful
CircleCI-app `setup` and `verify` checks and every advertised latest check/status,
then requests a normal exact-head squash merge. It never uses admin override or
force pushes. The resulting signed merge must have the qualified source tree.
Pending/failed checks and changed bases are reported without merging. Repeated
runs can advance the same PR once checks complete; there is no polling loop that
holds a CI executor open.
`--enforce` independently checks the actual current target branch, which must
pass its own orb gates before shared classic protection enforcement. Existing
protections and other required checks are preserved. Supported gate profiles are
QBit, Core, frontend-core, frontend-material and repository-policy. Other profiles
remain explicitly unsupported for completion even when source enrollment works.
Existing active repository rulesets with app-bound gates and no bypass actors are verified without writes. Unsupported or incomplete rulesets remain unresolved; none are rewritten or layered with classic protection. API/plan restrictions remain visible.
A merged source update and an enforced branch are separate reported outcomes.
Neither proves artifact publication or deployment. Scheduling and ruleset
reconciliation remain outstanding.
+66
View File
@@ -0,0 +1,66 @@
"""Discovery covers all current branches and never turns an unknown into a target."""
from pathlib import Path
import sys
import unittest
from unittest import mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import discover_enrollment as discovery
def entry(name, archived=False):
return {'nameWithOwner': 'QRun-IO/' + name, 'isArchived': archived}
class DiscoveryTests(unittest.TestCase):
def test_paginates_branches_and_ignores_historical_releases(self):
requests = []
def request(path):
requests.append(path)
if path.endswith('page=1'):
return [{'name': 'release/' + str(i)} for i in range(99)] + [{'name': 'main'}]
return [{'name': 'develop'}]
inspect = mock.Mock(side_effect=lambda target, version: {**target, 'version': version})
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
self.assertEqual(['develop', 'main'], [p['branch'] for p in plans])
self.assertEqual(2, len(requests))
self.assertEqual([], findings)
def test_unknown_branch_and_profile_stay_unresolved_while_other_targets_progress(self):
def inspect(target, version):
if target['repository'].endswith('/custom'):
raise ValueError('Unknown orchestration')
return target
def request(path):
return [{'name': 'feature/new'}] if '/new/' in path else [{'name': 'main'}]
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request,
[entry('custom'), entry('new'), entry('good')])
self.assertEqual([{'repository': 'QRun-IO/good', 'branch': 'main'}], plans)
self.assertEqual(2, len(findings))
self.assertTrue(all(f['state'] == 'unresolved' for f in findings))
def test_archived_and_owner_repositories_are_never_modified(self):
inspect, request = mock.Mock(), mock.Mock()
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request,
[entry('old', True), entry('qqq-orb')])
self.assertEqual([], plans)
self.assertEqual(['archived', 'owner-bootstrap'], [f['state'] for f in findings])
inspect.assert_not_called()
request.assert_not_called()
def test_api_errors_and_duplicate_branches_never_become_empty_success(self):
for result in (ValueError('GitHub read unavailable'), [{'name': 'main'}, {'name': 'main'}]):
request = mock.Mock(side_effect=result) if isinstance(result, Exception) else mock.Mock(return_value=result)
inspect = mock.Mock()
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
self.assertEqual([], plans)
self.assertEqual('unresolved', findings[0]['state'])
inspect.assert_not_called()
if __name__ == '__main__':
unittest.main()
+123
View File
@@ -1,9 +1,12 @@
"""Shared enforcement only adds proven gates and preserves unrelated policy."""
import copy
import base64
import importlib.util
from pathlib import Path
import unittest
import yaml
spec = importlib.util.spec_from_file_location('enforcement', Path(__file__).resolve().parents[1] / 'tools/enforce_orb_checks.py')
enforcement = importlib.util.module_from_spec(spec)
spec.loader.exec_module(enforcement)
@@ -87,6 +90,126 @@ class EnforcementTests(unittest.TestCase):
enforcement.enforce(plan, request)
self.assertEqual([], state['writes'])
def ruleset_fixture(self):
parameters = {'strict_required_status_checks_policy': False,
'required_status_checks': [{'context': c['context'], 'integration_id': c['app_id']}
for c in enforcement.DESIRED] + [{'context': 'scanner', 'integration_id': 15368}]}
record = {'id': 123, 'enforcement': 'active', 'target': 'branch', 'source_type': 'Repository',
'source': 'QRun-IO/example', 'bypass_actors': [],
'rules': [{'type': 'required_status_checks', 'parameters': parameters}]}
effective = [{'type': 'required_status_checks', 'ruleset_source_type': 'Repository',
'ruleset_source': 'QRun-IO/example', 'ruleset_id': 123, 'parameters': copy.deepcopy(parameters)}]
return record, effective
def test_ruleset_verification_preserves_freshness_scanner_and_never_writes(self):
record, effective = self.ruleset_fixture()
original = copy.deepcopy(record)
def request(path, method='GET', payload=None):
self.assertEqual('GET', method)
if path.endswith('/rulesets/123'):
return copy.deepcopy(record)
if '/rules/branches/' in path:
return copy.deepcopy(effective)
if path.endswith('/protection'):
return None
if '/check-runs?' in path:
return {'check_runs': runs()}
if '/contents/' in path:
data = ({'profile': 'qbit', 'orb_version': '0.8', 'default_branch': 'develop'}
if '/.qqq-ci.yml?' in path else enforcement.standard.canonical_caller('qbit', '0.8'))
return {'encoding': 'base64', 'type': 'file', 'content': base64.b64encode(yaml.safe_dump(data).encode()).decode()}
if '/branches/' in path:
return {'commit': {'sha': 'a' * 40}}
return {'archived': False}
plan = enforcement.inspect('QRun-IO/example', 'develop', '0.8', request)
self.assertFalse(plan['changed'])
self.assertFalse(plan['required_status_checks']['strict'])
self.assertEqual('rulesets', plan['protection_mode'])
enforcement.enforce(plan, request)
self.assertEqual(original, record)
record['bypass_actors'] = [{'actor_type': 'RepositoryRole', 'actor_id': 5}]
with self.assertRaises(ValueError):
enforcement.enforce(plan, request)
def test_inactive_bypass_unbound_and_missing_ruleset_gates_reject(self):
record, effective = self.ruleset_fixture()
cases = [dict(record, enforcement='evaluate'), dict(record, bypass_actors=[{'actor_id': 1}]),
dict(record, rules=[])]
unbound = copy.deepcopy(record)
unbound['rules'][0]['parameters']['required_status_checks'][1]['integration_id'] = None
cases.append(unbound)
for candidate in cases:
with self.subTest(candidate=candidate), self.assertRaises(ValueError):
enforcement.verified_rulesets('QRun-IO/example', effective, lambda path: candidate)
def test_unsupported_ruleset_sources_are_not_inferred_as_protected(self):
record, effective = self.ruleset_fixture()
effective[0]['ruleset_source_type'] = 'Organization'
with self.assertRaises(ValueError):
enforcement.verified_rulesets('QRun-IO/example', effective, lambda path: record)
def test_empty_push_allowlist_gains_ruleset_without_changing_access(self):
classic = {'restrictions': {'users': [], 'teams': [], 'apps': []},
'enforce_admins': {'enabled': False}, 'required_signatures': {'enabled': True}}
state = {'ruleset': None, 'writes': []}
def request(path, method='GET', payload=None):
if method != 'GET':
self.assertEqual(('POST', 'repos/QRun-IO/example/rulesets'), (method, path))
state['writes'].append(copy.deepcopy(payload))
state['ruleset'] = {**copy.deepcopy(payload), 'id': 123,
'source_type': 'Repository', 'source': 'QRun-IO/example'}
return state['ruleset']
if '/rulesets?' in path:
return []
if path.endswith('/rulesets/123'):
return copy.deepcopy(state['ruleset'])
if '/rules/branches/' in path:
if state['ruleset'] is None:
return []
return [{**state['ruleset']['rules'][0], 'ruleset_source_type': 'Repository',
'ruleset_source': 'QRun-IO/example', 'ruleset_id': 123}]
if path.endswith('/protection'):
return copy.deepcopy(classic)
if '/check-runs?' in path:
return {'check_runs': runs()}
if '/contents/' in path:
data = ({'profile': 'core', 'orb_version': '0.8', 'default_branch': 'develop'}
if '/.qqq-ci.yml?' in path else enforcement.standard.canonical_caller('core', '0.8'))
return {'encoding': 'base64', 'type': 'file', 'content': base64.b64encode(yaml.safe_dump(data).encode()).decode()}
if '/branches/' in path:
return {'commit': {'sha': 'a' * 40}}
return {'archived': False}
plan = enforcement.inspect('QRun-IO/example', 'main', '0.8', request)
self.assertEqual('create-ruleset', plan['protection_mode'])
enforcement.enforce(plan, request)
self.assertEqual(1, len(state['writes']))
self.assertEqual([], state['writes'][0]['bypass_actors'])
self.assertEqual({'include': ['refs/heads/main'], 'exclude': []}, state['writes'][0]['conditions']['ref_name'])
self.assertFalse(plan['before']['enforce_admins']['enabled'])
self.assertEqual(classic, plan['before'])
repeat = enforcement.inspect('QRun-IO/example', 'main', '0.8', request)
self.assertFalse(repeat['changed'])
enforcement.enforce(repeat, request)
self.assertEqual(1, len(state['writes']))
def test_existing_named_ruleset_is_not_overwritten_or_duplicated(self):
before = {'restrictions': {'users': [], 'teams': [], 'apps': []}, 'enforce_admins': {'enabled': False}}
plan, base_request, state = self.fixture(before)
plan['protection_mode'] = 'create-ruleset'
def request(path, method='GET', payload=None):
if '/rulesets?' in path:
return [{'name': 'QQQ automated verification: develop'}]
return base_request(path, method, payload)
with self.assertRaisesRegex(ValueError, 'already exists'):
enforcement.enforce(plan, request)
self.assertEqual([], state['writes'])
if __name__ == '__main__':
unittest.main()
+92
View File
@@ -0,0 +1,92 @@
"""Automatic completion requires qualified source and preserves normal merge policy."""
import json
from pathlib import Path
import sys
import unittest
from unittest import mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import finish_enrollment as finish
class FinishTests(unittest.TestCase):
def setUp(self):
self.plan = {'repository': 'QRun-IO/example', 'branch': 'main', 'head': 'a' * 40,
'profile': 'qbit', 'version': '0.8'}
self.result = {'state': 'existing-pr', 'head': 'b' * 40, 'url': 'https://github.com/QRun-IO/example/pull/1'}
self.pr = {'number': 1, 'state': 'OPEN', 'headRefOid': 'b' * 40,
'baseRefOid': 'a' * 40, 'baseRefName': 'main', 'mergeCommit': None}
def checks(self):
return [{'id': i, 'name': name, 'app': {'id': 18001}, 'status': 'completed', 'conclusion': 'success'}
for i, name in enumerate(('setup', 'verify'))]
def test_all_checks_and_statuses_must_pass_and_gates_must_use_circle_app(self):
checks = self.checks()
checks[1]['app']['id'] = 123
checks.append({'id': 3, 'name': 'scanner', 'app': {'id': 15368}, 'status': 'completed', 'conclusion': 'failure'})
with mock.patch.object(finish.remote, 'api', side_effect=[{'check_runs': checks},
{'statuses': [{'context': 'legacy-test', 'state': 'pending'}]}]):
self.assertEqual(['legacy-test', 'scanner', 'verify'], finish.qualification('QRun-IO/example', 'b' * 40))
def test_failed_or_missing_checks_never_merge(self):
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', return_value=json.dumps(self.pr)) as command, mock.patch.object(finish.remote, 'api', return_value={'commit': {'sha': self.plan['head']}}), mock.patch.object(finish, 'qualification', return_value=['verify']):
self.assertEqual('waiting-checks', finish.merge_ready(self.plan, self.result)['state'])
self.assertEqual(1, command.call_count)
self.assertEqual(['gh', 'pr', 'view'], command.call_args.args[0][:3])
def test_stale_base_never_merges_even_when_checks_were_green(self):
self.pr['baseRefOid'] = 'c' * 40
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', return_value=json.dumps(self.pr)) as command:
self.assertEqual('stale-base', finish.merge_ready(self.plan, self.result)['state'])
self.assertEqual(1, command.call_count)
def test_success_uses_exact_head_normal_merge_and_verifies_result(self):
calls = []
def command(args):
calls.append(args)
if args[2] == 'merge':
return ''
if len(calls) == 1:
return json.dumps(self.pr)
return json.dumps({'state': 'MERGED', 'mergeCommit': {'oid': 'd' * 40}})
def api(path):
if '/branches/' in path:
return {'commit': {'sha': self.plan['head']}}
return {'commit': {'tree': {'sha': 'e' * 40}, 'verification': {'verified': True}}}
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', side_effect=command), mock.patch.object(finish.remote, 'api', side_effect=api), mock.patch.object(finish, 'qualification', return_value=[]):
result = finish.merge_ready(self.plan, self.result)
self.assertEqual('merged', result['state'])
self.assertEqual(['gh', 'pr', 'merge', '1', '--repo', 'QRun-IO/example', '--squash', '--match-head-commit', 'b' * 40], calls[1])
def test_enforcement_waits_for_current_head_and_uses_shared_preserving_writer(self):
with mock.patch.object(finish.enforcement, 'inspect', side_effect=ValueError('Current branch has no successful verify')), mock.patch.object(finish.enforcement, 'enforce') as enforce:
self.assertEqual('waiting-enforcement', finish.enforce_current(self.plan)['state'])
enforce.assert_not_called()
current = {'head': 'd' * 40, 'changed': True}
with mock.patch.object(finish.enforcement, 'inspect', return_value=current), mock.patch.object(finish.enforcement, 'enforce') as enforce:
self.assertEqual('enforced', finish.enforce_current(self.plan)['state'])
enforce.assert_called_once_with(current)
def test_unknown_gate_profile_never_merges_or_enforces(self):
self.plan['profile'] = 'all'
with mock.patch.object(finish.remote, 'command') as command, mock.patch.object(finish.enforcement, 'inspect') as inspect:
self.assertEqual('unsupported-gates', finish.merge_ready(self.plan, self.result)['state'])
self.assertEqual('unsupported-gates', finish.enforce_current(self.plan)['state'])
command.assert_not_called()
inspect.assert_not_called()
def test_merged_tree_mismatch_is_not_reported_as_success(self):
self.pr.update(state='MERGED', mergeCommit={'oid': 'd' * 40})
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', return_value=json.dumps(self.pr)), mock.patch.object(finish.remote, 'api', side_effect=[
{'commit': {'tree': {'sha': 'e' * 40}}},
{'commit': {'tree': {'sha': 'f' * 40}, 'verification': {'verified': True}}}]):
with self.assertRaises(ValueError):
finish.merge_ready(self.plan, self.result)
if __name__ == '__main__':
unittest.main()
+185
View File
@@ -0,0 +1,185 @@
"""Remote enrollment keeps immutable source checks and resumes signed submissions."""
import base64
from pathlib import Path
import sys
import unittest
from unittest import mock
import yaml
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import reconcile_enrollment as remote
class ReconcileTests(unittest.TestCase):
def setUp(self):
self.head = 'a' * 40
self.new = 'b' * 40
self.repo = 'QRun-IO/example'
self.target = {'repository': self.repo, 'branch': 'main'}
self.manifest = {'profile': 'qbit', 'orb_version': '0.7.4', 'default_branch': 'main',
'publish': {'snapshot': False, 'release_candidate': False, 'release': False}}
self.sources = {
'.qqq-ci.yml': yaml.safe_dump(self.manifest).encode(),
'.circleci/config.yml': yaml.safe_dump(remote.enroll_consumers.rollout.standard.canonical_caller('qbit', '0.7.4')).encode()}
def request(self, path, missing=False):
if path == 'repos/' + self.repo:
return {'archived': False, 'default_branch': 'main'}
if path.endswith('/branches/main'):
return {'commit': {'sha': self.head}}
if '/contents/' in path:
self.assertTrue(path.endswith('?ref=' + self.head))
name = path.split('/contents/')[1].split('?')[0]
raw = self.sources.get(name)
return None if raw is None else {'type': 'file', 'encoding': 'base64', 'content': base64.b64encode(raw).decode()}
raise AssertionError(path)
def plan(self):
with mock.patch.object(remote, 'api', side_effect=self.request):
return remote.inspect(self.target, '0.8')
def test_immutable_plan_is_deterministic_and_preserves_policy(self):
first, second = self.plan(), self.plan()
self.assertEqual(first['automation_branch'], second['automation_branch'])
self.assertEqual(self.sources, first['originals'])
self.assertEqual({**self.manifest, 'orb_version': '0.8'},
yaml.safe_load(first['replacements']['.qqq-ci.yml']))
self.assertTrue(first['changed'])
def test_missing_manifest_uses_legacy_converter(self):
self.sources.pop('.qqq-ci.yml')
self.sources['.circleci/config.yml'] = yaml.safe_dump({'version': 2.1,
'orbs': {'qqq-orb': 'kingsrook/qqq-orb@0.5.0'},
'workflows': {'verify': {'jobs': [{'qqq-orb/mvn_test_only': {}}]}}}).encode()
plan = self.plan()
self.assertIsNone(plan['originals']['.qqq-ci.yml'])
self.assertFalse(any(yaml.safe_load(plan['replacements']['.qqq-ci.yml'])['publish'].values()))
def submission_api(self, plan, path, missing=False):
if '/branches/main' in path:
return {'commit': {'sha': self.head}}
if '/git/ref/' in path:
return {'object': {'sha': self.new}}
if '/commits/' in path:
return {'commit': {'verification': {'verified': True}},
'parents': [{'sha': self.head}], 'files': [{'filename': p} for p in remote.PATHS]}
if '/contents/' in path:
self.assertTrue(path.endswith('?ref=' + self.new))
name = path.split('/contents/')[1].split('?')[0]
return {'type': 'file', 'encoding': 'base64', 'content': base64.b64encode(plan['replacements'][name]).decode()}
if '/pulls?' in path:
return [{'head': {'sha': self.new}, 'html_url': 'https://github.com/QRun-IO/example/pull/1'}]
raise AssertionError(path)
def test_matching_existing_submission_uses_no_git_or_pr_mutations(self):
plan = self.plan()
with mock.patch.object(remote, 'api', side_effect=lambda *a, **k: self.submission_api(plan, *a, **k)), mock.patch.object(remote, 'command') as command:
result = remote.apply(plan)
self.assertEqual('existing-pr', result['state'])
command.assert_not_called()
def test_unsigned_existing_branch_is_rejected_without_mutation(self):
plan = self.plan()
def request(path, missing=False):
value = self.submission_api(plan, path, missing)
if '/commits/' in path:
value['commit']['verification']['verified'] = False
return value
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'command') as command:
with self.assertRaises(ValueError):
remote.apply(plan)
command.assert_not_called()
def test_source_drift_prevents_submission(self):
plan = self.plan()
with mock.patch.object(remote, 'api', return_value={'commit': {'sha': 'c' * 40}}), mock.patch.object(remote, 'command') as command:
with self.assertRaises(ValueError):
remote.apply(plan)
command.assert_not_called()
def test_interrupted_push_creates_only_missing_pr(self):
plan = self.plan()
seen = []
def request(path, missing=False):
return [] if '/pulls?' in path else self.submission_api(plan, path, missing)
def command(args, cwd=None):
seen.append(args[:3])
self.assertEqual(['gh', 'pr', 'create'], args[:3])
self.assertIn(self.head, Path(args[args.index('--body-file') + 1]).read_text())
return 'https://github.com/QRun-IO/example/pull/2'
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'command', side_effect=command):
result = remote.apply(plan)
self.assertEqual('submitted', result['state'])
self.assertEqual([['gh', 'pr', 'create']], seen)
def test_new_api_submission_is_atomic_and_exact(self):
plan = self.plan()
mutations = []
def request(path, missing=False):
if '/git/ref/' in path:
return None
return self.submission_api(plan, path, missing)
def mutate(path, payload):
mutations.append((path, payload))
return {'data': {'createCommitOnBranch': {'commit': {'oid': self.new}}}}
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'mutation', side_effect=mutate), mock.patch.object(remote, 'command') as command:
self.assertEqual('existing-pr', remote.apply(plan)['state'])
command.assert_not_called()
self.assertEqual(('repos/' + self.repo + '/git/refs', {'ref': 'refs/heads/' + plan['automation_branch'], 'sha': self.head}), mutations[0])
commit = mutations[1][1]['variables']['input']
self.assertEqual(self.head, commit['expectedHeadOid'])
self.assertEqual({'repositoryNameWithOwner': self.repo, 'branchName': plan['automation_branch']}, commit['branch'])
self.assertEqual(plan['replacements'], {item['path']: base64.b64decode(item['contents']) for item in commit['fileChanges']['additions']})
def test_base_only_branch_resumes_without_creating_another_ref(self):
plan = self.plan()
result = {'data': {'createCommitOnBranch': {'commit': {'oid': self.new}}}}
with mock.patch.object(remote, 'api', return_value={'commit': {'sha': self.head}}), mock.patch.object(remote, 'mutation', return_value=result) as mutation:
self.assertEqual(self.new, remote.signed_api_commit(plan, {'object': {'sha': self.head}}))
self.assertEqual(1, mutation.call_count)
self.assertEqual('graphql', mutation.call_args.args[0])
def test_existing_completed_branch_is_not_mutated(self):
with mock.patch.object(remote, 'mutation') as mutation:
self.assertEqual(self.new, remote.signed_api_commit(self.plan(), {'object': {'sha': self.new}}))
mutation.assert_not_called()
def test_target_drift_prevents_commit_on_base_only_branch(self):
plan = self.plan()
with mock.patch.object(remote, 'api', return_value={'commit': {'sha': 'c' * 40}}), mock.patch.object(remote, 'mutation') as mutation:
with self.assertRaises(ValueError):
remote.signed_api_commit(plan, {'object': {'sha': self.head}})
mutation.assert_not_called()
def test_graphql_errors_fail_closed(self):
response = mock.Mock(returncode=0, stdout='{"errors": [{"message": "conflict"}]}')
with mock.patch.object(remote.subprocess, 'run', return_value=response):
with self.assertRaises(ValueError):
remote.mutation('graphql', {})
def test_unsigned_api_commit_cannot_create_pr(self):
plan = self.plan()
def request(path, missing=False):
if '/git/ref/' in path:
return {'object': {'sha': self.head}}
value = self.submission_api(plan, path, missing)
if '/commits/' in path:
value['commit']['verification']['verified'] = False
return value
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'mutation', return_value={'data': {'createCommitOnBranch': {'commit': {'oid': self.new}}}}), mock.patch.object(remote, 'command') as command:
with self.assertRaises(ValueError):
remote.apply(plan)
command.assert_not_called()
if __name__ == '__main__':
unittest.main()
+42
View File
@@ -0,0 +1,42 @@
"""Discover current enrollment targets without assuming unknown repositories conform."""
import audit_estate
def discover(organization, version, inspect, request, inventory=None):
"""Inspect every active main/develop branch; retain unsupported results explicitly."""
entries = audit_estate.discover_inventory(organization) if inventory is None else inventory
plans, findings = [], []
for entry in entries:
repo = entry['nameWithOwner']
if entry['isArchived']:
findings.append({'repository': repo, 'state': 'archived'})
continue
if repo.lower() == (organization + '/qqq-orb').lower():
findings.append({'repository': repo, 'state': 'owner-bootstrap',
'reason': 'Orb publisher is not its own consumer'})
continue
try:
branches, page = [], 1
while True:
batch = request(f'repos/{repo}/branches?per_page=100&page={page}')
if not isinstance(batch, list) or any(not isinstance(b, dict) or not isinstance(b.get('name'), str) for b in batch):
raise ValueError('Invalid branch inventory')
branches.extend(b['name'] for b in batch)
if len(batch) < 100:
break
page += 1
if len(set(branches)) != len(branches):
raise ValueError('Duplicate branch inventory')
selected = sorted(set(branches) & {'main', 'develop'})
if not selected:
raise ValueError('No current main/develop branch; explicit profile decision required')
except (ValueError, OSError) as error:
findings.append({'repository': repo, 'state': 'unresolved', 'reason': str(error)})
continue
for branch in selected:
target = {'repository': repo, 'branch': branch}
try:
plans.append(inspect(target, version))
except (ValueError, OSError) as error:
findings.append({**target, 'state': 'unresolved', 'reason': str(error)})
return plans, findings
+68 -4
View File
@@ -14,7 +14,7 @@ spec = importlib.util.spec_from_file_location('standard', ROOT / 'src/scripts/va
standard = importlib.util.module_from_spec(spec)
spec.loader.exec_module(standard)
DESIRED = [{'context': name, 'app_id': 18001} for name in ('setup', 'verify')]
PROFILES = {'qbit', 'core', 'frontend-core', 'frontend-material'}
PROFILES = {'qbit', 'core', 'frontend-core', 'frontend-material', 'repository-policy'}
def api(path, method='GET', payload=None):
@@ -54,6 +54,31 @@ def required_checks(before, runs):
return {'strict': True, 'checks': checks}
def verified_rulesets(repo, effective, request=api):
"""Recognize active repository rulesets with bound gates and no bypass actors."""
selected = [rule for rule in effective if rule['type'] == 'required_status_checks']
if not selected or any(rule.get('ruleset_source_type') != 'Repository'
or rule.get('ruleset_source') != repo for rule in selected):
raise ValueError('Required rulesets need unsupported source reconciliation')
records = []
for identifier in sorted({rule['ruleset_id'] for rule in selected}):
record = request(f'repos/{repo}/rulesets/{identifier}')
if (record.get('enforcement') != 'active' or record.get('target') != 'branch'
or record.get('source_type') != 'Repository' or record.get('source') != repo
or record.get('bypass_actors') != []):
raise ValueError('Ruleset must be active with no bypass actors')
records.append(record)
requirements = [rule['parameters'] for record in records for rule in record['rules']
if rule['type'] == 'required_status_checks']
for desired in DESIRED:
if not any(check['context'] == desired['context'] and check.get('integration_id') == desired['app_id']
for parameters in requirements for check in parameters['required_status_checks']):
raise ValueError('Ruleset is missing app-bound gate: ' + desired['context'])
# Retain the existing freshness policy: recognition does not add rebuilds or
# weaken any existing scanner, merge, or bypass requirement.
return records, any(parameters.get('strict_required_status_checks_policy', False) for parameters in requirements)
def inspect(repo, branch, version, request=api):
"""Read immutable canonical source, latest check runs, and current protections."""
if not re.fullmatch(r'[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+', repo) or branch not in ('main', 'develop'):
@@ -74,9 +99,12 @@ def inspect(repo, branch, version, request=api):
if standard.load_yaml(source[1])['profile'] not in PROFILES:
raise ValueError('Profile does not use the shared setup/verify gate pair')
rules = request(root + '/rules/branches/' + branch)
if rules:
raise ValueError('Ruleset-managed branch requires ruleset reconciliation')
rulesets, ruleset_strict = verified_rulesets(repo, rules, request) if rules else ([], False)
before = request(prefix + '/protection')
restrictions = (before or {}).get('restrictions')
use_ruleset = (not rulesets and restrictions is not None
and not any(restrictions.get(kind) for kind in ('users', 'teams', 'apps'))
and not (before or {}).get('enforce_admins', {}).get('enabled'))
runs = []
page = 1
while True:
@@ -86,6 +114,15 @@ def inspect(repo, branch, version, request=api):
break
page += 1
desired = required_checks(before, runs)
if use_ruleset:
return {'repository': repo, 'branch': branch, 'head': head, 'before': before,
'required_status_checks': {'strict': False, 'checks': DESIRED},
'changed': True, 'protection_mode': 'create-ruleset'}
if rulesets:
desired['strict'] = ruleset_strict or bool(((before or {}).get('required_status_checks') or {}).get('strict'))
return {'repository': repo, 'branch': branch, 'head': head, 'before': before,
'required_status_checks': desired, 'changed': False, 'protection_mode': 'rulesets',
'effective_rules': rules, 'rulesets': rulesets}
current = (before or {}).get('required_status_checks') or {}
ready = (current.get('strict') is True and current.get('checks') == desired['checks']
and (before or {}).get('enforce_admins', {}).get('enabled') is True)
@@ -99,8 +136,35 @@ def enforce(plan, request=api):
prefix = root + '/branches/' + plan['branch']
if request(prefix)['commit']['sha'] != plan['head'] or request(prefix + '/protection') != plan['before']:
raise ValueError('Branch or protection changed after preflight')
if request(root + '/rules/branches/' + plan['branch']):
rules = request(root + '/rules/branches/' + plan['branch'])
if plan.get('protection_mode') == 'rulesets':
if rules != plan['effective_rules'] or verified_rulesets(plan['repository'], rules, request)[0] != plan['rulesets']:
raise ValueError('Ruleset changed after preflight')
return
if rules:
raise ValueError('Ruleset changed after preflight')
if plan.get('protection_mode') == 'create-ruleset':
name = 'QQQ automated verification: ' + plan['branch']
page = 1
while True:
existing = request(root + '/rulesets?per_page=100&page=' + str(page))
if any(record['name'] == name for record in existing):
raise ValueError('Named ruleset already exists; reconcile its scope explicitly')
if len(existing) < 100:
break
page += 1
request(root + '/rulesets', 'POST', {
'name': name, 'target': 'branch', 'enforcement': 'active', 'bypass_actors': [],
'conditions': {'ref_name': {'include': ['refs/heads/' + plan['branch']], 'exclude': []}},
'rules': [{'type': 'required_status_checks', 'parameters': {
'strict_required_status_checks_policy': False, 'do_not_enforce_on_create': False,
'required_status_checks': [{'context': gate['context'], 'integration_id': gate['app_id']}
for gate in DESIRED]}}]})
effective = request(root + '/rules/branches/' + plan['branch'])
verified_rulesets(plan['repository'], effective, request)
if request(prefix + '/protection') != plan['before']:
raise ValueError('Classic access policy changed during ruleset creation')
return
if not plan['changed']:
return
before = plan['before']
+78
View File
@@ -0,0 +1,78 @@
"""Qualify signed enrollment PRs and enforce only successful current branch heads."""
import json
import enforce_orb_checks as enforcement
import reconcile_enrollment as remote
def qualification(repo, head):
"""Require both app-bound orb gates and every advertised latest check/status."""
checks, page = [], 1
while True:
batch = remote.api(f'repos/{repo}/commits/{head}/check-runs?filter=latest&per_page=100&page={page}')['check_runs']
checks.extend(batch)
if len(batch) < 100:
break
page += 1
reasons = []
for name in ('setup', 'verify'):
found = [check for check in checks if check['name'] == name and check['app']['id'] == 18001]
if not found or max(found, key=lambda check: check['id'])['conclusion'] != 'success':
reasons.append(name)
for check in checks:
if check['status'] != 'completed' or check['conclusion'] not in ('success', 'neutral', 'skipped'):
reasons.append(check['name'])
page = 1
while True:
batch = remote.api(f'repos/{repo}/commits/{head}/status?per_page=100&page={page}')['statuses']
reasons.extend(status['context'] for status in batch if status['state'] != 'success')
if len(batch) < 100:
break
page += 1
return sorted(set(reasons))
def merge_ready(plan, result):
"""Normal exact-head squash merge only; no administrative or force bypass."""
if plan['profile'] not in enforcement.PROFILES:
return {'state': 'unsupported-gates', 'profile': plan['profile']}
if result['state'] == 'unchanged':
return {'state': 'unchanged'}
repo, head = plan['repository'], result['head']
remote.verify_submission(plan, head)
pr = json.loads(remote.command(['gh', 'pr', 'view', result['url'], '--repo', repo,
'--json', 'number,state,headRefOid,baseRefOid,baseRefName,mergeCommit']))
if pr['headRefOid'] != head or pr['baseRefName'] != plan['branch']:
raise ValueError('PR differs from enrollment target')
if pr['state'] == 'CLOSED':
return {'state': 'closed', 'url': result['url']}
if pr['state'] != 'MERGED':
if pr['baseRefOid'] != plan['head'] or remote.api(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != plan['head']:
return {'state': 'stale-base', 'url': result['url']}
pending = qualification(repo, head)
if pending:
return {'state': 'waiting-checks', 'checks': pending, 'url': result['url']}
remote.command(['gh', 'pr', 'merge', str(pr['number']), '--repo', repo, '--squash', '--match-head-commit', head])
pr = json.loads(remote.command(['gh', 'pr', 'view', result['url'], '--repo', repo,
'--json', 'state,mergeCommit']))
if pr['state'] != 'MERGED':
raise ValueError('PR merge was not confirmed')
merged = pr['mergeCommit']['oid']
expected = remote.api(f'repos/{repo}/commits/{head}')['commit']['tree']['sha']
actual = remote.api(f'repos/{repo}/commits/{merged}')['commit']
if not actual['verification']['verified'] or actual['tree']['sha'] != expected:
raise ValueError('Merged signature or source tree differs from qualified source')
return {'state': 'merged', 'url': result['url'], 'head': head, 'merge': merged}
def enforce_current(plan):
"""Use shared narrow enforcement after the actual branch head passes its gates."""
if plan['profile'] not in enforcement.PROFILES:
return {'state': 'unsupported-gates', 'profile': plan['profile']}
try:
current = enforcement.inspect(plan['repository'], plan['branch'], plan['version'])
except ValueError as error:
return {'state': 'waiting-enforcement', 'reason': str(error)}
enforcement.enforce(current)
return {'state': 'enforced', 'head': current['head'], 'changed': current['changed'],
'protection_mode': current.get('protection_mode', 'classic')}
+217
View File
@@ -0,0 +1,217 @@
#!/usr/bin/env python3
"""Plan remote enrollment and submit signed, repeatable PRs with --apply."""
import argparse
import base64
import hashlib
import json
import os
from pathlib import Path
import re
import subprocess
import tempfile
import enroll_consumers
PATHS = ('.circleci/config.yml', '.qqq-ci.yml')
def command(args, cwd=None):
"""Use existing credentials and signing configuration without interactive prompts."""
env = dict(os.environ, GIT_TERMINAL_PROMPT='0')
result = subprocess.run(args, cwd=cwd, env=env, capture_output=True,
text=True, encoding='utf-8', timeout=120)
if result.returncode:
raise ValueError('Command failed: ' + args[0])
return result.stdout.strip()
def api(path, missing=False):
"""Only an explicit 404 is treated as absence; other API failures stop work."""
result = subprocess.run(['gh', 'api', path], capture_output=True,
text=True, encoding='utf-8', timeout=60)
try:
data = json.loads(result.stdout)
except ValueError:
raise ValueError('Invalid GitHub response') from None
if result.returncode:
if missing and str(data.get('status')) == '404':
return None
raise ValueError('GitHub read unavailable: ' + path.split('?')[0])
return data
def content(repo, path, head):
"""Read a regular declaration at an immutable commit, never at a moving branch."""
item = api(f'repos/{repo}/contents/{path}?ref={head}', missing=True)
if item is None:
return None
if item.get('type') != 'file' or item.get('encoding') != 'base64':
raise ValueError('Declaration is not a regular file')
return base64.b64decode(item['content'])
def mutation(path, payload):
"""Use the existing gh identity; never print credentials or response bodies."""
result = subprocess.run(['gh', 'api', path, '--method', 'POST', '--input', '-'],
input=json.dumps(payload), capture_output=True,
text=True, encoding='utf-8', timeout=60)
if result.returncode:
raise ValueError('GitHub mutation failed; inspect remote state before retrying')
try:
data = json.loads(result.stdout)
except ValueError:
raise ValueError('Invalid GitHub mutation response') from None
if not isinstance(data, dict) or data.get('errors'):
raise ValueError('GitHub mutation did not succeed')
return data
def signed_api_commit(plan, ref):
"""Resume a base-only branch; append atomically without local Git/SSH keys."""
repo, branch, base = plan['repository'], plan['automation_branch'], plan['head']
if ref is None:
mutation(f'repos/{repo}/git/refs', {'ref': 'refs/heads/' + branch, 'sha': base})
elif ref['object']['sha'] != base:
return ref['object']['sha']
if api(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != base:
raise ValueError('Target branch moved before submission')
additions = [{'path': path, 'contents': base64.b64encode(plan['replacements'][path]).decode('ascii')}
for path in PATHS if plan['originals'][path] != plan['replacements'][path]]
result = mutation('graphql', {
'query': 'mutation($input: CreateCommitOnBranchInput!) { createCommitOnBranch(input: $input) { commit { oid } } }',
'variables': {'input': {
'branch': {'repositoryNameWithOwner': repo, 'branchName': branch},
'expectedHeadOid': base,
'message': {'headline': 'ci: enroll in the shared QQQ orb',
'body': 'Co-Authored-By: Codex <noreply@openai.com>'},
'fileChanges': {'additions': additions}}}})
try:
head = result['data']['createCommitOnBranch']['commit']['oid']
except (KeyError, TypeError):
raise ValueError('GitHub commit response is incomplete') from None
if not isinstance(head, str) or not re.fullmatch(r'[0-9a-f]{40}', head):
raise ValueError('GitHub commit response has no valid SHA')
return head
def inspect(target, version):
"""Resolve observed repository identity and reuse the shared enrollment planner."""
if not isinstance(target, dict) or set(target) != {'repository', 'branch'}:
raise ValueError('Each target requires repository and branch')
repo, branch = target['repository'], target['branch']
if not isinstance(repo, str) or not re.fullmatch(r'[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+', repo):
raise ValueError('Invalid repository')
if branch not in ('main', 'develop'):
raise ValueError('Enrollment targets current main/develop only')
info = api('repos/' + repo)
if info['archived']:
raise ValueError('Archived repository')
head = api(f'repos/{repo}/branches/{branch}')['commit']['sha']
originals = {path: content(repo, path, head) for path in PATHS}
if originals[PATHS[0]] is None:
raise ValueError('No recognized CircleCI entrypoint')
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / '.circleci').mkdir()
for path, data in originals.items():
if data is not None:
(root / path).write_bytes(data)
plan = enroll_consumers.plan({'checkout': str(root), 'default_branch': info['default_branch']}, version)
replacements = {str(path.relative_to(root)).replace('\\', '/'): new for path, _, new in plan['files']}
digest = hashlib.sha256(head.encode() + b''.join(replacements[path] for path in PATHS)).hexdigest()[:20]
return {**target, 'head': head, 'originals': originals, 'replacements': replacements,
'profile': plan['profile'], 'version': version,
'changed': any(originals[path] != replacements[path] for path in PATHS),
'automation_branch': f'automation/qqq-enroll-{branch}-{digest}'}
def verify_submission(plan, head):
"""Accept only a signed, one-parent commit with exactly the planned changes."""
repo = plan['repository']
commit = api(f'repos/{repo}/commits/{head}')
expected = {path for path in PATHS if plan['originals'][path] != plan['replacements'][path]}
if (not commit['commit']['verification']['verified']
or [parent['sha'] for parent in commit['parents']] != [plan['head']]
or {item['filename'] for item in commit['files']} != expected):
raise ValueError('Existing automation branch differs from the signed plan')
for path in PATHS:
if content(repo, path, head) != plan['replacements'][path]:
raise ValueError('Submitted declaration differs from plan')
def apply(plan):
"""Never force push; reuse matching submissions after interrupted executions."""
if not plan['changed']:
return {'state': 'unchanged'}
repo, branch = plan['repository'], plan['automation_branch']
if api(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != plan['head']:
raise ValueError('Target branch changed after preflight')
ref = api(f'repos/{repo}/git/ref/heads/{branch}', missing=True)
head = signed_api_commit(plan, ref)
verify_submission(plan, head)
pulls = api(f'repos/{repo}/pulls?state=all&head={repo.split("/")[0]}:{branch}&base={plan["branch"]}')
if pulls:
if len(pulls) != 1 or pulls[0]['head']['sha'] != head:
raise ValueError('Ambiguous existing enrollment PR')
return {'state': 'existing-pr', 'head': head, 'url': pulls[0]['html_url']}
body = ('Uses the shared orb enrollment converter and preserves existing publication policy. '
'Only the canonical caller and manifest change. Exact-head hosted checks must pass '
'before merge; branch enforcement follows successful qualification.\n\n'
'Source base: ' + plan['head'])
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / 'body.md'
path.write_text(body, encoding='utf-8')
url = command(['gh', 'pr', 'create', '--repo', repo, '--base', plan['branch'], '--head', branch,
'--title', 'ci: enroll in the shared QQQ orb', '--body-file', str(path)])
return {'state': 'submitted', 'head': head, 'url': url}
def main():
"""Preflight every explicit target before mutations and save resumable results."""
parser = argparse.ArgumentParser(description=__doc__)
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument('--targets', type=Path)
source.add_argument('--organization')
parser.add_argument('--orb-version', required=True)
parser.add_argument('--apply', action='store_true')
parser.add_argument('--merge-ready', action='store_true')
parser.add_argument('--enforce', action='store_true')
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
if (args.merge_ready or args.enforce) and not args.apply:
parser.error('--merge-ready and --enforce require --apply')
if not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', args.orb_version):
raise ValueError('A stable minor track or exact patch is required')
findings = []
if args.organization:
from discover_enrollment import discover
plans, findings = discover(args.organization, args.orb_version, inspect, api)
else:
targets = json.loads(args.targets.read_text(encoding='utf-8'))
if not isinstance(targets, list) or not targets:
raise ValueError('A nonempty target list is required')
plans = [inspect(target, args.orb_version) for target in targets]
if len({(p['repository'].lower(), p['branch']) for p in plans}) != len(plans):
raise ValueError('Duplicate target')
rows = [{key: value for key, value in p.items() if key not in ('originals', 'replacements')} for p in plans]
report = findings + rows
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
if args.apply:
for plan, row in zip(plans, rows):
row['result'] = apply(plan)
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
if args.merge_ready or args.enforce:
import finish_enrollment
if args.merge_ready:
row['qualification'] = finish_enrollment.merge_ready(plan, row['result'])
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
if args.enforce:
row['enforcement'] = finish_enrollment.enforce_current(plan)
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
print(json.dumps({'targets': len(plans), 'changed': sum(p['changed'] for p in plans),
'unresolved': sum(row['state'] == 'unresolved' for row in findings),
'applied': args.apply}))
if __name__ == '__main__':
main()