feat: add docs-only Java CI and aggregate merge checks
Retain all applicable checks behind one result and use native CircleCI reporting. Co-Authored-By: Codex <noreply@openai.com> Signed-off-by: James Maes <james@kof22.com>
This commit is contained in:
16 files changed
+658
-7
No files matched your search
@@ -15,6 +15,8 @@ tag-only: &tag-only
|
||||
only: /^v[0-9]+\.[0-9]+\.[0-9]+$/
|
||||
|
||||
jobs:
|
||||
ci-ready:
|
||||
type: no-op
|
||||
test:
|
||||
docker:
|
||||
- image: cimg/node:22.12
|
||||
@@ -52,6 +54,14 @@ workflows:
|
||||
- test:
|
||||
filters: *filters
|
||||
|
||||
- ci-ready:
|
||||
requires:
|
||||
- orb-tools/lint
|
||||
- orb-tools/pack
|
||||
- orb-tools/review
|
||||
- test
|
||||
filters: *filters
|
||||
|
||||
- orb-tools/publish:
|
||||
orb_name: kof22/munitor
|
||||
vcs_type: << pipeline.project.type >>
|
||||
|
||||
@@ -52,6 +52,47 @@ Munitor validates branch names and will fail on non-conforming names.
|
||||
| `release/<version>` | Release prep |
|
||||
| `hotfix/<name>` | Production fixes |
|
||||
|
||||
## Optional docs-only Java checks
|
||||
|
||||
The new `java-webapp` docs-only path is opt-in and must be adopted **after** the
|
||||
aggregate required check is published and verified. Do not enable it against an
|
||||
older orb or while individual build jobs remain required:
|
||||
|
||||
```yaml
|
||||
ci:
|
||||
docs_only: true
|
||||
```
|
||||
|
||||
Only changes confined to root `README.md`, `CHANGELOG.md`, `CONTRIBUTING.md`, or
|
||||
Markdown files under `docs/` (including nested directories) qualify. Feature and
|
||||
hotfix branches use their complete diff from the merge base with `main`; `main`
|
||||
uses its first-parent diff. Tags, release and other branches, shallow history,
|
||||
failed comparisons, empty diffs, and any uncertain classification keep the full
|
||||
pipeline. The docs path retains secret scanning and emits the aggregate gate;
|
||||
it does not build or deploy an image.
|
||||
|
||||
Migration order: publish the gate-capable orb, adopt it with docs-only disabled,
|
||||
and verify the real GitHub context `ci/circleci: ci-ready` on a consumer run. Then
|
||||
replace that repository's required-check list with the verified aggregate gate,
|
||||
preserving all other branch rules. A single adoption PR may update both orb pins
|
||||
and enable `ci.docs_only`: those config changes force full CI. Verify the gate
|
||||
and migrate required checks before merging it or opening a docs-only PR. Munitor's
|
||||
own native no-op gate is expected to report `ci/circleci: lint-pack/ci-ready`;
|
||||
verify its actual context before changing its rules. Release and real-context
|
||||
verification are still pending; see [the adoption plan](docs/PLAN-agile-ci.md).
|
||||
|
||||
Use native [Website-Backend Insights](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows),
|
||||
[PR timings](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows/pr-checks),
|
||||
and [Test Insights](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows/pr-checks/tests)
|
||||
to measure results. Compare matching branches/windows and track the `docs-checks`
|
||||
median separately from full builds. The historical local report
|
||||
`C:/Users/james/.cache/ci-insights-report.md` ends October 17 and does not establish
|
||||
the speed of this change. No additional reporting job is needed.
|
||||
|
||||
Native auto-cancel is enabled for Website-Backend, whose default `main` is exempt.
|
||||
It remains disabled for Munitor: its default is `develop`, so `main` release work
|
||||
would not receive that exemption.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **CircleCI project**: Add your repo at https://app.circleci.com
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
# Agile CI adoption
|
||||
|
||||
Implemented locally: an opt-in `java-webapp` docs-only workflow, aggregate consumer
|
||||
gate, and native no-op gate for Munitor's own checks. Publication, real check-context
|
||||
verification, required-check migration and consumer opt-in remain pending.
|
||||
|
||||
## Contract
|
||||
|
||||
- `ci.docs_only: true` permits only root README/CHANGELOG/CONTRIBUTING.md and
|
||||
`docs/**/*.md` (including Markdown directly in docs). Secret scanning stays on.
|
||||
- Feature/hotfix classification uses the complete branch diff from the merge base
|
||||
with main, not the latest commit. Main uses its first parent. Tags, release/other
|
||||
branches, shallow history, empty or uncertain comparisons use full CI.
|
||||
- Docs checks emit the aggregate gate without image build/deploy. Full workflows
|
||||
retain their existing gates. No custom reporting framework or reporting job.
|
||||
|
||||
## Required order
|
||||
|
||||
1. Local make all, 20 BATS, 31 classifier cases, eight source/packed graph cases, and CircleCI config validation passed. Finish hosted CI and publish the gate-capable orb.
|
||||
2. Adopt it in the consumer with docs-only disabled. Observe successful actual
|
||||
GitHub context `ci/circleci: ci-ready`; do not infer the name from YAML alone.
|
||||
3. Replace the consumer required-check list with that verified aggregate context,
|
||||
preserving review, signature, conversation, bypass and all other branch rules.
|
||||
4. Enable `ci.docs_only: true`; prove docs-only and mixed/code changes select the
|
||||
expected paths and retain the required gate.
|
||||
5. Separately verify Munitor's native no-op context (expected
|
||||
`ci/circleci: lint-pack/ci-ready`) before migrating its required checks.
|
||||
|
||||
All five adoption steps are pending; an implemented template is not rollout proof.
|
||||
To avoid duplicate full builds, the pin and opt-in may share one adoption PR:
|
||||
the manifest/config changes force full CI. Verify its gate and migrate required
|
||||
checks before merging or opening a docs-only PR.
|
||||
|
||||
## Measure using existing CircleCI views
|
||||
|
||||
[Project Insights](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows),
|
||||
[PR checks](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows/pr-checks),
|
||||
and [Tests](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows/pr-checks/tests)
|
||||
already provide the dashboard. The historical API baseline was retrieved
|
||||
October 19 but its returned data ends October 17 (PR P50 13m58s, P95 34m37s).
|
||||
It is not a measurement of this implementation. After Insights catches up, compare
|
||||
equivalent windows/branches and report the docs-checks median separately; do not
|
||||
blend docs runs into a claimed full-build improvement or sum parallel job medians.
|
||||
|
||||
Backend native auto-cancel is on; main is its exempt default branch. Munitor stays
|
||||
off because develop is its default and main/release work is not similarly exempt.
|
||||
@@ -1,3 +1,18 @@
|
||||
## Agile CI (2026-10-19)
|
||||
|
||||
Fast Java CI shipped as Munitor 0.3.9 and Website-Backend PR330. The warm
|
||||
consumer PR workflow fell from 37m04 to 20m45; cache restore fell from 5m32 to
|
||||
4.76s (48GiB to 565MiB). Backend PR332 then preserved all 777 test identities
|
||||
while reducing its verified PR workflow to 14m49 and tests to 8m10. Timing-sensitive
|
||||
performance tests run serially after the parallel suite; thresholds are unchanged.
|
||||
|
||||
Current feature/agile-ci adds conservative opt-in documentation-only validation
|
||||
and one aggregate merge result. Backend native auto-cancel is enabled; Munitor
|
||||
remains off because its default branch is develop. Native Insights provides
|
||||
runtime/flaky-test reporting without another service. Local classifier and
|
||||
workflow-graph regressions pass; publication and consumer adoption are pending.
|
||||
See docs/PLAN-agile-ci.md for migration order and measurement limits.
|
||||
|
||||
## Fast Java CI (2026-10-19)
|
||||
|
||||
Active user priority: fast, agile, lightweight CI/CD; backend ticket goal paused.
|
||||
|
||||
+3
-1
@@ -2,7 +2,9 @@
|
||||
|
||||
## Pending
|
||||
|
||||
- [ ] Validate and publish fast Java CI patch, canary actual cache size/timings, then adopt stable orb (docs/PLAN-lean-java-ci.md).
|
||||
- [ ] Publish and adopt aggregate CI result and docs-only opt-in; verify real check names before changing required checks (docs/PLAN-agile-ci.md).
|
||||
|
||||
- [x] Publish fast Java CI as 0.3.9; backend PR330 adopted it and measured the warm workflow at 20m45 (from 37m04).
|
||||
|
||||
- [ ] Validate and release supplemental-image freshness fix, then adopt it in backend PR311 and verify all supplemental Trivy scans.
|
||||
|
||||
|
||||
@@ -38,8 +38,14 @@ echo "Packing generate_config.sh..."
|
||||
echo 'MUNITOR_EXTRACT_EOF'
|
||||
echo ''
|
||||
|
||||
# Embed conservative documentation-only classification.
|
||||
echo "cat > /tmp/munitor/docs_only.sh << 'MUNITOR_DOCS_EOF'"
|
||||
cat "${SRC}/docs_only.sh"
|
||||
echo 'MUNITOR_DOCS_EOF'
|
||||
echo ''
|
||||
|
||||
# Embed each template
|
||||
for tpl in java-webapp gradle-webapp node-api node-webapp terraform sdk-distribution validate-cd-repo argocd-apps obsidian-plugin; do
|
||||
for tpl in java-docs java-webapp gradle-webapp node-api node-webapp terraform sdk-distribution validate-cd-repo argocd-apps obsidian-plugin; do
|
||||
echo "cat > /tmp/munitor/templates/${tpl}.yml.tpl << 'MUNITOR_TPL_EOF'"
|
||||
cat "${SRC}/templates/${tpl}.yml.tpl"
|
||||
echo 'MUNITOR_TPL_EOF'
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
description: >
|
||||
Single merge result. Workflow dependencies enforce the applicable checks.
|
||||
executor: base
|
||||
steps:
|
||||
- run:
|
||||
name: Required checks passed
|
||||
command: echo "All applicable CI checks passed."
|
||||
@@ -0,0 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
# Source this helper. Any uncertainty keeps the full pipeline.
|
||||
munitor_docs_only() (
|
||||
[[ -z "${CIRCLE_TAG:-}" ]] || return 1
|
||||
local base paths path seen=0
|
||||
case "${CIRCLE_BRANCH:-}" in
|
||||
feature/*|hotfix/*)
|
||||
git fetch --no-tags origin '+refs/heads/main:refs/remotes/origin/main' >/dev/null 2>&1 || return 1
|
||||
base=$(git merge-base HEAD refs/remotes/origin/main 2>/dev/null) || return 1
|
||||
;;
|
||||
main)
|
||||
base=$(git rev-parse --verify 'HEAD^1' 2>/dev/null) || return 1
|
||||
;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
[[ "$(git rev-parse --is-shallow-repository 2>/dev/null)" == false ]] || return 1
|
||||
paths=$(mktemp 2>/dev/null) || return 1
|
||||
trap 'rm -f -- "$paths"' EXIT
|
||||
git diff --no-ext-diff --no-renames --name-only -z "$base" HEAD >"$paths" 2>/dev/null || return 1
|
||||
while IFS= read -r -d '' path; do
|
||||
case "$path" in
|
||||
README.md|CHANGELOG.md|CONTRIBUTING.md|docs/*.md) ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
seen=1
|
||||
done <"$paths"
|
||||
[[ "$seen" -eq 1 ]]
|
||||
)
|
||||
@@ -97,7 +97,7 @@ validate_required_fields "${PIPELINE_TYPE}" "${CONFIG_FILE}"
|
||||
# --------------------------------------------------------------------------
|
||||
# Warn about unknown top-level keys (catches typos like java_verion)
|
||||
# --------------------------------------------------------------------------
|
||||
KNOWN_KEYS="pipeline orb_version org image_name server_module java_version node_version sonar docker cd coverage e2e sbom owasp contexts npm node services test terraform sast health kustomize kube_linter_config package_manager yamllint_paths supplemental_images"
|
||||
KNOWN_KEYS="pipeline orb_version org image_name server_module java_version node_version sonar docker cd coverage e2e sbom owasp contexts npm node services test terraform sast health kustomize kube_linter_config package_manager yamllint_paths supplemental_images ci"
|
||||
ACTUAL_KEYS=$(yq 'keys | .[]' "${CONFIG_FILE}" 2>/dev/null || true)
|
||||
for key in ${ACTUAL_KEYS}; do
|
||||
if ! echo "${KNOWN_KEYS}" | grep -qw "${key}"; then
|
||||
@@ -110,6 +110,17 @@ done
|
||||
# --------------------------------------------------------------------------
|
||||
TEMPLATE_FILE="${TEMPLATE_DIR}/${PIPELINE_TYPE}.yml.tpl"
|
||||
|
||||
# Opt in only after the consumer requires ci-ready instead of individual build jobs.
|
||||
# Missing history or any non-documentation path keeps the complete pipeline.
|
||||
if [[ "${PIPELINE_TYPE}" == "java-webapp" && "$(yq '.ci.docs_only // false' "${CONFIG_FILE}")" == "true" ]]; then
|
||||
# shellcheck source=docs_only.sh
|
||||
source "${SCRIPT_DIR}/docs_only.sh"
|
||||
if munitor_docs_only; then
|
||||
TEMPLATE_FILE="${TEMPLATE_DIR}/java-docs.yml.tpl"
|
||||
echo "Documentation-only change: retaining secret scanning; no application build or deployment."
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ ! -f "${TEMPLATE_FILE}" ]]; then
|
||||
echo "ERROR: No template found for pipeline type '${PIPELINE_TYPE}'."
|
||||
echo "Available templates:"
|
||||
|
||||
@@ -347,6 +347,54 @@ get_envsubst_vars() {
|
||||
}
|
||||
MUNITOR_EXTRACT_EOF
|
||||
|
||||
cat > /tmp/munitor/docs_only.sh << 'MUNITOR_DOCS_EOF'
|
||||
#!/usr/bin/env bash
|
||||
# Source this helper. Any uncertainty keeps the full pipeline.
|
||||
munitor_docs_only() (
|
||||
[[ -z "${CIRCLE_TAG:-}" ]] || return 1
|
||||
local base paths path seen=0
|
||||
case "${CIRCLE_BRANCH:-}" in
|
||||
feature/*|hotfix/*)
|
||||
git fetch --no-tags origin '+refs/heads/main:refs/remotes/origin/main' >/dev/null 2>&1 || return 1
|
||||
base=$(git merge-base HEAD refs/remotes/origin/main 2>/dev/null) || return 1
|
||||
;;
|
||||
main)
|
||||
base=$(git rev-parse --verify 'HEAD^1' 2>/dev/null) || return 1
|
||||
;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
[[ "$(git rev-parse --is-shallow-repository 2>/dev/null)" == false ]] || return 1
|
||||
paths=$(mktemp 2>/dev/null) || return 1
|
||||
trap 'rm -f -- "$paths"' EXIT
|
||||
git diff --no-ext-diff --no-renames --name-only -z "$base" HEAD >"$paths" 2>/dev/null || return 1
|
||||
while IFS= read -r -d '' path; do
|
||||
case "$path" in
|
||||
README.md|CHANGELOG.md|CONTRIBUTING.md|docs/*.md) ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
seen=1
|
||||
done <"$paths"
|
||||
[[ "$seen" -eq 1 ]]
|
||||
)
|
||||
MUNITOR_DOCS_EOF
|
||||
|
||||
cat > /tmp/munitor/templates/java-docs.yml.tpl << 'MUNITOR_TPL_EOF'
|
||||
version: 2.1
|
||||
|
||||
orbs:
|
||||
munitor: ${MUNITOR_ORB_SLUG}@${MUNITOR_ORB_VERSION}
|
||||
|
||||
workflows:
|
||||
docs-checks:
|
||||
jobs:
|
||||
- munitor/secrets_scan:
|
||||
name: secrets-scan
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- secrets-scan
|
||||
MUNITOR_TPL_EOF
|
||||
|
||||
cat > /tmp/munitor/templates/java-webapp.yml.tpl << 'MUNITOR_TPL_EOF'
|
||||
version: 2.1
|
||||
|
||||
@@ -427,6 +475,23 @@ workflows:
|
||||
- /hotfix\/.*/
|
||||
##ENDIF_E2E##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- build-and-test
|
||||
- image-scan
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
##IF_E2E##
|
||||
- e2e-tests
|
||||
##ENDIF_E2E##
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
|
||||
##INCLUDE_DEPLOY java-webapp-deploy develop develop ${MUNITOR_CD_ENV_DEVELOP}##
|
||||
##INCLUDE_DEPLOY java-webapp-deploy staging staging ${MUNITOR_CD_ENV_STAGING}##
|
||||
|
||||
@@ -585,6 +650,24 @@ workflows:
|
||||
- /release\/.*/
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_SBOM##
|
||||
- sbom
|
||||
##ENDIF_SBOM##
|
||||
##IF_CD_NON_PROD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD_NON_PROD##
|
||||
##IF_GITHUB_RELEASE##
|
||||
- github-release
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
|
||||
production:
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
@@ -648,6 +731,20 @@ workflows:
|
||||
branches:
|
||||
only: main
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_CD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD##
|
||||
##IF_GITHUB_RELEASE##
|
||||
- github-release
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
filters:
|
||||
branches:
|
||||
only: main
|
||||
MUNITOR_TPL_EOF
|
||||
|
||||
cat > /tmp/munitor/templates/gradle-webapp.yml.tpl << 'MUNITOR_TPL_EOF'
|
||||
@@ -2660,6 +2757,20 @@ cat > /tmp/munitor/templates/partials/java-webapp-deploy.yml.tpl << 'MUNITOR_PAR
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
##ENDIF_CD_NON_PROD##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_SBOM##
|
||||
- sbom
|
||||
##ENDIF_SBOM##
|
||||
##IF_CD_NON_PROD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD_NON_PROD##
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
MUNITOR_PARTIAL_EOF
|
||||
|
||||
cat > /tmp/munitor/templates/partials/gradle-webapp-deploy.yml.tpl << 'MUNITOR_PARTIAL_EOF'
|
||||
@@ -3248,7 +3359,7 @@ validate_required_fields "${PIPELINE_TYPE}" "${CONFIG_FILE}"
|
||||
# --------------------------------------------------------------------------
|
||||
# Warn about unknown top-level keys (catches typos like java_verion)
|
||||
# --------------------------------------------------------------------------
|
||||
KNOWN_KEYS="pipeline orb_version org image_name server_module java_version node_version sonar docker cd coverage e2e sbom owasp contexts npm node services test terraform sast health kustomize kube_linter_config package_manager yamllint_paths supplemental_images"
|
||||
KNOWN_KEYS="pipeline orb_version org image_name server_module java_version node_version sonar docker cd coverage e2e sbom owasp contexts npm node services test terraform sast health kustomize kube_linter_config package_manager yamllint_paths supplemental_images ci"
|
||||
ACTUAL_KEYS=$(yq 'keys | .[]' "${CONFIG_FILE}" 2>/dev/null || true)
|
||||
for key in ${ACTUAL_KEYS}; do
|
||||
if ! echo "${KNOWN_KEYS}" | grep -qw "${key}"; then
|
||||
@@ -3261,6 +3372,17 @@ done
|
||||
# --------------------------------------------------------------------------
|
||||
TEMPLATE_FILE="${TEMPLATE_DIR}/${PIPELINE_TYPE}.yml.tpl"
|
||||
|
||||
# Opt in only after the consumer requires ci-ready instead of individual build jobs.
|
||||
# Missing history or any non-documentation path keeps the complete pipeline.
|
||||
if [[ "${PIPELINE_TYPE}" == "java-webapp" && "$(yq '.ci.docs_only // false' "${CONFIG_FILE}")" == "true" ]]; then
|
||||
# shellcheck source=docs_only.sh
|
||||
source "${SCRIPT_DIR}/docs_only.sh"
|
||||
if munitor_docs_only; then
|
||||
TEMPLATE_FILE="${TEMPLATE_DIR}/java-docs.yml.tpl"
|
||||
echo "Documentation-only change: retaining secret scanning; no application build or deployment."
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ ! -f "${TEMPLATE_FILE}" ]]; then
|
||||
echo "ERROR: No template found for pipeline type '${PIPELINE_TYPE}'."
|
||||
echo "Available templates:"
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
version: 2.1
|
||||
|
||||
orbs:
|
||||
munitor: ${MUNITOR_ORB_SLUG}@${MUNITOR_ORB_VERSION}
|
||||
|
||||
workflows:
|
||||
docs-checks:
|
||||
jobs:
|
||||
- munitor/secrets_scan:
|
||||
name: secrets-scan
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- secrets-scan
|
||||
@@ -77,6 +77,23 @@ workflows:
|
||||
- /hotfix\/.*/
|
||||
##ENDIF_E2E##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- build-and-test
|
||||
- image-scan
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
##IF_E2E##
|
||||
- e2e-tests
|
||||
##ENDIF_E2E##
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
|
||||
##INCLUDE_DEPLOY java-webapp-deploy develop develop ${MUNITOR_CD_ENV_DEVELOP}##
|
||||
##INCLUDE_DEPLOY java-webapp-deploy staging staging ${MUNITOR_CD_ENV_STAGING}##
|
||||
|
||||
@@ -235,6 +252,24 @@ workflows:
|
||||
- /release\/.*/
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_SBOM##
|
||||
- sbom
|
||||
##ENDIF_SBOM##
|
||||
##IF_CD_NON_PROD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD_NON_PROD##
|
||||
##IF_GITHUB_RELEASE##
|
||||
- github-release
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
|
||||
production:
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
@@ -298,3 +333,17 @@ workflows:
|
||||
branches:
|
||||
only: main
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_CD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD##
|
||||
##IF_GITHUB_RELEASE##
|
||||
- github-release
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
filters:
|
||||
branches:
|
||||
only: main
|
||||
@@ -129,3 +129,17 @@
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
##ENDIF_CD_NON_PROD##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_SBOM##
|
||||
- sbom
|
||||
##ENDIF_SBOM##
|
||||
##IF_CD_NON_PROD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD_NON_PROD##
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
@@ -0,0 +1,114 @@
|
||||
#!/usr/bin/env bash
|
||||
# Exercise the real generator and validate its complete dependency graph.
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
GENERATOR="${SCRIPT_DIR}/../src/scripts/generate_config.sh"
|
||||
WORK=$(mktemp -d)
|
||||
trap 'rm -rf -- "$WORK"' EXIT
|
||||
export CIRCLE_BRANCH=main CIRCLE_TAG=""
|
||||
export GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null
|
||||
export GIT_AUTHOR_NAME=Fixture GIT_AUTHOR_EMAIL=fixture@example.invalid
|
||||
export GIT_COMMITTER_NAME=Fixture GIT_COMMITTER_EMAIL=fixture@example.invalid
|
||||
PASS=0
|
||||
|
||||
render_check() {
|
||||
local config="$1" mode="$2" label="$3" generator="${4:-$GENERATOR}"
|
||||
MUNITOR_CONFIG="$config" MUNITOR_OUTPUT_FILE="${WORK}/generated.yml" \
|
||||
bash "$generator" >"${WORK}/generator.log" 2>&1 || { cat "${WORK}/generator.log"; return 1; }
|
||||
yq -o=json '.' "${WORK}/generated.yml" >"${WORK}/generated.json"
|
||||
python3 - "${WORK}/generated.json" "$mode" <<'PY'
|
||||
import json, sys
|
||||
data = json.load(open(sys.argv[1], encoding='utf-8'))
|
||||
workflows = data['workflows']
|
||||
assert workflows, 'no generated workflows'
|
||||
if sys.argv[2] == 'docs':
|
||||
assert set(workflows) == {'docs-checks'}, workflows.keys()
|
||||
else:
|
||||
assert {'pr-checks', 'production', 'release-candidate'} <= set(workflows), workflows.keys()
|
||||
assert 'docs-checks' not in workflows
|
||||
for workflow, body in workflows.items():
|
||||
jobs = {}
|
||||
filters = {}
|
||||
for invocation in body['jobs']:
|
||||
if isinstance(invocation, str):
|
||||
command, options = invocation, {}
|
||||
else:
|
||||
assert len(invocation) == 1
|
||||
command, options = next(iter(invocation.items()))
|
||||
options = options or {}
|
||||
name = options.get('name', command)
|
||||
assert name not in jobs, (workflow, 'duplicate job', name)
|
||||
jobs[name] = (command, options.get('requires', []))
|
||||
filters[name] = options.get('filters', {})
|
||||
assert 'ci-ready' in jobs, (workflow, 'missing ci-ready')
|
||||
assert jobs['ci-ready'][0] == 'munitor/ci_ready'
|
||||
for name, job_filters in filters.items():
|
||||
assert job_filters == filters['ci-ready'], (workflow, name, 'filters differ from ci-ready', job_filters, filters['ci-ready'])
|
||||
for name, (_, dependencies) in jobs.items():
|
||||
assert isinstance(dependencies, list)
|
||||
assert set(dependencies) <= jobs.keys(), (workflow, name, 'dangling dependency', dependencies)
|
||||
def closure(name, ancestors):
|
||||
assert name not in ancestors, (workflow, 'dependency cycle', name)
|
||||
reached = set()
|
||||
for dependency in jobs[name][1]:
|
||||
reached.add(dependency)
|
||||
reached.update(closure(dependency, ancestors | {name}))
|
||||
return reached
|
||||
assert closure('ci-ready', set()) == set(jobs) - {'ci-ready'}, (workflow, 'ungated job')
|
||||
if sys.argv[2] == 'docs':
|
||||
assert set(jobs) == {'secrets-scan', 'ci-ready'}
|
||||
assert jobs['secrets-scan'][0] == 'munitor/secrets_scan'
|
||||
else:
|
||||
assert any(command == 'munitor/mvn_build_and_test' for command, _ in jobs.values()), workflow
|
||||
print('validated', len(workflows), 'workflow graphs')
|
||||
PY
|
||||
printf 'PASS: %s\n' "$label"
|
||||
PASS=$((PASS + 1))
|
||||
}
|
||||
|
||||
render_check "${SCRIPT_DIR}/fixtures/java-webapp.munitor.yml" full 'full Java fixture gates every job'
|
||||
render_check "${SCRIPT_DIR}/fixtures/java-webapp-minimal.munitor.yml" full 'minimal Java fixture has no dangling optional dependencies'
|
||||
# Disable all optional publication/scanner additions as well as e2e/SBOM.
|
||||
cat >"${WORK}/minimal.yml" <<'YAML'
|
||||
pipeline: java-webapp
|
||||
orb_version: dev:snapshot
|
||||
image_name: fixture
|
||||
java_version: "21"
|
||||
docker:
|
||||
registry: ghcr.io/example
|
||||
e2e: false
|
||||
sbom: false
|
||||
owasp: false
|
||||
sast: false
|
||||
YAML
|
||||
render_check "${WORK}/minimal.yml" full 'optional jobs absent without dangling gates'
|
||||
|
||||
git init -q -b main "${WORK}/repo"
|
||||
cd "${WORK}/repo"
|
||||
cp "${WORK}/minimal.yml" .munitor.yml
|
||||
printf '\nci:\n docs_only: true\n' >> .munitor.yml
|
||||
printf 'original\n' > README.md
|
||||
mkdir src
|
||||
printf 'original\n' > src/App.java
|
||||
git add .
|
||||
git -c commit.gpgsign=false commit -qm baseline
|
||||
printf 'documentation\n' >> README.md
|
||||
git add README.md
|
||||
git -c commit.gpgsign=false commit -qm documentation
|
||||
render_check "$PWD/.munitor.yml" docs 'opted-in main README-only change retains only secrets and ci-ready'
|
||||
# The shipped script stages /tmp/munitor helpers; invoke sequentially, never in parallel.
|
||||
render_check "$PWD/.munitor.yml" docs 'packed generator retains docs-only secrets and ci-ready' \
|
||||
"${SCRIPT_DIR}/../src/scripts/packed_generate_config.sh"
|
||||
|
||||
# Opt-out must retain the full workflow even with the same docs-only Git diff.
|
||||
yq '.ci.docs_only = false' .munitor.yml >"${WORK}/optout.yml"
|
||||
render_check "${WORK}/optout.yml" full 'docs change with opt-out still builds'
|
||||
printf 'source change\n' >> src/App.java
|
||||
git add src/App.java
|
||||
git -c commit.gpgsign=false commit -qm source
|
||||
render_check "$PWD/.munitor.yml" full 'source change still builds'
|
||||
printf '\n# configuration change\n' >> .munitor.yml
|
||||
git add .munitor.yml
|
||||
git -c commit.gpgsign=false commit -qm configuration
|
||||
render_check "$PWD/.munitor.yml" full 'configuration change still builds'
|
||||
printf '%s ci-ready generator checks passed\n' "$PASS"
|
||||
@@ -0,0 +1,171 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# shellcheck source=src/scripts/docs_only.sh
|
||||
source "${SCRIPT_DIR}/../src/scripts/docs_only.sh"
|
||||
WORK=$(mktemp -d)
|
||||
trap 'rm -rf -- "$WORK"' EXIT
|
||||
export GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null
|
||||
export GIT_AUTHOR_NAME=Fixture GIT_AUTHOR_EMAIL=fixture@example.invalid
|
||||
export GIT_COMMITTER_NAME=Fixture GIT_COMMITTER_EMAIL=fixture@example.invalid
|
||||
export GIT_TERMINAL_PROMPT=0
|
||||
export CIRCLE_BRANCH=feature/docs CIRCLE_TAG=""
|
||||
PASS=0
|
||||
git init -q --bare "${WORK}/origin.git"
|
||||
git init -q -b main "${WORK}/seed"
|
||||
cd "${WORK}/seed"
|
||||
mkdir -p src docs/nested
|
||||
printf 'base\n' > README.md
|
||||
printf 'base\n' > src/app.java
|
||||
printf 'base\n' > docs/old.md
|
||||
git add .
|
||||
git -c commit.gpgsign=false commit -qm base
|
||||
git remote add origin "${WORK}/origin.git"
|
||||
git push -q origin main
|
||||
git --git-dir="${WORK}/origin.git" symbolic-ref HEAD refs/heads/main
|
||||
|
||||
fresh() {
|
||||
local repo="${WORK}/case-${PASS}"
|
||||
git clone -q "${WORK}/origin.git" "$repo"
|
||||
cd "$repo"
|
||||
git switch -qc feature/docs
|
||||
export CIRCLE_BRANCH=feature/docs CIRCLE_TAG=""
|
||||
}
|
||||
|
||||
commit() {
|
||||
git add -A
|
||||
git -c commit.gpgsign=false commit -qm change
|
||||
}
|
||||
|
||||
check() {
|
||||
local expected="$1" name="$2" status=0 output
|
||||
output=$(munitor_docs_only 2>&1) || status=$?
|
||||
if [[ "$status" -ne "$expected" || -n "$output" ]]; then
|
||||
printf 'FAIL: %s (status %s, expected %s; output must be empty)\n' "$name" "$status" "$expected"
|
||||
exit 1
|
||||
fi
|
||||
printf 'PASS: %s\n' "$name"
|
||||
PASS=$((PASS + 1))
|
||||
}
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
mkdir -p docs/nested
|
||||
printf 'docs\n' > 'docs/nested/space name.md'
|
||||
printf 'docs\n' > $'docs/nested/line\nbreak.md'
|
||||
printf 'docs\n' > CHANGELOG.md
|
||||
printf 'docs\n' > CONTRIBUTING.md
|
||||
commit
|
||||
check 0 'feature docs with nested, spaced and newline paths'
|
||||
CIRCLE_BRANCH=hotfix/docs check 0 'hotfix docs'
|
||||
for branch in develop staging release/1.0 unknown ''; do
|
||||
CIRCLE_BRANCH="$branch" check 1 "ineligible branch: $branch"
|
||||
done
|
||||
CIRCLE_TAG=v1.0 check 1 'tag cannot skip release'
|
||||
|
||||
fresh
|
||||
check 1 'empty branch diff'
|
||||
printf 'code\n' >> src/app.java
|
||||
commit
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
check 1 'earlier code commit followed by latest docs commit'
|
||||
|
||||
fresh
|
||||
printf 'code\n' >> src/app.java
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
check 1 'mixed source and docs'
|
||||
|
||||
fresh
|
||||
git mv src/app.java docs/disguised.md
|
||||
commit
|
||||
check 1 'code renamed into allowed docs path'
|
||||
|
||||
fresh
|
||||
git mv docs/old.md docs/new.md
|
||||
commit
|
||||
check 0 'docs-only rename'
|
||||
|
||||
fresh
|
||||
git rm -q src/app.java
|
||||
commit
|
||||
check 1 'deleted code'
|
||||
|
||||
fresh
|
||||
git rm -q docs/old.md
|
||||
commit
|
||||
check 0 'deleted documentation'
|
||||
|
||||
for path in pom.xml .circleci/config.yml docs/image.png docs/example.java readme.md nested/README.md; do
|
||||
fresh
|
||||
mkdir -p "$(dirname "$path")"
|
||||
printf 'not docs\n' > "$path"
|
||||
commit
|
||||
check 1 "non-allowlisted path: $path"
|
||||
done
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
git remote remove origin
|
||||
check 1 'missing origin fails closed'
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
git remote set-url origin "${WORK}/missing.git"
|
||||
check 1 'fetch failure fails closed despite cached origin/main'
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
# Advance origin main after this checkout; an old tracking ref is not enough.
|
||||
cd "${WORK}/seed"
|
||||
printf 'main\n' >> README.md
|
||||
commit
|
||||
git push -q origin main
|
||||
latest=$(git rev-parse HEAD)
|
||||
cd "${WORK}/case-${PASS}"
|
||||
check 0 'fetch updated main then compare merge-base'
|
||||
[[ "$(git rev-parse origin/main)" == "$latest" ]]
|
||||
|
||||
fresh
|
||||
git switch -q main
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
CIRCLE_BRANCH=main check 0 'main docs commit compares first parent'
|
||||
printf 'code\n' >> src/app.java
|
||||
commit
|
||||
CIRCLE_BRANCH=main check 1 'main source commit'
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
git switch -q main
|
||||
git -c commit.gpgsign=false merge --no-ff -qm 'docs merge' feature/docs
|
||||
CIRCLE_BRANCH=main check 0 'main docs merge uses first parent'
|
||||
|
||||
fresh
|
||||
printf 'code\n' >> src/app.java
|
||||
commit
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
git switch -q main
|
||||
git -c commit.gpgsign=false merge --no-ff -qm 'mixed merge' feature/docs
|
||||
CIRCLE_BRANCH=main check 1 'main merge includes whole source change'
|
||||
|
||||
git init -q -b main "${WORK}/root-only"
|
||||
cd "${WORK}/root-only"
|
||||
printf 'docs\n' > README.md
|
||||
commit
|
||||
CIRCLE_BRANCH=main check 1 'root commit has no parent'
|
||||
git remote add origin "${WORK}/origin.git"
|
||||
CIRCLE_BRANCH=feature/docs check 1 'unrelated history has no merge base'
|
||||
|
||||
git clone -q --depth=1 "file://${WORK}/origin.git" "${WORK}/shallow"
|
||||
cd "${WORK}/shallow"
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
CIRCLE_BRANCH=main check 1 'shallow history fails closed'
|
||||
printf '%s docs-only checks passed\n' "$PASS"
|
||||
@@ -843,8 +843,8 @@ echo ""
|
||||
echo "=== Error Handling Tests ==="
|
||||
|
||||
echo -n " TEST: all scripts use set -euo pipefail... "
|
||||
# munitor_helpers.sh is a sourced library, not a standalone script -- exclude it
|
||||
SCRIPTS_WITHOUT_STRICT=$(find "${SRC_SCRIPTS}" -name "*.sh" ! -name "munitor_helpers.sh" -exec grep -L 'set -euo pipefail' {} \;)
|
||||
# Sourced libraries must not change the caller shell options.
|
||||
SCRIPTS_WITHOUT_STRICT=$(find "${SRC_SCRIPTS}" -name "*.sh" ! -name "munitor_helpers.sh" ! -name "docs_only.sh" -exec grep -L 'set -euo pipefail' {} \;)
|
||||
if [[ -z "${SCRIPTS_WITHOUT_STRICT}" ]]; then
|
||||
pass
|
||||
else
|
||||
@@ -901,8 +901,9 @@ echo "=== Munitor Helpers Integration Tests ==="
|
||||
# Scripts that are excluded from the munitor_header requirement:
|
||||
# - munitor_helpers.sh: the library itself
|
||||
# - extract_munitor_vars.sh: low-level helper, sourced by other scripts
|
||||
# - docs_only.sh: status-only predicate sourced by the generator
|
||||
# - packed_*: auto-generated, embed helpers differently
|
||||
EXCLUDED_PATTERN="munitor_helpers.sh|extract_munitor_vars.sh|packed_"
|
||||
EXCLUDED_PATTERN="munitor_helpers.sh|extract_munitor_vars.sh|docs_only.sh|packed_"
|
||||
|
||||
echo -n " TEST: all scripts source munitor_helpers... "
|
||||
MISSING_SOURCE=""
|
||||
|
||||
Reference in new issue
Block a user