Compare commits
19
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a56ef1d8e7
|
||
|
|
674fcd1bc7 | ||
|
|
7e89e28a7a | ||
|
|
154d453a00
|
||
|
|
64831e83f1 | ||
|
|
77cdf931c2
|
||
|
|
d7c278b344
|
||
|
|
c8fd974bfd
|
||
|
|
6cd50b0a90 | ||
|
|
5d4a9ea420 | ||
|
|
f402fb8347
|
||
|
|
11d5704b47 | ||
|
|
576de90e03
|
||
|
|
9678be0cff | ||
|
|
baca55bd77 | ||
|
|
01176862d4
|
||
|
|
f7487b356a | ||
|
|
34ddd25ac9 | ||
|
|
4aeccb801e
|
No files matched your search
+21
-11
@@ -15,6 +15,8 @@ tag-only: &tag-only
|
||||
only: /^v[0-9]+\.[0-9]+\.[0-9]+$/
|
||||
|
||||
jobs:
|
||||
ci-ready:
|
||||
type: no-op
|
||||
test:
|
||||
docker:
|
||||
- image: cimg/node:22.12
|
||||
@@ -52,27 +54,29 @@ workflows:
|
||||
- test:
|
||||
filters: *filters
|
||||
|
||||
test-deploy:
|
||||
jobs:
|
||||
- orb-tools/pack:
|
||||
filters: *tag-only
|
||||
- ci-ready:
|
||||
requires:
|
||||
- orb-tools/lint
|
||||
- orb-tools/pack
|
||||
- orb-tools/review
|
||||
- test
|
||||
filters: *filters
|
||||
|
||||
- orb-tools/publish:
|
||||
orb_name: kof22/munitor
|
||||
vcs_type: << pipeline.project.type >>
|
||||
pub_type: production
|
||||
requires:
|
||||
- orb-tools/lint
|
||||
- orb-tools/pack
|
||||
- orb-tools/review
|
||||
- test
|
||||
context:
|
||||
- orb-publishing
|
||||
filters: *tag-only
|
||||
|
||||
# develop owns the shared dev:snapshot tag. Feature branches publish only an
|
||||
# immutable dev:<sha> tag so open branches cannot overwrite each other's
|
||||
# snapshot (issue #26).
|
||||
snapshot:
|
||||
jobs:
|
||||
- orb-tools/pack:
|
||||
filters: *filters
|
||||
# All publishers consume the same packed workspace after all checks pass.
|
||||
# Only develop updates dev:snapshot; feature branches publish dev:<sha>.
|
||||
- orb-tools/publish:
|
||||
name: publish-develop-snapshot
|
||||
orb_name: kof22/munitor
|
||||
@@ -80,7 +84,10 @@ workflows:
|
||||
pub_type: dev
|
||||
dev_tags: "dev:${CIRCLE_SHA1},dev:snapshot"
|
||||
requires:
|
||||
- orb-tools/lint
|
||||
- orb-tools/pack
|
||||
- orb-tools/review
|
||||
- test
|
||||
context:
|
||||
- orb-publishing
|
||||
filters:
|
||||
@@ -94,7 +101,10 @@ workflows:
|
||||
pub_type: dev
|
||||
dev_tags: "dev:${CIRCLE_SHA1}"
|
||||
requires:
|
||||
- orb-tools/lint
|
||||
- orb-tools/pack
|
||||
- orb-tools/review
|
||||
- test
|
||||
context:
|
||||
- orb-publishing
|
||||
filters:
|
||||
|
||||
@@ -30,7 +30,7 @@ lint:
|
||||
test-scripts:
|
||||
@echo "Running script tests..."
|
||||
@for f in tests/test_*.sh; do \
|
||||
if [ -f "$$f" ]; then \
|
||||
if [ -f "$$f" ] && [ "$$f" != "tests/test_generate_config.sh" ]; then \
|
||||
echo " Running $$f"; \
|
||||
bash "$$f" || exit 1; \
|
||||
fi; \
|
||||
|
||||
@@ -6,7 +6,7 @@ Latin: *munitor* (fortifier, protector) -- KofTwentyTwo's unified CircleCI orb f
|
||||
| --- | --- |
|
||||
| **Tier** | Internal until classified -- listed under *Awaiting classification* in the KofTwentyTwo standards [MAINTAINERS.md](https://github.com/KofTwentyTwo/standards/blob/main/MAINTAINERS.md); Product-tier requirements are being adopted |
|
||||
| **Standards** | Targets [KofTwentyTwo standards](https://github.com/KofTwentyTwo/standards) 1.0.0 |
|
||||
| **Orb** | [`kof22/munitor`](https://circleci.com/developer/orbs/orb/kof22/munitor), latest stable `0.3.6` |
|
||||
| **Orb** | [`kof22/munitor`](https://circleci.com/developer/orbs/orb/kof22/munitor), latest stable `0.3.9` |
|
||||
| **Security** | [SECURITY.md](SECURITY.md) |
|
||||
| **Contributing** | [CONTRIBUTING.md](CONTRIBUTING.md) |
|
||||
|
||||
@@ -52,6 +52,47 @@ Munitor validates branch names and will fail on non-conforming names.
|
||||
| `release/<version>` | Release prep |
|
||||
| `hotfix/<name>` | Production fixes |
|
||||
|
||||
## Optional docs-only Java checks
|
||||
|
||||
The new `java-webapp` docs-only path is opt-in and must be adopted **after** the
|
||||
aggregate required check is published and verified. Do not enable it against an
|
||||
older orb or while individual build jobs remain required:
|
||||
|
||||
```yaml
|
||||
ci:
|
||||
docs_only: true
|
||||
```
|
||||
|
||||
Only changes confined to root `README.md`, `CHANGELOG.md`, `CONTRIBUTING.md`, or
|
||||
Markdown files under `docs/` (including nested directories) qualify. Feature and
|
||||
hotfix branches use their complete diff from the merge base with `main`; `main`
|
||||
uses its first-parent diff. Tags, release and other branches, shallow history,
|
||||
failed comparisons, empty diffs, and any uncertain classification keep the full
|
||||
pipeline. The docs path retains secret scanning and emits the aggregate gate;
|
||||
it does not build or deploy an image.
|
||||
|
||||
Migration order: publish the gate-capable orb, adopt it with docs-only disabled,
|
||||
and verify the real GitHub context `ci/circleci: ci-ready` on a consumer run. Then
|
||||
replace that repository's required-check list with the verified aggregate gate,
|
||||
preserving all other branch rules. A single adoption PR may update both orb pins
|
||||
and enable `ci.docs_only`: those config changes force full CI. Verify the gate
|
||||
and migrate required checks before merging it or opening a docs-only PR. Munitor's
|
||||
own native no-op gate is expected to report `ci/circleci: lint-pack/ci-ready`;
|
||||
verify its actual context before changing its rules. Release and real-context
|
||||
verification are still pending; see [the adoption plan](docs/PLAN-agile-ci.md).
|
||||
|
||||
Use native [Website-Backend Insights](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows),
|
||||
[PR timings](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows/pr-checks),
|
||||
and [Test Insights](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows/pr-checks/tests)
|
||||
to measure results. Compare matching branches/windows and track the `docs-checks`
|
||||
median separately from full builds. The historical local report
|
||||
`C:/Users/james/.cache/ci-insights-report.md` ends October 17 and does not establish
|
||||
the speed of this change. No additional reporting job is needed.
|
||||
|
||||
Native auto-cancel is enabled for Website-Backend, whose default `main` is exempt.
|
||||
It remains disabled for Munitor: its default is `develop`, so `main` release work
|
||||
would not receive that exemption.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **CircleCI project**: Add your repo at https://app.circleci.com
|
||||
@@ -70,16 +111,16 @@ Munitor validates branch names and will fail on non-conforming names.
|
||||
|
||||
Munitor is a public CircleCI orb: any CircleCI organization can use it without forking. This checklist covers what a new consumer needs, for example a Java/Maven web service. A QQQ application should also read the [QQQ adoption checklist](https://github.com/QRun-IO/qqq/blob/develop/docs/migration/adoption-checklist.adoc) (section 7 covers release tooling).
|
||||
|
||||
**Version.** Use `kof22/munitor@0.3.6` (or `@0.3` to float on the 0.3 line) in `.circleci/config.yml`, and the same value as `orb_version` in `.munitor.yml`. Keep the setup orb and the generated pipeline on the same line.
|
||||
**Version.** Use `kof22/munitor@0.3.9` (or `@0.3` to float on the 0.3 line) in `.circleci/config.yml`, and the same value as `orb_version` in `.munitor.yml`. Keep the setup orb and the generated pipeline on the same line.
|
||||
|
||||
### 1. Repository files
|
||||
|
||||
- `.circleci/config.yml`: the setup config from [Quick Start](#quick-start), pinned to `kof22/munitor@0.3.6`.
|
||||
- `.circleci/config.yml`: the setup config from [Quick Start](#quick-start), pinned to `kof22/munitor@0.3.9`.
|
||||
- `.munitor.yml`: pick a [pipeline type](#pipeline-types). A Java service is `java-webapp`. KofTwentyTwo's Website-Backend is a working example; a generalized version:
|
||||
|
||||
```yaml
|
||||
pipeline: java-webapp
|
||||
orb_version: "0.3.6"
|
||||
orb_version: "0.3.9"
|
||||
image_name: my-service
|
||||
java_version: "21"
|
||||
docker:
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
# Agile CI adoption
|
||||
|
||||
Implemented locally: an opt-in `java-webapp` docs-only workflow, aggregate consumer
|
||||
gate, and native no-op gate for Munitor's own checks. Publication, real check-context
|
||||
verification, required-check migration and consumer opt-in remain pending.
|
||||
|
||||
## Contract
|
||||
|
||||
- `ci.docs_only: true` permits only root README/CHANGELOG/CONTRIBUTING.md and
|
||||
`docs/**/*.md` (including Markdown directly in docs). Secret scanning stays on.
|
||||
- Feature/hotfix classification uses the complete branch diff from the merge base
|
||||
with main, not the latest commit. Main uses its first parent. Tags, release/other
|
||||
branches, shallow history, empty or uncertain comparisons use full CI.
|
||||
- Docs checks emit the aggregate gate without image build/deploy. Full workflows
|
||||
retain their existing gates. No custom reporting framework or reporting job.
|
||||
|
||||
## Required order
|
||||
|
||||
1. Local make all, 20 BATS, 31 classifier cases, eight source/packed graph cases, and CircleCI config validation passed. Finish hosted CI and publish the gate-capable orb.
|
||||
2. Adopt it in the consumer with docs-only disabled. Observe successful actual
|
||||
GitHub context `ci/circleci: ci-ready`; do not infer the name from YAML alone.
|
||||
3. Replace the consumer required-check list with that verified aggregate context,
|
||||
preserving review, signature, conversation, bypass and all other branch rules.
|
||||
4. Enable `ci.docs_only: true`; prove docs-only and mixed/code changes select the
|
||||
expected paths and retain the required gate.
|
||||
5. Separately verify Munitor's native no-op context (expected
|
||||
`ci/circleci: lint-pack/ci-ready`) before migrating its required checks.
|
||||
|
||||
All five adoption steps are pending; an implemented template is not rollout proof.
|
||||
To avoid duplicate full builds, the pin and opt-in may share one adoption PR:
|
||||
the manifest/config changes force full CI. Verify its gate and migrate required
|
||||
checks before merging or opening a docs-only PR.
|
||||
|
||||
## Measure using existing CircleCI views
|
||||
|
||||
[Project Insights](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows),
|
||||
[PR checks](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows/pr-checks),
|
||||
and [Tests](https://app.circleci.com/insights/github/KofTwentyTwo/Website-Backend/workflows/pr-checks/tests)
|
||||
already provide the dashboard. The historical API baseline was retrieved
|
||||
October 19 but its returned data ends October 17 (PR P50 13m58s, P95 34m37s).
|
||||
It is not a measurement of this implementation. After Insights catches up, compare
|
||||
equivalent windows/branches and report the docs-checks median separately; do not
|
||||
blend docs runs into a claimed full-build improvement or sum parallel job medians.
|
||||
|
||||
Backend native auto-cancel is on; main is its exempt default branch. Munitor stays
|
||||
off because develop is its default and main/release work is not similarly exempt.
|
||||
@@ -0,0 +1,41 @@
|
||||
# Fast Java CI
|
||||
|
||||
Backend ticket work is paused while this performance change is delivered.
|
||||
|
||||
## Measured baseline
|
||||
|
||||
Website-Backend PR311, CircleCI jobs6314–6318 on2026-10-19:
|
||||
|
||||
- Build/test job:30m15s. Compilation49s; tests14m24s; Java installation4m27s.
|
||||
- Maven cache48GiB: restore5m32s, save4m19s.
|
||||
- Quality, coverage and security each restore the same48GiB again (5–6min).
|
||||
- Coverage computation itself takes5.4s. Quality tools take57s combined.
|
||||
- OWASP is disabled in this consumer, but security still installs Java and restores Maven.
|
||||
- Test results are uploaded by both the test command and its enclosing job:777 results appear twice.
|
||||
|
||||
Evidence: coordinator cache/cicd-baseline.json, cicd-test-timings.json and cicd-github-friction.md.
|
||||
|
||||
## First implementation
|
||||
|
||||
1. Replace the old broad Maven fallback with an exact v2 key based on checked-out dependency definitions. Keep that key stable across release-version rewrites and invalidate for nested POM or Maven configuration changes. Never restore the48GiB v1 cache.
|
||||
2. Skip Java/Maven/NVD setup when OWASP is disabled; preserve Trivy.
|
||||
3. Upload Maven test results once.
|
||||
4. Pack the orb once per pipeline; publish only after its existing checks pass, preserving branch/tag behavior and required check names.
|
||||
5. Run template tests only in their dedicated Make target, rather than also discovering them in the generic script loop.
|
||||
6. Run Checkstyle, SpotBugs, PMD and coverage on the existing build worker for generated PR/develop/staging/release workflows. Keep the standalone jobs available for custom consumers and preserve production behavior. Generated downstream jobs depend on the completed build gate.
|
||||
7. Render each immutable test fixture once per test run; retain all277 assertions.
|
||||
|
||||
Maven caches are dependency reuse, not artifact handoff. A real three-module experiment failed SpotBugs with current target outputs and an older installed-module cache. Preserve locally installed version directories, mutable SNAPSHOT dependencies and ancestor metadata in a workspace archive, then overlay it after dependency-cache restore. The experiment passes for release versions, snapshots and classifier dependencies. Maven Resolver identifies local installs with empty repository IDs in `_remote.repositories`; this implementation depends on that Maven3 format. Do not replace this archive with the entire local repository.
|
||||
|
||||
Cold dependency downloads on changed POMs are deliberate: no cross-version fallback accumulation. Do not delete caches, artifacts, tags or change branch protections.
|
||||
|
||||
## Verification and rollout
|
||||
|
||||
- Focused behavioral/config regressions, then required make all and BATS suites once the combined patch is complete.
|
||||
- Publish immutable feature orb; run a backend feature canary with both pins matching that SHA.
|
||||
- Compare real cache size and build/check durations against the baseline. No claimed speedup before measurement; do not merge the consumer until tests and all existing security gates pass.
|
||||
- Release the proven orb and adopt its stable version through normal PRs. Track the already-running backend release separately; do not resume paused ticket work.
|
||||
|
||||
## Follow-up decisions after measurement
|
||||
|
||||
If tests dominate after cache repair, inspect measured slow fixtures and isolate/parallelize only tests proven independent. Exact-source artifact promotion is a separate future change requiring verified source identity; do not skip merged-source validation on the strength of an earlier PR run. Avoid a new orchestration framework or speculative configuration options.
|
||||
@@ -1,3 +1,36 @@
|
||||
## Agile CI (2026-10-19)
|
||||
|
||||
Fast Java CI shipped as Munitor 0.3.9 and Website-Backend PR330. The warm
|
||||
consumer PR workflow fell from 37m04 to 20m45; cache restore fell from 5m32 to
|
||||
4.76s (48GiB to 565MiB). Backend PR332 then preserved all 777 test identities
|
||||
while reducing its verified PR workflow to 14m49 and tests to 8m10. Timing-sensitive
|
||||
performance tests run serially after the parallel suite; thresholds are unchanged.
|
||||
|
||||
Current feature/agile-ci adds conservative opt-in documentation-only validation
|
||||
and one aggregate merge result. Backend native auto-cancel is enabled; Munitor
|
||||
remains off because its default branch is develop. Native Insights provides
|
||||
runtime/flaky-test reporting without another service. Local classifier and
|
||||
workflow-graph regressions pass; publication and consumer adoption are pending.
|
||||
See docs/PLAN-agile-ci.md for migration order and measurement limits.
|
||||
|
||||
## Fast Java CI (2026-10-19)
|
||||
|
||||
Active user priority: fast, agile, lightweight CI/CD; backend ticket goal paused.
|
||||
Measured48GiB Maven cache restored four times and saved once per backend PR.
|
||||
Work on feature/lean-java-ci: stable exact dependency cache keys with new namespace,
|
||||
conditional OWASP setup, single test-result upload, and one orb pack per pipeline.
|
||||
The first patch passed make all and15BATS. Further simplification combines quality
|
||||
and coverage with the build, renders fixtures once, and carries installed reactor
|
||||
artifacts in a small workspace archive. A real multi-module experiment proved this
|
||||
artifact handoff necessary, including snapshot/classifier coverage. Expanded make
|
||||
all and20BATS pass, including all277template assertions. Real consumer timing and
|
||||
stable publication/adoption remain pending. No deployed speedup claimed.
|
||||
See docs/PLAN-lean-java-ci.md.
|
||||
|
||||
## Supplemental runtime freshness (2026-10-19)
|
||||
|
||||
Backend PR311 image scan6307 failed because db-backup reused a cached apk-upgrade layer (libexpat2.8.5-r0, fixed2.9.0-r0). Supplemental builds now use --pull --no-cache in both PR scan-only and release publishing paths. Trivy and immutable-tag guards remain unchanged. Targeted tests cover both Dockerfile forms, fresh-build flags, scan-before-push ordering and scan failure propagation. Validation pending; no release or consumer adoption yet.
|
||||
|
||||
# Munitor session state
|
||||
|
||||
## JVM runtime package freshness (2026-10-19)
|
||||
@@ -98,3 +131,16 @@ These fixes require normal feature-to-develop review and stable tag publication
|
||||
from main before website consumers can select the new version. Nothing deployed.
|
||||
|
||||
Runtime freshness validation: coordinating agent completed make all (lint, pack, orb validation, scripts,277template assertions) and all7BATS tests in Ubuntu. Real consumer image scan remains required after release.
|
||||
|
||||
## Release 0.3.7 preparation (2026-10-19)
|
||||
Includes merged PR38 (filesystem Trivy findings fail CI; current Semgrep image and zero-scan/configuration failures) and PR39 (pull fresh runtime bases and bypass cached JVM package-upgrade layers). Both feature PRs passed all CircleCI gates; PR39 also passed full local make all and seven BATS tests. Publish the stable orb only from a signed v0.3.7 tag on the reviewed main merge. Website-Backend must update both orb pins and pass real image/SAST scans before rollout. No vulnerability waiver or immutable-tag overwrite is part of this release.
|
||||
|
||||
Validation completed: focused scan tests12/12, full make all (lint, packed orb validation, script/template tests), and7BATS passed under WSL/Nix on2026-10-19. Real backend supplemental image scans remain required after stable release adoption.
|
||||
|
||||
## Release candidate 0.3.8 (2026-10-19)
|
||||
|
||||
Promotes PR42 supplemental build freshness fix (develop11d5704) to main. Full local make all and seven BATS passed; all PR42 CI checks passed. Release metadata changes only; source/tests match the verified feature. Stable publication still requires the approved main merge and signed v0.3.8 tag. Backend adoption and real supplemental scans remain pending.
|
||||
|
||||
## Release candidate 0.3.9 (2026-10-19)
|
||||
|
||||
Promotes PR45 (develop64831e8): consolidated Java quality/coverage, bounded dependency caches with current artifact/SNAPSHOT handoff, conditional OWASP setup, and deduplicated orb/test work. Source and tests are identical to verified77cdf93: make all and20BATS passed; actual CircleCI compiler preserves required build-and-test name. Cold backend canary PR330 passed every test/scan: combined build19m02s vs old build/test30m15s, Maven cache565MiB vs old48GiB. Stable publication and backend warm-cache adoption remain pending. Production build retains its prior operations under a distinct name to avoid PR check suffixes.
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
## Pending
|
||||
|
||||
- [ ] Publish and adopt aggregate CI result and docs-only opt-in; verify real check names before changing required checks (docs/PLAN-agile-ci.md).
|
||||
|
||||
- [x] Publish fast Java CI as 0.3.9; backend PR330 adopted it and measured the warm workflow at 20m45 (from 37m04).
|
||||
|
||||
- [ ] Validate and release supplemental-image freshness fix, then adopt it in backend PR311 and verify all supplemental Trivy scans.
|
||||
|
||||
- [ ] Release JVM runtime cache fix and adopt the stable orb in Website-Backend; verify fresh image Trivy before release.
|
||||
|
||||
- [ ] Run PR image-scan script/template/BATS tests on Linux and verify a consumer PR with Docker/Trivy before publishing.
|
||||
|
||||
@@ -38,8 +38,14 @@ echo "Packing generate_config.sh..."
|
||||
echo 'MUNITOR_EXTRACT_EOF'
|
||||
echo ''
|
||||
|
||||
# Embed conservative documentation-only classification.
|
||||
echo "cat > /tmp/munitor/docs_only.sh << 'MUNITOR_DOCS_EOF'"
|
||||
cat "${SRC}/docs_only.sh"
|
||||
echo 'MUNITOR_DOCS_EOF'
|
||||
echo ''
|
||||
|
||||
# Embed each template
|
||||
for tpl in java-webapp gradle-webapp node-api node-webapp terraform sdk-distribution validate-cd-repo argocd-apps obsidian-plugin; do
|
||||
for tpl in java-docs java-webapp gradle-webapp node-api node-webapp terraform sdk-distribution validate-cd-repo argocd-apps obsidian-plugin; do
|
||||
echo "cat > /tmp/munitor/templates/${tpl}.yml.tpl << 'MUNITOR_TPL_EOF'"
|
||||
cat "${SRC}/templates/${tpl}.yml.tpl"
|
||||
echo 'MUNITOR_TPL_EOF'
|
||||
|
||||
@@ -1,7 +1,12 @@
|
||||
description: >
|
||||
Restore the Maven local repository cache based on pom.xml checksum.
|
||||
Restore only the cache for the checked-out Maven dependency definitions.
|
||||
steps:
|
||||
- run:
|
||||
name: Identify Maven dependency cache
|
||||
command: <<include(scripts/maven_cache_key.sh)>>
|
||||
- restore_cache:
|
||||
keys:
|
||||
- maven-v1-{{ checksum "pom.xml" }}
|
||||
- maven-v1-
|
||||
- maven-v2-{{ checksum ".munitor-maven-cache-key" }}
|
||||
- run:
|
||||
name: Restore this build's installed Maven artifacts
|
||||
command: <<include(scripts/maven_workspace.sh)>>
|
||||
@@ -1,7 +1,12 @@
|
||||
description: >
|
||||
Save the Maven local repository to cache for future builds.
|
||||
Save the Maven repository under the same immutable key used at restore time.
|
||||
steps:
|
||||
- save_cache:
|
||||
key: maven-v1-{{ checksum "pom.xml" }}
|
||||
key: maven-v2-{{ checksum ".munitor-maven-cache-key" }}
|
||||
paths:
|
||||
- ~/.m2/repository
|
||||
- run:
|
||||
name: Preserve installed Maven artifacts for this workspace
|
||||
environment:
|
||||
MAVEN_WORKSPACE_ACTION: save
|
||||
command: <<include(scripts/maven_workspace.sh)>>
|
||||
@@ -3,8 +3,8 @@ description: >
|
||||
parameters:
|
||||
version:
|
||||
type: string
|
||||
default: "1.61.1"
|
||||
default: "1.181.0"
|
||||
description: Semgrep image version tag.
|
||||
docker:
|
||||
- image: returntocorp/semgrep:<< parameters.version >>
|
||||
- image: semgrep/semgrep:<< parameters.version >>
|
||||
resource_class: medium
|
||||
@@ -0,0 +1,7 @@
|
||||
description: >
|
||||
Single merge result. Workflow dependencies enforce the applicable checks.
|
||||
executor: base
|
||||
steps:
|
||||
- run:
|
||||
name: Required checks passed
|
||||
command: echo "All applicable CI checks passed."
|
||||
@@ -3,6 +3,14 @@ description: >
|
||||
version, builds, tests, saves cache, and persists the workspace for
|
||||
downstream jobs.
|
||||
parameters:
|
||||
quality_checks:
|
||||
type: boolean
|
||||
default: false
|
||||
description: Run quality and coverage checks on the build worker before sharing artifacts.
|
||||
min_instruction:
|
||||
type: string
|
||||
default: "70"
|
||||
description: Minimum instruction coverage when quality_checks is enabled.
|
||||
java_version:
|
||||
type: string
|
||||
default: "21"
|
||||
@@ -29,8 +37,20 @@ steps:
|
||||
- mvn_set_version
|
||||
- mvn_build
|
||||
- mvn_test
|
||||
- store_test_results:
|
||||
path: target/surefire-reports
|
||||
- when:
|
||||
condition: << parameters.quality_checks >>
|
||||
steps:
|
||||
- mvn_checkstyle
|
||||
- mvn_spotbugs
|
||||
- mvn_pmd
|
||||
- mvn_jacoco:
|
||||
min_instruction: << parameters.min_instruction >>
|
||||
- store_artifacts:
|
||||
path: target/checkstyle-result.xml
|
||||
destination: checkstyle-result.xml
|
||||
- store_artifacts:
|
||||
path: target/site/jacoco
|
||||
destination: coverage-report
|
||||
- run:
|
||||
name: Persist version metadata
|
||||
command: <<include(scripts/persist_version_metadata.sh)>>
|
||||
|
||||
@@ -23,12 +23,12 @@ executor:
|
||||
steps:
|
||||
- attach_workspace:
|
||||
at: .
|
||||
- install_java:
|
||||
version: << parameters.java_version >>
|
||||
- restore_maven_cache
|
||||
- when:
|
||||
condition: << parameters.owasp >>
|
||||
steps:
|
||||
- install_java:
|
||||
version: << parameters.java_version >>
|
||||
- restore_maven_cache
|
||||
- restore_cache:
|
||||
keys:
|
||||
- nvd-cache-v1-{{ .Branch }}
|
||||
|
||||
@@ -83,7 +83,8 @@ for i in $(seq 0 $((COUNT - 1))); do
|
||||
|
||||
# Build
|
||||
echo "Building ${FULL_TAG} ..."
|
||||
BUILD_ARGS=(-f "${DOCKERFILE}" -t "${FULL_TAG}")
|
||||
# Refresh base images and rerun package upgrades in supplemental Dockerfiles.
|
||||
BUILD_ARGS=(--pull --no-cache -f "${DOCKERFILE}" -t "${FULL_TAG}")
|
||||
if [[ -n "${CIRCLE_SHA1:-}" ]]; then
|
||||
BUILD_ARGS+=(--label "org.opencontainers.image.revision=${CIRCLE_SHA1}")
|
||||
fi
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
# Source this helper. Any uncertainty keeps the full pipeline.
|
||||
munitor_docs_only() (
|
||||
[[ -z "${CIRCLE_TAG:-}" ]] || return 1
|
||||
local base paths path seen=0
|
||||
case "${CIRCLE_BRANCH:-}" in
|
||||
feature/*|hotfix/*)
|
||||
git fetch --no-tags origin '+refs/heads/main:refs/remotes/origin/main' >/dev/null 2>&1 || return 1
|
||||
base=$(git merge-base HEAD refs/remotes/origin/main 2>/dev/null) || return 1
|
||||
;;
|
||||
main)
|
||||
base=$(git rev-parse --verify 'HEAD^1' 2>/dev/null) || return 1
|
||||
;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
[[ "$(git rev-parse --is-shallow-repository 2>/dev/null)" == false ]] || return 1
|
||||
paths=$(mktemp 2>/dev/null) || return 1
|
||||
trap 'rm -f -- "$paths"' EXIT
|
||||
git diff --no-ext-diff --no-renames --name-only -z "$base" HEAD >"$paths" 2>/dev/null || return 1
|
||||
while IFS= read -r -d '' path; do
|
||||
case "$path" in
|
||||
README.md|CHANGELOG.md|CONTRIBUTING.md|docs/*.md) ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
seen=1
|
||||
done <"$paths"
|
||||
[[ "$seen" -eq 1 ]]
|
||||
)
|
||||
@@ -97,7 +97,7 @@ validate_required_fields "${PIPELINE_TYPE}" "${CONFIG_FILE}"
|
||||
# --------------------------------------------------------------------------
|
||||
# Warn about unknown top-level keys (catches typos like java_verion)
|
||||
# --------------------------------------------------------------------------
|
||||
KNOWN_KEYS="pipeline orb_version org image_name server_module java_version node_version sonar docker cd coverage e2e sbom owasp contexts npm node services test terraform sast health kustomize kube_linter_config package_manager yamllint_paths supplemental_images"
|
||||
KNOWN_KEYS="pipeline orb_version org image_name server_module java_version node_version sonar docker cd coverage e2e sbom owasp contexts npm node services test terraform sast health kustomize kube_linter_config package_manager yamllint_paths supplemental_images ci"
|
||||
ACTUAL_KEYS=$(yq 'keys | .[]' "${CONFIG_FILE}" 2>/dev/null || true)
|
||||
for key in ${ACTUAL_KEYS}; do
|
||||
if ! echo "${KNOWN_KEYS}" | grep -qw "${key}"; then
|
||||
@@ -110,6 +110,17 @@ done
|
||||
# --------------------------------------------------------------------------
|
||||
TEMPLATE_FILE="${TEMPLATE_DIR}/${PIPELINE_TYPE}.yml.tpl"
|
||||
|
||||
# Opt in only after the consumer requires ci-ready instead of individual build jobs.
|
||||
# Missing history or any non-documentation path keeps the complete pipeline.
|
||||
if [[ "${PIPELINE_TYPE}" == "java-webapp" && "$(yq '.ci.docs_only // false' "${CONFIG_FILE}")" == "true" ]]; then
|
||||
# shellcheck source=docs_only.sh
|
||||
source "${SCRIPT_DIR}/docs_only.sh"
|
||||
if munitor_docs_only; then
|
||||
TEMPLATE_FILE="${TEMPLATE_DIR}/java-docs.yml.tpl"
|
||||
echo "Documentation-only change: retaining secret scanning; no application build or deployment."
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ ! -f "${TEMPLATE_FILE}" ]]; then
|
||||
echo "ERROR: No template found for pipeline type '${PIPELINE_TYPE}'."
|
||||
echo "Available templates:"
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
MUNITOR_HELPERS="${MUNITOR_HELPERS:-$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)/munitor_helpers.sh}"
|
||||
# shellcheck source=munitor_helpers.sh
|
||||
if [[ -f "${MUNITOR_HELPERS}" ]]; then
|
||||
source "${MUNITOR_HELPERS}"
|
||||
elif ! type munitor_header &>/dev/null; then
|
||||
munitor_header() { echo "=== Munitor: ${1:-unknown} ==="; }
|
||||
fi
|
||||
munitor_header "maven_cache_key"
|
||||
|
||||
# Hash the checked-out dependency definitions, not release-version rewrites.
|
||||
# The same index travels with the workspace to downstream Maven jobs.
|
||||
git ls-files -s -- pom.xml '**/pom.xml' '.mvn/*' > .munitor-maven-cache-key
|
||||
if [[ ! -s .munitor-maven-cache-key ]]; then
|
||||
echo "ERROR: Maven cache requires tracked dependency definitions." >&2
|
||||
exit 1
|
||||
fi
|
||||
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
MUNITOR_HELPERS="${MUNITOR_HELPERS:-$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)/munitor_helpers.sh}"
|
||||
# shellcheck source=munitor_helpers.sh
|
||||
if [[ -f "${MUNITOR_HELPERS}" ]]; then
|
||||
source "${MUNITOR_HELPERS}"
|
||||
elif ! type munitor_header &>/dev/null; then
|
||||
munitor_header() { echo "=== Munitor: ${1:-unknown} ==="; }
|
||||
fi
|
||||
munitor_header "maven_workspace"
|
||||
|
||||
archive="${PWD}/.munitor-maven-installed.tgz"
|
||||
repository="${HOME}/.m2/repository"
|
||||
case "${MAVEN_WORKSPACE_ACTION:-restore}" in
|
||||
save)
|
||||
# Maven marks local installs with an empty repository ID (artifact>=).
|
||||
# Carry build outputs and mutable SNAPSHOT dependencies separately from the cache.
|
||||
files=$(mktemp)
|
||||
trap 'rm -f "${files}"' EXIT
|
||||
cd "${repository}"
|
||||
while IFS= read -r -d '' tracking; do
|
||||
if [[ "${tracking}" == */*-SNAPSHOT/_remote.repositories ]] ||
|
||||
grep -qE '^[^#].*>=[[:space:]]*$' "${tracking}"; then
|
||||
directory=$(dirname "${tracking}")
|
||||
printf '%s\0' "${directory}" >> "${files}"
|
||||
while [[ "${directory}" != . ]]; do
|
||||
directory=$(dirname "${directory}")
|
||||
if [[ -f "${directory}/maven-metadata-local.xml" ]]; then
|
||||
printf '%s\0' "${directory}/maven-metadata-local.xml" >> "${files}"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
done < <(find . -name _remote.repositories -type f -print0)
|
||||
tar -czf "${archive}" --null -T "${files}"
|
||||
;;
|
||||
restore)
|
||||
if [[ -f "${archive}" ]]; then
|
||||
mkdir -p "${repository}"
|
||||
tar -xzf "${archive}" -C "${repository}"
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: Unknown Maven workspace action." >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
@@ -347,6 +347,54 @@ get_envsubst_vars() {
|
||||
}
|
||||
MUNITOR_EXTRACT_EOF
|
||||
|
||||
cat > /tmp/munitor/docs_only.sh << 'MUNITOR_DOCS_EOF'
|
||||
#!/usr/bin/env bash
|
||||
# Source this helper. Any uncertainty keeps the full pipeline.
|
||||
munitor_docs_only() (
|
||||
[[ -z "${CIRCLE_TAG:-}" ]] || return 1
|
||||
local base paths path seen=0
|
||||
case "${CIRCLE_BRANCH:-}" in
|
||||
feature/*|hotfix/*)
|
||||
git fetch --no-tags origin '+refs/heads/main:refs/remotes/origin/main' >/dev/null 2>&1 || return 1
|
||||
base=$(git merge-base HEAD refs/remotes/origin/main 2>/dev/null) || return 1
|
||||
;;
|
||||
main)
|
||||
base=$(git rev-parse --verify 'HEAD^1' 2>/dev/null) || return 1
|
||||
;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
[[ "$(git rev-parse --is-shallow-repository 2>/dev/null)" == false ]] || return 1
|
||||
paths=$(mktemp 2>/dev/null) || return 1
|
||||
trap 'rm -f -- "$paths"' EXIT
|
||||
git diff --no-ext-diff --no-renames --name-only -z "$base" HEAD >"$paths" 2>/dev/null || return 1
|
||||
while IFS= read -r -d '' path; do
|
||||
case "$path" in
|
||||
README.md|CHANGELOG.md|CONTRIBUTING.md|docs/*.md) ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
seen=1
|
||||
done <"$paths"
|
||||
[[ "$seen" -eq 1 ]]
|
||||
)
|
||||
MUNITOR_DOCS_EOF
|
||||
|
||||
cat > /tmp/munitor/templates/java-docs.yml.tpl << 'MUNITOR_TPL_EOF'
|
||||
version: 2.1
|
||||
|
||||
orbs:
|
||||
munitor: ${MUNITOR_ORB_SLUG}@${MUNITOR_ORB_VERSION}
|
||||
|
||||
workflows:
|
||||
docs-checks:
|
||||
jobs:
|
||||
- munitor/secrets_scan:
|
||||
name: secrets-scan
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- secrets-scan
|
||||
MUNITOR_TPL_EOF
|
||||
|
||||
cat > /tmp/munitor/templates/java-webapp.yml.tpl << 'MUNITOR_TPL_EOF'
|
||||
version: 2.1
|
||||
|
||||
@@ -358,6 +406,8 @@ workflows:
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
name: build-and-test
|
||||
quality_checks: true
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
@@ -395,27 +445,6 @@ workflows:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
- munitor/mvn_code_quality:
|
||||
name: code-quality
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
- munitor/mvn_coverage:
|
||||
name: coverage
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
- munitor/security_scan:
|
||||
name: security-scan
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
@@ -446,6 +475,23 @@ workflows:
|
||||
- /hotfix\/.*/
|
||||
##ENDIF_E2E##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- build-and-test
|
||||
- image-scan
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
##IF_E2E##
|
||||
- e2e-tests
|
||||
##ENDIF_E2E##
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
|
||||
##INCLUDE_DEPLOY java-webapp-deploy develop develop ${MUNITOR_CD_ENV_DEVELOP}##
|
||||
##INCLUDE_DEPLOY java-webapp-deploy staging staging ${MUNITOR_CD_ENV_STAGING}##
|
||||
|
||||
@@ -453,6 +499,8 @@ workflows:
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
name: build-and-test
|
||||
quality_checks: true
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
@@ -473,25 +521,6 @@ workflows:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
- munitor/mvn_code_quality:
|
||||
name: code-quality
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
- munitor/mvn_coverage:
|
||||
name: coverage
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
- munitor/security_scan:
|
||||
name: security-scan
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
@@ -526,7 +555,6 @@ workflows:
|
||||
sonar_project_key: ${MUNITOR_SONAR_PROJECT_KEY}
|
||||
requires:
|
||||
- build-and-test
|
||||
- coverage
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_SONAR}
|
||||
filters:
|
||||
@@ -547,8 +575,7 @@ workflows:
|
||||
supplemental_images: '${MUNITOR_SUPPLEMENTAL_IMAGES_JSON}'
|
||||
##ENDIF_SUPPLEMENTAL##
|
||||
requires:
|
||||
- code-quality
|
||||
- coverage
|
||||
- build-and-test
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
@@ -571,8 +598,7 @@ workflows:
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
requires:
|
||||
- code-quality
|
||||
- coverage
|
||||
- build-and-test
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
@@ -624,10 +650,28 @@ workflows:
|
||||
- /release\/.*/
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_SBOM##
|
||||
- sbom
|
||||
##ENDIF_SBOM##
|
||||
##IF_CD_NON_PROD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD_NON_PROD##
|
||||
##IF_GITHUB_RELEASE##
|
||||
- github-release
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
|
||||
production:
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
name: build-and-test
|
||||
name: production-build-and-test
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
@@ -647,7 +691,7 @@ workflows:
|
||||
supplemental_images: '${MUNITOR_SUPPLEMENTAL_IMAGES_JSON}'
|
||||
##ENDIF_SUPPLEMENTAL##
|
||||
requires:
|
||||
- build-and-test
|
||||
- production-build-and-test
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_REGISTRY}
|
||||
filters:
|
||||
@@ -687,6 +731,20 @@ workflows:
|
||||
branches:
|
||||
only: main
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_CD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD##
|
||||
##IF_GITHUB_RELEASE##
|
||||
- github-release
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
filters:
|
||||
branches:
|
||||
only: main
|
||||
MUNITOR_TPL_EOF
|
||||
|
||||
cat > /tmp/munitor/templates/gradle-webapp.yml.tpl << 'MUNITOR_TPL_EOF'
|
||||
@@ -2572,6 +2630,8 @@ cat > /tmp/munitor/templates/partials/java-webapp-deploy.yml.tpl << 'MUNITOR_PAR
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
name: build-and-test
|
||||
quality_checks: true
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
@@ -2589,23 +2649,6 @@ cat > /tmp/munitor/templates/partials/java-webapp-deploy.yml.tpl << 'MUNITOR_PAR
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
- munitor/mvn_code_quality:
|
||||
name: code-quality
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
- munitor/mvn_coverage:
|
||||
name: coverage
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
- munitor/security_scan:
|
||||
name: security-scan
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
@@ -2638,7 +2681,6 @@ cat > /tmp/munitor/templates/partials/java-webapp-deploy.yml.tpl << 'MUNITOR_PAR
|
||||
sonar_project_key: ${MUNITOR_SONAR_PROJECT_KEY}
|
||||
requires:
|
||||
- build-and-test
|
||||
- coverage
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_SONAR}
|
||||
filters:
|
||||
@@ -2656,8 +2698,7 @@ cat > /tmp/munitor/templates/partials/java-webapp-deploy.yml.tpl << 'MUNITOR_PAR
|
||||
supplemental_images: '${MUNITOR_SUPPLEMENTAL_IMAGES_JSON}'
|
||||
##ENDIF_SUPPLEMENTAL##
|
||||
requires:
|
||||
- code-quality
|
||||
- coverage
|
||||
- build-and-test
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
@@ -2679,8 +2720,7 @@ cat > /tmp/munitor/templates/partials/java-webapp-deploy.yml.tpl << 'MUNITOR_PAR
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
requires:
|
||||
- code-quality
|
||||
- coverage
|
||||
- build-and-test
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
@@ -2717,6 +2757,20 @@ cat > /tmp/munitor/templates/partials/java-webapp-deploy.yml.tpl << 'MUNITOR_PAR
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
##ENDIF_CD_NON_PROD##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_SBOM##
|
||||
- sbom
|
||||
##ENDIF_SBOM##
|
||||
##IF_CD_NON_PROD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD_NON_PROD##
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
MUNITOR_PARTIAL_EOF
|
||||
|
||||
cat > /tmp/munitor/templates/partials/gradle-webapp-deploy.yml.tpl << 'MUNITOR_PARTIAL_EOF'
|
||||
@@ -3305,7 +3359,7 @@ validate_required_fields "${PIPELINE_TYPE}" "${CONFIG_FILE}"
|
||||
# --------------------------------------------------------------------------
|
||||
# Warn about unknown top-level keys (catches typos like java_verion)
|
||||
# --------------------------------------------------------------------------
|
||||
KNOWN_KEYS="pipeline orb_version org image_name server_module java_version node_version sonar docker cd coverage e2e sbom owasp contexts npm node services test terraform sast health kustomize kube_linter_config package_manager yamllint_paths supplemental_images"
|
||||
KNOWN_KEYS="pipeline orb_version org image_name server_module java_version node_version sonar docker cd coverage e2e sbom owasp contexts npm node services test terraform sast health kustomize kube_linter_config package_manager yamllint_paths supplemental_images ci"
|
||||
ACTUAL_KEYS=$(yq 'keys | .[]' "${CONFIG_FILE}" 2>/dev/null || true)
|
||||
for key in ${ACTUAL_KEYS}; do
|
||||
if ! echo "${KNOWN_KEYS}" | grep -qw "${key}"; then
|
||||
@@ -3318,6 +3372,17 @@ done
|
||||
# --------------------------------------------------------------------------
|
||||
TEMPLATE_FILE="${TEMPLATE_DIR}/${PIPELINE_TYPE}.yml.tpl"
|
||||
|
||||
# Opt in only after the consumer requires ci-ready instead of individual build jobs.
|
||||
# Missing history or any non-documentation path keeps the complete pipeline.
|
||||
if [[ "${PIPELINE_TYPE}" == "java-webapp" && "$(yq '.ci.docs_only // false' "${CONFIG_FILE}")" == "true" ]]; then
|
||||
# shellcheck source=docs_only.sh
|
||||
source "${SCRIPT_DIR}/docs_only.sh"
|
||||
if munitor_docs_only; then
|
||||
TEMPLATE_FILE="${TEMPLATE_DIR}/java-docs.yml.tpl"
|
||||
echo "Documentation-only change: retaining secret scanning; no application build or deployment."
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ ! -f "${TEMPLATE_FILE}" ]]; then
|
||||
echo "ERROR: No template found for pipeline type '${PIPELINE_TYPE}'."
|
||||
echo "Available templates:"
|
||||
|
||||
+34
-11
@@ -15,22 +15,45 @@ munitor_header "run_semgrep"
|
||||
|
||||
CONFIG="${SEMGREP_CONFIG:-auto}"
|
||||
FAIL_ON_FINDINGS="${SEMGREP_FAIL_ON_FINDINGS:-true}"
|
||||
REPORT="${SEMGREP_REPORT:-/tmp/semgrep-results.json}"
|
||||
|
||||
echo "Running Semgrep SAST scan (config: ${CONFIG}, fail_on_findings: ${FAIL_ON_FINDINGS})..."
|
||||
|
||||
EXIT_CODE=0
|
||||
semgrep ci \
|
||||
--config "${CONFIG}" \
|
||||
--json \
|
||||
--output /tmp/semgrep-results.json || {
|
||||
EXIT_CODE=$?
|
||||
if [[ "${FAIL_ON_FINDINGS}" == "true" ]]; then
|
||||
echo "ERROR: Semgrep found issues."
|
||||
echo "Review the report at: /tmp/semgrep-results.json"
|
||||
exit "${EXIT_CODE}"
|
||||
else
|
||||
echo "WARNING: Semgrep found issues (warn-only mode, not failing build)."
|
||||
echo "Review the report at: /tmp/semgrep-results.json"
|
||||
fi
|
||||
}
|
||||
--output "${REPORT}" || EXIT_CODE=$?
|
||||
|
||||
# Summarise the report, and fail a scan that ran no rules or scanned no files:
|
||||
# semgrep exits 0 when the downloaded rules fail to parse (an outdated semgrep),
|
||||
# which used to pass this job without scanning anything.
|
||||
python3 - "${REPORT}" <<'PY'
|
||||
import json, sys
|
||||
report = json.load(open(sys.argv[1]))
|
||||
scanned = len(report.get("paths", {}).get("scanned", []))
|
||||
results = report.get("results", [])
|
||||
errors = report.get("errors", [])
|
||||
rule_errors = [e for e in errors if "rule" in str(e.get("type", "")).lower()
|
||||
or "config" in str(e.get("message", "")).lower()]
|
||||
print(f"Semgrep: {scanned} files scanned, {len(results)} findings, {len(errors)} errors.")
|
||||
for r in results[:50]:
|
||||
print(f" {r.get('check_id')} {r.get('path')}:{r.get('start', {}).get('line')}")
|
||||
if scanned == 0 or rule_errors:
|
||||
for e in rule_errors[:5]:
|
||||
print(f" error: {str(e.get('message', e))[:300]}")
|
||||
print("ERROR: Semgrep did not run its rules (no files scanned or invalid rule configuration).")
|
||||
sys.exit(2)
|
||||
PY
|
||||
|
||||
if [[ "${EXIT_CODE}" -ne 0 ]]; then
|
||||
if [[ "${FAIL_ON_FINDINGS}" == "true" ]]; then
|
||||
echo "ERROR: Semgrep found issues."
|
||||
echo "Review the report at: ${REPORT}"
|
||||
exit "${EXIT_CODE}"
|
||||
fi
|
||||
echo "WARNING: Semgrep found issues (warn-only mode, not failing build)."
|
||||
echo "Review the report at: ${REPORT}"
|
||||
fi
|
||||
|
||||
echo "Semgrep scan completed."
|
||||
@@ -36,13 +36,17 @@ fi
|
||||
|
||||
echo "Running Trivy filesystem scan..."
|
||||
|
||||
# --exit-code 1: without it trivy exits 0 even when it reports HIGH/CRITICAL
|
||||
# findings, so this job could never fail.
|
||||
trivy fs \
|
||||
--format json \
|
||||
--output /tmp/trivy-results.json \
|
||||
--severity HIGH,CRITICAL \
|
||||
--exit-code 1 \
|
||||
. || {
|
||||
EXIT_CODE=$?
|
||||
echo "Trivy found vulnerabilities. See /tmp/trivy-results.json"
|
||||
trivy fs --severity HIGH,CRITICAL . || true
|
||||
exit "${EXIT_CODE}"
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
version: 2.1
|
||||
|
||||
orbs:
|
||||
munitor: ${MUNITOR_ORB_SLUG}@${MUNITOR_ORB_VERSION}
|
||||
|
||||
workflows:
|
||||
docs-checks:
|
||||
jobs:
|
||||
- munitor/secrets_scan:
|
||||
name: secrets-scan
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- secrets-scan
|
||||
@@ -8,6 +8,8 @@ workflows:
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
name: build-and-test
|
||||
quality_checks: true
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
@@ -45,27 +47,6 @@ workflows:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
- munitor/mvn_code_quality:
|
||||
name: code-quality
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
- munitor/mvn_coverage:
|
||||
name: coverage
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
- munitor/security_scan:
|
||||
name: security-scan
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
@@ -96,6 +77,23 @@ workflows:
|
||||
- /hotfix\/.*/
|
||||
##ENDIF_E2E##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- build-and-test
|
||||
- image-scan
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
##IF_E2E##
|
||||
- e2e-tests
|
||||
##ENDIF_E2E##
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /feature\/.*/
|
||||
- /hotfix\/.*/
|
||||
|
||||
##INCLUDE_DEPLOY java-webapp-deploy develop develop ${MUNITOR_CD_ENV_DEVELOP}##
|
||||
##INCLUDE_DEPLOY java-webapp-deploy staging staging ${MUNITOR_CD_ENV_STAGING}##
|
||||
|
||||
@@ -103,6 +101,8 @@ workflows:
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
name: build-and-test
|
||||
quality_checks: true
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
@@ -123,25 +123,6 @@ workflows:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
- munitor/mvn_code_quality:
|
||||
name: code-quality
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
- munitor/mvn_coverage:
|
||||
name: coverage
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
- munitor/security_scan:
|
||||
name: security-scan
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
@@ -176,7 +157,6 @@ workflows:
|
||||
sonar_project_key: ${MUNITOR_SONAR_PROJECT_KEY}
|
||||
requires:
|
||||
- build-and-test
|
||||
- coverage
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_SONAR}
|
||||
filters:
|
||||
@@ -197,8 +177,7 @@ workflows:
|
||||
supplemental_images: '${MUNITOR_SUPPLEMENTAL_IMAGES_JSON}'
|
||||
##ENDIF_SUPPLEMENTAL##
|
||||
requires:
|
||||
- code-quality
|
||||
- coverage
|
||||
- build-and-test
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
@@ -221,8 +200,7 @@ workflows:
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
requires:
|
||||
- code-quality
|
||||
- coverage
|
||||
- build-and-test
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
@@ -274,10 +252,28 @@ workflows:
|
||||
- /release\/.*/
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_SBOM##
|
||||
- sbom
|
||||
##ENDIF_SBOM##
|
||||
##IF_CD_NON_PROD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD_NON_PROD##
|
||||
##IF_GITHUB_RELEASE##
|
||||
- github-release
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
filters:
|
||||
branches:
|
||||
only:
|
||||
- /release\/.*/
|
||||
|
||||
production:
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
name: build-and-test
|
||||
name: production-build-and-test
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
@@ -297,7 +293,7 @@ workflows:
|
||||
supplemental_images: '${MUNITOR_SUPPLEMENTAL_IMAGES_JSON}'
|
||||
##ENDIF_SUPPLEMENTAL##
|
||||
requires:
|
||||
- build-and-test
|
||||
- production-build-and-test
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_REGISTRY}
|
||||
filters:
|
||||
@@ -337,3 +333,17 @@ workflows:
|
||||
branches:
|
||||
only: main
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_CD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD##
|
||||
##IF_GITHUB_RELEASE##
|
||||
- github-release
|
||||
##ENDIF_GITHUB_RELEASE##
|
||||
filters:
|
||||
branches:
|
||||
only: main
|
||||
@@ -2,6 +2,8 @@
|
||||
jobs:
|
||||
- munitor/mvn_build_and_test:
|
||||
name: build-and-test
|
||||
quality_checks: true
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
@@ -19,23 +21,6 @@
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
- munitor/mvn_code_quality:
|
||||
name: code-quality
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
- munitor/mvn_coverage:
|
||||
name: coverage
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
min_instruction: "${MUNITOR_COVERAGE_MIN}"
|
||||
requires:
|
||||
- build-and-test
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
- munitor/security_scan:
|
||||
name: security-scan
|
||||
java_version: "${MUNITOR_JAVA_VERSION}"
|
||||
@@ -68,7 +53,6 @@
|
||||
sonar_project_key: ${MUNITOR_SONAR_PROJECT_KEY}
|
||||
requires:
|
||||
- build-and-test
|
||||
- coverage
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_SONAR}
|
||||
filters:
|
||||
@@ -86,8 +70,7 @@
|
||||
supplemental_images: '${MUNITOR_SUPPLEMENTAL_IMAGES_JSON}'
|
||||
##ENDIF_SUPPLEMENTAL##
|
||||
requires:
|
||||
- code-quality
|
||||
- coverage
|
||||
- build-and-test
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
@@ -109,8 +92,7 @@
|
||||
context:
|
||||
- ${MUNITOR_CONTEXT_GITHUB}
|
||||
requires:
|
||||
- code-quality
|
||||
- coverage
|
||||
- build-and-test
|
||||
- secrets-scan
|
||||
- sast-scan
|
||||
- security-scan
|
||||
@@ -147,3 +129,17 @@
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
##ENDIF_CD_NON_PROD##
|
||||
|
||||
- munitor/ci_ready:
|
||||
name: ci-ready
|
||||
requires:
|
||||
- docker-build-push
|
||||
##IF_SBOM##
|
||||
- sbom
|
||||
##ENDIF_SBOM##
|
||||
##IF_CD_NON_PROD##
|
||||
- update-cd-repo
|
||||
##ENDIF_CD_NON_PROD##
|
||||
filters:
|
||||
branches:
|
||||
only: __BRANCH_FILTER__
|
||||
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/env bats
|
||||
|
||||
setup() {
|
||||
ROOT="${BATS_TEST_DIRNAME}/.."
|
||||
CONFIG="${BATS_TEST_TMPDIR}/rendered.yml"
|
||||
}
|
||||
|
||||
check_workflows() {
|
||||
local expected="$1"
|
||||
yq -o=json '.' "$CONFIG" | EXPECTED_COVERAGE="$expected" node -e '
|
||||
const assert = require("node:assert/strict");
|
||||
const config = JSON.parse(require("node:fs").readFileSync(0,"utf8"));
|
||||
for (const [workflow, definition] of Object.entries(config.workflows)) {
|
||||
const jobs = definition.jobs.map(entry => {
|
||||
const [type, options] = Object.entries(entry)[0];
|
||||
assert.ok(!["munitor/mvn_code_quality", "munitor/mvn_coverage"].includes(type));
|
||||
return {type, ...options};
|
||||
});
|
||||
const names = jobs.map(job => job.name);
|
||||
assert.equal(new Set(names).size, names.length, workflow);
|
||||
const buildName = workflow === "production" ? "production-build-and-test" : "build-and-test";
|
||||
const build = jobs.find(job => job.name === buildName);
|
||||
assert.ok(build, workflow);
|
||||
assert.equal(jobs.filter(job => job.type === "munitor/mvn_build_and_test").length, 1);
|
||||
assert.ok(!names.includes(workflow === "production" ? "build-and-test" : "production-build-and-test"));
|
||||
const expectedFilter = {
|
||||
"pr-checks": ["/feature\\/.*/", "/hotfix\\/.*/"],
|
||||
develop: "develop", staging: "staging",
|
||||
"release-candidate": ["/release\\/.*/"], production: "main"
|
||||
}[workflow];
|
||||
assert.deepEqual(build.filters.branches.only, expectedFilter, workflow);
|
||||
if (workflow === "production") {
|
||||
assert.equal(build.quality_checks, undefined);
|
||||
assert.equal(build.min_instruction, undefined);
|
||||
} else {
|
||||
assert.equal(build.quality_checks, true, workflow);
|
||||
assert.equal(String(build.min_instruction), process.env.EXPECTED_COVERAGE, workflow);
|
||||
for (const gate of ["secrets-scan", "sast-scan", "security-scan"]) assert.ok(names.includes(gate), workflow);
|
||||
}
|
||||
for (const job of jobs) {
|
||||
assert.deepEqual(job.filters, build.filters, `${workflow}/${job.name} filters`);
|
||||
const dependencies = job.requires || [];
|
||||
assert.equal(new Set(dependencies).size, dependencies.length, job.name);
|
||||
for (const name of dependencies) assert.ok(names.includes(name), `${workflow}/${job.name}: ${name}`);
|
||||
if (["image-scan", "security-scan", "e2e-tests", "sonar-scan"].includes(job.name)) assert.ok(dependencies.includes(buildName));
|
||||
if (["docker-build-push", "sbom"].includes(job.name)) {
|
||||
assert.ok(dependencies.includes(buildName));
|
||||
if (workflow !== "production") for (const gate of ["secrets-scan", "sast-scan", "security-scan", "e2e-tests", "sonar-scan"].filter(name => names.includes(name))) assert.ok(dependencies.includes(gate), `${workflow}/${job.name}/${gate}`);
|
||||
}
|
||||
if (["update-cd-repo", "github-release"].includes(job.name)) assert.ok(dependencies.includes("docker-build-push"));
|
||||
}
|
||||
if (workflow === "pr-checks") assert.ok(names.includes("image-scan"));
|
||||
}
|
||||
for (const workflow of ["pr-checks", "develop", "staging", "release-candidate", "production"]) assert.ok(config.workflows[workflow]);
|
||||
'
|
||||
}
|
||||
|
||||
@test "full Java workflow folds quality into build and preserves optional gates" {
|
||||
MUNITOR_CONFIG="${ROOT}/tests/fixtures/java-webapp.munitor.yml" MUNITOR_OUTPUT_FILE="$CONFIG" bash "${ROOT}/src/scripts/generate_config.sh"
|
||||
check_workflows 80
|
||||
[ "$(yq '.workflows.develop.jobs[] | .[] | select(.name == "sonar-scan") | .name' "$CONFIG")" = "sonar-scan" ]
|
||||
}
|
||||
|
||||
@test "minimal Java workflow retains default coverage and production behavior" {
|
||||
MUNITOR_CONFIG="${ROOT}/tests/fixtures/java-webapp-minimal.munitor.yml" MUNITOR_OUTPUT_FILE="$CONFIG" bash "${ROOT}/src/scripts/generate_config.sh"
|
||||
check_workflows 70
|
||||
}
|
||||
|
||||
@test "quality steps finish before workspace persistence and forward coverage threshold" {
|
||||
yq -o=json '.' "${ROOT}/src/jobs/mvn_build_and_test.yml" | node -e '
|
||||
const assert = require("node:assert/strict");
|
||||
const job = JSON.parse(require("node:fs").readFileSync(0,"utf8"));
|
||||
assert.equal(job.parameters.quality_checks.default, false);
|
||||
assert.equal(job.parameters.min_instruction.default, "70");
|
||||
const index = name => job.steps.findIndex(step => step === name || typeof step === "object" && name in step);
|
||||
assert.ok(index("mvn_test") < index("when"));
|
||||
assert.ok(index("when") < index("persist_to_workspace"));
|
||||
const block = job.steps[index("when")].when;
|
||||
assert.equal(block.condition, "<< parameters.quality_checks >>");
|
||||
const commands = block.steps.map(step => typeof step === "string" ? step : Object.keys(step)[0]);
|
||||
assert.deepEqual(commands.slice(0,4), ["mvn_checkstyle", "mvn_spotbugs", "mvn_pmd", "mvn_jacoco"]);
|
||||
assert.equal(block.steps[3].mvn_jacoco.min_instruction, "<< parameters.min_instruction >>");
|
||||
'
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env bats
|
||||
|
||||
setup() {
|
||||
SCRIPT="${BATS_TEST_DIRNAME}/../src/scripts/maven_cache_key.sh"
|
||||
cd "$BATS_TEST_TMPDIR"
|
||||
git init -q
|
||||
printf '<project><version>1-SNAPSHOT</version></project>\n' > pom.xml
|
||||
git add pom.xml
|
||||
}
|
||||
|
||||
@test "release rewrites retain the restored cache identity" {
|
||||
bash "$SCRIPT"
|
||||
cp .munitor-maven-cache-key before
|
||||
printf '<project><version>1.2.3</version></project>\n' > pom.xml
|
||||
bash "$SCRIPT"
|
||||
cmp before .munitor-maven-cache-key
|
||||
}
|
||||
|
||||
@test "nested checked-out dependency changes invalidate the cache" {
|
||||
mkdir module
|
||||
printf '<project><dependencies/></project>\n' > module/pom.xml
|
||||
git add module/pom.xml
|
||||
bash "$SCRIPT"
|
||||
cp .munitor-maven-cache-key before
|
||||
printf '<project><dependencies>changed</dependencies></project>\n' > module/pom.xml
|
||||
git add module/pom.xml
|
||||
bash "$SCRIPT"
|
||||
! cmp -s before .munitor-maven-cache-key
|
||||
}
|
||||
|
||||
@test "source changes do not invalidate dependency cache" {
|
||||
bash "$SCRIPT"
|
||||
cp .munitor-maven-cache-key before
|
||||
printf 'class Example {}\n' > Example.java
|
||||
git add Example.java
|
||||
bash "$SCRIPT"
|
||||
cmp before .munitor-maven-cache-key
|
||||
}
|
||||
|
||||
@test "restore has no broad fallback and save uses the same key" {
|
||||
command_root="${BATS_TEST_DIRNAME}/../src/commands"
|
||||
restore=$(sed -n '/maven-v2-/p' "$command_root/restore_maven_cache.yml")
|
||||
save=$(sed -n '/maven-v2-/p' "$command_root/save_maven_cache.yml")
|
||||
[[ "$restore" == *'maven-v2-{{ checksum ".munitor-maven-cache-key" }}'* ]]
|
||||
[[ "$save" == *'maven-v2-{{ checksum ".munitor-maven-cache-key" }}'* ]]
|
||||
! grep -q 'maven-v1\|maven-v2-$' "$command_root/restore_maven_cache.yml"
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
#!/usr/bin/env bats
|
||||
|
||||
setup() {
|
||||
SCRIPT="${BATS_TEST_DIRNAME}/../src/scripts/maven_workspace.sh"
|
||||
export HOME="${BATS_TEST_TMPDIR}/home"
|
||||
cd "$BATS_TEST_TMPDIR"
|
||||
}
|
||||
|
||||
@test "workspace overlays current installed artifacts and snapshot metadata without copying remote dependencies" {
|
||||
repo="$HOME/.m2/repository"
|
||||
mkdir -p "$repo/example/library/2-SNAPSHOT" "$repo/vendor/dependency/1"
|
||||
printf 'library-2-SNAPSHOT.jar>=\r\nlibrary-2-SNAPSHOT.pom>=\r\nlibrary-2-SNAPSHOT-tests.jar>=\r\n' > "$repo/example/library/2-SNAPSHOT/_remote.repositories"
|
||||
for extension in jar pom; do
|
||||
printf 'current build\n' > "$repo/example/library/2-SNAPSHOT/library-2-SNAPSHOT.$extension"
|
||||
done
|
||||
printf 'current test classes\n' > "$repo/example/library/2-SNAPSHOT/library-2-SNAPSHOT-tests.jar"
|
||||
printf 'snapshot metadata\n' > "$repo/example/library/maven-metadata-local.xml"
|
||||
printf 'plugin prefix metadata\n' > "$repo/example/maven-metadata-local.xml"
|
||||
printf 'dependency-1.jar>central=\n' > "$repo/vendor/dependency/1/_remote.repositories"
|
||||
printf 'remote dependency\n' > "$repo/vendor/dependency/1/dependency-1.jar"
|
||||
mkdir -p "$repo/vendor/mutable/2-SNAPSHOT"
|
||||
printf 'mutable-2-SNAPSHOT.jar>central=\n' > "$repo/vendor/mutable/2-SNAPSHOT/_remote.repositories"
|
||||
printf 'updated snapshot\n' > "$repo/vendor/mutable/2-SNAPSHOT/mutable-2-SNAPSHOT.jar"
|
||||
MAVEN_WORKSPACE_ACTION=save bash "$SCRIPT"
|
||||
run tar -tzf .munitor-maven-installed.tgz
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" != *vendor/dependency* ]]
|
||||
export HOME="${BATS_TEST_TMPDIR}/downstream"
|
||||
bash "$SCRIPT"
|
||||
restored="$HOME/.m2/repository/example/library"
|
||||
cmp "$repo/example/library/2-SNAPSHOT/library-2-SNAPSHOT.jar" "$restored/2-SNAPSHOT/library-2-SNAPSHOT.jar"
|
||||
cmp "$repo/example/library/maven-metadata-local.xml" "$restored/maven-metadata-local.xml"
|
||||
cmp "$repo/example/maven-metadata-local.xml" "$restored/../maven-metadata-local.xml"
|
||||
[ -f "$restored/2-SNAPSHOT/library-2-SNAPSHOT-tests.jar" ]
|
||||
[ ! -e "$HOME/.m2/repository/vendor/dependency" ]
|
||||
cmp "$repo/vendor/mutable/2-SNAPSHOT/mutable-2-SNAPSHOT.jar" "$HOME/.m2/repository/vendor/mutable/2-SNAPSHOT/mutable-2-SNAPSHOT.jar"
|
||||
}
|
||||
|
||||
@test "initial build without a workspace archive needs no overlay" {
|
||||
run bash "$SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
// Public CI config only: no network, credentials or publication.
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const config = JSON.parse(fs.readFileSync(0, 'utf8'));
|
||||
const workflows = Object.values(config.workflows);
|
||||
assert.equal(workflows.length, 1, 'all publishers must share the checked pack workspace');
|
||||
const jobs = workflows.flatMap(workflow => workflow.jobs).map(job => {
|
||||
const [type, settings] = Object.entries(job)[0];
|
||||
return { type, ...settings, name: settings.name || type };
|
||||
});
|
||||
assert.equal(jobs.filter(job => job.type === 'orb-tools/pack').length, 1, 'pack once per pipeline');
|
||||
const gates = ['orb-tools/lint', 'orb-tools/pack', 'orb-tools/review', 'test'];
|
||||
for (const gate of gates) assert.equal(jobs.filter(job => job.name === gate).length, 1, `retain required context ${gate}`);
|
||||
const publishers = jobs.filter(job => job.type === 'orb-tools/publish');
|
||||
assert.equal(publishers.length, 3);
|
||||
for (const job of publishers) {
|
||||
assert.deepEqual([...job.requires].sort(), [...gates].sort(), `${job.name} must wait for every gate`);
|
||||
assert.equal(job.orb_name, 'kof22/munitor');
|
||||
assert.deepEqual(job.context, ['orb-publishing']);
|
||||
}
|
||||
function matches(pattern, value) {
|
||||
return pattern.startsWith('/') && pattern.endsWith('/')
|
||||
? new RegExp(`^(?:${pattern.slice(1, -1)})$`).test(value) : pattern === value;
|
||||
}
|
||||
function runs(job, kind, ref) {
|
||||
const filter = job.filters?.[kind];
|
||||
if (!filter) return kind === 'branches'; // CircleCI ignores tags unless opted in.
|
||||
const list = value => Array.isArray(value) ? value : [value];
|
||||
return (!filter.only || list(filter.only).some(pattern => matches(pattern, ref)))
|
||||
&& (!filter.ignore || !list(filter.ignore).some(pattern => matches(pattern, ref)));
|
||||
}
|
||||
const cases = [
|
||||
['branches', 'develop', ['publish-develop-snapshot']],
|
||||
['branches', 'feature/ci-speed', ['publish-feature-sha']],
|
||||
['branches', 'main', []], ['branches', 'hotfix/1.2.3', []],
|
||||
['branches', 'release/1.2.3', []], ['branches', 'fix/ci-speed', []],
|
||||
['tags', 'v1.2.3', ['orb-tools/publish']],
|
||||
['tags', 'v1.2.3-RC.1', []], ['tags', 'v1.2', []], ['tags', 'other', []],
|
||||
];
|
||||
for (const [kind, ref, expected] of cases) {
|
||||
assert.deepEqual(publishers.filter(job => runs(job, kind, ref)).map(job => job.name).sort(), expected.sort(), `${kind}:${ref}`);
|
||||
for (const gate of gates) assert.ok(runs(jobs.find(job => job.name === gate), kind, ref), `${gate} must run for ${ref}`);
|
||||
}
|
||||
assert.equal(jobs.find(job => job.name === 'publish-develop-snapshot').dev_tags, 'dev:${CIRCLE_SHA1},dev:snapshot');
|
||||
assert.equal(jobs.find(job => job.name === 'publish-feature-sha').dev_tags, 'dev:${CIRCLE_SHA1}');
|
||||
assert.equal(jobs.find(job => job.name === 'orb-tools/publish').pub_type, 'production');
|
||||
assert.equal(jobs.find(job => job.name === 'publish-feature-sha').pub_type, 'dev');
|
||||
assert.equal(jobs.find(job => job.name === 'publish-develop-snapshot').pub_type, 'dev');
|
||||
console.log('PASS: one pack, unchanged required contexts, all publish gates and 10 branch/tag cases');
|
||||
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env bats
|
||||
|
||||
setup() {
|
||||
export JOB="${BATS_TEST_DIRNAME}/../src/jobs/security_scan.yml"
|
||||
}
|
||||
|
||||
# Resolve only this job's compile-time OWASP condition, rejecting unknown ones.
|
||||
resolved_steps() {
|
||||
yq -o=json '.' "$JOB" | jq --argjson enabled "$1" '
|
||||
def expand:
|
||||
.[] | if type == "object" and has("when") then
|
||||
if .when.condition != "<< parameters.owasp >>" then error("unknown condition")
|
||||
elif $enabled then .when.steps | expand else empty end
|
||||
else . end;
|
||||
[.steps | expand]'
|
||||
}
|
||||
|
||||
@test "OWASP disabled skips Java Maven and NVD but retains Trivy and its artifact" {
|
||||
run resolved_steps false
|
||||
[ "$status" -eq 0 ]
|
||||
echo "$output" | jq -e '
|
||||
map(if type == "string" then . else keys[0] end) ==
|
||||
["attach_workspace", "restore_trivy_cache", "run_trivy", "save_trivy_cache", "store_artifacts"]' >/dev/null
|
||||
echo "$output" | jq -e '.[-1].store_artifacts.path == "/tmp/trivy-results.json"' >/dev/null
|
||||
}
|
||||
|
||||
@test "OWASP enabled retains ordered Java Maven NVD scan and report workflow" {
|
||||
run resolved_steps true
|
||||
[ "$status" -eq 0 ]
|
||||
echo "$output" | jq -e '
|
||||
map(if type == "string" then . else keys[0] end) ==
|
||||
["attach_workspace", "install_java", "restore_maven_cache", "restore_cache",
|
||||
"mvn_owasp", "save_cache", "restore_trivy_cache", "run_trivy", "save_trivy_cache",
|
||||
"store_artifacts", "store_artifacts"]' >/dev/null
|
||||
echo "$output" | jq -e '
|
||||
.[1].install_java.version == "<< parameters.java_version >>" and
|
||||
.[3].restore_cache.keys == ["nvd-cache-v1-{{ .Branch }}", "nvd-cache-v1-"] and
|
||||
.[5].save_cache.paths == ["~/.dependency-check"] and
|
||||
.[9].store_artifacts.path == "target/dependency-check-report.html" and
|
||||
.[10].store_artifacts.path == "/tmp/trivy-results.json"' >/dev/null
|
||||
}
|
||||
|
||||
@test "OWASP remains enabled by default" {
|
||||
[ "$(yq '.parameters.owasp.default' "$JOB")" = true ]
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
#!/usr/bin/env bash
|
||||
# Exercise the real generator and validate its complete dependency graph.
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
GENERATOR="${SCRIPT_DIR}/../src/scripts/generate_config.sh"
|
||||
WORK=$(mktemp -d)
|
||||
trap 'rm -rf -- "$WORK"' EXIT
|
||||
export CIRCLE_BRANCH=main CIRCLE_TAG=""
|
||||
export GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null
|
||||
export GIT_AUTHOR_NAME=Fixture GIT_AUTHOR_EMAIL=fixture@example.invalid
|
||||
export GIT_COMMITTER_NAME=Fixture GIT_COMMITTER_EMAIL=fixture@example.invalid
|
||||
PASS=0
|
||||
|
||||
render_check() {
|
||||
local config="$1" mode="$2" label="$3" generator="${4:-$GENERATOR}"
|
||||
MUNITOR_CONFIG="$config" MUNITOR_OUTPUT_FILE="${WORK}/generated.yml" \
|
||||
bash "$generator" >"${WORK}/generator.log" 2>&1 || { cat "${WORK}/generator.log"; return 1; }
|
||||
yq -o=json '.' "${WORK}/generated.yml" >"${WORK}/generated.json"
|
||||
python3 - "${WORK}/generated.json" "$mode" <<'PY'
|
||||
import json, sys
|
||||
data = json.load(open(sys.argv[1], encoding='utf-8'))
|
||||
workflows = data['workflows']
|
||||
assert workflows, 'no generated workflows'
|
||||
if sys.argv[2] == 'docs':
|
||||
assert set(workflows) == {'docs-checks'}, workflows.keys()
|
||||
else:
|
||||
assert {'pr-checks', 'production', 'release-candidate'} <= set(workflows), workflows.keys()
|
||||
assert 'docs-checks' not in workflows
|
||||
for workflow, body in workflows.items():
|
||||
jobs = {}
|
||||
filters = {}
|
||||
for invocation in body['jobs']:
|
||||
if isinstance(invocation, str):
|
||||
command, options = invocation, {}
|
||||
else:
|
||||
assert len(invocation) == 1
|
||||
command, options = next(iter(invocation.items()))
|
||||
options = options or {}
|
||||
name = options.get('name', command)
|
||||
assert name not in jobs, (workflow, 'duplicate job', name)
|
||||
jobs[name] = (command, options.get('requires', []))
|
||||
filters[name] = options.get('filters', {})
|
||||
assert 'ci-ready' in jobs, (workflow, 'missing ci-ready')
|
||||
assert jobs['ci-ready'][0] == 'munitor/ci_ready'
|
||||
for name, job_filters in filters.items():
|
||||
assert job_filters == filters['ci-ready'], (workflow, name, 'filters differ from ci-ready', job_filters, filters['ci-ready'])
|
||||
for name, (_, dependencies) in jobs.items():
|
||||
assert isinstance(dependencies, list)
|
||||
assert set(dependencies) <= jobs.keys(), (workflow, name, 'dangling dependency', dependencies)
|
||||
def closure(name, ancestors):
|
||||
assert name not in ancestors, (workflow, 'dependency cycle', name)
|
||||
reached = set()
|
||||
for dependency in jobs[name][1]:
|
||||
reached.add(dependency)
|
||||
reached.update(closure(dependency, ancestors | {name}))
|
||||
return reached
|
||||
assert closure('ci-ready', set()) == set(jobs) - {'ci-ready'}, (workflow, 'ungated job')
|
||||
if sys.argv[2] == 'docs':
|
||||
assert set(jobs) == {'secrets-scan', 'ci-ready'}
|
||||
assert jobs['secrets-scan'][0] == 'munitor/secrets_scan'
|
||||
else:
|
||||
assert any(command == 'munitor/mvn_build_and_test' for command, _ in jobs.values()), workflow
|
||||
print('validated', len(workflows), 'workflow graphs')
|
||||
PY
|
||||
printf 'PASS: %s\n' "$label"
|
||||
PASS=$((PASS + 1))
|
||||
}
|
||||
|
||||
render_check "${SCRIPT_DIR}/fixtures/java-webapp.munitor.yml" full 'full Java fixture gates every job'
|
||||
render_check "${SCRIPT_DIR}/fixtures/java-webapp-minimal.munitor.yml" full 'minimal Java fixture has no dangling optional dependencies'
|
||||
# Disable all optional publication/scanner additions as well as e2e/SBOM.
|
||||
cat >"${WORK}/minimal.yml" <<'YAML'
|
||||
pipeline: java-webapp
|
||||
orb_version: dev:snapshot
|
||||
image_name: fixture
|
||||
java_version: "21"
|
||||
docker:
|
||||
registry: ghcr.io/example
|
||||
e2e: false
|
||||
sbom: false
|
||||
owasp: false
|
||||
sast: false
|
||||
YAML
|
||||
render_check "${WORK}/minimal.yml" full 'optional jobs absent without dangling gates'
|
||||
|
||||
git init -q -b main "${WORK}/repo"
|
||||
cd "${WORK}/repo"
|
||||
cp "${WORK}/minimal.yml" .munitor.yml
|
||||
printf '\nci:\n docs_only: true\n' >> .munitor.yml
|
||||
printf 'original\n' > README.md
|
||||
mkdir src
|
||||
printf 'original\n' > src/App.java
|
||||
git add .
|
||||
git -c commit.gpgsign=false commit -qm baseline
|
||||
printf 'documentation\n' >> README.md
|
||||
git add README.md
|
||||
git -c commit.gpgsign=false commit -qm documentation
|
||||
render_check "$PWD/.munitor.yml" docs 'opted-in main README-only change retains only secrets and ci-ready'
|
||||
# The shipped script stages /tmp/munitor helpers; invoke sequentially, never in parallel.
|
||||
render_check "$PWD/.munitor.yml" docs 'packed generator retains docs-only secrets and ci-ready' \
|
||||
"${SCRIPT_DIR}/../src/scripts/packed_generate_config.sh"
|
||||
|
||||
# Opt-out must retain the full workflow even with the same docs-only Git diff.
|
||||
yq '.ci.docs_only = false' .munitor.yml >"${WORK}/optout.yml"
|
||||
render_check "${WORK}/optout.yml" full 'docs change with opt-out still builds'
|
||||
printf 'source change\n' >> src/App.java
|
||||
git add src/App.java
|
||||
git -c commit.gpgsign=false commit -qm source
|
||||
render_check "$PWD/.munitor.yml" full 'source change still builds'
|
||||
printf '\n# configuration change\n' >> .munitor.yml
|
||||
git add .munitor.yml
|
||||
git -c commit.gpgsign=false commit -qm configuration
|
||||
render_check "$PWD/.munitor.yml" full 'configuration change still builds'
|
||||
printf '%s ci-ready generator checks passed\n' "$PASS"
|
||||
@@ -57,11 +57,11 @@ else
|
||||
fail
|
||||
fi
|
||||
|
||||
echo -n " TEST: push=false builds and scans every supplemental without publishing... "
|
||||
echo -n " TEST: push=false refreshes both Dockerfile forms and scans without publishing... "
|
||||
: > "${CALL_LOG}"
|
||||
echo 'export DOCKER_ENV_TAG=develop' >> "${BASH_ENV}"
|
||||
if PUSH_IMAGES=false bash "${SRC_SCRIPTS}/build_push_supplemental.sh" > "${WORK}/supplemental.log" &&
|
||||
[[ $(grep -c '^docker build ' "${CALL_LOG}") == 2 ]] &&
|
||||
[[ $(grep -c '^docker build --pull --no-cache -f ' "${CALL_LOG}") == 2 ]] &&
|
||||
[[ $(grep -c '^trivy image .*--severity HIGH,CRITICAL --exit-code 1 ' "${CALL_LOG}") == 2 ]] &&
|
||||
! grep -Eq '^docker (push|login|tag|buildx) ' "${CALL_LOG}" &&
|
||||
[[ -s "${WORK}/migrations.json" && -s "${WORK}/worker.json" ]]; then
|
||||
@@ -70,11 +70,16 @@ else
|
||||
fail
|
||||
fi
|
||||
|
||||
echo -n " TEST: default push still publishes version and environment tags... "
|
||||
echo -n " TEST: default push refreshes both Dockerfile forms and scans before publishing tags... "
|
||||
: > "${CALL_LOG}"
|
||||
unset PUSH_IMAGES
|
||||
if bash "${SRC_SCRIPTS}/build_push_supplemental.sh" > "${WORK}/supplemental.log" &&
|
||||
[[ $(grep -c '^docker push ' "${CALL_LOG}") == 4 ]]; then
|
||||
[[ $(grep -c '^docker build --pull --no-cache -f ' "${CALL_LOG}") == 2 ]] &&
|
||||
[[ $(grep -c '^docker push ' "${CALL_LOG}") == 4 ]] &&
|
||||
awk '
|
||||
/^trivy image / { image=$NF; sub(/:[^:]+$/, "", image); scanned[image]=1 }
|
||||
/^docker push / { image=$NF; sub(/:[^:]+$/, "", image); if (!scanned[image]) exit 1 }
|
||||
' "${CALL_LOG}"; then
|
||||
pass
|
||||
else
|
||||
fail
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# shellcheck source=src/scripts/docs_only.sh
|
||||
source "${SCRIPT_DIR}/../src/scripts/docs_only.sh"
|
||||
WORK=$(mktemp -d)
|
||||
trap 'rm -rf -- "$WORK"' EXIT
|
||||
export GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null
|
||||
export GIT_AUTHOR_NAME=Fixture GIT_AUTHOR_EMAIL=fixture@example.invalid
|
||||
export GIT_COMMITTER_NAME=Fixture GIT_COMMITTER_EMAIL=fixture@example.invalid
|
||||
export GIT_TERMINAL_PROMPT=0
|
||||
export CIRCLE_BRANCH=feature/docs CIRCLE_TAG=""
|
||||
PASS=0
|
||||
git init -q --bare "${WORK}/origin.git"
|
||||
git init -q -b main "${WORK}/seed"
|
||||
cd "${WORK}/seed"
|
||||
mkdir -p src docs/nested
|
||||
printf 'base\n' > README.md
|
||||
printf 'base\n' > src/app.java
|
||||
printf 'base\n' > docs/old.md
|
||||
git add .
|
||||
git -c commit.gpgsign=false commit -qm base
|
||||
git remote add origin "${WORK}/origin.git"
|
||||
git push -q origin main
|
||||
git --git-dir="${WORK}/origin.git" symbolic-ref HEAD refs/heads/main
|
||||
|
||||
fresh() {
|
||||
local repo="${WORK}/case-${PASS}"
|
||||
git clone -q "${WORK}/origin.git" "$repo"
|
||||
cd "$repo"
|
||||
git switch -qc feature/docs
|
||||
export CIRCLE_BRANCH=feature/docs CIRCLE_TAG=""
|
||||
}
|
||||
|
||||
commit() {
|
||||
git add -A
|
||||
git -c commit.gpgsign=false commit -qm change
|
||||
}
|
||||
|
||||
check() {
|
||||
local expected="$1" name="$2" status=0 output
|
||||
output=$(munitor_docs_only 2>&1) || status=$?
|
||||
if [[ "$status" -ne "$expected" || -n "$output" ]]; then
|
||||
printf 'FAIL: %s (status %s, expected %s; output must be empty)\n' "$name" "$status" "$expected"
|
||||
exit 1
|
||||
fi
|
||||
printf 'PASS: %s\n' "$name"
|
||||
PASS=$((PASS + 1))
|
||||
}
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
mkdir -p docs/nested
|
||||
printf 'docs\n' > 'docs/nested/space name.md'
|
||||
printf 'docs\n' > $'docs/nested/line\nbreak.md'
|
||||
printf 'docs\n' > CHANGELOG.md
|
||||
printf 'docs\n' > CONTRIBUTING.md
|
||||
commit
|
||||
check 0 'feature docs with nested, spaced and newline paths'
|
||||
CIRCLE_BRANCH=hotfix/docs check 0 'hotfix docs'
|
||||
for branch in develop staging release/1.0 unknown ''; do
|
||||
CIRCLE_BRANCH="$branch" check 1 "ineligible branch: $branch"
|
||||
done
|
||||
CIRCLE_TAG=v1.0 check 1 'tag cannot skip release'
|
||||
|
||||
fresh
|
||||
check 1 'empty branch diff'
|
||||
printf 'code\n' >> src/app.java
|
||||
commit
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
check 1 'earlier code commit followed by latest docs commit'
|
||||
|
||||
fresh
|
||||
printf 'code\n' >> src/app.java
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
check 1 'mixed source and docs'
|
||||
|
||||
fresh
|
||||
git mv src/app.java docs/disguised.md
|
||||
commit
|
||||
check 1 'code renamed into allowed docs path'
|
||||
|
||||
fresh
|
||||
git mv docs/old.md docs/new.md
|
||||
commit
|
||||
check 0 'docs-only rename'
|
||||
|
||||
fresh
|
||||
git rm -q src/app.java
|
||||
commit
|
||||
check 1 'deleted code'
|
||||
|
||||
fresh
|
||||
git rm -q docs/old.md
|
||||
commit
|
||||
check 0 'deleted documentation'
|
||||
|
||||
for path in pom.xml .circleci/config.yml docs/image.png docs/example.java readme.md nested/README.md; do
|
||||
fresh
|
||||
mkdir -p "$(dirname "$path")"
|
||||
printf 'not docs\n' > "$path"
|
||||
commit
|
||||
check 1 "non-allowlisted path: $path"
|
||||
done
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
git remote remove origin
|
||||
check 1 'missing origin fails closed'
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
git remote set-url origin "${WORK}/missing.git"
|
||||
check 1 'fetch failure fails closed despite cached origin/main'
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
# Advance origin main after this checkout; an old tracking ref is not enough.
|
||||
cd "${WORK}/seed"
|
||||
printf 'main\n' >> README.md
|
||||
commit
|
||||
git push -q origin main
|
||||
latest=$(git rev-parse HEAD)
|
||||
cd "${WORK}/case-${PASS}"
|
||||
check 0 'fetch updated main then compare merge-base'
|
||||
[[ "$(git rev-parse origin/main)" == "$latest" ]]
|
||||
|
||||
fresh
|
||||
git switch -q main
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
CIRCLE_BRANCH=main check 0 'main docs commit compares first parent'
|
||||
printf 'code\n' >> src/app.java
|
||||
commit
|
||||
CIRCLE_BRANCH=main check 1 'main source commit'
|
||||
|
||||
fresh
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
git switch -q main
|
||||
git -c commit.gpgsign=false merge --no-ff -qm 'docs merge' feature/docs
|
||||
CIRCLE_BRANCH=main check 0 'main docs merge uses first parent'
|
||||
|
||||
fresh
|
||||
printf 'code\n' >> src/app.java
|
||||
commit
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
git switch -q main
|
||||
git -c commit.gpgsign=false merge --no-ff -qm 'mixed merge' feature/docs
|
||||
CIRCLE_BRANCH=main check 1 'main merge includes whole source change'
|
||||
|
||||
git init -q -b main "${WORK}/root-only"
|
||||
cd "${WORK}/root-only"
|
||||
printf 'docs\n' > README.md
|
||||
commit
|
||||
CIRCLE_BRANCH=main check 1 'root commit has no parent'
|
||||
git remote add origin "${WORK}/origin.git"
|
||||
CIRCLE_BRANCH=feature/docs check 1 'unrelated history has no merge base'
|
||||
|
||||
git clone -q --depth=1 "file://${WORK}/origin.git" "${WORK}/shallow"
|
||||
cd "${WORK}/shallow"
|
||||
printf 'docs\n' >> README.md
|
||||
commit
|
||||
CIRCLE_BRANCH=main check 1 'shallow history fails closed'
|
||||
printf '%s docs-only checks passed\n' "$PASS"
|
||||
@@ -8,8 +8,8 @@ PASS=0
|
||||
FAIL=0
|
||||
|
||||
GENERATE_SCRIPT="${PROJECT_DIR}/src/scripts/generate_config.sh"
|
||||
GENERATED_CONFIG=$(mktemp)
|
||||
trap 'rm -f "${GENERATED_CONFIG}"' EXIT
|
||||
GENERATED_DIR=$(mktemp -d)
|
||||
trap 'rm -rf "${GENERATED_DIR}"' EXIT
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Helper: render a fixture by calling the real generate_config.sh script
|
||||
@@ -17,11 +17,15 @@ trap 'rm -f "${GENERATED_CONFIG}"' EXIT
|
||||
# --------------------------------------------------------------------------
|
||||
render_fixture() {
|
||||
local fixture="$1"
|
||||
local generated_config="${GENERATED_DIR}/$(basename "${fixture}")"
|
||||
|
||||
# Run the real script; capture stderr for diagnostics, return generated YAML
|
||||
MUNITOR_CONFIG="${fixture}" MUNITOR_OUTPUT_FILE="${GENERATED_CONFIG}" \
|
||||
bash "${GENERATE_SCRIPT}" >/dev/null 2>&1
|
||||
cat "${GENERATED_CONFIG}"
|
||||
# Fixtures are immutable and uniquely named: render once, assert many times.
|
||||
if [[ ! -f "${generated_config}" ]]; then
|
||||
MUNITOR_CONFIG="${fixture}" MUNITOR_OUTPUT_FILE="${generated_config}.tmp" \
|
||||
bash "${GENERATE_SCRIPT}" >/dev/null 2>&1 || return 1
|
||||
mv "${generated_config}.tmp" "${generated_config}"
|
||||
fi
|
||||
cat "${generated_config}"
|
||||
}
|
||||
|
||||
run_test() {
|
||||
@@ -672,17 +676,17 @@ run_negative_context_test "java-webapp: production has no security-scan" \
|
||||
"security_scan"
|
||||
|
||||
# java-webapp: release-candidate has quality gates
|
||||
run_context_test "java-webapp: release-candidate has code-quality" \
|
||||
run_context_test "java-webapp: release-candidate enables quality checks" \
|
||||
"${FIXTURES_DIR}/java-webapp.munitor.yml" \
|
||||
"release-candidate:" \
|
||||
"production:" \
|
||||
"code-quality"
|
||||
"quality_checks: true"
|
||||
|
||||
run_context_test "java-webapp: release-candidate has coverage" \
|
||||
run_context_test "java-webapp: release-candidate forwards coverage threshold" \
|
||||
"${FIXTURES_DIR}/java-webapp.munitor.yml" \
|
||||
"release-candidate:" \
|
||||
"production:" \
|
||||
"mvn_coverage"
|
||||
'min_instruction: "80"'
|
||||
|
||||
run_context_test "java-webapp: release-candidate has security-scan" \
|
||||
"${FIXTURES_DIR}/java-webapp.munitor.yml" \
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Protect required CI contexts and publication gates when consolidating workflows.
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
yq -o=json '.' "${SCRIPT_DIR}/../.circleci/config.yml" | node "${SCRIPT_DIR}/own_ci_regression.cjs"
|
||||
@@ -0,0 +1,93 @@
|
||||
#!/usr/bin/env bash
|
||||
# The filesystem Trivy scan and the Semgrep scan must fail when they find
|
||||
# something, and Semgrep must fail when it ran no rules. Local tool stubs; no
|
||||
# network needed.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
SRC_SCRIPTS="$(cd "${SCRIPT_DIR}/../src/scripts" && pwd)"
|
||||
WORK=$(mktemp -d)
|
||||
trap 'rm -rf "${WORK}"' EXIT
|
||||
PASS=0
|
||||
FAIL=0
|
||||
pass() { echo "PASS"; PASS=$((PASS + 1)); }
|
||||
fail() { echo "FAIL"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
mkdir -p "${WORK}/home/bin" "${WORK}/bin" "${WORK}/repo"
|
||||
cat > "${WORK}/home/bin/trivy" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
printf 'trivy %s\n' "$*" >> "${CALL_LOG}"
|
||||
# Real trivy exits 0 on findings unless --exit-code is given.
|
||||
for a in "$@"; do [[ "$a" == --exit-code ]] && exit "${SCAN_EXIT:-0}"; done
|
||||
exit 0
|
||||
EOF
|
||||
cat > "${WORK}/bin/semgrep" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' "${SEMGREP_FAKE_REPORT}" > "${SEMGREP_REPORT}"
|
||||
exit "${SEMGREP_FAKE_EXIT:-0}"
|
||||
EOF
|
||||
chmod +x "${WORK}/home/bin/trivy" "${WORK}/bin/semgrep"
|
||||
export PATH="${WORK}/bin:${PATH}" HOME="${WORK}/home" CALL_LOG="${WORK}/calls"
|
||||
export SEMGREP_REPORT="${WORK}/semgrep.json"
|
||||
cd "${WORK}/repo"
|
||||
|
||||
echo "=== Scan gate tests ==="
|
||||
echo -n " TEST: trivy fs fails the job on HIGH/CRITICAL findings... "
|
||||
if ! SCAN_EXIT=1 bash "${SRC_SCRIPTS}/run_trivy.sh" > "${WORK}/trivy.log" 2>&1 &&
|
||||
grep -q '^trivy fs .*--severity HIGH,CRITICAL --exit-code 1 ' "${CALL_LOG}"; then
|
||||
pass
|
||||
else
|
||||
fail
|
||||
fi
|
||||
|
||||
echo -n " TEST: trivy fs passes a clean scan... "
|
||||
if SCAN_EXIT=0 bash "${SRC_SCRIPTS}/run_trivy.sh" > "${WORK}/trivy.log" 2>&1; then
|
||||
pass
|
||||
else
|
||||
fail
|
||||
fi
|
||||
|
||||
if command -v python3 &>/dev/null; then
|
||||
CLEAN='{"results":[],"errors":[],"paths":{"scanned":["a.java","b.java"]}}'
|
||||
FOUND='{"results":[{"check_id":"java.lang.security.x","path":"a.java","start":{"line":3}}],"errors":[],"paths":{"scanned":["a.java"]}}'
|
||||
NORULES='{"results":[],"errors":[{"type":"InvalidRuleSchemaError","message":"invalid configuration file found (1 configs were invalid)"}],"paths":{"scanned":[]}}'
|
||||
|
||||
echo -n " TEST: semgrep passes a clean scan... "
|
||||
if SEMGREP_FAKE_REPORT="${CLEAN}" bash "${SRC_SCRIPTS}/run_semgrep.sh" > "${WORK}/sg.log" 2>&1 &&
|
||||
grep -q '2 files scanned, 0 findings' "${WORK}/sg.log"; then
|
||||
pass
|
||||
else
|
||||
fail
|
||||
fi
|
||||
|
||||
echo -n " TEST: semgrep fails on findings and lists them... "
|
||||
if ! SEMGREP_FAKE_REPORT="${FOUND}" SEMGREP_FAKE_EXIT=1 bash "${SRC_SCRIPTS}/run_semgrep.sh" > "${WORK}/sg.log" 2>&1 &&
|
||||
grep -q 'java.lang.security.x a.java:3' "${WORK}/sg.log"; then
|
||||
pass
|
||||
else
|
||||
fail
|
||||
fi
|
||||
|
||||
echo -n " TEST: semgrep warn-only mode passes with findings... "
|
||||
if SEMGREP_FAIL_ON_FINDINGS=false SEMGREP_FAKE_REPORT="${FOUND}" SEMGREP_FAKE_EXIT=1 \
|
||||
bash "${SRC_SCRIPTS}/run_semgrep.sh" > "${WORK}/sg.log" 2>&1; then
|
||||
pass
|
||||
else
|
||||
fail
|
||||
fi
|
||||
|
||||
echo -n " TEST: semgrep fails when no rules ran, even with exit 0 and warn-only... "
|
||||
if ! SEMGREP_FAIL_ON_FINDINGS=false SEMGREP_FAKE_REPORT="${NORULES}" SEMGREP_FAKE_EXIT=0 \
|
||||
bash "${SRC_SCRIPTS}/run_semgrep.sh" > "${WORK}/sg.log" 2>&1 &&
|
||||
grep -q 'did not run its rules' "${WORK}/sg.log"; then
|
||||
pass
|
||||
else
|
||||
fail
|
||||
fi
|
||||
else
|
||||
echo " SKIP: semgrep gate tests need python3"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Results: ${PASS} passed, ${FAIL} failed"
|
||||
[[ "${FAIL}" -eq 0 ]]
|
||||
@@ -843,8 +843,8 @@ echo ""
|
||||
echo "=== Error Handling Tests ==="
|
||||
|
||||
echo -n " TEST: all scripts use set -euo pipefail... "
|
||||
# munitor_helpers.sh is a sourced library, not a standalone script -- exclude it
|
||||
SCRIPTS_WITHOUT_STRICT=$(find "${SRC_SCRIPTS}" -name "*.sh" ! -name "munitor_helpers.sh" -exec grep -L 'set -euo pipefail' {} \;)
|
||||
# Sourced libraries must not change the caller shell options.
|
||||
SCRIPTS_WITHOUT_STRICT=$(find "${SRC_SCRIPTS}" -name "*.sh" ! -name "munitor_helpers.sh" ! -name "docs_only.sh" -exec grep -L 'set -euo pipefail' {} \;)
|
||||
if [[ -z "${SCRIPTS_WITHOUT_STRICT}" ]]; then
|
||||
pass
|
||||
else
|
||||
@@ -901,8 +901,9 @@ echo "=== Munitor Helpers Integration Tests ==="
|
||||
# Scripts that are excluded from the munitor_header requirement:
|
||||
# - munitor_helpers.sh: the library itself
|
||||
# - extract_munitor_vars.sh: low-level helper, sourced by other scripts
|
||||
# - docs_only.sh: status-only predicate sourced by the generator
|
||||
# - packed_*: auto-generated, embed helpers differently
|
||||
EXCLUDED_PATTERN="munitor_helpers.sh|extract_munitor_vars.sh|packed_"
|
||||
EXCLUDED_PATTERN="munitor_helpers.sh|extract_munitor_vars.sh|docs_only.sh|packed_"
|
||||
|
||||
echo -n " TEST: all scripts source munitor_helpers... "
|
||||
MISSING_SOURCE=""
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
bats "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/security_scan_owasp.bats"
|
||||
@@ -0,0 +1,11 @@
|
||||
#!/usr/bin/env bats
|
||||
|
||||
@test "combined script and template targets run each suite once" {
|
||||
mkdir -p "$BATS_TEST_TMPDIR/tests"
|
||||
printf 'echo SCRIPT_EXECUTED\n' > "$BATS_TEST_TMPDIR/tests/test_example.sh"
|
||||
printf 'echo TEMPLATE_EXECUTED\n' > "$BATS_TEST_TMPDIR/tests/test_generate_config.sh"
|
||||
run make --no-print-directory -f "$BATS_TEST_DIRNAME/../Makefile" -C "$BATS_TEST_TMPDIR" test-scripts test-templates
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(echo "$output" | grep -c '^SCRIPT_EXECUTED$')" -eq 1 ]
|
||||
[ "$(echo "$output" | grep -c '^TEMPLATE_EXECUTED$')" -eq 1 ]
|
||||
}
|
||||
Reference in new issue
Block a user