diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..5c2aebc --- /dev/null +++ b/.gitattributes @@ -0,0 +1,2 @@ +*.sh text eol=lf +*.bats text eol=lf diff --git a/CLAUDE.md b/CLAUDE.md index c180de1..b6554cc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,6 +30,7 @@ GitHub Issues (org: KofTwentyTwo). Commit format: `feat(#N): description` or `Cl - Namespace: `kof22` - Orb: `kof22/munitor` - Latest stable: `kof22/munitor@0.3.1` +- Release candidate: `kof22/munitor@0.3.2` (stable tag publication pending) - Dev snapshot: `kof22/munitor@dev:snapshot` `0.3.0` includes first-class pnpm support, repository-owned Dockerfiles for diff --git a/README.md b/README.md index 4f14d84..7a6c4b1 100644 --- a/README.md +++ b/README.md @@ -569,3 +569,19 @@ Pipeline generation flow: 5. Runs `envsubst` for variable substitution 6. Processes `##IF_*##` / `##ENDIF_*##` conditional blocks 7. Validates output YAML and passes to CircleCI continuation + +## Temporary npm audit exceptions + +The default npm audit gate remains unchanged. For a reviewed development-only HIGH +advisory with no patched package release, a repository may commit +`.npm-audit-exceptions.json` with schemaVersion 1 and an exceptions array. Each entry +requires a register ID, exact GHSA advisory ID, package, version, development scope +and UTC expiry date. See Website-Frontend's EX-0008 for the first consumer. + +The full npm audit runs alongside a production audit. All production HIGH/CRITICAL +and all CRITICAL findings fail. Only the named advisory and its propagated HIGH +findings can pass, and every affected lockfile path must have dev true and the exact +excepted version. Audit errors, unknown chains and expired exceptions fail. A new +upstream package release also fails and requires review. Trivy remains unchanged. +This mechanism requires npm audit report version 2 and a package-path lockfile; +unsupported reports fail. BATS and Node are required for the audit regression tests. diff --git a/docs/SESSION-STATE.md b/docs/SESSION-STATE.md index f9a566f..013f043 100644 --- a/docs/SESSION-STATE.md +++ b/docs/SESSION-STATE.md @@ -1,5 +1,17 @@ # Munitor session state +## Release candidate 0.3.2 (2026-10-10) + +Munitor PR19 merged to develop at c8bbf2e. Release branch `release/0.3.2` +promotes the bounded publishing CI fixes to main. Full make all and the feature +CircleCI lint/pack/review/snapshot publication passed. Stable publication still +requires the v0.3.2 tag on main and successful production orb publication. + +EX-0008 is registered by standards PR27 (5c21fc7); Website-Frontend may select the +new stable orb after publication. Default audit policy, Trivy and GitOps contracts +are preserved. Website-Backend's Java installation now avoids interactive Ubuntu +service-restart dialogs. Website consumers and production remain pending. + ## Current status The `develop` branch contains the complete `0.3.0` feature set. PR #6 added @@ -23,3 +35,17 @@ ArgoCD. Run `make all` before release. The stable tag must publish successfully through CircleCI before consumer repositories move from immutable development pins to `0.3`. + +## Publishing release CI fixes (2026-10-10) + +Feature branch `feature/scoped-audit-exceptions` prepares development-only npm +exceptions with expiry, exact advisory/package/version checks, an independent +production audit and reassessment on a new upstream release. Website-Frontend's +proposed EX-0008 covers braces 3.0.3 through 2026-11-10. Regression tests exercise +acceptance and rejection paths. The default audit and Trivy remain unchanged. + +Website-Backend CI job 1135 timed out in Ubuntu's interactive needrestart prompt +after Java installation. APT now receives noninteractive debconf and automatic +needrestart settings; a stubbed integration test verifies both APT invocations. +These fixes require normal feature-to-develop review and stable tag publication +from main before website consumers can select the new version. Nothing deployed. diff --git a/docs/TODO.md b/docs/TODO.md index 540dba6..0c08e3b 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -2,6 +2,9 @@ ## Pending +- [x] Review publishing CI fixes and merge PR19 to develop. +- [ ] Publish v0.3.2 from main; update website consumer orb pins. + - [ ] Follow makers4 and terraform-proxmox-talos on CircleCI when those repos are next active. - [ ] Adopt supplemental images on an appropriate consumer. @@ -13,7 +16,7 @@ - [x] Validate first-class pnpm support and merge PR #6. - [x] Add opt-in repository Dockerfile preservation for node-webapp (GH-7) - [x] Implement first-class pnpm support (GH-5, 17 commits, 219 tests) -- [x] Rename all npm_* orb components to node_* (backward-incompatible, pre-1.0) +- [x] Rename all `npm_*` orb components to `node_*` (backward-incompatible, pre-1.0) - [x] Thread package_manager through full job/command/script chain - [x] Add corepack activation for pnpm/yarn in install_node - [x] Add pnpm Dockerfile variants for node-api and node-webapp diff --git a/src/scripts/install_java.sh b/src/scripts/install_java.sh index e30710f..aa6fe70 100755 --- a/src/scripts/install_java.sh +++ b/src/scripts/install_java.sh @@ -26,8 +26,9 @@ echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] \ $(. /etc/os-release && echo "${VERSION_CODENAME}") main" \ | sudo tee /etc/apt/sources.list.d/adoptium.list > /dev/null -sudo apt-get update -qq -sudo apt-get install -y -qq "temurin-${VERSION}-jdk" +# CI cannot answer debconf or needrestart's service-restart dialog. +sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get update -qq +sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get install -y -qq "temurin-${VERSION}-jdk" # Set JAVA_HOME for subsequent steps JAVA_HOME_DIR="/usr/lib/jvm/temurin-${VERSION}-jdk-$(dpkg --print-architecture)" diff --git a/src/scripts/node_audit.sh b/src/scripts/node_audit.sh index 6e43d8c..a0175b9 100644 --- a/src/scripts/node_audit.sh +++ b/src/scripts/node_audit.sh @@ -15,6 +15,106 @@ munitor_header "node_audit" echo "Running npm audit..." +if [[ -f .npm-audit-exceptions.json ]]; then + # The optional, repository-reviewed exception file never changes runtime policy. + REPORT_DIR=$(mktemp -d) + trap 'rm -rf "${REPORT_DIR}"' EXIT + export REPORT_DIR + npm audit --json --audit-level=high > "${REPORT_DIR}/full.json" || AUDIT_STATUS=$? + npm audit --json --omit=dev --audit-level=high > "${REPORT_DIR}/production.json" || PRODUCTION_STATUS=$? + export AUDIT_STATUS="${AUDIT_STATUS:-0}" PRODUCTION_STATUS="${PRODUCTION_STATUS:-0}" + node <<'AUDIT_JS' +const fs = require('node:fs'); +const { execFileSync } = require('node:child_process'); +function fail(message) { throw new Error(message); } +function readReport(name, status) { + const report = JSON.parse(fs.readFileSync(`${process.env.REPORT_DIR}/${name}.json`, 'utf8')); + if (report.error || report.auditReportVersion !== 2 || !report.vulnerabilities || !report.metadata?.vulnerabilities) + fail(`${name}: npm audit returned an error or unsupported report`); + if (![0, 1].includes(Number(status))) fail(`${name}: npm audit failed (${status})`); + const counts = report.metadata.vulnerabilities; + if (Object.values(report.vulnerabilities).some(v => !v || !['info', 'low', 'moderate', 'high', 'critical'].includes(v.severity))) + fail(`${name}: invalid vulnerability severity`); + for (const severity of ['info', 'low', 'moderate', 'high', 'critical']) { + if (!Number.isInteger(counts[severity]) || counts[severity] < 0) fail(`${name}: invalid severity counts`); + if (Object.values(report.vulnerabilities).filter(v => v.severity === severity).length !== counts[severity]) + fail(`${name}: inconsistent severity counts`); + } + return report; +} +try { + const full = readReport('full', process.env.AUDIT_STATUS); + const production = readReport('production', process.env.PRODUCTION_STATUS); + if (production.metadata.vulnerabilities.high || production.metadata.vulnerabilities.critical) + fail('Production dependencies contain high/critical vulnerabilities; exceptions do not apply'); + if (full.metadata.vulnerabilities.critical) fail('Critical vulnerabilities cannot be excepted'); + const config = JSON.parse(fs.readFileSync('.npm-audit-exceptions.json', 'utf8')); + if (config.schemaVersion !== 1 || !Array.isArray(config.exceptions) || !config.exceptions.length) + fail('Invalid exception configuration'); + const today = new Date().toISOString().slice(0, 10); + const lock = JSON.parse(fs.readFileSync('package-lock.json', 'utf8')); + if (!lock.packages) fail('Exceptions require a package-lock.json with package paths'); + const allowed = new Map(); + for (const entry of config.exceptions) { + if (!/^EX-\d{4}$/.test(entry.id) || !/^GHSA-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}$/.test(entry.advisory) + || entry.scope !== 'development' || !/^\d{4}-\d{2}-\d{2}$/.test(entry.expires) + || !Number.isFinite(Date.parse(entry.expires)) + || new Date(entry.expires).toISOString().slice(0, 10) !== entry.expires || today >= entry.expires + || typeof entry.package !== 'string' || !/^(@[a-z0-9._-]+\/)?[a-z0-9._-]+$/.test(entry.package) + || !/^\d+\.\d+\.\d+$/.test(entry.version) || allowed.has(entry.advisory)) + fail('Invalid, duplicated or expired development exception'); + allowed.set(entry.advisory, entry); + } + function roots(name, seen = new Set()) { + if (seen.has(name)) fail(`Cyclic vulnerability chain: ${name}`); + const vulnerability = full.vulnerabilities[name]; + if (!vulnerability || !Array.isArray(vulnerability.via) || !vulnerability.via.length) + fail(`Unresolved vulnerability chain: ${name}`); + const next = new Set([...seen, name]); + return vulnerability.via.flatMap(via => { + if (typeof via === 'string') return roots(via, next); + if (!via || typeof via.url !== 'string' || typeof via.severity !== 'string') + fail(`Invalid advisory: ${name}`); + return [{ advisory: via, vulnerability }]; + }); + } + const used = new Set(); + for (const [name, vulnerability] of Object.entries(full.vulnerabilities)) { + if (vulnerability.severity !== 'high') continue; + for (const { advisory, vulnerability: root } of roots(name)) { + if (!['high', 'critical'].includes(advisory.severity)) continue; + const id = advisory.url.replace('https://github.com/advisories/', ''); + const exception = allowed.get(id); + if (!exception || advisory.url !== `https://github.com/advisories/${id}` + || advisory.severity !== 'high' || advisory.name !== exception.package + || root.name !== exception.package || !Array.isArray(root.nodes) || !root.nodes.length) + fail(`Unexcepted high vulnerability: ${name} (${advisory.url})`); + for (const path of root.nodes) { + if (!lock.packages[path] || lock.packages[path].dev !== true || lock.packages[path].version !== exception.version) + fail(`Exception does not cover package path or version: ${path}`); + } + used.add(id); + } + if (!roots(name).some(r => ['high', 'critical'].includes(r.advisory.severity))) + fail(`High vulnerability has no resolvable high advisory: ${name}`); + } + for (const id of used) { + const entry = allowed.get(id); + // Any newer release requires reassessment, even if npm proposes a parent downgrade. + const latest = execFileSync('npm', ['view', entry.package, 'version', '--json'], + { encoding: 'utf8', timeout: 30000 }); + if (JSON.parse(latest) !== entry.version) fail(`New ${entry.package} release available; reassess ${entry.id}`); + console.log(`Accepted ${entry.id}: ${id}, development only, expires ${entry.expires}`); + } + console.log('Full audit checked; production audit has no high/critical findings.'); +} catch (error) { + console.error(`npm audit rejected: ${error.message}`); + process.exit(1); +} +AUDIT_JS + exit 0 +fi + npm audit --audit-level=high || { EXIT_CODE=$? echo "npm audit found high/critical vulnerabilities." diff --git a/tests/fixtures/npm-audit/build-fixture.cjs b/tests/fixtures/npm-audit/build-fixture.cjs new file mode 100644 index 0000000..de349d6 --- /dev/null +++ b/tests/fixtures/npm-audit/build-fixture.cjs @@ -0,0 +1,31 @@ +const fs = require('node:fs'); +const [directory, scenario = 'allowed'] = process.argv.slice(2); +const advisory = { + name: 'braces', severity: 'high', url: 'https://github.com/advisories/GHSA-vfj7-8cjw-p6xm' +}; +const root = { name: 'braces', severity: 'high', via: [advisory], nodes: ['node_modules/braces'] }; +const full = { auditReportVersion: 2, vulnerabilities: { + braces: root, parent: { name: 'parent', severity: 'high', via: ['braces'] } +}, metadata: { vulnerabilities: { info: 0, low: 0, moderate: 0, high: 2, critical: 0 } } }; +const production = { auditReportVersion: 2, vulnerabilities: {}, + metadata: { vulnerabilities: { info: 0, low: 0, moderate: 0, high: 0, critical: 0 } } }; +const exception = { id: 'EX-0008', advisory: 'GHSA-vfj7-8cjw-p6xm', package: 'braces', + version: '3.0.3', scope: 'development', expires: '2099-01-01' }; +const lock = { lockfileVersion: 3, packages: { 'node_modules/braces': { version: '3.0.3', dev: true } } }; +if (scenario === 'expired') exception.expires = '2020-01-01'; +if (scenario === 'invalid-date') exception.expires = '2099-02-30'; +if (scenario === 'different') advisory.url = 'https://github.com/advisories/GHSA-aaaa-bbbb-cccc'; +if (scenario === 'mixed') root.via.push({ ...advisory, url: 'https://github.com/advisories/GHSA-aaaa-bbbb-cccc' }); +if (scenario === 'critical') { root.severity = advisory.severity = 'critical'; full.metadata.vulnerabilities.high--; full.metadata.vulnerabilities.critical++; } +if (scenario === 'production') { production.vulnerabilities.braces = root; production.metadata.vulnerabilities.high = 1; } +if (scenario === 'runtime-path') lock.packages['node_modules/braces'].dev = false; +if (scenario === 'different-version') lock.packages['node_modules/braces'].version = '3.0.2'; +if (scenario === 'unknown-chain') full.vulnerabilities.parent.via = ['missing']; +if (scenario === 'cycle') root.via = ['parent']; +if (scenario === 'counts') full.metadata.vulnerabilities.high = 0; +if (scenario === 'registry-error') full.error = { code: 'ENETUNREACH' }; +fs.mkdirSync(directory, { recursive: true }); +for (const [name, data] of Object.entries({ 'full.json': full, 'production.json': production, + '.npm-audit-exceptions.json': { schemaVersion: 1, exceptions: [exception] }, 'package-lock.json': lock })) { + fs.writeFileSync(`${directory}/${name}`, JSON.stringify(data)); +} diff --git a/tests/install_java.bats b/tests/install_java.bats new file mode 100644 index 0000000..433f6ba --- /dev/null +++ b/tests/install_java.bats @@ -0,0 +1,32 @@ +#!/usr/bin/env bats + +setup() { + export PROJECT_DIR="${BATS_TEST_DIRNAME}/.." + export WORK="${BATS_TEST_TMPDIR}/work" + mkdir -p "${WORK}/bin" + export BASH_ENV="${WORK}/bash-env" JAVA_VERSION=21 + export APT_CALLS="${WORK}/apt-calls" + cat > "${WORK}/bin/sudo" <<'MOCK' +#!/usr/bin/env bash +if [[ "$1" == env ]]; then + [[ "$2" == DEBIAN_FRONTEND=noninteractive && "$3" == NEEDRESTART_MODE=a && "$4" == apt-get ]] || exit 2 + printf '%s\n' "$*" >> "$APT_CALLS" +elif [[ "$1" == tee ]]; then + cat > /dev/null +fi +MOCK + printf '#!/usr/bin/env bash\necho key\n' > "${WORK}/bin/curl" + printf '#!/usr/bin/env bash\necho amd64\n' > "${WORK}/bin/dpkg" + printf '#!/usr/bin/env bash\necho java-version-21\n' > "${WORK}/bin/java" + chmod +x "${WORK}/bin/"* + export PATH="${WORK}/bin:${PATH}" +} + +@test "APT update and Java installation cannot open an interactive restart prompt" { + run bash "${PROJECT_DIR}/src/scripts/install_java.sh" + [ "$status" -eq 0 ] + [ "$(wc -l < "$APT_CALLS")" -eq 2 ] + grep -q 'apt-get update -qq' "$APT_CALLS" + grep -q 'apt-get install -y -qq temurin-21-jdk' "$APT_CALLS" + grep -q "export JAVA_HOME='/usr/lib/jvm/temurin-21-jdk-amd64'" "$BASH_ENV" +} diff --git a/tests/node_audit.bats b/tests/node_audit.bats new file mode 100644 index 0000000..e573673 --- /dev/null +++ b/tests/node_audit.bats @@ -0,0 +1,55 @@ +#!/usr/bin/env bats + +setup() { + export PROJECT_DIR="${BATS_TEST_DIRNAME}/.." + export WORK="${BATS_TEST_TMPDIR}/work" + node "${BATS_TEST_DIRNAME}/fixtures/npm-audit/build-fixture.cjs" "${WORK}" + mkdir -p "${WORK}/bin" + cat > "${WORK}/bin/npm" <<'MOCK' +#!/usr/bin/env bash +if [[ "$1" == view ]]; then printf '"%s"\n' "${LATEST_VERSION:-3.0.3}"; exit 0; fi +if [[ "$*" == *--omit=dev* ]]; then cat production.json; exit 0; fi +cat full.json +exit "${MOCK_AUDIT_STATUS:-1}" +MOCK + chmod +x "${WORK}/bin/npm" + export PATH="${WORK}/bin:${PATH}" + cd "${WORK}" +} + +@test "accepts only the registered advisory and its propagated development findings" { + run bash "${PROJECT_DIR}/src/scripts/node_audit.sh" + [ "$status" -eq 0 ] + [[ "$output" == *"Accepted EX-0008"* ]] +} + +@test "fails closed for expired, invalid, unrelated, mixed, critical and runtime findings" { + for scenario in expired invalid-date different mixed critical production runtime-path different-version unknown-chain cycle counts registry-error; do + node "${BATS_TEST_DIRNAME}/fixtures/npm-audit/build-fixture.cjs" "$WORK" "$scenario" + run bash "${PROJECT_DIR}/src/scripts/node_audit.sh" + if [ "$status" -eq 0 ]; then echo "Unexpected acceptance: $scenario"; return 1; fi + done +} + +@test "a new upstream release requires reassessment" { + export LATEST_VERSION=3.0.4 + run bash "${PROJECT_DIR}/src/scripts/node_audit.sh" + [ "$status" -ne 0 ] + [[ "$output" == *"New braces release available"* ]] +} + +@test "malformed reports and unexpected audit exit codes fail closed" { + export MOCK_AUDIT_STATUS=2 + run bash "${PROJECT_DIR}/src/scripts/node_audit.sh" + [ "$status" -ne 0 ] + printf 'invalid json' > full.json + run bash "${PROJECT_DIR}/src/scripts/node_audit.sh" + [ "$status" -ne 0 ] +} + +@test "repositories without exceptions retain their existing audit gate" { + rm .npm-audit-exceptions.json + run bash "${PROJECT_DIR}/src/scripts/node_audit.sh" + [ "$status" -eq 1 ] + [[ "$output" == *"npm audit found high/critical"* ]] +} diff --git a/tests/test_install_java.sh b/tests/test_install_java.sh new file mode 100644 index 0000000..78a40a5 --- /dev/null +++ b/tests/test_install_java.sh @@ -0,0 +1,3 @@ +#!/usr/bin/env bash +set -euo pipefail +bats "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/install_java.bats" diff --git a/tests/test_node_audit.sh b/tests/test_node_audit.sh new file mode 100644 index 0000000..1756db1 --- /dev/null +++ b/tests/test_node_audit.sh @@ -0,0 +1,3 @@ +#!/usr/bin/env bash +set -euo pipefail +bats "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/node_audit.bats"