james.maesandOpenAI GPT-6 Codex
f7262f97cb
Add mandatory parser fuzzing (K22-TEST-25) and public Product Best Practices participation (K22-REPO-12), with scope records, criterion evidence, and adoption templates. The existing required conformance job runs both PowerShell fuzz seeds on PR/main and longer campaigns on weekly schedules and release tags.
Fixes #18. Preserve Scorecard detection limits: native PowerShell property-based fuzzing is running, but is not a claim of a supported integration or a passing badge. EX-0007 covers standards enrollment only, reviewed 2026-10-17 and expiring 2026-11-10 UTC, while the provider blocks maintainer login.
BREAKING CHANGE: Public Product repositories must enroll and maintain an evidence-backed assessment; complex untrusted-input parsers must have required CI fuzz campaigns and documented scope. Publish these new MUSTs in a MAJOR release. Existing immutable v0.1.2 and reserved failed tags remain unchanged; this merge does not publish another release.
Validation: all nine required PR checks passed at aa40477a11, including both native fuzz seeds. Extended local campaigns passed 20000 cases and 320000 assertions. Checker/package regressions, Markdown/links, workflow lint/security, secret scans, configured analysis of new PowerShell code, and full authenticated conformance passed.
AI review: https://github.com/KofTwentyTwo/standards/pull/19#pullrequestreview-5480144854. No blocking code findings remain. Substantial AI assistance: OpenAI GPT-6 Codex, under explicit maintainer implementation and merge authorization. No independent human review or completed badge assessment is claimed.
Signed-off-by: James Maes <james@kof22.com>
Co-Authored-By: OpenAI GPT-6 Codex <noreply@openai.com>
2.9 KiB
Conformance parser fuzzing
Scope and properties
K22-TEST-25 applies to shipped tooling even though most repository content is prose.
tools/Test-RepoConformance.Fuzz.ps1 generates synthetic workflow and exception input
for the pure helpers in tools/Test-RepoConformance.ps1. It varies quotes, indentation,
comments, LF/CRLF, scalar indicators, action references and attacker expressions.
Independent construction supplies the expected security decision: only full SHA
pins pass; credential persistence and untrusted expressions are detected; expressions
in environment fields remain outside script bodies; exceptions match their exact
scope and must be open and unexpired. Bounded malformed ASCII inputs also exercise
parser robustness. These properties test unsafe acceptance, not only crashes.
This is native generated-input fuzzing, not coverage-guided instrumentation or a complete YAML validator. Zizmor remains the authoritative workflow security gate. Scorecard currently detects selected integrations; it may continue reporting zero for this PowerShell harness. Do not claim OSS-Fuzz or ClusterFuzzLite enrollment.
Running and replaying
PowerShell 7.4 or later is the only dependency; no network or profile is used.
pwsh -NoProfile -File tools/Test-RepoConformance.Fuzz.ps1 -Seed 22023 -Iterations 1000
pwsh -NoProfile -File tools/Test-RepoConformance.Fuzz.ps1 -Seed 99173 -Iterations 1000
# Weekly and pre-MINOR/MAJOR campaigns: run both seeds at 10000 iterations.
pwsh -NoProfile -File tools/Test-RepoConformance.Fuzz.ps1 -Seed 22023 -Iterations 10000
pwsh -NoProfile -File tools/Test-RepoConformance.Fuzz.ps1 -Seed 99173 -Iterations 10000
Failures print synthetic input, property, iteration and seed as JSON. Replay with
that seed and at least that iteration count against the same checkout and PowerShell
runtime. Minimize the input into a named -SelfTest regression, then fix the parser
and rerun both seeds. Report suspected vulnerabilities through SECURITY.md.
Inputs are bounded to small workflow fragments and fewer than 128 random ASCII
characters; iterations are limited to 100000. Generated scripts are never executed.
The harness does not write files, launch tools, or request repository credentials.
The required ci / conformance job runs both seeds on PRs and main, and 10000
iterations per seed on weekly schedules and release-tag validation, within its
10-minute timeout. Future failing seeds must be retained alongside regression cases.
Review record
- Owner: @KofTwentyTwo; reviewed: 2026-10-10; next review: 2027-01-08.
- Seed 22023 exposed quoted-checkout and explicit-indentation block-scalar gaps; both now have deterministic self-test regressions. This is not a coverage claim.
- Reassess targets when parser boundaries change; generated data alone does not establish completeness or substitute for ordinary package and checker tests.