feat(assurance)!: require applicable fuzzing and badge participation #19
No files matched your search
@@ -93,6 +93,16 @@ jobs:
|
||||
shell: pwsh
|
||||
run: ./tools/Test-RepoConformance.ps1 -SelfTest
|
||||
|
||||
- name: Conformance parser fuzzing
|
||||
shell: pwsh
|
||||
env:
|
||||
CI_EVENT_NAME: ${{ github.event_name }}
|
||||
CI_REF_TYPE: ${{ github.ref_type }}
|
||||
run: |
|
||||
$iterations = if ($env:CI_EVENT_NAME -eq 'schedule' -or $env:CI_REF_TYPE -eq 'tag') { 10000 } else { 1000 }
|
||||
./tools/Test-RepoConformance.Fuzz.ps1 -Seed 22023 -Iterations $iterations
|
||||
./tools/Test-RepoConformance.Fuzz.ps1 -Seed 99173 -Iterations $iterations
|
||||
|
||||
- name: Standards package tests
|
||||
shell: pwsh
|
||||
run: ./tools/Test-StandardsPackage.ps1
|
||||
|
||||
@@ -41,6 +41,7 @@ Documentation and package checks:
|
||||
npx markdownlint-cli2 "**/*.md"
|
||||
lychee --offline --include-fragments --no-progress './**/*.md'
|
||||
pwsh tools/Test-RepoConformance.ps1 -SelfTest
|
||||
pwsh tools/Test-RepoConformance.Fuzz.ps1 -Seed 22023 -Iterations 1000
|
||||
pwsh tools/Test-StandardsPackage.ps1
|
||||
pwsh tools/Test-RepoConformance.ps1 -LocalPath . -StaticOnly
|
||||
```
|
||||
@@ -59,6 +60,9 @@ pwsh tools/Test-RepoConformance.ps1 -Repository KofTwentyTwo/standards -LocalPat
|
||||
```
|
||||
|
||||
The same checks run in CI on every pull request and must pass before merging.
|
||||
Parser changes must pass both CI seeds and retain any discovered failure as a
|
||||
deterministic regression test. See [fuzzing](docs/security/fuzzing.md) for extended
|
||||
campaigns and [badge assessment](docs/security/best-practices.md) for enrollment evidence.
|
||||
|
||||
## License of contributions
|
||||
|
||||
|
||||
@@ -33,6 +33,10 @@ v1.0* once the checker passes on it.
|
||||
the [reusable workflows](#use-the-reusable-workflows), and keep it green with the
|
||||
[conformance checker](#check-a-repositorys-conformance).
|
||||
|
||||
Security assurance is recorded in the [Best Practices assessment](docs/security/best-practices.md)
|
||||
and [parser fuzzing guide](docs/security/fuzzing.md). Badge enrollment is temporarily
|
||||
blocked under [EX-0007](exceptions/register.md#ex-0007); no passing badge is claimed.
|
||||
|
||||
## Contents
|
||||
|
||||
- [How to use](#how-to-use)
|
||||
@@ -197,13 +201,18 @@ IDE does on save is what CI checks.
|
||||
`protect-release-tags` rulesets, squash-only merging, the security features
|
||||
(private vulnerability reporting, secret scanning with push protection, Dependabot,
|
||||
code scanning, immutable releases), and read-only Actions tokens.
|
||||
6. **Check.** Run the conformance checker and fix every FAIL:
|
||||
6. **Assurance.** Complete `docs/security/assurance.json` and its evidence templates.
|
||||
Public Product repositories register their own Best Practices assessment and
|
||||
link the actual badge. Identify complex untrusted-input parsers; add their fuzz
|
||||
commands to a required PR/main CI job and weekly/release campaigns, or document
|
||||
why none apply. Review scope decisions; copied placeholders do not conform.
|
||||
7. **Check.** Run the conformance checker and fix every FAIL:
|
||||
|
||||
```powershell
|
||||
pwsh ./tools/Test-RepoConformance.ps1 -Repository KofTwentyTwo/<repo> -LocalPath ../<repo>
|
||||
```
|
||||
|
||||
7. **Declare it.** Add *"Conforms to KofTwentyTwo standards vX.Y"* to the README and
|
||||
8. **Declare it.** Add *"Conforms to KofTwentyTwo standards vX.Y"* to the README and
|
||||
list the repository in [MAINTAINERS.md](MAINTAINERS.md#repositories).
|
||||
|
||||
### An existing repository
|
||||
|
||||
@@ -38,7 +38,7 @@ this crosswalk is updated when it becomes final.
|
||||
| **PW.5** Create source code by adhering to secure coding practices | [Coding standards](../standards/coding/README.md), K22-AI-30 |
|
||||
| **PW.6** Configure build processes to improve executable security | Language profiles (warnings as errors, analyzers, deterministic builds), K22-CI-30 |
|
||||
| **PW.7** Review and/or analyze human-readable code | K22-SDLC-12, K22-CI-22 |
|
||||
| **PW.8** Test executable code | [Testing standard](../standards/testing.md) |
|
||||
| **PW.8** Test executable code | [Testing standard](../standards/testing.md), including K22-TEST-25 parser fuzzing where applicable |
|
||||
| **PW.9** Configure software to have secure settings by default | K22-SEC-70, K22-SEC-71 |
|
||||
| **RV.1** Identify and confirm vulnerabilities on an ongoing basis | K22-CI-21, K22-CI-22 (weekly scans), K22-DEP-40, K22-SEC-40 |
|
||||
| **RV.2** Assess, prioritize, and remediate vulnerabilities | K22-SEC-41, K22-SEC-42 |
|
||||
@@ -72,4 +72,5 @@ Scorecard checks and the requirement that drives each:
|
||||
| Vulnerabilities | K22-SEC-41 |
|
||||
| Binary-Artifacts | K22-REPO-05 |
|
||||
| Packaging | K22-REL-11 |
|
||||
| Fuzzing | K22-TEST-23 (where applicable) |
|
||||
| Fuzzing | K22-TEST-25 (mandatory where applicable); K22-TEST-23 negative-test guidance. Detection supports selected integrations and can miss native harnesses |
|
||||
| CII-Best-Practices | K22-REPO-12 (public Product enrollment and maintained assessment); actual badge level is assigned by the provider |
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"badge": {
|
||||
"applicability": "required",
|
||||
"reason": "Public open-source Product repository distributing standards and executable tooling.",
|
||||
"projectUrl": null,
|
||||
"evidence": "docs/security/best-practices.md"
|
||||
},
|
||||
"fuzzing": {
|
||||
"applicability": "required",
|
||||
"reason": "The conformance checker parses contributor-controlled workflows and exception records; unsafe acceptance can hide failed requirements.",
|
||||
"targets": ["tools/Test-RepoConformance.ps1"],
|
||||
"command": "./tools/Test-RepoConformance.Fuzz.ps1",
|
||||
"workflow": ".github/workflows/ci.yml",
|
||||
"evidence": "docs/security/fuzzing.md"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
# OpenSSF Best Practices evidence
|
||||
|
||||
## Enrollment and review
|
||||
|
||||
**Repository:** <https://github.com/KofTwentyTwo/standards>. **Owner:** @KofTwentyTwo.
|
||||
**Prepared:** 2026-10-10. **Enrollment review:** 2026-10-17; **exception expiry:**
|
||||
2026-11-10 ([EX-0007](../../exceptions/register.md#ex-0007)). Once enrolled, reassess
|
||||
within 90 days and before each MINOR or MAJOR release.
|
||||
|
||||
The maintainer reported an anti-spam login restriction and deferred enrollment.
|
||||
There is no registered project ID, submitted assessment, progress percentage, or
|
||||
passing badge. The table below is prepared evidence for the current
|
||||
[passing criteria](https://www.bestpractices.dev/en/criteria/0), not certification.
|
||||
Use those criterion IDs in the provider's form; answers still require owner review.
|
||||
|
||||
## Criterion evidence matrix
|
||||
|
||||
IDs below omit the provider's `0.` level prefix. **Evidence** means source exists;
|
||||
**Review** means operational evidence or human judgment is still needed;
|
||||
**Proposed N/A** needs the provider's applicability rules and owner approval.
|
||||
|
||||
| Criteria | Prepared evidence / remaining decision | State |
|
||||
| --- | --- | --- |
|
||||
| `description_good`, `documentation_basics` | [README](../../README.md) describes standards, tooling and adoption | Evidence |
|
||||
| `interact`, `discussion`, `report_process`, `report_tracker`, `report_archive` | Public [issues](https://github.com/KofTwentyTwo/standards/issues), templates and [CONTRIBUTING](../../CONTRIBUTING.md) | Evidence |
|
||||
| `contribution`, `contribution_requirements` | CONTRIBUTING documents proposals, signed commits, DCO, review and gates | Evidence |
|
||||
| `floss_license`, `floss_license_osi`, `license_location` | Root [MIT](../../LICENSE) for code, configs and templates; [CC BY 4.0](../../LICENSE-docs) for prose. Explain the separate documentation license | Evidence |
|
||||
| `documentation_interface` | [Tool help and local commands](../../CONTRIBUTING.md#running-the-checks-locally), [release instructions](../releases.md); owner checks shipped tools' help completeness | Review |
|
||||
| `sites_https`, `delivery_mitm` | GitHub repository and release delivery use HTTPS; [verification](../../standards/releases.md#verifying-a-release) checks provenance | Evidence |
|
||||
| `english` | Repository documentation and contribution guidance are in English | Evidence |
|
||||
| `maintained`, `report_responses`, `enhancement_responses` | Recent PRs/issues exist; a new project has limited response history. Assess actual responsiveness, not a policy promise | Review |
|
||||
| `repo_public`, `repo_track`, `repo_interim`, `repo_distributed` | Public Git repository retains reviewed changes; signed history and protected main | Evidence |
|
||||
| `version_unique`, `version_semver`, `version_tags`, `release_notes` | Immutable [v0.1.2](https://github.com/KofTwentyTwo/standards/releases/tag/v0.1.2), committed version and [changelog](../../CHANGELOG.md); failed unpublished tags are reserved | Evidence |
|
||||
| `release_notes_vulns` | [Release standard](../../standards/releases.md) requires security fix notes; no published vulnerability fix history to attest to yet | Review |
|
||||
| `vulnerability_report_process`, `vulnerability_report_private` | [SECURITY](../../SECURITY.md) links private GitHub reporting and supported versions | Evidence |
|
||||
| `vulnerability_report_response` | SECURITY defines targets; maintainer confirms monitored contact and actual response capability | Review |
|
||||
| `build`, `build_common_tools`, `build_floss_tools` | [Bundle builder](../../tools/Build-StandardsPackage.ps1) uses Git archives and PowerShell; prerequisites and release steps are documented | Evidence |
|
||||
| `test`, `test_invocation`, `test_policy`, `tests_are_added`, `tests_documented_added` | CONTRIBUTING requires behavioral regression tests and documents checker, package and fuzz commands | Evidence |
|
||||
| `test_continuous_integration` | [CI](../../.github/workflows/ci.yml) runs tests on PR/main, schedules and tag publication gates | Evidence |
|
||||
| `test_most` | Parser properties, helper self-tests and package tests exist; no measured whole-tool coverage. Owner must assess most functionality, not infer it from case counts | Review |
|
||||
| `warnings`, `warnings_fixed`, `warnings_strict` | Markdown/workflow gates reject findings. Broad PowerShell warning coverage is not established by Actions CodeQL; assess this gap separately | Review |
|
||||
| `know_secure_design`, `know_common_errors` | [Threat model](threat-model.md), [security policy](../../policies/security-program.md) and parser regressions support discussion; a primary human developer must personally attest to the required knowledge | Review |
|
||||
| `crypto_published`, `crypto_call`, `crypto_floss`, `crypto_keylength`, `crypto_working`, `crypto_weaknesses` | No custom product cryptography. GitHub TLS, SSH signing and attestations are delegated mechanisms; assess whether delegation meets each criterion or permits N/A | Review |
|
||||
| `crypto_pfs` | No application server or product-controlled TLS configuration; document the hosted-service boundary if N/A is permitted | Proposed N/A |
|
||||
| `crypto_password_storage` | No application passwords, password database or credential-storage implementation | Proposed N/A |
|
||||
| `crypto_random` | The seeded fuzz PRNG generates only synthetic tests, never security keys/nonces. Cryptographic randomness is delegated to signing/hosting providers | Review |
|
||||
| `delivery_unsigned` | Release assets have checksums, verified source, build provenance and GitHub immutable-release attestation; see [verification commands](../releases.md) | Evidence |
|
||||
| `vulnerabilities_fixed_60_days`, `vulnerabilities_critical_fixed` | [Security program](../../policies/security-program.md#vulnerability-management) defines deadlines; maintainer must check current findings and known unpatched issues before answering | Review |
|
||||
| `no_leaked_credentials` | Required [gitleaks and secret scans](../../.github/workflows/security.yml); scans support, but cannot prove, a universal absence claim | Review |
|
||||
| `static_analysis`, `static_analysis_common_vulnerabilities`, `static_analysis_often` | [CodeQL Actions](../../.github/workflows/codeql.yml), zizmor and actionlint in required security gates; document PowerShell analysis coverage accurately | Evidence |
|
||||
| `static_analysis_fixed` | Gates reject applicable findings; review unresolved alerts and [Scorecard dispositions](scorecard-dispositions.md) before claiming all important findings fixed | Review |
|
||||
| `dynamic_analysis`, `dynamic_analysis_enable_assertions` | Executed [fuzz properties](fuzzing.md), deterministic helper and isolated package tests; assertions fail the job | Evidence |
|
||||
| `dynamic_analysis_unsafe` | Shipped tooling is managed PowerShell, not unsafe native-language implementation; owner validates applicability | Proposed N/A |
|
||||
| `dynamic_analysis_fixed` | Generated-input findings gained deterministic regressions; review all future campaign failures before assessment updates | Review |
|
||||
|
||||
## Completing enrollment
|
||||
|
||||
1. Retry normal GitHub login at <https://www.bestpractices.dev/en> after the provider
|
||||
restriction clears; register the exact repository above. Do not bypass anti-spam.
|
||||
2. Review every proposed answer against the live criteria and actual evidence,
|
||||
including human knowledge, unresolved findings, and observed response history.
|
||||
3. Put the actual project URL in [assurance.json](assurance.json) and add the
|
||||
provider's linked badge to README. Record the resulting assessment level honestly.
|
||||
4. Close EX-0007 through a normal PR and set the next assessment review date.
|
||||
Work toward passing by resolving genuine gaps; registration is not passing.
|
||||
|
||||
Downstream repositories use the [assessment template](../../templates/repo/docs/security/best-practices.md)
|
||||
and their own evidence and project ID; EX-0007 does not apply to them.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Conformance parser fuzzing
|
||||
|
||||
## Scope and properties
|
||||
|
||||
K22-TEST-25 applies to shipped tooling even though most repository content is prose.
|
||||
`tools/Test-RepoConformance.Fuzz.ps1` generates synthetic workflow and exception input
|
||||
for the pure helpers in `tools/Test-RepoConformance.ps1`. It varies quotes, indentation,
|
||||
comments, LF/CRLF, scalar indicators, action references and attacker expressions.
|
||||
Independent construction supplies the expected security decision: only full SHA
|
||||
pins pass; credential persistence and untrusted expressions are detected; expressions
|
||||
in environment fields remain outside script bodies; exceptions match their exact
|
||||
scope and must be open and unexpired. Bounded malformed ASCII inputs also exercise
|
||||
parser robustness. These properties test unsafe acceptance, not only crashes.
|
||||
|
||||
This is native generated-input fuzzing, not coverage-guided instrumentation or a
|
||||
complete YAML validator. Zizmor remains the authoritative workflow security gate.
|
||||
Scorecard currently detects selected integrations; it may continue reporting zero
|
||||
for this PowerShell harness. Do not claim OSS-Fuzz or ClusterFuzzLite enrollment.
|
||||
|
||||
## Running and replaying
|
||||
|
||||
PowerShell 7.4 or later is the only dependency; no network or profile is used.
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -File tools/Test-RepoConformance.Fuzz.ps1 -Seed 22023 -Iterations 1000
|
||||
pwsh -NoProfile -File tools/Test-RepoConformance.Fuzz.ps1 -Seed 99173 -Iterations 1000
|
||||
# Weekly and pre-MINOR/MAJOR campaigns: run both seeds at 10000 iterations.
|
||||
pwsh -NoProfile -File tools/Test-RepoConformance.Fuzz.ps1 -Seed 22023 -Iterations 10000
|
||||
pwsh -NoProfile -File tools/Test-RepoConformance.Fuzz.ps1 -Seed 99173 -Iterations 10000
|
||||
```
|
||||
|
||||
Failures print synthetic input, property, iteration and seed as JSON. Replay with
|
||||
that seed and at least that iteration count against the same checkout and PowerShell
|
||||
runtime. Minimize the input into a named `-SelfTest` regression, then fix the parser
|
||||
and rerun both seeds. Report suspected vulnerabilities through [SECURITY.md](../../SECURITY.md).
|
||||
|
||||
Inputs are bounded to small workflow fragments and fewer than 128 random ASCII
|
||||
characters; iterations are limited to 100000. Generated scripts are never executed.
|
||||
The harness does not write files, launch tools, or request repository credentials.
|
||||
The required `ci / conformance` job runs both seeds on PRs and `main`, and 10000
|
||||
iterations per seed on weekly schedules and release-tag validation, within its
|
||||
10-minute timeout. Future failing seeds must be retained alongside regression cases.
|
||||
|
||||
## Review record
|
||||
|
||||
- **Owner:** @KofTwentyTwo; **reviewed:** 2026-10-10; **next review:** 2027-01-08.
|
||||
- Seed 22023 exposed quoted-checkout and explicit-indentation block-scalar gaps;
|
||||
both now have deterministic self-test regressions. This is not a coverage claim.
|
||||
- Reassess targets when parser boundaries change; generated data alone does not
|
||||
establish completeness or substitute for ordinary package and checker tests.
|
||||
@@ -15,7 +15,11 @@ changes or predict future maintenance.
|
||||
Keep the accepted risk visible; dismissal is not a claim that the missing second
|
||||
maintainer exists. Do not disable Scorecard or weaken protections. Reopen and
|
||||
reassess these dispositions when their assumptions change. The LOW best-practices
|
||||
badge and MEDIUM fuzzing recommendations remain open for follow-up.
|
||||
badge and MEDIUM fuzzing recommendations remain visible. The
|
||||
[badge assessment](best-practices.md) records deferred enrollment under EX-0007;
|
||||
[native parser fuzzing](fuzzing.md) now supplies generated security properties and
|
||||
regressions. Scorecard may not recognize this PowerShell harness; do not label a
|
||||
score of zero as proof that tests did not run or claim a supported integration.
|
||||
|
||||
No OpenVEX statement is fabricated for these process findings: no vulnerable shipped
|
||||
component or exploit was identified. Any actual vulnerability present but not
|
||||
|
||||
@@ -11,6 +11,7 @@ Open and closed exceptions to the KofTwentyTwo standards. The process is in
|
||||
| [EX-0004](#ex-0004) | Open | K22-CI-30, K22-REL-04 (shared release workflow) | gclo | When the shared reusable release workflow ships, at the latest 2027-04-09 |
|
||||
| [EX-0005](#ex-0005) | Open | K22-AI-10 (agents via PR only), K22-AI-11 (in part: unconfirmed pushes) | second-brain, nix, Praetor, Jellyfin | Reviewed yearly, next on 2027-10-10 |
|
||||
| [EX-0006](#ex-0006) | Open | K22-CI-16 (tag-only environment secrets) | standards | 2027-04-10; review 2027-01-10 |
|
||||
| [EX-0007](#ex-0007) | Open | K22-REPO-12 (enrollment) | KofTwentyTwo/standards | 2026-11-10; review 2026-10-17 |
|
||||
|
||||
## EX-0001
|
||||
|
||||
@@ -169,3 +170,21 @@ Open and closed exceptions to the KofTwentyTwo standards. The process is in
|
||||
- **Expires:** 2027-04-10, or earlier when version proposals use credential federation.
|
||||
- **Owner:** the maintainer.
|
||||
- **Review date:** 2027-01-10.
|
||||
|
||||
## EX-0007
|
||||
|
||||
**Best Practices enrollment temporarily blocked by provider anti-spam**
|
||||
|
||||
- **Requirements:** `K22-REPO-12` enrollment and the actual README badge only.
|
||||
- **Scope:** `KofTwentyTwo/standards`; no downstream repository is covered.
|
||||
- **Reason:** on 2026-10-10 the maintainer could not sign in because of the provider's
|
||||
anti-spam restriction and explicitly deferred enrollment. No project ID exists.
|
||||
- **Risk:** consumers cannot inspect a registered provider assessment yet.
|
||||
- **Compensating controls:** publish the [criterion evidence](../docs/security/best-practices.md)
|
||||
and pending answers; retain existing security/PR gates and parser fuzzing. Do not
|
||||
claim participation or a passing badge. Retry normal login once the restriction
|
||||
clears, register the exact repository URL, and close this exception through a PR.
|
||||
- **Expires:** 2026-11-10 (UTC), or successful enrollment and README badge publication,
|
||||
whichever occurs first. **Review:** 2026-10-17.
|
||||
- **Owner:** @KofTwentyTwo. Approval is recorded by the normal exception-register PR;
|
||||
this entry does not authorize bypassing CI or publishing an unreviewed assessment.
|
||||
@@ -2,6 +2,7 @@
|
||||
"$schema": "https://raw.githubusercontent.com/googleapis/release-please/main/schemas/config.json",
|
||||
"release-type": "simple",
|
||||
"include-component-in-tag": false,
|
||||
"bump-minor-pre-major": false,
|
||||
"initial-version": "0.1.0",
|
||||
"changelog-sections": [
|
||||
{ "type": "security", "section": "Security", "hidden": false },
|
||||
|
||||
@@ -88,6 +88,20 @@ released software, links to the latest release and the verification instructions
|
||||
[releases](releases.md#verifying-a-release).
|
||||
*Why:* users can see the security posture before they install. *Verified by:* review.
|
||||
|
||||
**K22-REPO-12 (MUST)** Public, open-source Product repositories enroll in the
|
||||
[OpenSSF Best Practices program](https://www.bestpractices.dev/en), link their actual
|
||||
project badge from the README, and maintain an evidence-backed assessment in
|
||||
`docs/security/best-practices.md`. Reassess at least every 90 days and before each
|
||||
MINOR or MAJOR release. Record applicability, evidence path, and the actual project
|
||||
URL in `docs/security/assurance.json` using the [starter record](../templates/repo/docs/security/assurance.json).
|
||||
Private repositories record their scope and reason instead; Internal repositories
|
||||
follow the tier rules above. Registration obstacles require a dated exception, not
|
||||
a placeholder badge or a claim of passing. A passing badge is the adoption target;
|
||||
enrollment alone does not demonstrate that every criterion is met.
|
||||
*Why:* a public assessment makes security practice gaps visible and accountable.
|
||||
*Verified by:* conformance checker (record, evidence and badge link); maintainer
|
||||
review of the provider's project page, repository identity, and criterion answers.
|
||||
|
||||
## Branch and tag protection
|
||||
|
||||
Protection is configured with **repository rulesets**, never with legacy branch
|
||||
|
||||
@@ -80,3 +80,29 @@ release checklist.
|
||||
applications expose a data-directory override (for example an environment variable)
|
||||
that tests use.
|
||||
*Why:* running tests must be safe on any machine. *Verified by:* review.
|
||||
|
||||
## Parser fuzzing
|
||||
|
||||
**K22-TEST-25 (MUST)** Repositories with complex parsers of untrusted input (including
|
||||
workflow, configuration, protocol, archive, and document parsers) maintain fuzz
|
||||
targets for those boundaries. Use coverage-guided fuzzing where supported; otherwise
|
||||
use generated or mutated inputs with independent security properties. A crash-only
|
||||
smoke test is insufficient when acceptance of unsafe input is the relevant failure.
|
||||
|
||||
Run bounded campaigns in a merge-gating CI job on every PR and push to `main`, and
|
||||
longer campaigns at least weekly and before each MINOR or MAJOR release. Document
|
||||
targets, input limits, invariants, a local replay command, seeds or corpus, CI job,
|
||||
and failure handling in `docs/security/fuzzing.md`; declare the evidence, targets,
|
||||
command and workflow in `docs/security/assurance.json`. Preserve minimized failures
|
||||
as regression tests and fix or triage them before merging. Never execute generated
|
||||
attacker scripts, use real secrets, or write to a user's profile.
|
||||
|
||||
Repositories without a complex untrusted-input parser document that decision and
|
||||
its rationale in the same files; review applicability when input boundaries change
|
||||
and at least every 90 days. Documentation-only status does not exempt shipped tooling.
|
||||
*Why:* malformed inputs can cause unsafe acceptance as well as crashes; repeatable
|
||||
fuzzing exercises combinations ordinary examples miss. *Verified by:* conformance
|
||||
checker (scope record, target paths, command and CI reference); required CI execution
|
||||
and review of properties, corpus and applicability. See the
|
||||
[native PowerShell example](../docs/security/fuzzing.md). Scorecard recognition is
|
||||
useful evidence, but is not a substitute for running appropriate targets.
|
||||
@@ -51,6 +51,11 @@ sha256sum --check SHA256SUMS
|
||||
## Security
|
||||
|
||||
Report vulnerabilities privately; see [SECURITY.md](SECURITY.md).
|
||||
Maintain the [Best Practices assessment](docs/security/best-practices.md) and
|
||||
[parser fuzzing assessment](docs/security/fuzzing.md), with their actual scope and
|
||||
configuration in [assurance.json](docs/security/assurance.json). After enrollment,
|
||||
insert the provider's actual linked badge here; placeholders do not demonstrate
|
||||
participation.
|
||||
|
||||
## Contributing
|
||||
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"badge": {
|
||||
"applicability": "required",
|
||||
"reason": "Replace with this repository's public Product scope, or a reviewed private/Internal rationale.",
|
||||
"projectUrl": null,
|
||||
"evidence": "docs/security/best-practices.md"
|
||||
},
|
||||
"fuzzing": {
|
||||
"applicability": "required",
|
||||
"reason": "Replace with actual untrusted-input boundaries, or a reviewed not-applicable rationale.",
|
||||
"targets": [],
|
||||
"command": "",
|
||||
"workflow": "",
|
||||
"evidence": "docs/security/fuzzing.md"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
# OpenSSF Best Practices assessment
|
||||
|
||||
## Project and ownership
|
||||
|
||||
For a public open-source Product repository, register its exact canonical URL at
|
||||
<https://www.bestpractices.dev/en>. Replace `badge.projectUrl` in `assurance.json`
|
||||
with the actual `https://www.bestpractices.dev/projects/<numeric-id>` URL and add the
|
||||
provider's real linked badge to the README. Record the owner, review date and next
|
||||
review within 90 days. Private/Internal scope decisions need a documented rationale.
|
||||
|
||||
## Criterion evidence
|
||||
|
||||
Use the current [passing criteria](https://www.bestpractices.dev/en/criteria/0).
|
||||
For each answer, link actual source, policy, test or observed operational evidence.
|
||||
Separate implemented practice from intentions, pending human attestations, and
|
||||
justified not-applicable answers. Passing is the target, not an automatic claim
|
||||
made by a copied template. Record remaining gaps with owners and follow-up dates.
|
||||
|
||||
## Maintenance
|
||||
|
||||
Reassess at least every 90 days and before each MINOR or MAJOR release. Keep the
|
||||
provider's answers consistent with this evidence. A registration restriction needs
|
||||
a dated approved exception; never use another repository's badge or invent an ID.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Parser fuzzing assessment
|
||||
|
||||
## Applicability
|
||||
|
||||
Record the reviewed complex untrusted-input boundaries and their owner. If none
|
||||
exist, explain why shipped tooling, importers, configuration and protocol handling
|
||||
do not introduce one. Set `fuzzing.applicability` in `assurance.json` to `required`
|
||||
or `not-applicable` with the actual rationale. Unfilled templates do not conform.
|
||||
|
||||
## Targets and properties
|
||||
|
||||
List each target, synthetic input limits, security invariants and corpus or seeds.
|
||||
Use the language's native coverage-guided fuzzer where practical. Generated-input
|
||||
property tests are an alternative when instrumentation is unavailable; assertions
|
||||
must detect unsafe acceptance as well as crashes.
|
||||
|
||||
## Run and replay
|
||||
|
||||
Provide exact local commands, prerequisites, bounded PR/main campaign budgets,
|
||||
weekly and pre-MINOR/MAJOR budgets, and the required CI job. Reference the target
|
||||
paths, command and workflow in `assurance.json`; ensure the command actually runs
|
||||
there rather than merely appearing in a comment. Explain how to replay and minimize
|
||||
failures into regression tests. Never execute generated attacker scripts.
|
||||
|
||||
## Review
|
||||
|
||||
Name the owner, review date and next review (within 90 days). Reassess input
|
||||
boundaries when they change. Do not claim a Scorecard-supported integration unless
|
||||
it is genuinely configured and running; a score alone is not a test result.
|
||||
@@ -0,0 +1,135 @@
|
||||
#Requires -Version 7.4
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Runs bounded, reproducible property tests against the conformance parsers.
|
||||
.DESCRIPTION
|
||||
Constructs workflows with independently known security properties, varying quotes,
|
||||
indentation, comments, line endings, block scalars, and attacker expressions. Inputs
|
||||
are data only: no generated script is executed and no GitHub access is needed.
|
||||
A failure prints the seed, iteration, property, and synthetic input for replay.
|
||||
.PARAMETER Seed
|
||||
Seed for a deterministic sequence of generated cases.
|
||||
.PARAMETER Iterations
|
||||
Number of generated cases; bounded for pull-request CI.
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[int] $Seed = 22023,
|
||||
[ValidateRange(1, 100000)]
|
||||
[int] $Iterations = 1000
|
||||
)
|
||||
|
||||
Set-StrictMode -Version Latest
|
||||
$ErrorActionPreference = 'Stop'
|
||||
. (Join-Path $PSScriptRoot 'Test-RepoConformance.ps1') -LoadHelpers
|
||||
$random = [System.Random]::new($Seed)
|
||||
$assertions = 0
|
||||
$caseNumber = 0
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Fail with replay data when an independently generated security property is violated.
|
||||
#>
|
||||
function Assert-Property
|
||||
{
|
||||
param([bool] $Condition, [string] $Property, [string] $InputText)
|
||||
if(-not $Condition)
|
||||
{
|
||||
$replay = [ordered]@{ seed = $Seed; iteration = $caseNumber; property = $Property; input = $InputText }
|
||||
throw ($replay | ConvertTo-Json -Compress)
|
||||
}
|
||||
$script:assertions++
|
||||
}
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Generate synthetic hexadecimal action refs using the injected deterministic PRNG.
|
||||
#>
|
||||
function Get-HexString
|
||||
{
|
||||
param([int] $Length)
|
||||
$characters = for($position = 0; $position -lt $Length; $position++)
|
||||
{
|
||||
'0123456789abcdef'[$random.Next(16)]
|
||||
}
|
||||
return -join $characters
|
||||
}
|
||||
|
||||
for($caseNumber = 1; $caseNumber -le $Iterations; $caseNumber++)
|
||||
{
|
||||
$newline = @("`n", "`r`n")[$random.Next(2)]
|
||||
$indent = ' ' * $random.Next(2, 5)
|
||||
$quote = @('', "'", '"')[$random.Next(3)]
|
||||
$sha = Get-HexString 40
|
||||
$uses = 'actions/checkout@' + $sha
|
||||
$pinnedWorkflow = @(
|
||||
'name: generated'
|
||||
'on: pull_request'
|
||||
'permissions: {}'
|
||||
'jobs:'
|
||||
($indent + 'check:')
|
||||
($indent * 2 + 'runs-on: ubuntu-24.04')
|
||||
($indent * 2 + 'steps:')
|
||||
($indent * 3 + '- uses: ' + $quote + $uses + $quote + ' # pinned')
|
||||
($indent * 3 + ' with:')
|
||||
($indent * 3 + ' persist-credentials: false')
|
||||
) -join $newline
|
||||
$parsedUses = @(Get-WorkflowUses $pinnedWorkflow)
|
||||
Assert-Property -Condition ($parsedUses.Count -eq 1) -Property 'one generated action is found' -InputText $pinnedWorkflow
|
||||
Assert-Property -Condition ((Get-UsesPinState $parsedUses[0].Uses) -eq 'PINNED') -Property 'quoted full SHA remains pinned' -InputText $pinnedWorkflow
|
||||
Assert-Property -Condition (@(Find-PersistedCheckout $pinnedWorkflow).Count -eq 0) -Property 'explicit credential disabling is found' -InputText $pinnedWorkflow
|
||||
$unsafeCheckout = $pinnedWorkflow.Replace('persist-credentials: false', 'persist-credentials: true')
|
||||
Assert-Property -Condition (@(Find-PersistedCheckout $unsafeCheckout).Count -eq 1) -Property 'quoted unsafe checkout is never missed' -InputText $unsafeCheckout
|
||||
|
||||
$badRef = @('v7', 'main', (Get-HexString 39), (Get-HexString 41))[$random.Next(4)]
|
||||
Assert-Property -Condition ((Get-UsesPinState ('actions/checkout@' + $badRef)) -ne 'PINNED') -Property 'non-SHA refs never pass pinning' -InputText $badRef
|
||||
|
||||
$expression = @('github.event.pull_request.title', 'github.head_ref', 'inputs.payload')[$random.Next(3)]
|
||||
$indicator = @('|', '|-', '|+', '>', '>-', '>+', '|2', '|2-', '>2+')[$random.Next(9)]
|
||||
$expressionText = '${{ ' + $expression + ' }}'
|
||||
$injectionWorkflow = @(
|
||||
'permissions: {}'
|
||||
'jobs:'
|
||||
($indent + 'check:')
|
||||
($indent * 2 + 'steps:')
|
||||
($indent * 3 + '- run: ' + $indicator + ' # generated scalar')
|
||||
($indent * 3 + ' echo "' + $expressionText + '"')
|
||||
) -join $newline
|
||||
$findings = @(Find-TemplateInjection $injectionWorkflow -IncludeInputs)
|
||||
Assert-Property -Condition ($findings.Count -eq 1) -Property 'untrusted expression in a block scalar is found' -InputText $injectionWorkflow
|
||||
Assert-Property -Condition ($findings[0].Expression -eq $expression) -Property 'finding preserves the controlled expression' -InputText $injectionWorkflow
|
||||
$safeWorkflow = $injectionWorkflow.Replace($expressionText, '${{ github.sha }}')
|
||||
Assert-Property -Condition (@(Find-TemplateInjection $safeWorkflow -IncludeInputs).Count -eq 0) -Property 'trusted expression is not reported' -InputText $safeWorkflow
|
||||
$envWorkflow = @(
|
||||
'jobs:'
|
||||
($indent + 'check:')
|
||||
($indent * 2 + 'steps:')
|
||||
($indent * 3 + '- env:')
|
||||
($indent * 3 + ' TITLE: ' + $expressionText)
|
||||
($indent * 3 + ' run: echo "$TITLE"')
|
||||
) -join $newline
|
||||
Assert-Property -Condition (@(Find-TemplateInjection $envWorkflow -IncludeInputs).Count -eq 0) -Property 'expressions in env stay outside run bodies' -InputText $envWorkflow
|
||||
|
||||
$shortName = 'generated-' + $random.Next(100000).ToString([Globalization.CultureInfo]::InvariantCulture)
|
||||
$requirement = 'K22-CI-' + $random.Next(10, 25).ToString('00', [Globalization.CultureInfo]::InvariantCulture)
|
||||
$exception = '| [EX-0099](#ex-0099) | Open | ' + $requirement + ' | KofTwentyTwo/' + $shortName + ' | 2099-01-01 |'
|
||||
$register = @(Read-ExceptionRegister $exception)
|
||||
Assert-Property -Condition ($register.Count -eq 1) -Property 'generated exception row is parsed' -InputText $exception
|
||||
Assert-Property -Condition ($null -ne (Find-Exception $register $requirement ('KofTwentyTwo/' + $shortName))) -Property 'exact exception scope matches' -InputText $exception
|
||||
Assert-Property -Condition ($null -eq (Find-Exception $register $requirement ('KofTwentyTwo/' + $shortName + '-other'))) -Property 'partial repository names cannot gain an exception' -InputText $exception
|
||||
Assert-Property -Condition ($null -eq (Find-Exception $register $requirement ('AnotherOwner/' + $shortName))) -Property 'explicit scope cannot authorize another owner' -InputText $exception
|
||||
$closed = @(Read-ExceptionRegister $exception.Replace('| Open |', '| Closed |'))
|
||||
Assert-Property -Condition ($null -eq (Find-Exception $closed $requirement ('KofTwentyTwo/' + $shortName))) -Property 'closed exceptions cannot authorize a failure' -InputText $exception
|
||||
$expired = @(Read-ExceptionRegister $exception.Replace('2099-01-01', '2026-11-10'))
|
||||
Assert-Property -Condition ($null -eq (Find-Exception $expired $requirement ('KofTwentyTwo/' + $shortName) -Today ([datetime] '2026-11-10'))) -Property 'expired exceptions cannot authorize a failure' -InputText $exception
|
||||
|
||||
$noise = -join (1..$random.Next(1, 128) | ForEach-Object { [char] $random.Next(32, 127) })
|
||||
foreach($parser in 'Get-WorkflowUses', 'Get-TopLevelPermissions', 'Get-RunBlocks', 'Find-TemplateInjection', 'Find-PersistedCheckout', 'Read-ExceptionRegister')
|
||||
{
|
||||
& $parser $noise | Out-Null
|
||||
}
|
||||
Get-AssuranceCheck $noise @() @{} '' | Out-Null
|
||||
$assertions++
|
||||
}
|
||||
|
||||
Write-Output "PASS: conformance fuzz seed=$Seed iterations=$Iterations assertions=$assertions"
|
||||
@@ -4,8 +4,9 @@
|
||||
Checks a repository against the KofTwentyTwo repository and CI/CD standards.
|
||||
|
||||
.DESCRIPTION
|
||||
Evaluates every automatable requirement in standards/repository.md (K22-REPO-*)
|
||||
and standards/ci-cd.md (K22-CI-*) and prints one row per requirement:
|
||||
Evaluates automatable requirements in standards/repository.md (K22-REPO-*),
|
||||
standards/ci-cd.md (K22-CI-*), and the K22-TEST-25 fuzzing scope record, printing
|
||||
one row per requirement:
|
||||
|
||||
PASS the requirement is met
|
||||
FAIL a MUST is not met (or a SHOULD, reported as WARN instead)
|
||||
@@ -45,6 +46,9 @@
|
||||
.PARAMETER SelfTest
|
||||
Run the checker's own unit tests and exit.
|
||||
|
||||
.PARAMETER LoadHelpers
|
||||
Load pure parser helpers for the fuzz harness without checking a repository.
|
||||
|
||||
.EXAMPLE
|
||||
./tools/Test-RepoConformance.ps1 -Repository KofTwentyTwo/AppKit -LocalPath ../AppKit
|
||||
|
||||
@@ -58,7 +62,8 @@ param(
|
||||
[switch] $StaticOnly,
|
||||
[string] $StandardsPath = (Split-Path -Parent $PSScriptRoot),
|
||||
[switch] $Json,
|
||||
[switch] $SelfTest
|
||||
[switch] $SelfTest,
|
||||
[switch] $LoadHelpers
|
||||
)
|
||||
|
||||
Set-StrictMode -Version Latest
|
||||
@@ -72,7 +77,8 @@ $script:Levels = [ordered]@{
|
||||
'K22-REPO-01' = 'MUST'; 'K22-REPO-02' = 'MUST'; 'K22-REPO-03' = 'MUST'
|
||||
'K22-REPO-04' = 'MUST'; 'K22-REPO-05' = 'MUST'; 'K22-REPO-06' = 'MUST'
|
||||
'K22-REPO-07' = 'MUST'; 'K22-REPO-08' = 'MUST'; 'K22-REPO-09' = 'MUST'
|
||||
'K22-REPO-10' = 'SHOULD'; 'K22-REPO-11' = 'SHOULD'; 'K22-REPO-20' = 'MUST'
|
||||
'K22-REPO-10' = 'SHOULD'; 'K22-REPO-11' = 'SHOULD'; 'K22-REPO-12' = 'MUST'
|
||||
'K22-TEST-25' = 'MUST'; 'K22-REPO-20' = 'MUST'
|
||||
'K22-REPO-21' = 'MUST'; 'K22-REPO-22' = 'MUST'; 'K22-REPO-30' = 'MUST'
|
||||
'K22-REPO-31' = 'MUST'; 'K22-REPO-32' = 'SHOULD'; 'K22-REPO-40' = 'MUST'
|
||||
'K22-REPO-41' = 'SHOULD'; 'K22-REPO-42' = 'MUST'
|
||||
@@ -169,7 +175,7 @@ function Get-RunBlocks
|
||||
{
|
||||
$indent = $Matches[1].Length
|
||||
$rest = $Matches[2]
|
||||
if ($rest -and $rest -notmatch '^[|>][+-]?\s*$')
|
||||
if ($rest -and $rest -notmatch '^[|>](?:[1-9][+-]?|[+-][1-9]?|[+-]?)\s*(?:#.*)?$')
|
||||
{
|
||||
[pscustomobject]@{ Line = $i + 1; Body = $rest }
|
||||
continue
|
||||
@@ -226,7 +232,7 @@ function Find-PersistedCheckout
|
||||
$lines = $Text -split "`r?`n"
|
||||
for ($i = 0; $i -lt $lines.Count; $i++)
|
||||
{
|
||||
if ($lines[$i] -match '^(\s*)(-\s+)?uses:\s*actions/checkout@')
|
||||
if ($lines[$i] -match '^(\s*)(-\s+)?uses:\s*["'']?actions/checkout@')
|
||||
{
|
||||
$stepIndent = $Matches[1].Length
|
||||
$ok = $false
|
||||
@@ -263,6 +269,7 @@ function Read-ExceptionRegister
|
||||
Requirements = @($qualifiers.Keys)
|
||||
Qualifiers = $qualifiers
|
||||
Scope = $cells[3]
|
||||
Expires = if ($cells.Count -ge 5) { $cells[4] } else { '' }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -270,10 +277,18 @@ function Read-ExceptionRegister
|
||||
# Returns the open exception covering a requirement for a repository, or $null.
|
||||
function Find-Exception
|
||||
{
|
||||
param([object[]] $Register, [string] $RequirementId, [string] $RepositoryName, [string[]] $Aspects = @())
|
||||
param([object[]] $Register, [string] $RequirementId, [string] $RepositoryName, [string[]] $Aspects = @(), [datetime] $Today = [datetime]::UtcNow.Date)
|
||||
foreach ($ex in $Register)
|
||||
{
|
||||
if ($ex.Status -notmatch '^Open$') { continue }
|
||||
# Review dates are not expiry dates. Enforce an explicit ISO expiry or an
|
||||
# "at the latest" deadline; condition-only expiry still needs human review.
|
||||
if ($ex.Expires -match '(?i)(?:^|at (?:the )?latest\s+)(\d{4}-\d{2}-\d{2})')
|
||||
{
|
||||
$deadline = [datetime]::MinValue
|
||||
if (-not [datetime]::TryParseExact($Matches[1], 'yyyy-MM-dd', [Globalization.CultureInfo]::InvariantCulture, [Globalization.DateTimeStyles]::None, [ref] $deadline)) { continue }
|
||||
if ($Today.Date -ge $deadline.Date) { continue }
|
||||
}
|
||||
if ($ex.Requirements -notcontains $RequirementId) { continue }
|
||||
# A qualified exception covers a failure only when every failing aspect is
|
||||
# the qualified one.
|
||||
@@ -285,7 +300,8 @@ function Find-Exception
|
||||
}
|
||||
$short = if ($RepositoryName) { ($RepositoryName -split '/')[-1] } else { '' }
|
||||
if ($ex.Scope -match '(?i)^all repositories' -or
|
||||
($short -and $ex.Scope -match "(?i)(^|[\s/,(])$([regex]::Escape($short))($|[\s,)])"))
|
||||
($RepositoryName -and $ex.Scope -match "(?i)(^|[\s,(])$([regex]::Escape($RepositoryName))($|[\s,)])") -or
|
||||
($short -and $ex.Scope -match "(?i)(^|[\s,(])$([regex]::Escape($short))($|[\s,)])"))
|
||||
{
|
||||
return $ex
|
||||
}
|
||||
@@ -293,6 +309,78 @@ function Find-Exception
|
||||
return $null
|
||||
}
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Validate declared assurance evidence without executing commands or claiming certification.
|
||||
#>
|
||||
function Get-AssuranceCheck
|
||||
{
|
||||
param([string] $Text, [string[]] $Paths, [hashtable] $Workflows, [string] $Readme)
|
||||
try
|
||||
{
|
||||
$record = ConvertFrom-Json -InputObject $Text -AsHashtable -ErrorAction Stop
|
||||
if ($record -isnot [System.Collections.IDictionary] -or $record['schemaVersion'] -isnot [long] -or $record['schemaVersion'] -ne 1) { throw 'unsupported schema' }
|
||||
}
|
||||
catch
|
||||
{
|
||||
foreach ($id in 'K22-REPO-12', 'K22-TEST-25')
|
||||
{
|
||||
[pscustomobject]@{ Id = $id; Status = 'FAIL'; Detail = 'missing or invalid docs/security/assurance.json (schemaVersion 1)'; Aspects = @('record') }
|
||||
}
|
||||
return
|
||||
}
|
||||
foreach ($pair in @(@('badge', 'K22-REPO-12'), @('fuzzing', 'K22-TEST-25')))
|
||||
{
|
||||
$kind, $id = $pair
|
||||
$entry = $record[$kind]
|
||||
$status, $detail, $aspect = 'FAIL', 'invalid assurance scope, rationale or evidence path', 'record'
|
||||
if ($entry -is [System.Collections.IDictionary] -and
|
||||
$entry['applicability'] -in @('required', 'not-applicable') -and
|
||||
$entry['reason'] -is [string] -and -not [string]::IsNullOrWhiteSpace($entry['reason']) -and
|
||||
$entry['evidence'] -is [string] -and $entry['evidence'] -cmatch '^docs/security/[A-Za-z0-9._/-]+\.md$' -and
|
||||
$entry['evidence'] -notmatch '(^|/)\.\.?(/|$)' -and $Paths -ccontains $entry['evidence'])
|
||||
{
|
||||
if ($entry['applicability'] -eq 'not-applicable')
|
||||
{
|
||||
$status, $detail = 'REVIEW', "$kind scope declared not applicable; verify rationale and input/public-Product scope in $($entry.evidence)"
|
||||
}
|
||||
elseif ($kind -eq 'badge')
|
||||
{
|
||||
$aspect = 'enrollment'
|
||||
$detail = 'badge enrollment missing; register this repository or obtain a dated enrollment exception'
|
||||
if ($entry['projectUrl'] -is [string] -and $entry['projectUrl'] -cmatch '^https://www\.bestpractices\.dev/(?:en/)?projects/([1-9][0-9]*)/?$')
|
||||
{
|
||||
$projectId = $Matches[1]
|
||||
$badgePattern = '\[!\[[^\]]*\]\(https://www\.bestpractices\.dev/projects/' + $projectId + '/badge\)\]\(https://www\.bestpractices\.dev/(?:en/)?projects/' + $projectId + '/?\)'
|
||||
if ($Readme -cmatch $badgePattern)
|
||||
{
|
||||
$status, $detail = 'REVIEW', "badge linked; verify $($entry.projectUrl) belongs to this repository, current answers and 90-day review in $($entry.evidence)"
|
||||
}
|
||||
else { $detail = 'README lacks the actual linked badge matching the declared project ID' }
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
$aspect = 'fuzzing'
|
||||
$targets = @($entry['targets'])
|
||||
$validTargets = $targets.Count -gt 0 -and @($targets | Where-Object {
|
||||
$_ -isnot [string] -or $_ -match '(^[/\\]|^[A-Za-z]:|(^|[/\\])\.\.?([/\\]|$))' -or $Paths -cnotcontains $_
|
||||
}).Count -eq 0
|
||||
$validWorkflow = $entry['workflow'] -is [string] -and $entry['workflow'] -cmatch '^\.github/workflows/[^/]+\.ya?ml$' -and $Workflows.ContainsKey($entry['workflow'])
|
||||
$validCommand = $entry['command'] -is [string] -and -not [string]::IsNullOrWhiteSpace($entry['command']) -and $entry['command'] -notmatch '[\r\n]'
|
||||
$runs = if ($validWorkflow) { @(Get-RunBlocks $Workflows[$entry['workflow']] | ForEach-Object Body) -join "`n" } else { '' }
|
||||
$commandInRun = $validCommand -and $runs -cmatch ('(?m)^\s*' + [regex]::Escape($entry['command'].Trim()) + '(?:\s|$)')
|
||||
if ($validTargets -and $validWorkflow -and $commandInRun)
|
||||
{
|
||||
$status, $detail = 'REVIEW', "fuzz targets and run command found; verify required PR/main gate, weekly/release budgets, properties and replay in $($entry.evidence)"
|
||||
}
|
||||
else { $detail = 'declared fuzz targets must exist and the command must begin a run line in the declared workflow' }
|
||||
}
|
||||
}
|
||||
[pscustomobject]@{ Id = $id; Status = $status; Detail = $detail; Aspects = @($aspect) }
|
||||
}
|
||||
}
|
||||
|
||||
# Lock files expected for each manifest found (K22-REPO-09).
|
||||
$script:LockRules = @(
|
||||
@{ Manifest = '(^|/)[^/]+\.(cs|fs|vb)proj$|(^|/)Directory\.Packages\.props$'; Locks = 'packages\.lock\.json$'; Name = 'NuGet (packages.lock.json)' }
|
||||
@@ -395,6 +483,8 @@ jobs:
|
||||
Assert ($inj[1].Expression -eq 'github.head_ref') 'injection: head_ref'
|
||||
$persisted = @(Find-PersistedCheckout $wf)
|
||||
Assert ($persisted.Count -eq 1 -and $persisted[0] -eq 9) "checkout: persisted ($($persisted -join ','))"
|
||||
Assert (@(Find-PersistedCheckout "jobs:`n check:`n steps:`n - uses: 'actions/checkout@v7'`n with:`n persist-credentials: true").Count -eq 1) 'checkout: quoted unsafe action'
|
||||
Assert (@(Find-TemplateInjection "jobs:`n check:`n steps:`n - run: |2 # explicit indentation`n echo `${{ github.head_ref }}").Count -eq 1) 'injection: block scalar indentation and comment'
|
||||
|
||||
$register = @"
|
||||
| ID | Status | Requirements | Scope | Expires |
|
||||
@@ -416,6 +506,24 @@ jobs:
|
||||
Assert ($null -eq (Find-Exception $parsed 'K22-REPO-05' 'KofTwentyTwo/AppKit')) 'exception: closed'
|
||||
Assert ((Find-Exception $parsed 'K22-CODE-CS-03' 'KofTwentyTwo/AppKit').Id -eq 'EX-0004') 'exception: short name scope'
|
||||
Assert ($null -eq (Find-Exception $parsed 'K22-CODE-CS-03' 'KofTwentyTwo/AppKitExtras')) 'exception: no partial name match'
|
||||
$expired = @(Read-ExceptionRegister '| [EX-0099](#ex-0099) | Open | K22-REPO-12 | standards | 2026-11-10; review 2026-10-17 |')
|
||||
Assert ($null -ne (Find-Exception $expired 'K22-REPO-12' 'standards' -Today ([datetime] '2026-11-09'))) 'exception: valid before deadline'
|
||||
Assert ($null -eq (Find-Exception $expired 'K22-REPO-12' 'standards' -Today ([datetime] '2026-11-10'))) 'exception: expiry cannot silently authorize failure'
|
||||
$exactScope = @(Read-ExceptionRegister '| [EX-0099](#ex-0099) | Open | K22-REPO-12 | KofTwentyTwo/standards | 2099-01-01 |')
|
||||
Assert ($null -eq (Find-Exception $exactScope 'K22-REPO-12' 'AnotherOwner/standards')) 'exception: explicit owner scope cannot match another owner'
|
||||
Assert (@(Get-AssuranceCheck '{}' @() @{} '').Count -eq 2) 'assurance: missing schema returns both failures'
|
||||
Assert (@(Get-AssuranceCheck '{"schemaVersion":true}' @() @{} '' | Where-Object Status -eq 'FAIL').Count -eq 2) 'assurance: boolean cannot masquerade as schema version 1'
|
||||
Assert (@(Get-AssuranceCheck '{"schemaVersion":1,"badge":{},"fuzzing":{}}' @() @{} '' | Where-Object Status -eq 'FAIL').Count -eq 2) 'assurance: incomplete entries fail without crashing'
|
||||
$assurance = '{"schemaVersion":1,"badge":{"applicability":"required","reason":"public Product","projectUrl":"https://www.bestpractices.dev/projects/123","evidence":"docs/security/best-practices.md"},"fuzzing":{"applicability":"required","reason":"parser","targets":["tools/parser.ps1"],"command":"./tools/fuzz.ps1","workflow":".github/workflows/ci.yml","evidence":"docs/security/fuzzing.md"}}'
|
||||
$assurancePaths = @('tools/parser.ps1', 'docs/security/best-practices.md', 'docs/security/fuzzing.md')
|
||||
$assuranceWorkflow = @{ '.github/workflows/ci.yml' = "jobs:`n tests:`n steps:`n - run: ./tools/fuzz.ps1 -Iterations 10" }
|
||||
$assuranceReadme = '[](https://www.bestpractices.dev/projects/123)'
|
||||
$assuranceRows = @(Get-AssuranceCheck $assurance $assurancePaths $assuranceWorkflow $assuranceReadme)
|
||||
Assert (@($assuranceRows | Where-Object Status -ne 'REVIEW').Count -eq 0) 'assurance: evidence needs human verification, never fabricated certification'
|
||||
Assert (@(Get-AssuranceCheck $assurance $assurancePaths $assuranceWorkflow '' | Where-Object { $_.Id -eq 'K22-REPO-12' -and $_.Status -eq 'FAIL' }).Count -eq 1) 'assurance: missing badge fails'
|
||||
$assuranceWorkflow['.github/workflows/ci.yml'] = "jobs:`n tests:`n steps:`n - run: |`n # ./tools/fuzz.ps1"
|
||||
Assert (@(Get-AssuranceCheck $assurance $assurancePaths $assuranceWorkflow $assuranceReadme | Where-Object { $_.Id -eq 'K22-TEST-25' -and $_.Status -eq 'FAIL' }).Count -eq 1) 'assurance: comment does not execute a fuzz command'
|
||||
Assert (@(Get-AssuranceCheck ($assurance.Replace('tools/parser.ps1', '../parser.ps1')) $assurancePaths $assuranceWorkflow $assuranceReadme | Where-Object { $_.Id -eq 'K22-TEST-25' -and $_.Status -eq 'FAIL' }).Count -eq 1) 'assurance: traversal paths cannot satisfy targets'
|
||||
|
||||
Assert (@(Get-MissingLocks @('src/a/a.csproj', 'README.md')) -contains 'NuGet (packages.lock.json)') 'locks: missing nuget'
|
||||
Assert (@(Get-MissingLocks @('src/a/a.csproj', 'src/a/packages.lock.json')).Count -eq 0) 'locks: nuget ok'
|
||||
@@ -450,6 +558,7 @@ jobs:
|
||||
exit 0
|
||||
}
|
||||
|
||||
if ($LoadHelpers) { return }
|
||||
if ($SelfTest) { Invoke-SelfTest }
|
||||
|
||||
# =================================================================================
|
||||
@@ -534,6 +643,13 @@ function Find-FirstPath
|
||||
}
|
||||
|
||||
$repoName = if ($Repository) { $Repository } else { Split-Path -Leaf $script:LocalRoot }
|
||||
if (-not $Repository)
|
||||
{
|
||||
# Local static checks can identify exact exception scope without contacting
|
||||
# GitHub. A clone with no canonical GitHub origin keeps its directory name.
|
||||
$origin = & git -C $script:LocalRoot remote get-url origin 2>$null
|
||||
if ($LASTEXITCODE -eq 0 -and $origin -match '^(?:https://github\.com/|git@github\.com:|ssh://git@(?:github\.com(?::22)?|ssh\.github\.com:443)/)([^/]+/[^/]+?)(?:\.git)?$') { $repoName = $Matches[1] }
|
||||
}
|
||||
|
||||
# =================================================================================
|
||||
# Static checks: required files (K22-REPO-01 .. 11)
|
||||
@@ -662,6 +778,15 @@ $workflows = @{}
|
||||
foreach ($p in $workflowPaths) { $workflows[$p] = Get-FileText $p }
|
||||
$allWorkflowText = ($workflows.Values -join "`n")
|
||||
|
||||
# Assurance records fail closed when required evidence is absent. A positive static
|
||||
# check is REVIEW: identity, applicability and actual CI execution need verification.
|
||||
$assuranceReadme = if ($readmePath) { Get-FileText $readmePath } else { '' }
|
||||
if ($assuranceReadme -match '(?im)\bTier(?:\*\*)?\s*[:|]\s*(?:\*\*)?\s*Internal\b') { $script:Levels['K22-REPO-12'] = 'SHOULD' }
|
||||
foreach ($check in Get-AssuranceCheck (Get-FileText 'docs/security/assurance.json') $script:Paths $workflows $assuranceReadme)
|
||||
{
|
||||
Add-Result $check.Id $check.Status $check.Detail $check.Aspects
|
||||
}
|
||||
|
||||
function Test-Calls
|
||||
{
|
||||
# True when any workflow calls the named standards reusable workflow (by remote
|
||||
|
||||
Reference in new issue
Block a user