detect: confine header detection to avoid deleting adjacent comments (#30) #41

Merged
KofTwentyTwo merged 1 commits from feature/GH-30-detect-overmatch into develop 2026-06-09 15:17:56 +00:00
KofTwentyTwo commented 2026-06-09 13:46:08 +00:00 (Migrated from github.com)

Closes #30

Problem (safety-critical)

License-header detection could over-match and absorb adjacent source content, which on apply (relicense / year-refresh) caused render.replaceSpan to delete a real doc comment. A false positive in detection means the tool removes content it does not own.

Two distinct over-match paths, both on line-comment file types:

  1. [HIGH] lineCommentRegion tolerated truly-blank lines between comment lines, so the layout license header / blank line / unrelated file doc comment collapsed into one detected span. The doc comment was then deleted on the next relicense or year-refresh.
  2. [MED] spdxIdentifierTag matched SPDX-License-Identifier: anywhere in the leading comment via strings.Index, so a comment that merely mentioned the tag in prose was treated as a managed header.

Fix

  1. lineCommentRegion now spans only contiguous comment lines: any non-comment line, including a truly-blank line, ends the run.
  2. spdxIdentifierTag now requires the tag to begin a comment line (leading whitespace allowed), not appear mid-prose.

When in doubt the detector now under-detects (leaves a foreign comment alone) rather than over-detecting.

Why this is safe for idempotency / round-trip

The renderer never emits a truly-blank internal line in a managed header. For line-comment styles, the separator between REUSE tags and the notice block is rendered as a bare comment prefix (e.g. # or //), which is still a comment line and therefore stays in-region. Confining detection to contiguous comment lines is thus byte-idempotent for every header this tool writes, while it stops absorbing foreign comments that a blank line has separated from the header.

Verified by:

  • The existing TestSpliceIdempotent, TestApplyFile (re-apply is idempotent), and TestApplyFileIdempotentAfterPreserveFirstConstructs / ...ShebangPHPIdempotent suites — all still green.
  • A new end-to-end applier test TestApplyFilePreservesDocCommentBelowBlankLine: inserts a header into a Python file whose body opens with a doc comment, relicenses MIT -> Apache-2.0, and asserts the doc comment below the blank separator survives the replace.

TDD

Red-green: failing tests written and observed failing for the right reason before the fix.

  • detect: blank line ends the region (doc comment excluded); bare-prefix separator stays in-region; SPDX-License-Identifier mentioned mid-prose is not a managed header; own-line tag (incl. leading whitespace) still qualifies; Detect-level span guard.
  • applier: the relicense round-trip survival test above.
  • The previously-enshrining region test (detect_test.go) was rewritten to require the corrected contiguous behavior, with the safety rationale inline.

Deferred

The [LOW] sub-issue (block detection spans only the first /* */, so a stacked two-block header is not fully de-stacked) is intentionally deferred. Expanding block detection to span multiple blocks would increase over-detection risk, which contradicts the safety mandate to prefer under-detection. Spanning only the first block remains the safe choice.

Gate

  • gofmt -l . clean
  • go vet ./... clean
  • go test ./... -race -coverpkg=./internal/...,./cmd/... -covermode=atomic passing
  • coverage threshold: 100% (2079/2079)
  • golangci-lint run: 0 issues
Closes #30 ## Problem (safety-critical) License-header detection could over-match and absorb adjacent source content, which on `apply` (relicense / year-refresh) caused `render.replaceSpan` to **delete a real doc comment**. A false positive in detection means the tool removes content it does not own. Two distinct over-match paths, both on line-comment file types: 1. **[HIGH]** `lineCommentRegion` tolerated truly-blank lines between comment lines, so the layout `license header` / blank line / `unrelated file doc comment` collapsed into one detected span. The doc comment was then deleted on the next relicense or year-refresh. 2. **[MED]** `spdxIdentifierTag` matched `SPDX-License-Identifier:` *anywhere* in the leading comment via `strings.Index`, so a comment that merely mentioned the tag in prose was treated as a managed header. ## Fix 1. `lineCommentRegion` now spans only **contiguous** comment lines: any non-comment line, **including a truly-blank line**, ends the run. 2. `spdxIdentifierTag` now requires the tag to **begin a comment line** (leading whitespace allowed), not appear mid-prose. When in doubt the detector now **under-detects** (leaves a foreign comment alone) rather than over-detecting. ## Why this is safe for idempotency / round-trip The renderer never emits a truly-blank internal line in a managed header. For line-comment styles, the separator between REUSE tags and the notice block is rendered as a **bare comment prefix** (e.g. `#` or `//`), which is still a comment line and therefore stays in-region. Confining detection to contiguous comment lines is thus byte-idempotent for every header this tool writes, while it stops absorbing foreign comments that a blank line has separated from the header. Verified by: - The existing `TestSpliceIdempotent`, `TestApplyFile` (re-apply is idempotent), and `TestApplyFileIdempotentAfterPreserveFirstConstructs` / `...ShebangPHPIdempotent` suites — all still green. - A new end-to-end applier test `TestApplyFilePreservesDocCommentBelowBlankLine`: inserts a header into a Python file whose body opens with a doc comment, relicenses MIT -> Apache-2.0, and asserts the doc comment below the blank separator survives the replace. ## TDD Red-green: failing tests written and observed failing for the right reason before the fix. - detect: blank line ends the region (doc comment excluded); bare-prefix separator stays in-region; `SPDX-License-Identifier` mentioned mid-prose is not a managed header; own-line tag (incl. leading whitespace) still qualifies; Detect-level span guard. - applier: the relicense round-trip survival test above. - The previously-enshrining region test (`detect_test.go`) was rewritten to require the corrected contiguous behavior, with the safety rationale inline. ## Deferred The **[LOW]** sub-issue (block detection spans only the first `/* */`, so a stacked two-block header is not fully de-stacked) is intentionally deferred. Expanding block detection to span multiple blocks would *increase* over-detection risk, which contradicts the safety mandate to prefer under-detection. Spanning only the first block remains the safe choice. ## Gate - `gofmt -l .` clean - `go vet ./...` clean - `go test ./... -race -coverpkg=./internal/...,./cmd/... -covermode=atomic` passing - coverage threshold: **100% (2079/2079)** - `golangci-lint run`: 0 issues
Sign in to join this conversation.