Python coding equals pragma is not preserved before inserted headers #24

New Issue

2026-06-04T16:19:30Z

KofTwentyTwo commented

2026-06-04 16:19:30 +00:00

(Migrated from github.com)

Finding

Python encoding pragmas using coding= are recognized by detection, but insertion only preserves pragmas containing coding:. A header can be inserted before a valid encoding declaration.

Evidence

docs/license-tool-requirements.md:24 requires encoding pragma preservation.
docs/license-tool-design.md:94 lists preserve-first encoding pragma coverage as a tested edge case.
internal/detect/detect.go:511 to internal/detect/detect.go:520 treats both coding: and coding= as coding pragmas.
internal/render/render.go:359 to internal/render/render.go:363 preserves only lines containing coding: during insertion.
internal/render/render_test.go:414 to internal/render/render_test.go:427 covers coding: utf-8, but not coding=utf-8.

Reproduction

Create a Python file whose first line is # coding=utf-8.
Run apply in write mode.
Observe that the managed header is inserted before the coding pragma.

Why It Matters

Python encoding declarations must be on the first or second line. Moving one down can change interpreter and tooling behavior for files that rely on a declared encoding, and it violates the preserve-first contract.

Suggested Fix

Share the same coding-pragma predicate between detection and insertion, or update render insertion to match coding= as well. Add regression coverage for both forms.

## Finding Python encoding pragmas using `coding=` are recognized by detection, but insertion only preserves pragmas containing `coding:`. A header can be inserted before a valid encoding declaration. ## Evidence - `docs/license-tool-requirements.md:24` requires encoding pragma preservation. - `docs/license-tool-design.md:94` lists preserve-first encoding pragma coverage as a tested edge case. - `internal/detect/detect.go:511` to `internal/detect/detect.go:520` treats both `coding:` and `coding=` as coding pragmas. - `internal/render/render.go:359` to `internal/render/render.go:363` preserves only lines containing `coding:` during insertion. - `internal/render/render_test.go:414` to `internal/render/render_test.go:427` covers `coding: utf-8`, but not `coding=utf-8`. ## Reproduction 1. Create a Python file whose first line is `# coding=utf-8`. 2. Run apply in write mode. 3. Observe that the managed header is inserted before the coding pragma. ## Why It Matters Python encoding declarations must be on the first or second line. Moving one down can change interpreter and tooling behavior for files that rely on a declared encoding, and it violates the preserve-first contract. ## Suggested Fix Share the same coding-pragma predicate between detection and insertion, or update render insertion to match `coding=` as well. Add regression coverage for both forms.

KofTwentyTwo commented

2026-06-04 16:31:32 +00:00

(Migrated from github.com)

Status: confirmed — detection treats coding= and coding: as pragmas, but the insertion path matches only coding:, so a header is spliced before a coding= pragma.

Severity: medium (matches the bug label). Justification: silent, write-mode source corruption of a semantically-load-bearing line, but scoped to Python/Ruby files whose first/second line uses the coding= form (less common than coding:). The shebang+pragma case is the worst variant: it pushes the pragma from line 2 to line 5, outside Python's first-or-second-line encoding window. Not high because it is form-specific and does not affect the more common coding: spelling.

Verified root cause: internal/render/render.go:preserveBoundary:359-363 — the PreserveCodingPragma case uses an inline predicate strings.HasPrefix(t, "#") && strings.Contains(t, "coding:"), which omits the coding= form. The detection side is correct and symmetric: internal/detect/detect.go:isCodingPragma:519 returns on coding: or coding=, consumed at internal/detect/detect.go:208. The two paths drift. (Note: render uses a private inline closure, not the isCodingPragma helper — they are separate functions in separate packages.)

Reproduced with a binary built from HEAD (v0.2.1) in a temp git repo, apply --write --license MIT:

# coding=utf-8 (line 1) → header inserted at byte 0; pragma demoted to line 5. BUG.
# coding: utf-8 (line 1) → header inserted after the pragma. Correct (control).
#!/usr/bin/env python3 + # coding=utf-8 → shebang preserved, but header inserted before the coding= pragma, demoting it from line 2 to line 5. BUG.

Suggested fix: Make the insertion predicate match detection. The cleanest local fix is to export/share the predicate so render.go:359-363 reuses the same coding:-or-coding= test instead of duplicating a narrower copy. This mirrors the v0.2.1 universal-shebang precedent (commit f4a111f), where the fix had to land on both boundary computations to stay consistent. Concretely the change touches:

internal/render/render.go:362 (add the coding= alternative, or call a shared helper), and
it should not need a change in internal/detect/detect.go (already correct) — but any shared-predicate refactor must keep both call sites (detect:208 and render:359) pointed at the same function.
The filetype table (internal/filetype/filetype.go:106,118 — Python and Ruby carry after(PreserveCodingPragma)) is already correct and needs no change; the rule exists, only render's matcher is too narrow.

Keep the 100% gate green: internal/detect/detect_test.go:282-283 ("coding pragma with equals form") and :710 (isCodingPragma("# coding=latin-1")) already cover the coding= form on the detect side, but internal/render/render_test.go:414-427 (TestSpliceInsertAfterCodingPragma) only exercises coding:. Add a render-side regression for # coding=utf-8 (both first-line and shebang+second-line variants) so the fixed branch is exercised and the asymmetry can't silently return.

Corrections to the issue: minor only. (1) The body's "Share the same coding-pragma predicate between detection and insertion" framing is exactly right, but note render does not currently call any helper — it inlines its own closure, so the fix is to introduce/share one, not to "reuse an existing" helper. (2) Impact is slightly broader than the title implies: Ruby (.rb) also carries PreserveCodingPragma (filetype.go:118), so the bug is not Python-only. Otherwise every cited line ref (detect.go:511-520, render.go:359-363, render_test.go:414-427, requirements.md:24, design.md:94) checks out.

Open questions: (1) Scope — fix narrowly in render (one-line predicate change + test) now, or fold into the shared-predicate refactor tracked by #27? (2) Should the shared predicate also normalize the broader set the docstring hints at (e.g. # vim: declarations are mentioned in the detect.go comment but neither path actually matches non-coding vim modelines today) — or keep the fix strictly to coding=/coding: parity? Recommend the latter to avoid scope creep.

Overlaps: #27 (deduplicate header placement/sentinel across detect and render) explicitly names this exact coding= drift as its motivating example — a structural fix for #27 subsumes #24; a targeted fix for #24 should be designed not to conflict with it. Same defect family (header inserted before a must-lead construct) as #7 (CSS @charset), #8 (HTML <!DOCTYPE>), and #6 (Go //go:build).

Automated maintainer review.

**Status**: confirmed — detection treats `coding=` and `coding:` as pragmas, but the insertion path matches only `coding:`, so a header is spliced *before* a `coding=` pragma. **Severity**: medium (matches the `bug` label). Justification: silent, write-mode source corruption of a semantically-load-bearing line, but scoped to Python/Ruby files whose first/second line uses the `coding=` form (less common than `coding:`). The shebang+pragma case is the worst variant: it pushes the pragma from line 2 to line 5, outside Python's first-or-second-line encoding window. Not high because it is form-specific and does not affect the more common `coding:` spelling. **Verified root cause**: `internal/render/render.go:preserveBoundary:359-363` — the `PreserveCodingPragma` case uses an inline predicate `strings.HasPrefix(t, "#") && strings.Contains(t, "coding:")`, which omits the `coding=` form. The detection side is correct and symmetric: `internal/detect/detect.go:isCodingPragma:519` returns on `coding:` **or** `coding=`, consumed at `internal/detect/detect.go:208`. The two paths drift. (Note: render uses a private inline closure, not the `isCodingPragma` helper — they are separate functions in separate packages.) Reproduced with a binary built from `HEAD` (v0.2.1) in a temp git repo, `apply --write --license MIT`: - `# coding=utf-8` (line 1) → header inserted at byte 0; pragma demoted to line 5. **BUG.** - `# coding: utf-8` (line 1) → header inserted *after* the pragma. Correct (control). - `#!/usr/bin/env python3` + `# coding=utf-8` → shebang preserved, but header inserted before the `coding=` pragma, demoting it from line 2 to line 5. **BUG.** **Suggested fix**: Make the insertion predicate match detection. The cleanest local fix is to export/share the predicate so render.go:359-363 reuses the same `coding:`-or-`coding=` test instead of duplicating a narrower copy. This mirrors the v0.2.1 universal-shebang precedent (commit f4a111f), where the fix had to land on **both** boundary computations to stay consistent. Concretely the change touches: - `internal/render/render.go:362` (add the `coding=` alternative, or call a shared helper), and - it should *not* need a change in `internal/detect/detect.go` (already correct) — but any shared-predicate refactor must keep both call sites (detect:208 and render:359) pointed at the same function. - The filetype table (`internal/filetype/filetype.go:106,118` — Python and Ruby carry `after(PreserveCodingPragma)`) is already correct and needs no change; the rule exists, only render's matcher is too narrow. Keep the 100% gate green: `internal/detect/detect_test.go:282-283` ("coding pragma with equals form") and `:710` (`isCodingPragma("# coding=latin-1")`) already cover the `coding=` form on the detect side, but `internal/render/render_test.go:414-427` (`TestSpliceInsertAfterCodingPragma`) only exercises `coding:`. Add a render-side regression for `# coding=utf-8` (both first-line and shebang+second-line variants) so the fixed branch is exercised and the asymmetry can't silently return. **Corrections to the issue**: minor only. (1) The body's "Share the same coding-pragma predicate between detection and insertion" framing is exactly right, but note render does not currently call any helper — it inlines its own closure, so the fix is to introduce/share one, not to "reuse an existing" helper. (2) Impact is slightly broader than the title implies: Ruby (`.rb`) also carries `PreserveCodingPragma` (`filetype.go:118`), so the bug is not Python-only. Otherwise every cited line ref (detect.go:511-520, render.go:359-363, render_test.go:414-427, requirements.md:24, design.md:94) checks out. **Open questions**: (1) Scope — fix narrowly in render (one-line predicate change + test) now, or fold into the shared-predicate refactor tracked by #27? (2) Should the shared predicate also normalize the broader set the docstring hints at (e.g. `# vim:` declarations are mentioned in the detect.go comment but neither path actually matches non-`coding` vim modelines today) — or keep the fix strictly to `coding=`/`coding:` parity? Recommend the latter to avoid scope creep. **Overlaps**: #27 (deduplicate header placement/sentinel across detect and render) explicitly names this exact `coding=` drift as its motivating example — a structural fix for #27 subsumes #24; a targeted fix for #24 should be designed not to conflict with it. Same defect family (header inserted before a must-lead construct) as #7 (CSS `@charset`), #8 (HTML `<!DOCTYPE>`), and #6 (Go `//go:build`). _Automated maintainer review._

KofTwentyTwo commented

2026-06-04 20:28:20 +00:00

(Migrated from github.com)

Resolved on develop in commit 742c124e6ec2afa9312075db3112bc2def3c91ff.

Validation:

CI: success, https://github.com/KofTwentyTwo/license-tool/actions/runs/26977377097
CodeQL: success, https://github.com/KofTwentyTwo/license-tool/actions/runs/26977376421
Coverage gate: 100% (2087/2087)

Closing as fixed.

Resolved on `develop` in commit `742c124e6ec2afa9312075db3112bc2def3c91ff`. Validation: - CI: success, https://github.com/KofTwentyTwo/license-tool/actions/runs/26977377097 - CodeQL: success, https://github.com/KofTwentyTwo/license-tool/actions/runs/26977376421 - Coverage gate: 100% (2087/2087) Closing as fixed.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/license-tool#24