zscripts-token-savers/tests/PlainTextTwins.Tests.ps1
Kelly Michels 573e01021b
docs(security): security notes, and a twin for every root .md (#78)
Defers to the org policy for how to report and covers what is particular to a
repository that is a sanitised mirror: the most valuable report here is not a
crash, it is something REAL that should not be here - a credential, an internal
hostname, an operator path, an identifier naming a private project. Mail those
rather than filing an issue, because a public issue about a leaked secret
publishes it a second time.

It also says what the automated check is and is not. The sanitisation suite is
a DENYLIST: it proves the absence of known patterns, not the absence of
secrets. Green tests are why a human report is still worth sending.

And the ordinary warning for what these actually are - automation that
archives a tree, uploads it, rebuilds containers and restarts services. Read
before running, nothing here is a sandbox, the config is yours to replace.

TWINS ARE NOW DISCOVERED, NOT LISTED. PAIRS was hand-kept and two files had
outgrown it: ELEVATOR_PITCH.md and TOKEN_SAVINGS.md had no twin at all. Adding
a document and remembering to add it to a list are two acts, and the second is
the one that gets skipped.

The Pester test had the same shape in reverse - it scraped PAIRS out of the
generator's source, so it could only prove the list was self-consistent and a
document nobody listed was invisible to it. It now asks the REPOSITORY what
markdown it has. Proven by deleting SECURITY.txt and watching two tests fail.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 00:35:29 -05:00

74 lines
3.0 KiB
PowerShell

# The .txt twins are generated, and this is what makes that true.
#
# Invoke-Pester .\tests
#
# scripts/plaintext_twins.py has always shipped a --check mode, and its
# docstring claimed "the test suite runs --check". Nothing ran it. So README.txt
# could drift from README.md silently, and CHANGELOG.txt - which no generator
# covered at all - actually did.
#
# A twin that disagrees with the file it mirrors is worse than no twin: it is a
# second document that looks authoritative and is wrong.
# -Skip is evaluated during DISCOVERY, before BeforeAll runs, so a python
# lookup done in BeforeAll leaves the flag $null and the real check silently
# skips - which is how this test first "passed" while verifying nothing.
BeforeDiscovery {
$script:Python = $null
foreach ($candidate in @('python', 'python3', 'py')) {
$cmd = Get-Command $candidate -ErrorAction SilentlyContinue
if ($cmd) { $script:Python = $cmd.Source; break }
}
}
BeforeAll {
$script:RepoRoot = Split-Path -Parent $PSScriptRoot
$script:Generator = Join-Path $script:RepoRoot "scripts\plaintext_twins.py"
$script:Python = $null
foreach ($candidate in @('python', 'python3', 'py')) {
$cmd = Get-Command $candidate -ErrorAction SilentlyContinue
if ($cmd) { $script:Python = $cmd.Source; break }
}
}
Describe "plain-text twins" {
It "the generator is where the tests and the docs say it is" {
Test-Path -LiteralPath $script:Generator | Should -BeTrue
}
# Asks the REPOSITORY what markdown it has, not the generator what it was
# told about. Scraping the generator's own list could only ever prove the
# list was self-consistent - a document nobody added to it was invisible to
# the check, which is how ELEVATOR_PITCH.md and TOKEN_SAVINGS.md sat here
# with no twin while this test passed.
It "every .md at the repository root has a twin" {
$mds = Get-ChildItem -LiteralPath $script:RepoRoot -Filter *.md -File
$mds.Count | Should -BeGreaterThan 0 -Because "the repo documents itself in markdown"
foreach ($md in $mds) {
$txt = [IO.Path]::ChangeExtension($md.FullName, ".txt")
Test-Path -LiteralPath $txt |
Should -BeTrue -Because "$($md.Name) owes a twin at $(Split-Path -Leaf $txt)"
}
}
It "every twin is in sync with its markdown" -Skip:(-not $script:Python) {
Push-Location $script:RepoRoot
try {
$output = & $script:Python $script:Generator --check 2>&1
$code = $LASTEXITCODE
}
finally { Pop-Location }
$code | Should -Be 0 -Because ($output -join "`n")
}
It "reports the python that was missing rather than passing quietly" -Skip:([bool]$script:Python) {
# Not a pass. If this is the test you are reading, --check never ran:
# install python, or regenerate the twins by hand before shipping.
Set-ItResult -Inconclusive -Because "no python on PATH, so the twins were not verified"
}
}