diff --git a/CHANGELOG.md b/CHANGELOG.md index 8fb51d6..eba426f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,30 @@ Notable changes to the Evomedia.net Token Savers. ## Unreleased +### Changed +- **Every release since 1.0.0 has its own section again** - 21 entries had + piled up under `Unreleased` while 22 builds shipped, so this file said + nothing had been released since 1.0.0 and a reader at any tag found no + section for the version they were holding. Which release carried which + entry was derived from git, not guessed: for every line, the commit that + introduced it, then the earliest tag containing that commit. No entry text + changed - only headings were added and whole entries moved under the + release that carried them. +- **`scripts/readme_txt.py` is now `scripts/plaintext_twins.py` and covers + every markdown file that owes a twin**, `CHANGELOG.txt` included. It was + kept by hand, so it drifted the moment this file was reorganised. The + renderer also drops `` markers, which are invisible in markdown + and read as stray punctuation in a text file. + +### Fixed +- **The `--check` that keeps the twins honest is actually run now** - + `readme_txt.py` shipped one and its docstring claimed "the test suite runs + --check", but nothing invoked it, so a twin could disagree with its + markdown indefinitely. `tests/PlainTextTwins.Tests.ps1` runs it, and + reports *inconclusive* rather than passing when python is unavailable. + +## v1.0.0.0.22 - 2026-08-31 + ### Changed - **The sanitization denylist moved to `tests/sanitization-patterns.psd1`**, so the test suite here and the publisher in the private toolkit read one @@ -19,6 +43,18 @@ Notable changes to the Evomedia.net Token Savers. names, operator paths. It therefore reported "clean" on files this suite rejects. No rule changed; only where they live. +## v1.0.0.0.21 - 2026-08-31 + +### Added +- **`tests/VerifyPlan.Tests.ps1`** — the verification channel-selection rules + are pure functions in `ZHelpers.ps1` (`Get-VerifyAttempts`, + `Get-VerifyTimeout`) and Pester pins them, including "a project with no + domain must never produce an edge attempt" and the PowerShell 5.1 + one-element-unroll trap. + +- **`scripts/readme_txt.py` and `README.txt`** — a generated plain-text twin + of the README for terminals and pagers. `README.txt` is generated, never + edited by hand. ### Changed - **`zdeploy` verification picks its channel on every retry, and never asks @@ -38,16 +74,22 @@ Notable changes to the Evomedia.net Token Savers. config so a project that is slow to boot — e.g. one that runs database migrations in its entrypoint — can widen its own window instead of warning on every routine success. + - **An interrupted deploy can no longer destroy server-side `.env` files** — the preserve/restore of operator files is transactional: the restore comes from a tarball taken before the tree is replaced, so a deploy that dies mid-flight leaves the previous files in place instead of an empty directory. + - **A successful deploy no longer reports failure** — `docker compose restart` writes routine progress to stderr, which PowerShell 5.1 turns into a terminating error under `$ErrorActionPreference = 'Stop'`; four ssh calls bypassed the wrapper that flattens this. All remote steps now run through it and are judged by exit code alone. + +## v1.0.0.0.20 - 2026-08-30 + +### Changed - **`zversion bump` is once per *release*, not once per PR** — the usage text and the `bump` help line both said "one per PR, one per defect fix". The build number names something that shipped, so a release carrying five PRs @@ -57,15 +99,9 @@ Notable changes to the Evomedia.net Token Savers. records what the rule was when `zversion` shipped, is deliberately left as written. +## v1.0.0.0.19 - 2026-08-30 + ### Added -- **`tests/VerifyPlan.Tests.ps1`** — the verification channel-selection rules - are pure functions in `ZHelpers.ps1` (`Get-VerifyAttempts`, - `Get-VerifyTimeout`) and Pester pins them, including "a project with no - domain must never produce an edge attempt" and the PowerShell 5.1 - one-element-unroll trap. -- **`scripts/readme_txt.py` and `README.txt`** — a generated plain-text twin - of the README for terminals and pagers. `README.txt` is generated, never - edited by hand. - **Read a live build from inside the docker network, not through the public proxy** — `zdeploy`, `zec2` and `zec2online` now prefer `docker exec curl http:///api/build-version` when a @@ -84,6 +120,7 @@ Notable changes to the Evomedia.net Token Savers. knows a version. Purely additive: projects without those two keys behave exactly as before. +## v1.0.0.0.14 - 2026-08-20 ### Fixed - **`scp` no longer receives an ssh-only flag** — the stdin-hang fix added @@ -98,6 +135,8 @@ Notable changes to the Evomedia.net Token Savers. without checking; it now points at `scp`'s own output, where the real diagnosis already was. +## v1.0.0.0.8 - 2026-08-12 + ### Fixed - **Deploys can no longer hang forever on an ssh prompt** — every deploy-path `ssh`/`scp` now carries `BatchMode=yes` plus connect and @@ -109,16 +148,19 @@ Notable changes to the Evomedia.net Token Savers. there is no prompt on this path worth answering. `ServerAlive*` bounds a session that dies mid-command (dropped VPN, sleeping laptop, rebooting host) to about a minute instead of hanging. + - **Vendored archives survive the archive filter** — files under a `vendor/` directory are exempt from the "no archives in the zip" rule. A project that vendors a dependency as `vendor/*.tgz` needs it in the deploy zip; dropping it makes a Dockerfile's `COPY vendor ./vendor` fail at image build, a confusing way to learn the filter ate a build input. + - **`unzip` install is idempotent** — the remote step ran `apt-get update && apt-get install -y unzip` on every deploy; it now checks `command -v unzip` first and skips the apt round-trip when the binary is already there. + - **`zdeploy` edge kind now ships asset subdirectories** (#42) — the edge deploy uploaded top-level files only, so a project self-hosting assets in folders (`fonts/`, `vendor/`) lost them on every deploy: docker @@ -127,15 +169,8 @@ Notable changes to the Evomedia.net Token Savers. loading. Every subdirectory except `nginx-logs/` and `.git/` now ships recursively, and the `ensure edge dir` chown is recursive so scp into docker-created root-owned dirs cannot fail. -- **`zdeploy` no longer deletes operator-managed files on deploy** (#2) — - the project-directory replacement preserved only `./.env`, silently - destroying every other server-side file (`.env.db`, staged signing - keys, certs) on every deploy. All `.env*` files at the project root are - now preserved by default, plus anything listed in the new - `deploy.preserve` array (files or directories); the vite kind, which - previously preserved nothing, gets the same protection. Found the hard - way: a first production deploy of an auth service wiped its staged DB - credentials and RSA signing keys. + +## v1.0.0.0.0 - 2026-07-28 ### Added - **Versioned releases: `zversion`, `zrelease`, `releases/`** — the toolkit now @@ -149,6 +184,7 @@ Notable changes to the Evomedia.net Token Savers. hash verifies the download, the bundled `CHECKSUMS.txt` verifies the extracted contents, so nobody needs to clone the repo to get a verifiable copy. Released zips are immutable — `zrelease` refuses to overwrite one. + - **`zchecksums` + `CHECKSUMS.txt`** — a SHA-256 manifest covering every `.ps1` and `.cmd`, so a download can be verified before anything is run. `zchecksums` checks them; `zchecksums -Update` regenerates after an intentional edit. The @@ -159,15 +195,18 @@ Notable changes to the Evomedia.net Token Savers. It's an integrity check, not a signature — the manifest sits in the same repo as the code, so it catches corruption and accidental drift, not a compromised repo. A Pester test fails if the manifest ever goes stale. + - **Test suite (Pester)** — the toolkit now has automated coverage of its own pure logic: `Get-ArchiveExcludes` (including the deploy-vs-backup rule that keeps `.env`/`uploads` out of deploys but *in* backups), config and project lookups, `remote.composeDir` fallback, EC2 target composition, and build-label formatting. Run with `Invoke-Pester .\tests` (Pester 5+). Verified by mutation testing — reintroducing each historical bug turns the suite red. + - **`ZCONFIG` environment variable** — overrides the path to `zconfig.json`, so a run can target an alternate config. Also gives the test suite a seam for injecting a fixture. + - **`zec2_rotatekeys` — safely rotate/reset server-side secrets** — a new tool for when a secret leaks or a deploy overwrites a production `.env` with dev values. `-Rotate KEY` regenerates a key **on the server** @@ -180,10 +219,12 @@ Notable changes to the Evomedia.net Token Savers. the container (`up -d --force-recreate`, so the new values actually load — a plain restart keeps the old environment). `-WhatIf` previews the plan without touching anything. + - **`zkill all`** — `zkill` now accepts `all`, stopping the dev server of every project that has a `ports.dev` (edge/docker stacks with no local dev server are skipped). Brings it in line with `zdeploy all` / `zbackup all`; the one-shot "stop everything I've got running locally". + - **`zdeploy` server-side health verification (`verify` block)** — projects not published through the edge proxy can declare `"verify": { "port": ..., "path": "/health", "expect": "..." }` and the @@ -191,16 +232,19 @@ Notable changes to the Evomedia.net Token Savers. over SSH) instead of hitting the public IP. Fixes a false PASS where the proxy's default vhost answered for apps that never started; projects with neither `domain` nor `verify` are now reported as NOT verified. + - **`zdeploy` optional `deploy.gitPull`** — `git pull --ff-only` in the project root before zipping. `zdeploy` zips the working tree and doesn't otherwise pull, so a checkout left behind `origin` after a merged PR would deploy stale code while still bumping the build number — success that changes nothing. A failed pull aborts the deploy instead. + - **Per-project `start` config block** — `zstart` honors optional pre-start steps from `zconfig.json`: `"gitPull": true` runs `git pull --ff-only` in the project root before starting (never boot a stale checkout), and `"env": { ... }` sets environment variables for the dev-server process. Example added to `zconfig.example.json`. + - **Switch-style argument tolerance** — a leading dash on a project key is ignored everywhere (`zdeploy -myapp` == `zdeploy myapp`), for hands that grew up on per-project switches. @@ -211,19 +255,33 @@ Notable changes to the Evomedia.net Token Savers. `all` does what bare invocation used to (matching `zdeploy`). The scheduled task created by `setup_backup_schedule.ps1` passes `all` — re-run it if your task was registered before this change. + - **`zbackup` parses more `DATABASE_URL` styles** — double/single-quoted values (Prisma convention), `postgres://` and `postgresql+driver://` schemes, and URLs without an explicit port (defaults to 5432) all work; previously these skipped the Postgres dump with "Could not parse DATABASE_URL". + - **`zkill` / port cleanup kills the whole process tree** — listeners on a project's port are now terminated children-first. Auto-reloading servers (uvicorn/watchfiles, nodemon) spawn workers that inherit the listening socket; killing only the parent left orphans serving stale code. + - **`zbackup` finds `DATABASE_URL` in `backend\.env` too** — projects with a frontend/backend split get their Postgres dump bundled without needing a root-level `.env`. +### Fixed +- **`zdeploy` no longer deletes operator-managed files on deploy** (#2) — + the project-directory replacement preserved only `./.env`, silently + destroying every other server-side file (`.env.db`, staged signing + keys, certs) on every deploy. All `.env*` files at the project root are + now preserved by default, plus anything listed in the new + `deploy.preserve` array (files or directories); the vite kind, which + previously preserved nothing, gets the same protection. Found the hard + way: a first production deploy of an auth service wiped its staged DB + credentials and RSA signing keys. + ## 1.0.0 Initial public release: `zstart` / `zkill` / `zrestart` (local dev servers), diff --git a/CHANGELOG.txt b/CHANGELOG.txt index f870e39..476fa97 100644 --- a/CHANGELOG.txt +++ b/CHANGELOG.txt @@ -9,80 +9,133 @@ Notable changes to the Evomedia.net Token Savers. Unreleased ---------- + Changed -- The sanitization denylist moved to tests/sanitization-patterns.psd1, so - the test suite here and the publisher in the private toolkit read one +------- +- Every release since 1.0.0 has its own section again - 21 entries had + piled up under Unreleased while 22 builds shipped, so this file said + nothing had been released since 1.0.0 and a reader at any tag found no + section for the version they were holding. Which release carried which + entry was derived from git, not guessed: for every line, the commit that + introduced it, then the earliest tag containing that commit. No entry text + changed - only headings were added and whole entries moved under the + release that carried them. +- **scripts/readme_txt.py is now scripts/plaintext_twins.py and covers + every markdown file that owes a twin**, CHANGELOG.txt included. It was + kept by hand, so it drifted the moment this file was reorganised. The + renderer also drops markers, which are invisible in markdown + and read as stray punctuation in a text file. + +Fixed +----- +- The --check that keeps the twins honest is actually run now - + readme_txt.py shipped one and its docstring claimed "the test suite runs + --check", but nothing invoked it, so a twin could disagree with its + markdown indefinitely. tests/PlainTextTwins.Tests.ps1 runs it, and + reports inconclusive rather than passing when python is unavailable. + +v1.0.0.0.22 - 2026-08-31 +------------------------ + +Changed +------- +- The sanitization denylist moved to tests/sanitization-patterns.psd1, + so the test suite here and the publisher in the private toolkit read one list instead of keeping two. They had two, and they disagreed: the publisher's scan looked only for secrets, while these rules are about - identity - internal project names, product domains, private-only script + identity — internal project names, product domains, private-only script names, operator paths. It therefore reported "clean" on files this suite rejects. No rule changed; only where they live. - -Changed -- zdeploy verification picks its channel on every retry, and never asks - the bare IP - the check used to choose its channel once, before the wait - loop, by probing; the probes raced the app restart the check exists to - wait through, so the whole window went to the edge fallback. For a - project with no domain that fallback had no Host header, and the proxy - can then only answer from its default vhost - a different product: that - is how one deploy's check compared another app's build number against - its own. Now channels are re-resolved each retry in trust order - (docker-network viaProxy -> localhost: -> edge with the project's - Host), the edge is skipped entirely when there is no host to route by, - and a project with no trustworthy channel is reported as unverifiable - instead of guessed at. The final warning also says which failure - happened: a version that never matched (stale/failed build) reads - differently from channels that never answered (probably still booting). - verify.timeoutSeconds joins the config so a project that is slow to - boot - e.g. one that runs database migrations in its entrypoint - can - widen its own window instead of warning on every routine success. -- An interrupted deploy can no longer destroy server-side .env files - the - preserve/restore of operator files is transactional: the restore comes - from a tarball taken before the tree is replaced, so a deploy that dies - mid-flight leaves the previous files in place instead of an empty - directory. -- A successful deploy no longer reports failure - docker compose restart - writes routine progress to stderr, which PowerShell 5.1 turns into a - terminating error under ErrorActionPreference = Stop; four ssh calls - bypassed the wrapper that flattens this. All remote steps now run - through it and are judged by exit code alone. -- zversion bump is once per RELEASE, not once per PR - the usage text and - the bump help line both said "one per PR, one per defect fix". The build - number names something that shipped, so a release carrying five PRs moves - it by one; PRs that never shipped on their own were never separate builds. - Help text only here, but it is the wording people follow: it stamped a - single evo.www release as two builds. The historical entry below, which - records what the rule was when zversion shipped, is deliberately left as - written. +v1.0.0.0.21 - 2026-08-31 +------------------------ Added -- tests/VerifyPlan.Tests.ps1 - the verification channel-selection rules +----- +- tests/VerifyPlan.Tests.ps1 — the verification channel-selection rules are pure functions in ZHelpers.ps1 (Get-VerifyAttempts, Get-VerifyTimeout) and Pester pins them, including "a project with no domain must never produce an edge attempt" and the PowerShell 5.1 one-element-unroll trap. -- scripts/readme_txt.py and README.txt - a generated plain-text twin of - the README for terminals and pagers. README.txt is generated, never + +- scripts/readme_txt.py and README.txt — a generated plain-text twin + of the README for terminals and pagers. README.txt is generated, never edited by hand. -- Read a live build from inside the docker network, not through the public - proxy — zdeploy, zec2 and zec2online now prefer + +Changed +------- +- **zdeploy verification picks its channel on every retry, and never asks + the bare IP** — the check used to choose its channel once, before the wait + loop, by probing; the probes raced the app restart the check exists to wait + through, so the whole window went to the edge fallback. For a project with + no domain that fallback had no Host header, and the proxy can then only + answer from its default vhost — a different product: that is how one + deploy's check compared another app's build number against its own. Now + channels are re-resolved each retry in trust order (docker-network + viaProxy → localhost: → edge with the project's Host), the edge + is skipped entirely when there is no host to route by, and a project with + no trustworthy channel is reported as unverifiable instead of guessed at. + The final warning also says which failure happened: a version that never + matched (stale/failed build) reads differently from channels that never + answered (probably still booting). verify.timeoutSeconds joins the + config so a project that is slow to boot — e.g. one that runs database + migrations in its entrypoint — can widen its own window instead of + warning on every routine success. + +- An interrupted deploy can no longer destroy server-side .env files — + the preserve/restore of operator files is transactional: the restore comes + from a tarball taken before the tree is replaced, so a deploy that dies + mid-flight leaves the previous files in place instead of an empty + directory. + +- A successful deploy no longer reports failure — `docker compose + restart` writes routine progress to stderr, which PowerShell 5.1 turns + into a terminating error under $ErrorActionPreference = 'Stop'; four ssh + calls bypassed the wrapper that flattens this. All remote steps now run + through it and are judged by exit code alone. + +v1.0.0.0.20 - 2026-08-30 +------------------------ + +Changed +------- +- **zversion bump is once per release, not once per PR** — the usage text + and the bump help line both said "one per PR, one per defect fix". The + build number names something that shipped, so a release carrying five PRs + moves it by one; PRs that never shipped on their own were never separate + builds. Help text only here, but it is the wording people follow: it stamped + a single evo.www release as two builds. The historical entry below, which + records what the rule was when zversion shipped, is deliberately left as + written. + +v1.0.0.0.19 - 2026-08-30 +------------------------ + +Added +----- +- **Read a live build from inside the docker network, not through the public + proxy** — zdeploy, zec2 and zec2online now prefer docker exec curl http:///api/build-version when a project sets verify.viaProxy and verify.upstream. - Two problems it closes. A build stamp is something many sites - deliberately do not serve publicly, and a checker that reads it over the - public URL stops working the moment that endpoint is blocked — - reporting "unknown", which is indistinguishable from "could not reach - it". And the proxy answers from whichever vhost matches the Host header, - so a container with no public route was getting another site's version - back and failing deploys that had worked. - Reading it from a container on the shared network also exercises the - real HTTP path, so it proves the app is serving rather than that its - database knows a version. Purely additive: projects without those two - keys behave exactly as before. + Two problems it closes. A build stamp is something many sites deliberately + do not serve publicly, and a checker that reads it over the public URL stops + working the moment that endpoint is blocked — reporting "unknown", which is + indistinguishable from "could not reach it". And the proxy answers from + whichever vhost matches the Host header, so a container with no public route + was getting another site's version back and failing deploys that had + worked. + + Reading it from a container on the shared network also exercises the real + HTTP path, so it proves the app is serving rather than that its database + knows a version. Purely additive: projects without those two keys behave + exactly as before. + +v1.0.0.0.14 - 2026-08-20 +------------------------ Fixed +----- - scp no longer receives an ssh-only flag — the stdin-hang fix added -n to Get-Ec2SshOpts, and the deploy path splats that same array into scp as well as ssh. OpenSSH's scp has no -n: it exits 1 with @@ -95,6 +148,8 @@ Fixed without checking; it now points at scp's own output, where the real diagnosis already was. +v1.0.0.0.8 - 2026-08-12 +----------------------- Fixed ----- @@ -108,16 +163,19 @@ Fixed there is no prompt on this path worth answering. ServerAlive* bounds a session that dies mid-command (dropped VPN, sleeping laptop, rebooting host) to about a minute instead of hanging. + - Vendored archives survive the archive filter — files under a vendor/ directory are exempt from the "no archives in the zip" rule. A project that vendors a dependency as vendor/*.tgz needs it in the deploy zip; dropping it makes a Dockerfile's COPY vendor ./vendor fail at image build, a confusing way to learn the filter ate a build input. + - unzip install is idempotent — the remote step ran apt-get update && apt-get install -y unzip on every deploy; it now checks command -v unzip first and skips the apt round-trip when the binary is already there. + - zdeploy edge kind now ships asset subdirectories (#42) — the edge deploy uploaded top-level files only, so a project self-hosting assets in folders (fonts/, vendor/) lost them on every deploy: docker @@ -126,15 +184,9 @@ Fixed loading. Every subdirectory except nginx-logs/ and .git/ now ships recursively, and the ensure edge dir chown is recursive so scp into docker-created root-owned dirs cannot fail. -- zdeploy no longer deletes operator-managed files on deploy (#2) — - the project-directory replacement preserved only ./.env, silently - destroying every other server-side file (.env.db, staged signing - keys, certs) on every deploy. All .env* files at the project root are - now preserved by default, plus anything listed in the new - deploy.preserve array (files or directories); the vite kind, which - previously preserved nothing, gets the same protection. Found the hard - way: a first production deploy of an auth service wiped its staged DB - credentials and RSA signing keys. + +v1.0.0.0.0 - 2026-07-28 +----------------------- Added ----- @@ -149,6 +201,7 @@ Added hash verifies the download, the bundled CHECKSUMS.txt verifies the extracted contents, so nobody needs to clone the repo to get a verifiable copy. Released zips are immutable — zrelease refuses to overwrite one. + - zchecksums + CHECKSUMS.txt — a SHA-256 manifest covering every .ps1 and .cmd, so a download can be verified before anything is run. zchecksums checks them; zchecksums -Update regenerates after an intentional edit. The @@ -159,15 +212,18 @@ Added It's an integrity check, not a signature — the manifest sits in the same repo as the code, so it catches corruption and accidental drift, not a compromised repo. A Pester test fails if the manifest ever goes stale. + - Test suite (Pester) — the toolkit now has automated coverage of its own pure logic: Get-ArchiveExcludes (including the deploy-vs-backup rule that keeps .env/uploads out of deploys but in backups), config and project lookups, remote.composeDir fallback, EC2 target composition, and build-label formatting. Run with Invoke-Pester .\tests (Pester 5+). Verified by mutation testing — reintroducing each historical bug turns the suite red. + - ZCONFIG environment variable — overrides the path to zconfig.json, so a run can target an alternate config. Also gives the test suite a seam for injecting a fixture. + - zec2_rotatekeys — safely rotate/reset server-side secrets — a new tool for when a secret leaks or a deploy overwrites a production .env with dev values. -Rotate KEY regenerates a key on the server @@ -180,10 +236,12 @@ Added the container (up -d --force-recreate, so the new values actually load — a plain restart keeps the old environment). -WhatIf previews the plan without touching anything. + - zkill all — zkill now accepts all, stopping the dev server of every project that has a ports.dev (edge/docker stacks with no local dev server are skipped). Brings it in line with zdeploy all / zbackup all; the one-shot "stop everything I've got running locally". + - zdeploy server-side health verification (verify block) — projects not published through the edge proxy can declare "verify": { "port": ..., "path": "/health", "expect": "..." } and the @@ -191,16 +249,19 @@ Added over SSH) instead of hitting the public IP. Fixes a false PASS where the proxy's default vhost answered for apps that never started; projects with neither domain nor verify are now reported as NOT verified. + - zdeploy optional deploy.gitPull — git pull --ff-only in the project root before zipping. zdeploy zips the working tree and doesn't otherwise pull, so a checkout left behind origin after a merged PR would deploy stale code while still bumping the build number — success that changes nothing. A failed pull aborts the deploy instead. + - Per-project start config block — zstart honors optional pre-start steps from zconfig.json: "gitPull": true runs git pull --ff-only in the project root before starting (never boot a stale checkout), and "env": { ... } sets environment variables for the dev-server process. Example added to zconfig.example.json. + - Switch-style argument tolerance — a leading dash on a project key is ignored everywhere (zdeploy -myapp == zdeploy myapp), for hands that grew up on per-project switches. @@ -212,19 +273,34 @@ Changed all does what bare invocation used to (matching zdeploy). The scheduled task created by setup_backup_schedule.ps1 passes all — re-run it if your task was registered before this change. + - zbackup parses more DATABASE_URL styles — double/single-quoted values (Prisma convention), postgres:// and postgresql+driver:// schemes, and URLs without an explicit port (defaults to 5432) all work; previously these skipped the Postgres dump with "Could not parse DATABASE_URL". + - zkill / port cleanup kills the whole process tree — listeners on a project's port are now terminated children-first. Auto-reloading servers (uvicorn/watchfiles, nodemon) spawn workers that inherit the listening socket; killing only the parent left orphans serving stale code. + - zbackup finds DATABASE_URL in backend\.env too — projects with a frontend/backend split get their Postgres dump bundled without needing a root-level .env. +Fixed +----- +- zdeploy no longer deletes operator-managed files on deploy (#2) — + the project-directory replacement preserved only ./.env, silently + destroying every other server-side file (.env.db, staged signing + keys, certs) on every deploy. All .env* files at the project root are + now preserved by default, plus anything listed in the new + deploy.preserve array (files or directories); the vite kind, which + previously preserved nothing, gets the same protection. Found the hard + way: a first production deploy of an auth service wiped its staged DB + credentials and RSA signing keys. + 1.0.0 ----- diff --git a/README.txt b/README.txt index 23a3a96..5fb7b26 100644 --- a/README.txt +++ b/README.txt @@ -1,8 +1,6 @@ - zscripts Token Savers ===================== diff --git a/scripts/plaintext_twins.py b/scripts/plaintext_twins.py new file mode 100644 index 0000000..52e397a --- /dev/null +++ b/scripts/plaintext_twins.py @@ -0,0 +1,104 @@ +# Evomedia.net Token Savers — https://github.com/evomedia-net/evo.zscripts +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +"""Render this repo's markdown to plain-text twins with the markup removed. + +The .txt files exist for terminals, pagers and anywhere markdown doesn't +render. They are generated - never edit one by hand: + + python scripts/plaintext_twins.py # rewrite every .txt twin + python scripts/plaintext_twins.py --check # exit 1 if any is out of sync + +tests/PlainTextTwins.Tests.ps1 runs --check, so a markdown edit that forgets +to regenerate fails the suite instead of shipping a twin that disagrees with +the file it mirrors. + +Was readme_txt.py, which did README only. CHANGELOG.txt was kept by hand and +drifted the moment CHANGELOG.md was reorganised - and its docstring claimed a +--check the suite never actually ran. Both are fixed here: one renderer, every +pair, and a test that invokes it. +""" + +from __future__ import annotations + +import re +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent + +# Every markdown file that owes the repo a plain-text twin. +PAIRS = ( + ("README.md", "README.txt"), + ("CHANGELOG.md", "CHANGELOG.txt"), +) + + +def _inline(text: str) -> str: + text = re.sub(r"!\[([^\]]*)\]\([^)]*\)", r"\1", text) # images -> alt text + text = re.sub(r"\[([^\]]+)\]\(([^)]+)\)", r"\1 (\2)", text) # links -> text (url) + text = re.sub(r"\*\*([^*]+)\*\*", r"\1", text) # bold + text = re.sub(r"(? str: + out: list[str] = [] + in_fence = False + for line in md.splitlines(): + if line.lstrip().startswith("```"): + # Drop the fence markers; the code itself stays, indented so it + # still reads as a block without the backticks. + in_fence = not in_fence + continue + if in_fence: + out.append((" " + line) if line else "") + continue + # An HTML comment is invisible in rendered markdown, but its markers + # are not invisible in a text file - they read as stray punctuation. + # Keep what the comment says, drop the around it. + if line.strip() in (""): + continue + heading = re.match(r"^(#{1,6})\s+(.*)$", line) + if heading: + text = _inline(heading.group(2)) + out.append(text) + out.append(("=" if len(heading.group(1)) == 1 else "-") * len(text)) + continue + out.append(_inline(line)) + text = "\n".join(out) + text = re.sub(r"\n{3,}", "\n\n", text) + return text.strip() + "\n" + + +def main() -> int: + check = "--check" in sys.argv + stale: list[str] = [] + for md_name, txt_name in PAIRS: + source = ROOT / md_name + if not source.exists(): + print(f"{md_name} is missing - nothing to render") + return 1 + rendered = render(source.read_text(encoding="utf-8")) + target = ROOT / txt_name + if check: + current = target.read_text(encoding="utf-8") if target.exists() else "" + if current != rendered: + stale.append(txt_name) + continue + target.write_text(rendered, encoding="utf-8", newline="\n") + print(f"Wrote {target} ({len(rendered.splitlines())} lines)") + + if check: + if stale: + print(f"out of sync: {', '.join(stale)}" + f" - run: python scripts/plaintext_twins.py") + return 1 + print(f"in sync: {', '.join(t for _, t in PAIRS)}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/readme_txt.py b/scripts/readme_txt.py deleted file mode 100644 index 4c802a0..0000000 --- a/scripts/readme_txt.py +++ /dev/null @@ -1,76 +0,0 @@ -# Evomedia.net Token Savers — https://github.com/evomedia-net/evo.zscripts -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. - -"""Render README.md to README.txt with the markdown markup removed. - -README.txt exists for terminals, pagers and anywhere markdown doesn't -render. It is generated - never edit it by hand: - - python scripts/readme_txt.py # rewrite README.txt - python scripts/readme_txt.py --check # exit 1 if it is out of sync - -The test suite runs --check, so a README.md edit that forgets to -regenerate fails CI rather than shipping a stale mirror. -""" - -from __future__ import annotations - -import re -import sys -from pathlib import Path - -ROOT = Path(__file__).resolve().parent.parent - - -def _inline(text: str) -> str: - text = re.sub(r"!\[([^\]]*)\]\([^)]*\)", r"\1", text) # images -> alt text - text = re.sub(r"\[([^\]]+)\]\(([^)]+)\)", r"\1 (\2)", text) # links -> text (url) - text = re.sub(r"\*\*([^*]+)\*\*", r"\1", text) # bold - text = re.sub(r"(? str: - out: list[str] = [] - in_fence = False - for line in md.splitlines(): - if line.lstrip().startswith("```"): - # Drop the fence markers; the code itself stays, indented so it - # still reads as a block without the backticks. - in_fence = not in_fence - continue - if in_fence: - out.append((" " + line) if line else "") - continue - heading = re.match(r"^(#{1,6})\s+(.*)$", line) - if heading: - text = _inline(heading.group(2)) - out.append(text) - out.append(("=" if len(heading.group(1)) == 1 else "-") * len(text)) - continue - out.append(_inline(line)) - text = "\n".join(out) - text = re.sub(r"\n{3,}", "\n\n", text) - return text.strip() + "\n" - - -def main() -> int: - source = (ROOT / "README.md").read_text(encoding="utf-8") - rendered = render(source) - target = ROOT / "README.txt" - if "--check" in sys.argv: - current = target.read_text(encoding="utf-8") if target.exists() else "" - if current != rendered: - print("README.txt is out of sync - run: python scripts/readme_txt.py") - return 1 - print("README.txt is in sync") - return 0 - target.write_text(rendered, encoding="utf-8", newline="\n") - print(f"Wrote {target} ({len(rendered.splitlines())} lines)") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/tests/PlainTextTwins.Tests.ps1 b/tests/PlainTextTwins.Tests.ps1 new file mode 100644 index 0000000..ecff5ab --- /dev/null +++ b/tests/PlainTextTwins.Tests.ps1 @@ -0,0 +1,68 @@ +# The .txt twins are generated, and this is what makes that true. +# +# Invoke-Pester .\tests +# +# scripts/plaintext_twins.py has always shipped a --check mode, and its +# docstring claimed "the test suite runs --check". Nothing ran it. So README.txt +# could drift from README.md silently, and CHANGELOG.txt - which no generator +# covered at all - actually did. +# +# A twin that disagrees with the file it mirrors is worse than no twin: it is a +# second document that looks authoritative and is wrong. + +# -Skip is evaluated during DISCOVERY, before BeforeAll runs, so a python +# lookup done in BeforeAll leaves the flag $null and the real check silently +# skips - which is how this test first "passed" while verifying nothing. +BeforeDiscovery { + $script:Python = $null + foreach ($candidate in @('python', 'python3', 'py')) { + $cmd = Get-Command $candidate -ErrorAction SilentlyContinue + if ($cmd) { $script:Python = $cmd.Source; break } + } +} + +BeforeAll { + $script:RepoRoot = Split-Path -Parent $PSScriptRoot + $script:Generator = Join-Path $script:RepoRoot "scripts\plaintext_twins.py" + + $script:Python = $null + foreach ($candidate in @('python', 'python3', 'py')) { + $cmd = Get-Command $candidate -ErrorAction SilentlyContinue + if ($cmd) { $script:Python = $cmd.Source; break } + } +} + +Describe "plain-text twins" { + + It "the generator is where the tests and the docs say it is" { + Test-Path -LiteralPath $script:Generator | Should -BeTrue + } + + It "every .md that owes a twin has one" { + $pairs = Select-String -Path $script:Generator -Pattern '^\s*\("([^"]+\.md)",\s*"([^"]+\.txt)"\),' | + ForEach-Object { [pscustomobject]@{ Md = $_.Matches[0].Groups[1].Value; Txt = $_.Matches[0].Groups[2].Value } } + + $pairs.Count | Should -BeGreaterThan 0 -Because "PAIRS in plaintext_twins.py is what this suite checks" + foreach ($p in $pairs) { + Test-Path -LiteralPath (Join-Path $script:RepoRoot $p.Md) | Should -BeTrue -Because "$($p.Md) is listed in PAIRS" + Test-Path -LiteralPath (Join-Path $script:RepoRoot $p.Txt) | Should -BeTrue -Because "$($p.Md) owes a twin at $($p.Txt)" + } + } + + It "every twin is in sync with its markdown" -Skip:(-not $script:Python) { + Push-Location $script:RepoRoot + try { + $output = & $script:Python $script:Generator --check 2>&1 + $code = $LASTEXITCODE + } + finally { Pop-Location } + + $code | Should -Be 0 -Because ($output -join "`n") + } + + It "reports the python that was missing rather than passing quietly" -Skip:([bool]$script:Python) { + # Not a pass. If this is the test you are reading, --check never ran: + # install python, or regenerate the twins by hand before shipping. + Set-ItResult -Inconclusive -Because "no python on PATH, so the twins were not verified" + } +}