mirror of
https://github.com/kellymichels/zscripts-token-savers
synced 2026-10-07 07:18:18 +00:00
The mirror's deploy pair had drifted ~280 lines behind: it lacked the
transactional .env preserve/restore (an interrupted deploy could destroy
server-side env files), the stderr-flattening step wrapper (a successful
deploy reported failure and skipped its own verification), and the
verification rework (channel re-picked every retry, edge only with a Host
to route by, verify.timeoutSeconds, honest split of "stale build" vs "no
channel answered").
The sync is byte-faithful to the private tree except where the mirror's
own Sanitization suite demands otherwise - and it caught the first copy:
three failures for private project names, a private domain, and a
private-only script name that rode along in comments. Each war story keeps
its lesson and loses its cast, per the convention already in the file
("EvoCivilCode: deploy/" was already published as "(deploy/, infra/,
...)"). That suite going red on an unsanitized copy is exactly what it
exists for.
Also in this change:
- tests/VerifyPlan.Tests.ps1 - the channel-selection rules are pure
functions and Pester pins them (no domain => no edge attempt; the PS 5.1
one-element-unroll trap). First verification tests in the mirror.
- README: the verify block now documents viaProxy/upstream (they shipped
in the docker-network read but were never in the README),
timeoutSeconds, and the channel order with why it re-resolves per retry.
- README.txt: generated plain-text twin, via scripts/readme_txt.py
(vendored from the fleet's reference implementation; the file is
generated, never edited by hand).
- CHANGELOG.md/.txt: entries merged into the existing Unreleased sections.
CHECKSUMS.txt refreshed (42 entries). Pester: 240 passed, 0 failed.
Both synced files parse clean.
Observed, untouched: the Unreleased section carries duplicate "### Fixed"
headings from earlier appends; folding them risks reordering entries whose
prose references their neighbours, so it is left for the next release cut
(zbump #110 rolls Unreleased into the version being cut).
227 lines
13 KiB
Plaintext
227 lines
13 KiB
Plaintext
Evomedia.net Token Savers — https://github.com/evomedia-net/evo.zscripts
|
|
Created by Kelly Michels · dev@evomedia.net
|
|
Licensed under the MIT License. See LICENSE.
|
|
|
|
Changelog
|
|
=========
|
|
|
|
Notable changes to the Evomedia.net Token Savers.
|
|
|
|
Unreleased
|
|
----------
|
|
Changed
|
|
- zdeploy verification picks its channel on every retry, and never asks
|
|
the bare IP - the check used to choose its channel once, before the wait
|
|
loop, by probing; the probes raced the app restart the check exists to
|
|
wait through, so the whole window went to the edge fallback. For a
|
|
project with no domain that fallback had no Host header, and the proxy
|
|
can then only answer from its default vhost - a different product: that
|
|
is how one deploy's check compared another app's build number against
|
|
its own. Now channels are re-resolved each retry in trust order
|
|
(docker-network viaProxy -> localhost:<port> -> edge with the project's
|
|
Host), the edge is skipped entirely when there is no host to route by,
|
|
and a project with no trustworthy channel is reported as unverifiable
|
|
instead of guessed at. The final warning also says which failure
|
|
happened: a version that never matched (stale/failed build) reads
|
|
differently from channels that never answered (probably still booting).
|
|
verify.timeoutSeconds joins the config so a project that is slow to
|
|
boot - e.g. one that runs database migrations in its entrypoint - can
|
|
widen its own window instead of warning on every routine success.
|
|
- An interrupted deploy can no longer destroy server-side .env files - the
|
|
preserve/restore of operator files is transactional: the restore comes
|
|
from a tarball taken before the tree is replaced, so a deploy that dies
|
|
mid-flight leaves the previous files in place instead of an empty
|
|
directory.
|
|
- A successful deploy no longer reports failure - docker compose restart
|
|
writes routine progress to stderr, which PowerShell 5.1 turns into a
|
|
terminating error under ErrorActionPreference = Stop; four ssh calls
|
|
bypassed the wrapper that flattens this. All remote steps now run
|
|
through it and are judged by exit code alone.
|
|
- zversion bump is once per RELEASE, not once per PR - the usage text and
|
|
the bump help line both said "one per PR, one per defect fix". The build
|
|
number names something that shipped, so a release carrying five PRs moves
|
|
it by one; PRs that never shipped on their own were never separate builds.
|
|
Help text only here, but it is the wording people follow: it stamped a
|
|
single evo.www release as two builds. The historical entry below, which
|
|
records what the rule was when zversion shipped, is deliberately left as
|
|
written.
|
|
|
|
Added
|
|
- tests/VerifyPlan.Tests.ps1 - the verification channel-selection rules
|
|
are pure functions in ZHelpers.ps1 (Get-VerifyAttempts,
|
|
Get-VerifyTimeout) and Pester pins them, including "a project with no
|
|
domain must never produce an edge attempt" and the PowerShell 5.1
|
|
one-element-unroll trap.
|
|
- scripts/readme_txt.py and README.txt - a generated plain-text twin of
|
|
the README for terminals and pagers. README.txt is generated, never
|
|
edited by hand.
|
|
- Read a live build from inside the docker network, not through the public
|
|
proxy — zdeploy, zec2 and zec2online now prefer
|
|
docker exec <viaProxy> curl http://<upstream>/api/build-version when a
|
|
project sets verify.viaProxy and verify.upstream.
|
|
Two problems it closes. A build stamp is something many sites
|
|
deliberately do not serve publicly, and a checker that reads it over the
|
|
public URL stops working the moment that endpoint is blocked —
|
|
reporting "unknown", which is indistinguishable from "could not reach
|
|
it". And the proxy answers from whichever vhost matches the Host header,
|
|
so a container with no public route was getting another site's version
|
|
back and failing deploys that had worked.
|
|
Reading it from a container on the shared network also exercises the
|
|
real HTTP path, so it proves the app is serving rather than that its
|
|
database knows a version. Purely additive: projects without those two
|
|
keys behave exactly as before.
|
|
|
|
|
|
Fixed
|
|
- scp no longer receives an ssh-only flag — the stdin-hang fix added
|
|
-n to Get-Ec2SshOpts, and the deploy path splats that same array into
|
|
scp as well as ssh. OpenSSH's scp has no -n: it exits 1 with
|
|
unknown option -- n and prints its usage block, so every upload failed.
|
|
Get-Ec2ScpOpts now supplies the shared connection options with -n
|
|
filtered out, derived from Get-Ec2SshOpts rather than duplicated so the
|
|
timeouts cannot drift apart between the two transports. All four scp
|
|
call sites use it, including the recursive directory upload.
|
|
The upload failure message also asserted "Likely server disk space"
|
|
without checking; it now points at scp's own output, where the real
|
|
diagnosis already was.
|
|
|
|
|
|
Fixed
|
|
-----
|
|
- Deploys can no longer hang forever on an ssh prompt — every
|
|
deploy-path ssh/scp now carries BatchMode=yes plus connect and
|
|
keepalive timeouts (Get-Ec2SshOpts in ZHelpers.ps1). Without
|
|
BatchMode, ssh prompts for a passphrase or password and waits
|
|
indefinitely; because the deploy pipes stderr through the pipeline, the
|
|
prompt never reaches the screen and the run just stops under whatever
|
|
step label printed last, with no explanation. Now it fails immediately —
|
|
there is no prompt on this path worth answering. ServerAlive* bounds a
|
|
session that dies mid-command (dropped VPN, sleeping laptop, rebooting
|
|
host) to about a minute instead of hanging.
|
|
- Vendored archives survive the archive filter — files under a
|
|
vendor/ directory are exempt from the "no archives in the zip" rule.
|
|
A project that vendors a dependency as vendor/*.tgz needs it in the
|
|
deploy zip; dropping it makes a Dockerfile's COPY vendor ./vendor
|
|
fail at image build, a confusing way to learn the filter ate a build
|
|
input.
|
|
- unzip install is idempotent — the remote step ran
|
|
apt-get update && apt-get install -y unzip on every deploy; it now
|
|
checks command -v unzip first and skips the apt round-trip when the
|
|
binary is already there.
|
|
- zdeploy edge kind now ships asset subdirectories (#42) — the edge
|
|
deploy uploaded top-level files only, so a project self-hosting assets
|
|
in folders (fonts/, vendor/) lost them on every deploy: docker
|
|
created empty root-owned mount points and nginx served 404s from them,
|
|
which shows up as fonts silently falling back and vendored JS never
|
|
loading. Every subdirectory except nginx-logs/ and .git/ now ships
|
|
recursively, and the ensure edge dir chown is recursive so scp into
|
|
docker-created root-owned dirs cannot fail.
|
|
- zdeploy no longer deletes operator-managed files on deploy (#2) —
|
|
the project-directory replacement preserved only ./.env, silently
|
|
destroying every other server-side file (.env.db, staged signing
|
|
keys, certs) on every deploy. All .env* files at the project root are
|
|
now preserved by default, plus anything listed in the new
|
|
deploy.preserve array (files or directories); the vite kind, which
|
|
previously preserved nothing, gets the same protection. Found the hard
|
|
way: a first production deploy of an auth service wiped its staged DB
|
|
credentials and RSA signing keys.
|
|
|
|
Added
|
|
-----
|
|
- Versioned releases: zversion, zrelease, releases/ — the toolkit now
|
|
carries one version in a 5-segment scheme,
|
|
v{major}.{rc}.{beta}.{alpha}.{build}. zversion bump (one per PR / defect
|
|
fix) and zversion bump-stage release|rc|beta|alpha (zeroes every lower
|
|
segment) rewrite build-version.json, stamp # Version: into every script
|
|
header — so a lone copied script still says which release it came from — and
|
|
regenerate CHECKSUMS.txt in the same step. zrelease packages the current
|
|
version as releases/zscripts-<version>.zip with a .sha256 beside it: one
|
|
hash verifies the download, the bundled CHECKSUMS.txt verifies the
|
|
extracted contents, so nobody needs to clone the repo to get a verifiable
|
|
copy. Released zips are immutable — zrelease refuses to overwrite one.
|
|
- zchecksums + CHECKSUMS.txt — a SHA-256 manifest covering every .ps1
|
|
and .cmd, so a download can be verified before anything is run. zchecksums
|
|
checks them; zchecksums -Update regenerates after an intentional edit. The
|
|
manifest is sha256sum format, so sha256sum -c CHECKSUMS.txt works on
|
|
Linux/macOS/WSL too, and the hashes match on every platform because
|
|
.gitattributes pins these files to CRLF everywhere. Flags changed files,
|
|
missing files, and scripts present on disk but absent from the manifest.
|
|
It's an integrity check, not a signature — the manifest sits in the same repo
|
|
as the code, so it catches corruption and accidental drift, not a compromised
|
|
repo. A Pester test fails if the manifest ever goes stale.
|
|
- Test suite (Pester) — the toolkit now has automated coverage of its own
|
|
pure logic: Get-ArchiveExcludes (including the deploy-vs-backup rule that
|
|
keeps .env/uploads out of deploys but in backups), config and project
|
|
lookups, remote.composeDir fallback, EC2 target composition, and build-label
|
|
formatting. Run with Invoke-Pester .\tests (Pester 5+). Verified by mutation
|
|
testing — reintroducing each historical bug turns the suite red.
|
|
- ZCONFIG environment variable — overrides the path to zconfig.json, so
|
|
a run can target an alternate config. Also gives the test suite a seam for
|
|
injecting a fixture.
|
|
- zec2_rotatekeys — safely rotate/reset server-side secrets — a new
|
|
tool for when a secret leaks or a deploy overwrites a production .env
|
|
with dev values. -Rotate KEY regenerates a key on the server
|
|
(openssl rand -hex 32) so the new value never leaves the box; -Set KEY
|
|
takes an operator-known value (e.g. DATABASE_URL, ADMIN_EMAIL) from a
|
|
masked prompt and streams it over SSH stdin — never a command argument,
|
|
never echoed. Backs the server .env up to a timestamped .bak first,
|
|
updates the key atomically (matches or appends), auto-detects
|
|
backend/.env from deploy.preserve, and with -Restart recreates
|
|
the container (up -d --force-recreate, so the new values actually load —
|
|
a plain restart keeps the old environment). -WhatIf previews the plan
|
|
without touching anything.
|
|
- zkill all — zkill now accepts all, stopping the dev server of
|
|
every project that has a ports.dev (edge/docker stacks with no local dev
|
|
server are skipped). Brings it in line with zdeploy all / zbackup all;
|
|
the one-shot "stop everything I've got running locally".
|
|
- zdeploy server-side health verification (verify block) — projects
|
|
not published through the edge proxy can declare
|
|
"verify": { "port": ..., "path": "/health", "expect": "..." } and the
|
|
deploy is checked from the server itself (curl localhost:<port><path>
|
|
over SSH) instead of hitting the public IP. Fixes a false PASS where the
|
|
proxy's default vhost answered for apps that never started; projects
|
|
with neither domain nor verify are now reported as NOT verified.
|
|
- zdeploy optional deploy.gitPull — git pull --ff-only in the
|
|
project root before zipping. zdeploy zips the working tree and doesn't
|
|
otherwise pull, so a checkout left behind origin after a merged PR would
|
|
deploy stale code while still bumping the build number — success that
|
|
changes nothing. A failed pull aborts the deploy instead.
|
|
- Per-project start config block — zstart honors optional pre-start
|
|
steps from zconfig.json: "gitPull": true runs git pull --ff-only in
|
|
the project root before starting (never boot a stale checkout), and
|
|
"env": { ... } sets environment variables for the dev-server process.
|
|
Example added to zconfig.example.json.
|
|
- Switch-style argument tolerance — a leading dash on a project key is
|
|
ignored everywhere (zdeploy -myapp == zdeploy myapp), for hands that
|
|
grew up on per-project switches.
|
|
|
|
Changed
|
|
-------
|
|
- zbackup / zbackup_and_sync require an explicit target — running
|
|
them bare now shows usage instead of quietly backing up every project;
|
|
all does what bare invocation used to (matching zdeploy). The
|
|
scheduled task created by setup_backup_schedule.ps1 passes all —
|
|
re-run it if your task was registered before this change.
|
|
- zbackup parses more DATABASE_URL styles — double/single-quoted
|
|
values (Prisma convention), postgres:// and postgresql+driver://
|
|
schemes, and URLs without an explicit port (defaults to 5432) all work;
|
|
previously these skipped the Postgres dump with "Could not parse
|
|
DATABASE_URL".
|
|
- zkill / port cleanup kills the whole process tree — listeners on a
|
|
project's port are now terminated children-first. Auto-reloading servers
|
|
(uvicorn/watchfiles, nodemon) spawn workers that inherit the listening
|
|
socket; killing only the parent left orphans serving stale code.
|
|
- zbackup finds DATABASE_URL in backend\.env too — projects with a
|
|
frontend/backend split get their Postgres dump bundled without needing a
|
|
root-level .env.
|
|
|
|
1.0.0
|
|
-----
|
|
|
|
Initial public release: zstart / zkill / zrestart (local dev servers),
|
|
zdeploy (zip → upload → compose build → live build-version verification,
|
|
with handlers for python / vite / nextjs / edge / docker project kinds),
|
|
zec2 / zec2online / zrepair (health checks and recovery), zbackup /
|
|
zbackup_ec2 / zsync (local, server-side, and offsite backups), all driven
|
|
by a single gitignored zconfig.json.
|