mirror of
https://github.com/kellymichels/zscripts-token-savers
synced 2026-10-07 07:18:18 +00:00
feat(version): read a live build from inside the docker network, not the public proxy (#59)
zdeploy, zec2 and zec2online now prefer
docker exec <viaProxy> curl http://<upstream>/api/build-version
when a project sets verify.viaProxy and verify.upstream.
Two problems it closes. A build stamp is something many sites deliberately
do not serve publicly, and a checker that reads it over the public URL stops
working the moment that endpoint is blocked -- reporting "unknown", which is
indistinguishable from "could not reach it". And the proxy answers from
whichever vhost matches the Host header, so a container with no public route
was getting another site's version back and failing deploys that had worked.
Reading it from a container on the shared network also exercises the real
HTTP path, so it proves the app is serving rather than that its database
knows a version. Purely additive: a project without those two keys behaves
exactly as before.
zec2 gains $PemKey and $SshTarget, which it had no need of until now -- the
read is inside a try/catch, so without them it would throw, be swallowed,
and fall through silently.
CHECKSUMS.txt regenerated (zchecksums -Update), zconfig.example.json
documents both shapes of the verify block, and CHANGELOG.md carries its
plain-text twin.
Pester: 231 passed, 0 failed -- including the sanitization suite.
This commit is contained in:
GitHub
parent
58c888aa56
commit
e738b62cdf
20
CHANGELOG.md
20
CHANGELOG.md
@ -10,6 +10,26 @@ Notable changes to the Evomedia.net Token Savers.
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Added
|
||||
- **Read a live build from inside the docker network, not through the public
|
||||
proxy** — `zdeploy`, `zec2` and `zec2online` now prefer
|
||||
`docker exec <viaProxy> curl http://<upstream>/api/build-version` when a
|
||||
project sets `verify.viaProxy` and `verify.upstream`.
|
||||
|
||||
Two problems it closes. A build stamp is something many sites deliberately
|
||||
do not serve publicly, and a checker that reads it over the public URL stops
|
||||
working the moment that endpoint is blocked — reporting "unknown", which is
|
||||
indistinguishable from "could not reach it". And the proxy answers from
|
||||
whichever vhost matches the Host header, so a container with no public route
|
||||
was getting *another site's* version back and failing deploys that had
|
||||
worked.
|
||||
|
||||
Reading it from a container on the shared network also exercises the real
|
||||
HTTP path, so it proves the app is serving rather than that its database
|
||||
knows a version. Purely additive: projects without those two keys behave
|
||||
exactly as before.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **`scp` no longer receives an ssh-only flag** — the stdin-hang fix added
|
||||
`-n` to `Get-Ec2SshOpts`, and the deploy path splats that same array into
|
||||
|
||||
@ -9,6 +9,24 @@ Notable changes to the Evomedia.net Token Savers.
|
||||
|
||||
Unreleased
|
||||
----------
|
||||
Added
|
||||
- Read a live build from inside the docker network, not through the public
|
||||
proxy — zdeploy, zec2 and zec2online now prefer
|
||||
docker exec <viaProxy> curl http://<upstream>/api/build-version when a
|
||||
project sets verify.viaProxy and verify.upstream.
|
||||
Two problems it closes. A build stamp is something many sites
|
||||
deliberately do not serve publicly, and a checker that reads it over the
|
||||
public URL stops working the moment that endpoint is blocked —
|
||||
reporting "unknown", which is indistinguishable from "could not reach
|
||||
it". And the proxy answers from whichever vhost matches the Host header,
|
||||
so a container with no public route was getting another site's version
|
||||
back and failing deploys that had worked.
|
||||
Reading it from a container on the shared network also exercises the
|
||||
real HTTP path, so it proves the app is serving rather than that its
|
||||
database knows a version. Purely additive: projects without those two
|
||||
keys behave exactly as before.
|
||||
|
||||
|
||||
Fixed
|
||||
- scp no longer receives an ssh-only flag — the stdin-hang fix added
|
||||
-n to Get-Ec2SshOpts, and the deploy path splats that same array into
|
||||
|
||||
@ -8,14 +8,14 @@
|
||||
addf4b7236629b03bf4f4259563e1bfc29bae2e883846c1b4852d882bc80f562 zchecksums.cmd
|
||||
0a4bd178fe82522ef3748110b8a0c471cdaf58db7ba4c1ff656bddfe59fa36ec zchecksums.ps1
|
||||
c5b310294c304f209c8327b3172f410f05a679693dfc20ee0f18763f6b8c60b5 zdeploy.cmd
|
||||
c780f4c4aa3308355aa830a1b5b48a56d9e4153f4646a80b5b2353bd80b1fff9 zdeploy.ps1
|
||||
08d67aa85c6e32416bef75b74eccaf6760b8d27c2b35eeee8a7512d6b8a64637 zdeploy.ps1
|
||||
4f5c648cf3a975d2897a601e22b4bfc3e0ce663ef15c2df7511da749a0595d69 zec2.cmd
|
||||
30efe84eea157937baa0148e8bdb7d7474680f9f7f0b1dbfef1604ea44da2413 zec2.ps1
|
||||
f11447e2039286b0ac6a0fe71ecd79d23167e46188b388ffe21fa3cf80a5cfef zec2.ps1
|
||||
c0ec3729dc8240609987287f8e130e48d07cc49162dfec28b553dca07444f644 zec2_rotatekeys.cmd
|
||||
207a2037da37787d81e69fc28fd6dc5386176459560b718ff902e023486f568f zec2_rotatekeys.ps1
|
||||
26047c3a8c04bcda94cda88dc27a0e96d25ed642e4914372c3f09e93a7ac4563 zec2online.cmd
|
||||
9698e94f409c935ddc73d4aae35cd25453044e34d5015523280cc5646ea0c677 zec2online.ps1
|
||||
1874633057f7fd82a591d25b90fa8b5915bf299499f491cd6b69b2b9aef1e1be ZHelpers.ps1
|
||||
86fc277b928dfa111f8773ccedc751805d0ab7b91aeea58e4b76d9c504405af4 zec2online.ps1
|
||||
9db5c546819b50dd940bfc89b5a653185139ac3bc2e476fe2f52886e2d4ec553 ZHelpers.ps1
|
||||
4a5495feb0d2e33dd264efa9fc02268d40510268bf7eed2212b394e73eb7cd6b zkill.cmd
|
||||
908357d024dd30e766f7a2b120cfa42d1d26ba4d2f55a413e70eebea5a6a296a zkill.ps1
|
||||
f25a11e6166647f63412918fc596d09c3a328ad469318589e597d6c3ab13d4ed ZKiller.ps1
|
||||
|
||||
62
ZHelpers.ps1
62
ZHelpers.ps1
@ -321,6 +321,68 @@ function Read-JsonBuildVersion {
|
||||
try { return Get-Content -LiteralPath $FilePath -Raw -Encoding UTF8 | ConvertFrom-Json } catch { return $null }
|
||||
}
|
||||
|
||||
function Get-ServerSideVersionCommand {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Shell command that reads a project's live build ON the server, or
|
||||
$null when the project has not configured one.
|
||||
|
||||
.DESCRIPTION
|
||||
Reading a live build number through the public proxy only works while
|
||||
that endpoint IS public - and a build stamp is something many sites
|
||||
deliberately do not serve to the world. Blocking it at the proxy
|
||||
without moving the readers first leaves every tool quietly reporting
|
||||
"unknown", which looks identical to "could not reach it".
|
||||
|
||||
Going through the proxy is also how a check reads the WRONG service:
|
||||
the proxy answers from whichever vhost matches the Host header, so a
|
||||
container with no public route gets another site's version back.
|
||||
|
||||
A service reached only on a shared docker network cannot be curled
|
||||
from the host when it publishes no port. It IS reachable by name from
|
||||
another container on that network, which also exercises the real HTTP
|
||||
path - so this proves the app is serving, not merely that its
|
||||
database knows a version.
|
||||
|
||||
Config, on the project's `verify` block:
|
||||
|
||||
"verify": {
|
||||
"path": "/api/build-version",
|
||||
"viaProxy": "edge_proxy_container",
|
||||
"upstream": "app_container:80"
|
||||
}
|
||||
|
||||
Returns $null when either key is missing, so every project without
|
||||
this config keeps the behaviour it has today.
|
||||
#>
|
||||
param($Proj)
|
||||
|
||||
$v = $Proj.verify
|
||||
if (-not $v) { return $null }
|
||||
if (-not $v.viaProxy -or -not $v.upstream) { return $null }
|
||||
$path = if ($v.path) { [string]$v.path } else { '/api/build-version' }
|
||||
return "sudo docker exec $([string]$v.viaProxy) curl -s -m 8 http://$([string]$v.upstream)$path"
|
||||
}
|
||||
|
||||
function Get-LabelFromVersionJson {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
The build label out of a version endpoint's JSON text, or $null.
|
||||
|
||||
.DESCRIPTION
|
||||
Apps disagree about the field name - some answer `build_version`,
|
||||
others `version`. Both mean "the build that is live", so both are
|
||||
accepted rather than making an app rename its own field.
|
||||
#>
|
||||
param([string]$Text)
|
||||
|
||||
if ([string]::IsNullOrWhiteSpace($Text)) { return $null }
|
||||
try { $obj = $Text.Trim() | ConvertFrom-Json -ErrorAction Stop } catch { return $null }
|
||||
if ($obj.build_version) { return [string]$obj.build_version }
|
||||
if ($obj.version) { return [string]$obj.version }
|
||||
return $null
|
||||
}
|
||||
|
||||
function Get-LabelFromBuildJsonObj {
|
||||
param($obj)
|
||||
if (-not $obj) { return $null }
|
||||
|
||||
@ -1,13 +1,11 @@
|
||||
{
|
||||
"_comment": "Copy this file to zconfig.json and fill in your values. zconfig.json is gitignored — never commit it.",
|
||||
|
||||
"_comment": "Copy this file to zconfig.json and fill in your values. zconfig.json is gitignored \u2014 never commit it.",
|
||||
"ec2": {
|
||||
"ip": "YOUR_SERVER_IP",
|
||||
"user": "YOUR_SSH_USER",
|
||||
"pemKey": "C:\\Users\\YourUser\\.ssh\\YourKey.pem",
|
||||
"stackRoot": "/home/YOUR_SSH_USER/stack"
|
||||
},
|
||||
|
||||
"paths": {
|
||||
"temp": "C:\\YourRoot\\temp",
|
||||
"backupsLocal": "C:\\YourRoot\\backups\\projects",
|
||||
@ -15,10 +13,8 @@
|
||||
"scriptsRoot": "C:\\YourRoot\\zscripts",
|
||||
"oneDriveBackups": ""
|
||||
},
|
||||
|
||||
"projects": {
|
||||
"_comment": "Rename these keys to your own project names — the key IS the command argument: zstart pyapp, zdeploy viteapp, zbackup nextapp. Add as many projects as you like. Keys starting with _ are ignored.",
|
||||
|
||||
"_comment": "Rename these keys to your own project names \u2014 the key IS the command argument: zstart pyapp, zdeploy viteapp, zbackup nextapp. Add as many projects as you like. Keys starting with _ are ignored.",
|
||||
"pyapp": {
|
||||
"label": "My Python App",
|
||||
"kind": "python",
|
||||
@ -26,68 +22,92 @@
|
||||
"startModule": "pyapp.main",
|
||||
"_startApp_note": "ASGI/FastAPI app? Use \"startApp\": \"app.main:app\" instead of startModule (runs uvicorn --port <dev> --reload).",
|
||||
"install": "-e .",
|
||||
"ports": { "dev": 8080 },
|
||||
"ports": {
|
||||
"dev": 8080
|
||||
},
|
||||
"domain": "pyapp.yourdomain.com",
|
||||
"start": {
|
||||
"_comment": "Optional zstart pre-steps: gitPull runs 'git pull --ff-only' first; env sets variables for the server process.",
|
||||
"gitPull": true,
|
||||
"env": { "MYAPP_DEBUG": "1" }
|
||||
"env": {
|
||||
"MYAPP_DEBUG": "1"
|
||||
}
|
||||
},
|
||||
"db": {
|
||||
"user": "pyapp_user",
|
||||
"name": "pyapp_db"
|
||||
},
|
||||
"db": { "user": "pyapp_user", "name": "pyapp_db" },
|
||||
"remote": {
|
||||
"path": "/home/YOUR_SSH_USER/stack/pyapp",
|
||||
"composeDir": "/home/YOUR_SSH_USER/stack/pyapp/docker",
|
||||
"appService": "app"
|
||||
},
|
||||
"verify": {
|
||||
"_comment": "Optional: after deploy, curl this ON the server (localhost:port+path) and require the substring. The accurate check for apps not published through the edge proxy.",
|
||||
"_comment": "Optional post-deploy build check. Two ways to reach the app, both ON the server rather than through the public proxy - which answers from whichever vhost matches the Host header, so a container with no public route gets another site's version back. Use port+expect when the app publishes a host port; use viaProxy+upstream when it does not, or when its version endpoint is deliberately not public.",
|
||||
"port": 8080,
|
||||
"path": "/health",
|
||||
"expect": "\"status\":\"ok\""
|
||||
"expect": "\"status\":\"ok\"",
|
||||
"viaProxy": "edge_proxy_container",
|
||||
"upstream": "app_container:80",
|
||||
"_viaProxy_note": "Runs: docker exec <viaProxy> curl -s http://<upstream><path>. Reads the build from inside the shared docker network, and exercises the real HTTP path - so it proves the app is serving, not just that its database knows a version. Omit both keys to keep the previous behaviour."
|
||||
},
|
||||
"deploy": {
|
||||
"zipName": "PyAppDeploy.zip",
|
||||
"gitPull": true,
|
||||
"exclude": ["docs"],
|
||||
"exclude": [
|
||||
"docs"
|
||||
],
|
||||
"_comment": "preserve: server-side files/dirs in the project dir that deploys must never delete (.env* files are always preserved)",
|
||||
"preserve": ["keys", "seed-data"]
|
||||
"preserve": [
|
||||
"keys",
|
||||
"seed-data"
|
||||
]
|
||||
}
|
||||
},
|
||||
|
||||
"viteapp": {
|
||||
"label": "My Vite Site",
|
||||
"kind": "vite",
|
||||
"localRoot": "C:\\YourRoot\\viteapp",
|
||||
"ports": { "dev": 5173 },
|
||||
"ports": {
|
||||
"dev": 5173
|
||||
},
|
||||
"domain": "www.yourdomain.com",
|
||||
"remote": {
|
||||
"path": "/home/YOUR_SSH_USER/stack/viteapp",
|
||||
"containerName": "viteapp"
|
||||
},
|
||||
"deploy": { "zipName": "ViteAppDeploy.zip" }
|
||||
"deploy": {
|
||||
"zipName": "ViteAppDeploy.zip"
|
||||
}
|
||||
},
|
||||
|
||||
"nextapp": {
|
||||
"label": "My Next.js App",
|
||||
"kind": "nextjs",
|
||||
"localRoot": "C:\\YourRoot\\nextapp",
|
||||
"ports": { "dev": 4173, "prod": 3000 },
|
||||
"ports": {
|
||||
"dev": 4173,
|
||||
"prod": 3000
|
||||
},
|
||||
"domain": "app.yourdomain.com",
|
||||
"db": { "user": "nextapp_user", "name": "nextapp_db" },
|
||||
"db": {
|
||||
"user": "nextapp_user",
|
||||
"name": "nextapp_db"
|
||||
},
|
||||
"migrations": "prisma",
|
||||
"remote": {
|
||||
"path": "/home/YOUR_SSH_USER/stack/nextapp",
|
||||
"appService": "web"
|
||||
},
|
||||
"verify": {
|
||||
"_comment": "Optional: after deploy, curl this ON the server (localhost:port+path). Add it when compose publishes the app to 127.0.0.1 only, as it usually does behind the edge proxy — probing the public IP on that port can never answer. Redirects are followed, so a Next.js '/' that 307s to '/login' still verifies. Takes precedence over the domain check.",
|
||||
"_comment": "Optional: after deploy, curl this ON the server (localhost:port+path). Add it when compose publishes the app to 127.0.0.1 only, as it usually does behind the edge proxy \u2014 probing the public IP on that port can never answer. Redirects are followed, so a Next.js '/' that 307s to '/login' still verifies. Takes precedence over the domain check.",
|
||||
"port": 3000,
|
||||
"path": "/",
|
||||
"expect": ""
|
||||
},
|
||||
"deploy": { "zipName": "NextAppDeploy.zip" }
|
||||
"deploy": {
|
||||
"zipName": "NextAppDeploy.zip"
|
||||
}
|
||||
},
|
||||
|
||||
"edge": {
|
||||
"label": "Edge Nginx Proxy",
|
||||
"kind": "edge",
|
||||
@ -98,7 +118,6 @@
|
||||
"path": "/home/YOUR_SSH_USER/stack/edge"
|
||||
}
|
||||
},
|
||||
|
||||
"analytics": {
|
||||
"label": "Analytics (any docker compose app)",
|
||||
"kind": "docker",
|
||||
|
||||
32
zdeploy.ps1
32
zdeploy.ps1
@ -256,19 +256,41 @@ function Wait-VerifyApiBuild {
|
||||
# no DNS yet, so neither answers even though the app is fine.
|
||||
$verifyHost = if ($Proj.deploy -and $Proj.deploy.verifyHost) { $Proj.deploy.verifyHost } else { $Proj.domain }
|
||||
if ($verifyHost) { $headers['Host'] = $verifyHost }
|
||||
# Preferred when the project configures it: read the version from a
|
||||
# container ON the shared docker network rather than through the public
|
||||
# proxy. A service that publishes no port cannot be curled from the host
|
||||
# at all, and the proxy answers from whichever vhost matches the Host
|
||||
# header - so a container with no public route gets another site's
|
||||
# version back. See Get-ServerSideVersionCommand.
|
||||
$execCmd = Get-ServerSideVersionCommand -Proj $Proj
|
||||
$useExec = $false
|
||||
if ($execCmd) {
|
||||
$probe = (ssh @SSH_OPTS -i $PEM_KEY $SSH_TARGET $execCmd | Out-String).Trim()
|
||||
if (Get-LabelFromVersionJson $probe) { $useExec = $true }
|
||||
}
|
||||
if ($useExec) {
|
||||
Write-Host " Asking on server: $($Proj.verify.upstream) (via $($Proj.verify.viaProxy))" -ForegroundColor DarkGray
|
||||
}
|
||||
|
||||
$deadline = (Get-Date).AddSeconds($TimeoutSec)
|
||||
while ((Get-Date) -lt $deadline) {
|
||||
try {
|
||||
if ($useExec) {
|
||||
$raw = ssh @SSH_OPTS -i $PEM_KEY $SSH_TARGET $execCmd
|
||||
$live = Get-LabelFromVersionJson ($raw | Out-String)
|
||||
} else {
|
||||
$r = Invoke-RestMethod -Uri "http://$EC2_IP/api/build-version" -Headers $headers -TimeoutSec 10 -ErrorAction Stop
|
||||
if ($r -and $r.build_version) {
|
||||
if ([string]$r.build_version -eq $ExpectedLabel) {
|
||||
Write-Host " PASS - live build $($r.build_version) matches expected." -ForegroundColor Green
|
||||
$live = if ($r -and $r.build_version) { [string]$r.build_version } else { $null }
|
||||
}
|
||||
if ($live) {
|
||||
if ($live -eq $ExpectedLabel) {
|
||||
Write-Host " PASS - live build $live matches expected." -ForegroundColor Green
|
||||
return $true
|
||||
}
|
||||
Write-Host " Live build is $($r.build_version), expected $ExpectedLabel - waiting..." -ForegroundColor DarkYellow
|
||||
Write-Host " Live build is $live, expected $ExpectedLabel - waiting..." -ForegroundColor DarkYellow
|
||||
}
|
||||
} catch {
|
||||
Write-Host " /api/build-version not ready yet - waiting..." -ForegroundColor DarkGray
|
||||
Write-Host " version endpoint not ready yet - waiting..." -ForegroundColor DarkGray
|
||||
}
|
||||
Start-Sleep -Seconds 3
|
||||
}
|
||||
|
||||
19
zec2.ps1
19
zec2.ps1
@ -22,6 +22,11 @@ Start-ZTracking
|
||||
|
||||
$cfg = Get-ZConfig
|
||||
if (-not $HostName) { $HostName = $cfg.ec2.ip }
|
||||
# Needed by the container-side version read below; same names zec2online.ps1
|
||||
# uses. Without them that read throws inside its try/catch and falls through
|
||||
# silently, which looks identical to a service that cannot be reached.
|
||||
$PemKey = $cfg.ec2.pemKey
|
||||
$SshTarget = Get-Ec2Target
|
||||
|
||||
if ($Projects.Count -eq 0) {
|
||||
Write-Host ""
|
||||
@ -98,8 +103,20 @@ function Show-Zec2LiveVersion {
|
||||
$r = Invoke-RestMethod -Uri "http://${HostName}/build-version.json" -Headers $headers -TimeoutSec 10 -ErrorAction Stop
|
||||
if ($r) { Write-Host " Live build: $(Get-LabelFromBuildJsonObj $r)" -ForegroundColor Gray }
|
||||
} else {
|
||||
# Container-side first where the project configures it: a build
|
||||
# stamp is not public on every site, and asking the proxy answers
|
||||
# from whichever vhost matches the Host header.
|
||||
$execCmd = Get-ServerSideVersionCommand -Proj $Proj
|
||||
$label = $null
|
||||
if ($execCmd -and (Test-Path $PemKey)) {
|
||||
$raw = (ssh -o StrictHostKeyChecking=no -o ConnectTimeout=15 -i $PemKey $SshTarget $execCmd | Out-String)
|
||||
$label = Get-LabelFromVersionJson $raw
|
||||
}
|
||||
if (-not $label) {
|
||||
$r = Invoke-RestMethod -Uri "http://${HostName}/api/build-version" -Headers $headers -TimeoutSec 10 -ErrorAction Stop
|
||||
if ($r -and $r.build_version) { Write-Host " Live build: $($r.build_version)" -ForegroundColor Gray }
|
||||
if ($r -and $r.build_version) { $label = [string]$r.build_version }
|
||||
}
|
||||
if ($label) { Write-Host " Live build: $label" -ForegroundColor Gray }
|
||||
}
|
||||
} catch {
|
||||
Write-Host " (Could not read live version endpoint - see 'Enabling deploy verification' in README)" -ForegroundColor DarkGray
|
||||
|
||||
@ -86,6 +86,17 @@ function Get-RemoteVersionLabel {
|
||||
param($Proj)
|
||||
$headers = @{}
|
||||
if ($Proj.domain) { $headers['Host'] = $Proj.domain }
|
||||
# Container-side first where the project configures it. A build stamp is
|
||||
# not public on every site, and the proxy answers from whichever vhost
|
||||
# matches the Host header - which is how a check reads another service.
|
||||
$execCmd = Get-ServerSideVersionCommand -Proj $Proj
|
||||
if ($execCmd -and (Test-Path $PemKey)) {
|
||||
try {
|
||||
$raw = ssh -o StrictHostKeyChecking=no -o ConnectTimeout=15 -i $PemKey $SshTarget $execCmd
|
||||
$label = Get-LabelFromVersionJson ($raw | Out-String)
|
||||
if ($label) { return $label }
|
||||
} catch { }
|
||||
}
|
||||
try {
|
||||
if ($Proj.kind -eq 'vite') {
|
||||
$r = Invoke-RestMethod -Uri "http://${HostName}/build-version.json" -Headers $headers -TimeoutSec 10 -ErrorAction Stop
|
||||
|
||||
Loading…
Reference in New Issue
Block a user