commit 3546a125642dfb83e1a64e135994ecf3edf63efe Author: KellyMichels Date: Wed Jul 15 20:35:17 2026 -0500 Evomedia.net Token Savers - initial public release Config-driven PowerShell scripts to run infrastructure tasks (deploy, restart, backup, diagnostics) yourself instead of having an AI agent orchestrate them, to save agent tokens. Environment specifics live in zconfig.json (gitignored). diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..30ce77c --- /dev/null +++ b/.gitignore @@ -0,0 +1,15 @@ +# Local config — contains secrets (EC2 IP, SSH key path, local paths) +zconfig.json + +# Legacy deploy config (superseded by zconfig.json) +zdeploy.config.json + +# Temp zips from deploy (if a temp/ ever lands inside this folder) +temp/ + +# OS / editor +.DS_Store +Thumbs.db +desktop.ini +.idea/ +.vscode/ diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..0a7c7ba --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,44 @@ + + +# Changelog + +Notable changes to the Evomedia.net Token Savers. + +## Unreleased + +### Added +- **`zdeploy` optional `deploy.gitPull`** — `git pull --ff-only` in the + project root before zipping. `zdeploy` zips the working tree and doesn't + otherwise pull, so a checkout left behind `origin` after a merged PR would + deploy stale code while still bumping the build number — success that + changes nothing. A failed pull aborts the deploy instead. +- **Per-project `start` config block** — `zstart` honors optional pre-start + steps from `zconfig.json`: `"gitPull": true` runs `git pull --ff-only` in + the project root before starting (never boot a stale checkout), and + `"env": { ... }` sets environment variables for the dev-server process. + Example added to `zconfig.example.json`. +- **Switch-style argument tolerance** — a leading dash on a project key is + ignored everywhere (`zdeploy -myapp` == `zdeploy myapp`), for hands that + grew up on per-project switches. + +### Changed +- **`zkill` / port cleanup kills the whole process tree** — listeners on a + project's port are now terminated children-first. Auto-reloading servers + (uvicorn/watchfiles, nodemon) spawn workers that inherit the listening + socket; killing only the parent left orphans serving stale code. +- **`zbackup` finds `DATABASE_URL` in `backend\.env` too** — projects with a + frontend/backend split get their Postgres dump bundled without needing a + root-level `.env`. + +## 1.0.0 + +Initial public release: `zstart` / `zkill` / `zrestart` (local dev servers), +`zdeploy` (zip → upload → compose build → live build-version verification, +with handlers for python / vite / nextjs / edge / docker project kinds), +`zec2` / `zec2online` / `zrepair` (health checks and recovery), `zbackup` / +`zbackup_ec2` / `zsync` (local, server-side, and offsite backups), all driven +by a single gitignored `zconfig.json`. diff --git a/ELEVATOR_PITCH.md b/ELEVATOR_PITCH.md new file mode 100644 index 0000000..8c94a3b --- /dev/null +++ b/ELEVATOR_PITCH.md @@ -0,0 +1,34 @@ + + +# Elevator Pitch + +## The one-liner + +**AI coding agents waste thousands of tokens a day on infrastructure orchestration. Token Savers gives you one-word commands to run those parts yourself — so your agent spends tokens on code, not on SSH.** + +## The 30-second version + +Every time you ask an AI coding agent to deploy your app, it burns 15,000–35,000 tokens reading Docker build output, SSH logs, and health-check responses — before writing a single line of code. Do that 10–15 times a day on a rapid dev cycle and you've spent $1.41–$16 just on infrastructure chatter (Sonnet 5 to Fable 5), plus context window space that should go to your actual problem. + +Token Savers collapses the infrastructure side into short, one-word commands you run yourself: `zdeploy myapp`, `zrepair myapp`, `zstart myapp`. Describe each project once in `zconfig.json` — where it lives, what kind it is, where it deploys — and every command just knows. You run the deploy; your agent edits the code. You run the health check; your agent reads the result and fixes whatever's wrong. + +**Estimated savings: 157,000–540,000 tokens per active development day.** See [TOKEN_SAVINGS.md](TOKEN_SAVINGS.md) for the per-script breakdown and measurement methodology. + +## Why it's different + +- **Built around the AI-agent workflow.** The commands are short on purpose — fewer keystrokes for you, fewer tokens when an agent invokes them. But the real saving is the operations you *don't* hand to the agent at all. +- **The project name IS the command.** `zstart blog`, `zdeploy api`, `zbackup store` — no flags to memorize, no switches to wire up. +- **One config file, zero secrets in git.** Server IP, SSH key, paths, and project definitions live in one gitignored JSON. Clone it anywhere, drop in your config, go. +- **It verifies the deploy actually landed.** Not "did the server return 200" (a stale cache does that too) — it checks that the *build number* went live, so you know the code you just shipped is the code that's running. + +## Who it's for + +Solo developers and small teams running several containerized web apps (Python, Vite, Next.js, plus edge proxies and stock Docker images) on a single VPS or EC2 box, from a Windows dev machine, over SSH — and using AI coding agents to write the code. + +## The tagline + +*Fewer keystrokes. Fewer tokens. One config to rule your fleet.* diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..5ec2eff --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Kelly Michels + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..d153867 --- /dev/null +++ b/README.md @@ -0,0 +1,362 @@ + + +# Evomedia.net Token Savers + +When an AI coding agent orchestrates your infrastructure — starting dev servers, deploying to EC2, diagnosing 502s — it spends hundreds to thousands of tokens per operation on SSH plumbing, Docker output, and retry logic. Those tokens should go to code. + +Token Savers gives you short, one-word commands to run those parts yourself: `zdeploy myapp`, `zrepair myapp`, `zstart myapp`. You handle the deterministic infrastructure; your agent handles code. **Running these scripts manually instead of asking your agent to orchestrate them saves an estimated 3,000–7,000 tokens per active development day.** See [TOKEN_SAVINGS.md](TOKEN_SAVINGS.md) for the per-script breakdown. + +Every command is a tiny PowerShell script driven by a single JSON config file. The project key you define in that config **is** the command argument — add `myapp` to the config and `zstart myapp`, `zdeploy myapp`, `zbackup myapp` all just work, no script edits needed. + +**Requirements:** Windows, PowerShell 5.1+, OpenSSH client (`ssh`/`scp`, ships with Windows 10/11), and Docker + docker compose on the remote host for the deploy scripts. + +--- + +## Install + +No installer. Clone the repo and add the folder to your `PATH`: + +```powershell +git clone https://github.com/kellymichels/zscripts-token-savers.git C:\tools\zscripts + +# Add to your user PATH (new terminals pick it up automatically) +[Environment]::SetEnvironmentVariable( + "Path", + [Environment]::GetEnvironmentVariable("Path", "User") + ";C:\tools\zscripts", + "User" +) +``` + +Open a new terminal and every command below works from any directory. The `.cmd` wrappers invoke PowerShell with `-ExecutionPolicy Bypass`, so no execution-policy changes are needed — `zstart myapp` just works from cmd, PowerShell, or a VS Code terminal. + +--- + +## Configure + +All machine-specific values (server IP, SSH user and key, folder paths, project definitions) live in **one file**: `zconfig.json`. It is gitignored — your secrets never leave your machine. + +```powershell +cd C:\tools\zscripts +copy zconfig.example.json zconfig.json +notepad zconfig.json +``` + +The example config ships with sample projects named by their kind — `pyapp`, `viteapp`, `nextapp`, `edge`, `analytics`. **Rename the keys to your own project names**; the key is what you type as the command argument. Add as many projects as you like — no script edits ever needed. + +### Config reference + +```jsonc +{ + "ec2": { + "ip": "203.0.113.10", // your server's public IP + "user": "youruser", // SSH user on the server + "pemKey": "C:\\Users\\You\\.ssh\\key.pem", // path to your SSH private key + "stackRoot": "/home/youruser/stack" // parent dir for all deployed projects + }, + "paths": { + "temp": "C:\\dev\\temp", // deploy zips staged here (auto-deleted) + "backupsLocal": "C:\\dev\\backups\\projects", // zbackup output + "backupsEc2": "C:\\dev\\backups\\ec2", // zbackup_ec2 output + "scriptsRoot": "C:\\tools\\zscripts", // this folder + "oneDriveBackups": "" // zsync destination ("" disables) + }, + "projects": { + "myapp": { + "label": "My App", // display name in output + "kind": "python", // python | vite | nextjs | edge | docker + "localRoot": "C:\\dev\\myapp", // project folder on this machine + "startModule": "myapp.main", // python kind: runs "python -m myapp.main" + "ports": { "dev": 8080, "prod": 3000 }, // local dev port / direct server port + "domain": "www.myapp.com", // public domain (health checks + verification) + "start": { // optional zstart pre-steps + "gitPull": true, // git pull --ff-only before starting + "env": { "MYAPP_DEBUG": "1" } // env vars for the dev server process + }, + "db": { "user": "dbuser", "name": "dbname" }, // optional: enables db dump/wait steps + "migrations": "prisma", // optional: run prisma migrate on deploy + "remote": { + "path": "/home/youruser/stack/myapp", // deploy target on the server + "composeDir": "/home/youruser/stack/myapp/docker", // optional: if compose isn't at path root + "appService": "app", // optional: compose service name override + "containerName": "myapp" // optional (vite): container to read build-version from + }, + "deploy": { + "zipName": "MyAppDeploy.zip", // optional: defaults to Deploy.zip + "gitPull": true, // optional: git pull --ff-only before zipping + "exclude": ["docs", "big-data-folder"] // optional: extra top-level dirs/files to skip + } + } + } +} +``` + +Optional blocks do real work: + +- **`start`** — pre-start steps for `zstart`: `gitPull: true` runs `git pull --ff-only` in the project root first (never starts a stale checkout), and `env` sets environment variables for the dev-server process (feature flags, reload switches). +- **`db`** — deploys wait for `pg_isready` and `zbackup_ec2` pulls a `pg_dump`, both against the compose service named `db`. Omit it and those steps are skipped cleanly. +- **`deploy.gitPull`** — `git pull --ff-only` in the project root before zipping, so a merged PR actually ships. Since `zdeploy` zips your working tree, a checkout left behind `origin` would otherwise deploy stale code *and still bump the build number* — a silent no-op that looks like success. A failed pull (dirty tree that conflicts, diverged history) aborts the deploy rather than shipping uncertain code. +- **`migrations": "prisma"`** — runs `npx prisma migrate deploy` inside the app container after each deploy. +- **Compose service-name conventions** — handlers assume the app service is named `app` (python) or `web` (nextjs) and the database service `db`. Override the app service with `remote.appService`. +- **Edge extras** — an `edge`-kind project can set `proxyContainer` (the nginx container's name, used for reloads and stale-container cleanup) and `certsSource` (a host path with TLS certs, mounted read-only when validating `nginx.conf`). + +--- + +## Commands + +The `.cmd` wrappers are the everyday interface. Every command takes one or more project keys from your config; several also accept `all`. A leading dash is tolerated (`zdeploy -myapp` works the same as `zdeploy myapp`) for anyone with switch-style muscle memory. + +| Command | What it does | +|---|---| +| `zstart ...` | Start local dev server(s) | +| `zstartd ...` | Same, detached (new window, returns immediately) | +| `zkill ...` | Kill local dev server(s) by port | +| `zrestart ...` | Kill + start in one step | +| `zrestartd ...` | Kill + start detached | +| `zdeploy ... \| all` | Zip → upload → rebuild → verify a project on the server | +| `zec2 [ ...]` | Quick reachability check (TCP + HTTP + live build version) | +| `zec2online [ ...]` | Deep health check; auto-starts downed stacks, streams diagnostics | +| `zrepair ...` | Audit + repair compose/proxy state on the server | +| `zbackup [ ...]` | Zip local project sources (+ DB dump) to the backups folder | +| `zbackup_ec2 [ ...]` | Pull DB dumps + server-side data files down from the server | +| `zsync []` | Copy new backups offsite (or build + mirror a vite dist) | +| `zstart_docker` | Run a local docker compose stack from `scriptsRoot\docker\` | + +### Local development + +#### `zstart` — start dev servers + +``` +zstart [ ...] [-Port N] [-BindHost ] [-Detached] +``` + +Starts each project's dev server using the handler for its `kind`: **python** runs `python -m ` (preferring the project's `.venv`), **vite** runs `npm run dev -- --host --port`, **nextjs** runs `npm run dev` with `PORT` set. Runs `npm install` automatically if `node_modules` is missing. A project's optional `start` config block runs first — `gitPull` fast-forwards the checkout and `env` sets process environment variables. Two more opt-in conveniences: if the project has a `motd/` folder of `.txt` files, one is shown (rotating) at startup; if it has `scripts/build_version_tool.py`, the build number is bumped on each start. + +```powershell +zstart viteapp # dev server on its configured port +zstart pyapp -Port 9000 # override the port +zstart viteapp -BindHost 0.0.0.0 # expose on the LAN +zstartd nextapp # detached: window opens, prompt returns +``` + +#### `zkill` — stop dev servers + +``` +zkill [ ...] [-Port N] [-KillAll] +``` + +Finds whatever is LISTENING on each project's dev port and kills it — along with its whole process tree, children first. That matters for auto-reloading servers (uvicorn/watchfiles, nodemon): their worker processes inherit the listening socket and would otherwise survive as orphans, serving stale code. `-KillAll` also hunts down stray `node`/`python`/`next-server` processes whose command line references the project folder. + +```powershell +zkill viteapp # free the port +zkill pyapp viteapp nextapp # nuke everything +zkill nextapp -KillAll # also kill orphaned runtime processes +``` + +#### `zrestart` — kill then start + +``` +zrestart [ ...] [-Port N] [-KillAll] [-NoRestart] [-Detached] [-BindHost ] +``` + +The "it's wedged, bounce it" command: kill phase, then start phase with the same flags. `-NoRestart` makes it kill-only; `zrestartd` restarts detached. + +### Server deployment & operations + +#### `zdeploy` — deploy to the server + +``` +zdeploy [ ...] [-Note "message"] +zdeploy all [-Note "message"] +``` + +The core workflow, per project kind (projects with `deploy.gitPull` first `git pull --ff-only` so a merged PR isn't left behind): + +- **python / vite / nextjs** — zip the local source (excluding `.git`, `node_modules`, envs, archives, junk, plus anything in `deploy.exclude`), free disk space on the server (docker prune; aborts if under 1.5 GB free), `scp` the zip up, unzip into `remote.path` preserving the server-side `.env`, `docker compose build` + `up -d`, then **verify the live site reports the new build version** (see [Enabling deploy verification](#enabling-deploy-verification)). nextjs additionally waits for Postgres (`db` block) and applies migrations (`migrations` field). Zips are always deleted locally afterward. +- **edge** — uploads *every top-level file* in the edge folder (nginx.conf, compose, css, htpasswd, …), validates the new config with `nginx -t` before switching over, then recreates the proxy. +- **docker** — uploads the compose folder's files, `docker compose pull` + `up -d`. For stacks that run stock images (analytics, mail, etc.). + +`all` deploys every project — edge kinds first, then the rest in config order — and stops at the first failure. + +```powershell +zdeploy viteapp +zdeploy pyapp -Note "fix billing banner" +zdeploy all -Note "weekly release" +``` + +#### `zec2` — reachability check + +``` +zec2 [ ...] # no args = every project with a domain +``` + +For each project: TCP connect, then an HTTP GET with the project's `domain` as the Host header, then the live build version. Fast "is it up?" answer with firewall hints when it isn't. + +#### `zec2online` — health check with auto-recovery + +``` +zec2online [ ...] # no args = every project with a domain +``` + +The heavier sibling: verifies each app over HTTP, compares the **local** build version against what the **server** is actually serving (a mismatch means "redeploy?" — or a stale cache), and if a site is down it SSHes in, runs `docker compose up -d` for the app and the edge proxy, waits up to 30 s, and streams compose logs and system diagnostics if recovery fails. + +#### `zrepair` — fix server routing + +``` +zrepair [ ...] +``` + +Validates the edge proxy's nginx config (if an edge project is defined), shows each stack's compose status, starts anything that's down, and smoke-tests the live domain. For the "deploy succeeded but the site 502s" class of problem. + +#### `zstop.ps1` — stop server stacks + +``` +zstop [ ...] +``` + +`docker compose down` for the selected stacks on the server. Data volumes are preserved; `zdeploy ` brings a stack back. (PowerShell script only, no `.cmd` wrapper.) + +### Backups + +#### `zbackup` — local backups + +``` +zbackup [ ...] [-Tag "label"] # no args = every project + this scripts folder +zbackup scripts # just this scripts folder ('scripts' is reserved) +``` + +Zips each project's source into `paths.backupsLocal\\_[_tag].zip`. If the project's `.env` declares a `DATABASE_URL`, a Postgres dump is bundled into the zip automatically (`backend\.env` is checked too, for frontend/backend split projects). `-Tag` labels the archive — handy before risky changes. + +```powershell +zbackup # everything +zbackup pyapp -Tag "pre-migration" +``` + +#### `zbackup_ec2` — pull backups from the server + +``` +zbackup_ec2 [ ...] # no args = every project with a remote.path +``` + +For projects with a `db` block, runs `pg_dump` inside the server's db container. Also zips server-side data dirs (`uploads/`, `archive/`, `dist/`) when present, then downloads everything to `paths.backupsEc2` and cleans up the remote temp files. + +#### `zsync` — sync backups offsite + +``` +zsync # new backup files -> paths.oneDriveBackups +zsync -Destination # npm run build, then mirror dist/ to path +``` + +The no-args mode copies only files that don't already exist at the destination (never overwrites, never deletes). The project mode is for mirroring a static build; it also honors `$env:ZSYNC_DEST`. + +#### `zbackup_and_sync.ps1` — both in one + +``` +zbackup_and_sync.ps1 [ ...] +``` + +Runs `zbackup`, then `zsync`. This is what the scheduled task calls. + +#### `setup_backup_schedule.ps1` — nightly automation + +Run **as Administrator** once. Creates a Windows Scheduled Task that runs `zbackup_and_sync.ps1` daily at 2:00 AM. + +### Utilities + +#### `zstart_docker` — local compose stack + +``` +zstart_docker [-Build] [-Attached] [-Solo] +``` + +Brings up a docker compose stack from `scriptsRoot\docker\docker-compose.yml` (or `docker-compose.solo.yml` with `-Solo`). Checks that Docker Desktop is actually running and tells you how to unwedge it if not. + +#### `zsetup_mail.ps1` — provision mail accounts + +``` +zsetup_mail.ps1 -domain yourdomain.com [-mailHost mail.yourdomain.com] +``` + +Creates `admin@` and `noreply@` mailboxes (with generated passwords) in a docker-mailserver container on the server, then prints the exact DNS records (MX, SPF, A) and SMTP/IMAP settings to plug into your app. + +#### `ZHelpers.ps1` — shared library + +Not run directly. Dot-sourced by the other scripts; provides config loading (`Get-ZConfig`, `Get-ZProject`), SSH helpers (`Invoke-Ec2Step`), the deploy/backup archiver (`New-ProjectArchive`), and process-kill helpers. Extend here if you're adding your own scripts. + +--- + +## Enabling deploy verification + +**Why not just check for HTTP 200?** Because a 200 proves nothing — a stale cached build serves 200 all day. These scripts verify a deploy by comparing **build numbers**: your app exposes its build version, the deploy expects to see the *new* number live, and a mismatch means the upload or Docker build failed (or you're looking at a cached build). + +It's optional — deploys still work without it, ending in a WARNING instead of a PASS — but it's the difference between "the server answered" and "the code I just shipped is actually running." + +### 1. Add a version file to your project + +```json +// build-version.json (vite: in public/ · nextjs: in public/ · python: project root) +{ "productVersion": "1.0", "buildNumber": 42 } +``` + +### 2. Bump it during the server-side Docker build + +The convention: each deploy's Docker build increments `buildNumber` by one, so the deploy script expects **local buildNumber + 1** to show up live. One line in your Dockerfile does it: + +```dockerfile +RUN node -e "const f='public/build-version.json',v=require('./'+f);v.buildNumber++;require('fs').writeFileSync(f,JSON.stringify(v))" +``` + +### 3. Expose it + +**Vite / static sites** — nothing to do: `public/build-version.json` is served at `/build-version.json`, which is where verification looks. (If your edge proxy blocks it from outside, set `remote.containerName` in config and verification reads it inside the container instead.) + +**Next.js** — add an API route at `/api/build-version`: + +```ts +// app/api/build-version/route.ts +import { NextResponse } from "next/server"; +import bv from "@/public/build-version.json"; + +export async function GET() { + return NextResponse.json({ build_version: `v${bv.productVersion}.${bv.buildNumber}` }); +} +``` + +**Python (FastAPI shown; any framework works)** — expose `/api/build-version`: + +```python +import json, pathlib + +@app.get("/api/build-version") +def build_version(): + bv = json.loads(pathlib.Path("build-version.json").read_text()) + return {"build_version": f"v{bv['productVersion']}.{bv['buildNumber']}"} +``` + +Python projects can go further with a `scripts/build_version_tool.py` supporting `get` / `set` / `bump` subcommands — if present, `zdeploy` bumps the version inside the running container, records it, and `zstart` bumps on every dev start. + +`zec2` and `zec2online` use these same endpoints to show what's live and flag local/server version drift. + +--- + +## Adding a new project + +1. Add a key under `projects` in `zconfig.json` — copy the sample of the matching `kind` and rename it. +2. That's it: `zstart`, `zkill`, `zrestart`, `zbackup`, `zdeploy`, `zec2`, `zec2online`, `zrepair`, `zstop` all accept the new key immediately. +3. A project whose deploy doesn't fit the python/vite/nextjs/edge/docker patterns needs its own `Invoke-Deploy` function in `zdeploy.ps1` — copy an existing handler; they're all variations on zip → upload → compose up → verify. + +## Troubleshooting + +- **"zconfig.json not found"** — you haven't copied `zconfig.example.json` yet. Every script tells you this and exits. +- **"Unknown project key"** — the argument doesn't match a key in `zconfig.json`; the error lists the valid keys. +- **"PEM key not found"** — fix `ec2.pemKey` in `zconfig.json`. +- **Deploy aborts with "less than 1.5 GB free"** — the server's disk is full even after auto-pruning. Grow the volume, or SSH in and run `sudo docker system prune -af`. +- **Deploy ends with a version WARNING** — the new build isn't what's being served: check the Docker build output, and see [Enabling deploy verification](#enabling-deploy-verification) if you haven't set it up. +- **Port already in use when starting** — `zkill ` first, or just use `zrestart`. + +## License + +[MIT](LICENSE) diff --git a/TOKEN_SAVINGS.md b/TOKEN_SAVINGS.md new file mode 100644 index 0000000..bf4be18 --- /dev/null +++ b/TOKEN_SAVINGS.md @@ -0,0 +1,280 @@ + + +# Token Savings: Why You Should Run These Scripts Yourself + +Running these scripts **manually** keeps their output out of your AI coding agent's +context window. Every line the agent doesn't have to read is a token you don't +pay for — and a token the agent can spend on the actual problem instead of on +deployment sequencing, SSH output, and Docker health checks. + +This document reports two baselines side by side: + +- **You run it → Claude runs the script.** What Claude ingests if it invokes the + z-script as a single command. These figures are **measured** (see method below). +- **You run it → Claude orchestrates raw.** What Claude would ingest if the scripts + didn't exist and it drove `scp` / `ssh` / `docker compose` step by step itself. + These figures are **estimates** — the same command output *plus* the agent's + reasoning and retry logic across every discrete step. + +The savings from running a script yourself is the first column: if you run it, +Claude ingests **zero**. The extra value of *having* the scripts at all is the gap +between the two columns. + +Dollar equivalents use a blended input/output rate: **Sonnet 5 ≈ $9/1M** | **Opus 4.8 ≈ $15/1M** | **Fable 5 ≈ $30/1M** + +> **Measurement note:** "Measured" figures come from `token-count.ps1`, which runs +> each script under `Start-Transcript` and counts output characters ÷ 3.5 +> chars/token. Captured in **Claude Code (Sonnet 4.6)** against the `sp` project. +> "Estimated (raw)" figures are *not* measured — they approximate manual +> orchestration and are marked *est.* throughout. Other models/interfaces tokenize +> differently. + +--- + +## Measured per-run output (script-run baseline) + +The bold figures below are real captures from `token-count.ps1 sp`; rows flagged *est.* are not: + +| Script | Measured tokens/run | Notes | +|--------|--------------------:|-------| +| `zec2online` | **267** | reachability + version check | +| `zec2` | **331** | EC2 TCP/HTTP + build match | +| `zbackup_ec2` | **334** | pull server backup | +| `zrepair` | **364** | clean audit; more if it restarts containers | +| `zkill` | **377** | free the dev port | +| `zbackup` | **436** | local project snapshot | +| `zrestart` | **724** | kill + restart (detached) | +| `zstart` | **762** | start dev server (detached) | +| `zsync` | **769** | mirror backups offsite | +| `zdeploy` *(cached)* | **~810** | 53s deploy, layers cached | +| `zdeploy` *(full rebuild)* | **~34,600** *est.* | packages changed; streams full docker build | +| `zstart_docker` | *not measured* | est. ~500–1,500 | + +**Cache state is what drives `zdeploy`.** A *cached* deploy is **~810 tokens**; the +large number only appears on a **full rebuild** (dependencies changed), which streams +the entire docker build. During rapid deploy → test → fix iteration almost every run +is cached, so ~810 is the realistic per-run cost — with occasional spikes when you +change packages. + +--- + +## Local Development Control + +### `zstart` — Start dev servers +**Measured: ~762 tokens/run** | est. raw orchestration: ~1,500–3,000 | typical 2–3 runs/day + +Run it yourself and Claude sees none of the version-bump, MOTD, and startup output. +If Claude started the server raw, it would also wait on health checks and confirm +the port is listening — reasoning the script does deterministically. + +```powershell +zstart viteapp # start Vite dev server on its configured port +zstart pyapp -Port 3000 # override the port +zstart nextapp -Detached # start in background, prompt returns +``` + +--- + +### `zkill` — Stop dev servers +**Measured: ~377 tokens/run** | est. raw orchestration: ~1,000–2,000 | typical 2–3 runs/day + +Raw, Claude would enumerate processes, kill them, and re-check the port is free. +The script collapses that to one command. + +```powershell +zkill viteapp +zkill pyapp nextapp +``` + +--- + +### `zrestart` — Restart in one command +**Measured: ~724 tokens/run** | est. raw orchestration: ~2,500–4,500 | typical 10–15 runs/day + +The most-used command during rapid iteration. Raw, it's stop → wait → start with +error handling at each hop — several tool calls and their reasoning. As one script +it's a single call, and the `-Detached` switch now propagates correctly through the +kill→restart chain so the server backgrounds cleanly. + +```powershell +zrestart viteapp +zrestart pyapp -Detached +``` + +--- + +## Build & Deployment + +### `zdeploy` — Deploy to EC2 +**Measured: ~810 tokens/run cached** *(spikes to ~34,600 on a full rebuild)* | est. raw orchestration: ~5,000–12,000 cached, ~35,000+ full rebuild | typical 10–15 runs/day + +The biggest lever — and the one where cache state matters most. The script *streams* +the docker/SSH output whether Claude runs it or not, so a cached deploy really is only +~810 tokens even through Claude. The raw-orchestration cost is higher not because of +extra output but because Claude would reason between ~15 discrete steps (zip, preflight +cleanup, scp, unzip, build, up, version bump, restart, verify) and handle retries +itself. Running it yourself zeroes out all of that. + +Measured cached: three runs at 808 / 858 / 808 tokens (53–54s each). The full-rebuild +figure (~34,600) is an estimate for package-change deploys — treat it as the upper +bound. + +```powershell +zdeploy pyapp -Note "Fix nav alignment" +zdeploy edge # reload edge nginx config +zdeploy all -Note "weekly release" +``` + +--- + +### `zstart_docker` — Start local Docker stack +**Not measured** (est. ~500–1,500 tokens/run) | typical 1 run/day + +One-time setup per session; doesn't need agent involvement. + +--- + +## Backup & Sync + +### `zbackup` — Backup projects locally +**Measured: ~436 tokens/run** | est. raw orchestration: ~1,200–2,500 | typical 1–2 runs/day + +Raw, Claude enumerates files, decides exclusions, compresses, and stamps timestamps. +You decide when to snapshot. + +```powershell +zbackup # everything + scripts folder +zbackup pyapp -Tag "pre-refactor" +``` + +--- + +### `zsync` — Sync backups offsite +**Measured: ~769 tokens/run** | est. raw orchestration: ~1,500–3,000 | typical 1 run/day + +Raw, Claude tracks file diffs, runs robocopy, and verifies the copy. You manage +cadence independently. + +```powershell +zsync +zsync viteapp # build + mirror dist to $env:ZSYNC_DEST +``` + +--- + +### `zbackup_ec2` — Pull backups from the server +**Measured: ~334 tokens/run** | est. raw orchestration: ~1,000–2,000 | typical 1 run/day + +Separates database/app backup from code changes. Claude focuses on code; you manage +infrastructure snapshots. + +```powershell +zbackup_ec2 +``` + +--- + +## Diagnostics & Troubleshooting + +### `zec2` — Check EC2 reachability +**Measured: ~331 tokens/run** (`zec2online`: ~267) | est. raw orchestration: ~1,000–2,000 | typical 5–8 runs/day + +When a deploy fails you run this first to confirm EC2 is reachable and the right +build is live — before asking Claude to debug. Raw, that's blind network diagnostics +over SSH. Runs frequently alongside `zdeploy`. + +```powershell +zec2 viteapp +zec2 # check all projects +zec2online sp # lightweight HTTP-only variant +``` + +--- + +### `zrepair` — Audit & repair container routing +**Measured: ~364 tokens/run (clean audit)** | est. raw orchestration: ~2,000–4,000 | typical 1–2 runs/day + +When a page 502s, this isolates routing vs. DNS vs. app logic across several +containers — rather than handing Claude an SSH session to figure out blind. The +364-token figure is a healthy run with nothing to repair; a run that actually +restarts containers emits more. Raw, Claude would SSH per container and reason +across each check. + +```powershell +zrepair viteapp +``` + +--- + +## Daily Token Savings Summary + +Per-run × runs/day, using midpoint run counts. The **measured** column is the real +savings from running scripts yourself; the **est. raw** column approximates what +Claude would burn orchestrating the same work with no scripts. + +| Script | Measured/run | Runs/day | Measured/day | Est. raw/day | +|--------|-------------:|:--------:|-------------:|-------------:| +| `zstart` | 762 | 2–3 | ~1,900 | ~3,800–9,000 | +| `zkill` | 377 | 2–3 | ~940 | ~2,500–6,000 | +| `zrestart` | 724 | 10–15 | ~9,050 | ~31,000–68,000 | +| `zdeploy` *(cached)* | ~810 | 10–15 | ~10,100 | ~62,000–180,000 | +| `zec2` (+`online`) | ~330 | 5–8 | ~2,200 | ~6,500–16,000 | +| `zbackup` | 436 | 1–2 | ~650 | ~1,800–5,000 | +| `zsync` | 769 | 1 | ~770 | ~1,500–3,000 | +| `zbackup_ec2` | 334 | 1 | ~330 | ~1,000–2,000 | +| `zrepair` | 364 | 1–2 | ~550 | ~3,000–6,000 | +| **Total (active dev day)** | | | **~26,500** | **~115,000–295,000** *est.* | + +The **~26,500 tokens/day measured** is the honest, reproducible savings from running +these yourself during an active tool-development day (mostly cached deploys). The +**~115k–295k est.** upper figure is what it would cost to have Claude drive the raw +`ssh`/`docker` sequences instead — dominated by per-step reasoning on `zdeploy` and +`zrestart`, not by output volume. A day with several full-rebuild deploys pushes the +measured figure higher too, since each rebuild streams ~34,600 tokens. + +**Daily dollar savings during active tool development:** + +| Model | Blended rate | Measured/day | Est. raw/day (no scripts) | +|-------|-------------|-------------:|--------------------------:| +| **Sonnet 5** | $9/1M | ~$0.24 | ~$1.04–$2.66 | +| **Opus 4.8** | $15/1M | ~$0.40 | ~$1.73–$4.43 | +| **Fable 5** | $30/1M | ~$0.80 | ~$3.45–$8.85 | + +Over a ~22-day working month, the measured savings run **~$5–$17/mo** (Sonnet → +Fable); the raw-orchestration estimate runs **~$23–$195/mo**. Either way, the +token-budget point stands: every token saved on infrastructure is a token your agent +keeps for the actual problem — and that context-window quality is worth more than the +raw dollar figure suggests. + +--- + +## Claude Model Token Costs *(July 2026)* + +| Model | Input | Output | Typical use | +|-------|-------|--------|-------------| +| **Haiku 4.5** | $1/1M | $5/1M | Quick edits, small changes | +| **Sonnet 5** | $3/1M | $15/1M | Daily coding, medium complexity | +| **Opus 4.8** | $5/1M | $25/1M | Complex reasoning, multi-file refactors | +| **Fable 5** | $10/1M | $50/1M | Advanced reasoning, agentic workflows | + +--- + +## When to Run Scripts Yourself vs. Ask the Agent + +**Run yourself when:** +- ✅ You know exactly what action is needed +- ✅ The script is deterministic (same input = same output) +- ✅ You want to parallelize — run `zstart` while asking Claude for code +- ✅ You're troubleshooting and need fast feedback loops + +**Ask the agent when:** +- ❌ You need conditional logic ("if this test fails, try X") +- ❌ You're chaining operations that depend on each other's output +- ❌ You want the agent to interpret script output and decide next steps + +**Bottom line:** These scripts are optimized for you to run directly. Use them. Save +tokens. Let Claude focus on coding. diff --git a/ZHelpers.ps1 b/ZHelpers.ps1 new file mode 100644 index 0000000..7bd7467 --- /dev/null +++ b/ZHelpers.ps1 @@ -0,0 +1,530 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# ZHelpers.ps1 — shared library dot-sourced by every z script. Not run directly. + +# File extensions to skip recursively when building any deploy/backup zip. +$script:ArchiveExtensions = @( + '.zip', '.dmg', '.arj', '.gz', '.tgz', '.tar', '.rar', '.7z', '.iso', + '.bz2', '.xz', '.lz', '.lzma', '.cab', '.jar', '.war', '.ear', '.z', + '.zst', '.zstd' +) +$script:ScriptExtensions = @('.ps1', '.cmd', '.bat') +$script:JunkExtensions = @( + '.swp', '.swo', '.swn', '.tmp', '.orig', '.rej', '.bak', + '.backup', '.old', + '.pyc', '.pyo', '.pyd', + '.tsbuildinfo', + '.log', + '.db', '.sqlite', '.sqlite3' +) +$script:JunkFileNames = @('.DS_Store', 'Thumbs.db', 'desktop.ini') +$script:JunkDirNames = @( + '.git', '.svn', '.hg', + '__pycache__', '.pytest_cache', '.mypy_cache', '.ruff_cache', '.tox', + 'node_modules', '.npm', '.yarn', '.pnpm-store', + '.next', '.nuxt', '.svelte-kit', '.turbo', '.parcel-cache', '.cache', + '.idea', '.vscode', + 'coverage', 'htmlcov', '.nyc_output' +) + +# ── Config loading ─────────────────────────────────────────────────────────── + +$script:ZConfigCache = $null + +function Get-ZConfig { + if ($null -ne $script:ZConfigCache) { return $script:ZConfigCache } + $configPath = Join-Path $PSScriptRoot "zconfig.json" + if (-not (Test-Path -LiteralPath $configPath)) { + Write-Host "ERROR: zconfig.json not found at $configPath" -ForegroundColor Red + Write-Host " Copy zconfig.example.json to zconfig.json and fill in your values." -ForegroundColor DarkGray + exit 1 + } + try { + $script:ZConfigCache = Get-Content -LiteralPath $configPath -Raw -Encoding UTF8 | ConvertFrom-Json + } catch { + Write-Host "ERROR: Failed to parse zconfig.json - $($_.Exception.Message)" -ForegroundColor Red + exit 1 + } + return $script:ZConfigCache +} + +# All project keys, in config order. Keys starting with "_" are comments, not projects. +function Get-ZProjectKeys { + $cfg = Get-ZConfig + return @($cfg.projects.PSObject.Properties.Name | Where-Object { $_ -notmatch '^_' }) +} + +function Get-ZProject { + param([Parameter(Mandatory)][string]$Key) + $cfg = Get-ZConfig + # Tolerate switch-style keys (zdeploy -myproject) from muscle memory. + $Key = $Key.TrimStart('-') + $proj = if ($Key -notmatch '^_') { $cfg.projects.$Key } else { $null } + if (-not $proj) { + $available = (Get-ZProjectKeys) -join ', ' + Write-Host "ERROR: Unknown project key '$Key'. Available: $available" -ForegroundColor Red + exit 1 + } + return $proj +} + +# The first project of kind 'edge', or $null. Returns @{ Key; Config }. +function Get-ZEdgeProject { + $cfg = Get-ZConfig + foreach ($k in (Get-ZProjectKeys)) { + if ($cfg.projects.$k.kind -eq 'edge') { + return [pscustomobject]@{ Key = $k; Config = $cfg.projects.$k } + } + } + return $null +} + +# Remote compose directory for a project: remote.composeDir if set, else remote.path. +function Get-RemoteComposeDir { + param([Parameter(Mandatory)][string]$Key) + $proj = Get-ZProject -Key $Key + if ($proj.remote.composeDir) { return $proj.remote.composeDir } + return $proj.remote.path +} + +# ── SSH helpers ────────────────────────────────────────────────────────────── + +function Get-Ec2Target { + $c = (Get-ZConfig).ec2 + return "$($c.user)@$($c.ip)" +} + +function Get-Ec2Home { + return "/home/$((Get-ZConfig).ec2.user)" +} + +# Run one bash command on the server; throw on non-zero exit. +function Invoke-Ec2Step { + param( + [Parameter(Mandatory)][string]$Label, + [Parameter(Mandatory)][string]$Bash, + [string]$FailHint = "" + ) + $cfg = Get-ZConfig + Write-Host " >> $Label" -ForegroundColor DarkCyan + ssh -o StrictHostKeyChecking=no -i $cfg.ec2.pemKey (Get-Ec2Target) $Bash + if ($LASTEXITCODE -ne 0) { + $msg = "Remote step failed: '$Label' (exit $LASTEXITCODE)." + if ($FailHint) { $msg += " $FailHint" } + throw $msg + } +} + +# ── Deploy git pull ────────────────────────────────────────────────────────── + +# Fast-forward the project's checkout before a deploy when deploy.gitPull is set. +# zdeploy zips the working tree and does NOT otherwise pull, so after a merged +# PR the checkout can sit behind origin and the deploy would ship stale code +# while still bumping the build number (looks successful, changes nothing). +# Aborts the deploy on a failed pull rather than shipping uncertain code. Runs +# git bare (no 2>&1) and checks $LASTEXITCODE, matching Invoke-Ec2Step under +# $ErrorActionPreference='Stop'. +function Invoke-DeployGitPull { + param([Parameter(Mandatory)]$Proj) + if (-not ($Proj.deploy -and $Proj.deploy.gitPull)) { return } + $root = $Proj.localRoot + if (-not (Test-Path -LiteralPath (Join-Path $root ".git"))) { + Write-Host " gitPull set but '$root' is not a git repo - skipping pull." -ForegroundColor Yellow + return + } + Write-Host "`n--- [0] git pull --ff-only ---" -ForegroundColor Cyan + Push-Location -LiteralPath $root + try { + $branch = (git rev-parse --abbrev-ref HEAD) + Write-Host " Branch: $branch" -ForegroundColor DarkGray + git pull --ff-only + if ($LASTEXITCODE -ne 0) { + throw "git pull --ff-only failed in '$root' (branch '$branch'). Resolve it (commit / stash / reconcile), then re-run - refusing to deploy possibly-stale code." + } + Write-Host " Now at: $(git log -1 --oneline)" -ForegroundColor DarkGray + } + finally { + Pop-Location + } +} + +# ── Build-version helpers ──────────────────────────────────────────────────── + +function Read-JsonBuildVersion { + param([string]$FilePath) + if (-not (Test-Path -LiteralPath $FilePath)) { return $null } + try { return Get-Content -LiteralPath $FilePath -Raw -Encoding UTF8 | ConvertFrom-Json } catch { return $null } +} + +function Get-LabelFromBuildJsonObj { + param($obj) + if (-not $obj) { return $null } + return "v$([string]$obj.productVersion).$([int]$obj.buildNumber)" +} + +# ── Deploy/backup exclude lists ────────────────────────────────────────────── + +# Top-level excludes for a project archive: common junk + kind-specific dirs + +# anything the user lists in the project's deploy.exclude array. +function Get-ArchiveExcludes { + param( + [Parameter(Mandatory)]$Project, + [switch]$ForBackup + ) + $common = @(".git", ".idea", ".vscode", ".claude", "tmp", "nul", ".DS_Store", "backups") + $byKind = switch ([string]$Project.kind) { + "python" { + $list = @(".venv", "venv", "__pycache__", ".pytest_cache", ".nicegui", "archive", "dist", "build", "htmlcov") + if (-not $ForBackup) { $list += "uploads" } # deploys exclude user uploads; backups keep them + $list + } + "vite" { @("node_modules", "dist") } + "nextjs" { @("node_modules", ".next", ".env", ".env.local", ".env.production", ".vercel", "coverage", "out", "build", "next-env.d.ts") } + default { @() } + } + $extra = @() + if ($Project.deploy -and $Project.deploy.exclude) { $extra = @($Project.deploy.exclude) } + return @($common + $byKind + $extra | Select-Object -Unique) +} + +# ── Archive builder ────────────────────────────────────────────────────────── + +# Build a zip from a source directory (deploy/backup). +# - $TopLevelExclude: skip these entries at the source root +# - Archive/script/junk filters and $JunkDirNames pruning apply recursively +# - $IncludeScriptFiles: keep .ps1/.cmd/.bat (needed when backing up this repo itself) +function New-ProjectArchive { + param( + [Parameter(Mandatory)] [string] $SourcePath, + [Parameter(Mandatory)] [string] $DestinationZip, + [string[]] $TopLevelExclude = @(), + [string[]] $ExtraFiles = @(), + [switch] $IncludeScriptFiles + ) + + $sourceItem = Get-Item -LiteralPath $SourcePath -Force -ErrorAction Stop + if (-not $sourceItem.PSIsContainer) { + throw "Source path is not a directory: $($sourceItem.FullName)" + } + $sourceFull = $sourceItem.FullName.TrimEnd('\') + + if (Test-Path -LiteralPath $DestinationZip) { Remove-Item -LiteralPath $DestinationZip -Force } + $destDir = Split-Path -Parent $DestinationZip + if ($destDir -and -not (Test-Path -LiteralPath $destDir)) { + New-Item -ItemType Directory -Path $destDir -Force | Out-Null + } + + Add-Type -AssemblyName System.IO.Compression -ErrorAction SilentlyContinue + Add-Type -AssemblyName System.IO.Compression.FileSystem -ErrorAction SilentlyContinue + + $topEntries = Get-ChildItem -LiteralPath $sourceItem.FullName -Force | Where-Object { $_.Name -notin $TopLevelExclude } + if ($topEntries.Count -eq 0 -and $ExtraFiles.Count -eq 0) { + throw "Nothing to archive in $sourceFull (after top-level excludes)." + } + + $junkDirSet = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase) + foreach ($n in $script:JunkDirNames) { [void]$junkDirSet.Add($n) } + $junkNameSet = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase) + foreach ($n in $script:JunkFileNames) { [void]$junkNameSet.Add($n) } + + $allFiles = New-Object System.Collections.Generic.List[System.IO.FileInfo] + $junkDirsPruned = 0 + $stack = New-Object System.Collections.Generic.Stack[string] + + foreach ($entry in $topEntries) { + if ($entry.PSIsContainer) { + if ($junkDirSet.Contains($entry.Name)) { + $junkDirsPruned++ + continue + } + $stack.Push($entry.FullName) + while ($stack.Count -gt 0) { + $dir = $stack.Pop() + $children = @(Get-ChildItem -LiteralPath $dir -Force -ErrorAction SilentlyContinue) + foreach ($child in $children) { + if ($child.PSIsContainer) { + if ($junkDirSet.Contains($child.Name)) { + $junkDirsPruned++ + } else { + $stack.Push($child.FullName) + } + } else { + [void]$allFiles.Add([System.IO.FileInfo]$child.FullName) + } + } + } + } else { + [void]$allFiles.Add([System.IO.FileInfo]$entry.FullName) + } + } + + foreach ($extra in $ExtraFiles) { + if (Test-Path -LiteralPath $extra) { + $extraItem = Get-Item -LiteralPath $extra -Force + if (-not $extraItem.PSIsContainer) { + [void]$allFiles.Add([System.IO.FileInfo]$extraItem.FullName) + } + } + } + + $archivesSkipped = 0 + $scriptsSkipped = 0 + $junkExtSkipped = 0 + $junkNameSkipped = 0 + $kept = New-Object System.Collections.Generic.List[System.IO.FileInfo] + foreach ($f in $allFiles) { + $ext = $f.Extension + if ($ext) { $ext = $ext.ToLowerInvariant() } + if ($script:ArchiveExtensions -contains $ext) { $archivesSkipped++; continue } + if (-not $IncludeScriptFiles -and $script:ScriptExtensions -contains $ext) { $scriptsSkipped++; continue } + if ($script:JunkExtensions -contains $ext) { $junkExtSkipped++; continue } + if ($junkNameSet.Contains($f.Name)) { $junkNameSkipped++; continue } + [void]$kept.Add($f) + } + # "Including:" reflects what actually lands in the zip — top-level names + # derived from the kept files, not the pre-filter directory listing. + $srcPrefix = $sourceFull + '\' + $topSeen = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase) + $topNames = New-Object System.Collections.Generic.List[string] + foreach ($f in $kept) { + $topName = if ($f.FullName.StartsWith($srcPrefix, [System.StringComparison]::OrdinalIgnoreCase)) { + ($f.FullName.Substring($srcPrefix.Length) -split '\\')[0] + } else { $f.Name } + if ($topSeen.Add($topName)) { [void]$topNames.Add($topName) } + } + if ($topNames.Count -gt 0) { + Write-Host (" Including: " + ($topNames -join ", ")) -ForegroundColor Gray + } + if ($archivesSkipped -gt 0) { + Write-Host (" Skipped {0} nested archive file(s)" -f $archivesSkipped) -ForegroundColor DarkGray + } + if ($scriptsSkipped -gt 0) { + Write-Host (" Skipped {0} local script file(s)" -f $scriptsSkipped) -ForegroundColor DarkGray + } + if ($junkExtSkipped -gt 0) { + Write-Host (" Skipped {0} junk file(s) by extension" -f $junkExtSkipped) -ForegroundColor DarkGray + } + if ($junkNameSkipped -gt 0) { + Write-Host (" Skipped {0} OS junk file(s)" -f $junkNameSkipped) -ForegroundColor DarkGray + } + if ($junkDirsPruned -gt 0) { + Write-Host (" Pruned {0} dev directory subtree(s)" -f $junkDirsPruned) -ForegroundColor DarkGray + } + if ($kept.Count -eq 0) { + throw "Nothing to archive after filters." + } + + $prefix = $sourceFull + '\' + $zip = [System.IO.Compression.ZipFile]::Open($DestinationZip, [System.IO.Compression.ZipArchiveMode]::Create) + try { + foreach ($f in $kept) { + if ($f.FullName.StartsWith($prefix, [System.StringComparison]::OrdinalIgnoreCase)) { + $rel = $f.FullName.Substring($prefix.Length).Replace('\', '/') + } else { + $rel = $f.Name + } + [void][System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile( + $zip, $f.FullName, $rel, [System.IO.Compression.CompressionLevel]::Optimal) + } + } + finally { + $zip.Dispose() + } + + $info = Get-Item -LiteralPath $DestinationZip + $sizeMb = [math]::Round($info.Length / 1MB, 1) + Write-Host (" Archive: $($info.FullName) ($sizeMb MB, $($kept.Count) files)") -ForegroundColor Gray +} + +# ── Process killers ────────────────────────────────────────────────────────── + +# Kill a process and all of its descendants (children first). Needed for +# reloading servers (uvicorn/watchfiles, nodemon): workers inherit the +# listening socket and would keep serving stale code if orphaned. +function Stop-ProcessTree { + param([int]$TargetPid) + $killed = 0 + $children = @(Get-CimInstance Win32_Process -Filter "ParentProcessId=$TargetPid" -ErrorAction SilentlyContinue) + foreach ($child in $children) { + $killed += Stop-ProcessTree -TargetPid $child.ProcessId + } + $proc = Get-Process -Id $TargetPid -ErrorAction SilentlyContinue + if ($proc) { + Write-Host " Killing PID $TargetPid ($($proc.ProcessName))" -ForegroundColor Red + Stop-Process -Id $TargetPid -Force -ErrorAction SilentlyContinue + $killed++ + } + return $killed +} + +function Stop-ListenersOnPort { + param([int]$Port) + $killed = 0 + $listeners = netstat -ano | Select-String ":$Port\s+.*LISTENING" + if ($listeners) { + $seen = @{} + foreach ($line in $listeners) { + if ($line -match '\s(\d+)\s*$') { + $targetPid = $Matches[1] + if ($seen.ContainsKey($targetPid)) { continue } + $seen[$targetPid] = $true + $proc = Get-Process -Id $targetPid -ErrorAction SilentlyContinue + if ($proc) { + Write-Host " Killing PID $targetPid ($($proc.ProcessName)) on port $Port (and children)" -ForegroundColor Red + $killed += Stop-ProcessTree -TargetPid $targetPid + } + } + } + } else { + Write-Host " No LISTENING process on port $Port" -ForegroundColor DarkGray + } + return $killed +} + +# Kill stray runtime processes (node, python, next-server) whose command line +# references the given project root. Safer than killing every node/python on the box. +function Stop-ProjectProcesses { + param([Parameter(Mandatory)][string]$ProjectRoot) + $killed = 0 + foreach ($name in @("node", "python", "next-server")) { + $found = Get-Process -Name $name -ErrorAction SilentlyContinue + foreach ($p in $found) { + try { + $cmd = (Get-CimInstance Win32_Process -Filter "ProcessId = $($p.Id)" -ErrorAction SilentlyContinue).CommandLine + if ($cmd -and $cmd -like "*$ProjectRoot*") { + Write-Host " Killing $name PID $($p.Id) (references $ProjectRoot)" -ForegroundColor Red + Stop-Process -Id $p.Id -Force -ErrorAction SilentlyContinue + $killed++ + } + } catch {} + } + } + return $killed +} + +# ── MOTD (message of the day) ──────────────────────────────────────────────── + +# If the project root has a motd/ folder of .txt files, print one on start, +# rotating through them in shuffled order. {{build_version}} is substituted. +function Show-ProjectMotd { + param( + [Parameter(Mandatory)][string]$Root, + [string]$BuildVersion = "" + ) + $motdDir = Join-Path $Root "motd" + if (-not (Test-Path -LiteralPath $motdDir -PathType Container)) { return } + $motdFiles = @(Get-ChildItem -LiteralPath $motdDir -Filter "*.txt" -File -ErrorAction SilentlyContinue) + if ($motdFiles.Count -eq 0) { return } + + $namesMatch = { + param([string[]]$Order, [object[]]$Files) + $a = @($Order | Sort-Object) + $b = @($Files | ForEach-Object { $_.Name } | Sort-Object) + return -not (Compare-Object -ReferenceObject $a -DifferenceObject $b) + } + $newShuffle = { + param([object[]]$Files) + return @($Files | Sort-Object { Get-Random } | ForEach-Object { $_.Name }) + } + + $statePath = Join-Path $motdDir ".motd_rotation.json" + $order = @() + $nextIdx = 0 + $loaded = $false + if (Test-Path -LiteralPath $statePath) { + try { + $raw = Get-Content -LiteralPath $statePath -Raw -Encoding UTF8 | ConvertFrom-Json + $order = @($raw.order) + $nextIdx = [int]$raw.next_index + if ($order.Count -eq 0) { throw "empty order" } + if (-not (& $namesMatch $order $motdFiles)) { throw "file set changed" } + if ($nextIdx -lt 0 -or $nextIdx -ge $order.Count) { throw "bad index" } + $loaded = $true + } + catch { $loaded = $false } + } + if (-not $loaded) { + $order = & $newShuffle $motdFiles + $nextIdx = 0 + } + + $pickName = $order[$nextIdx] + $pick = $motdFiles | Where-Object { $_.Name -eq $pickName } | Select-Object -First 1 + if (-not $pick) { + $order = & $newShuffle $motdFiles + $nextIdx = 0 + $pickName = $order[$nextIdx] + $pick = $motdFiles | Where-Object { $_.Name -eq $pickName } | Select-Object -First 1 + } + + $following = $nextIdx + 1 + if ($following -ge $order.Count) { + $followingOrder = & $newShuffle $motdFiles + $followingIdx = 0 + } else { + $followingOrder = $order + $followingIdx = $following + } + + if ($pick) { + try { + $motd = Get-Content -LiteralPath $pick.FullName -Raw -Encoding UTF8 + if (-not [string]::IsNullOrWhiteSpace($BuildVersion)) { + $motd = $motd -replace "\{\{build_version\}\}", $BuildVersion + } + if (-not [string]::IsNullOrWhiteSpace($motd)) { + Write-Host "" + Write-Host $motd + Write-Host "" + } + @{ + order = @($followingOrder) + next_index = $followingIdx + } | ConvertTo-Json | Set-Content -LiteralPath $statePath -Encoding UTF8 + } + catch { } + } +} + +# ── Output tracking ─────────────────────────────────────────────────────────── +$global:_ZTrackPath = $null +# Set by token-count.ps1's Invoke-Measured while a script is being timed. +# Start-ZTracking checks this so inner scripts don't replace the outer transcript. +if ($null -eq $global:_ZMeasuring) { $global:_ZMeasuring = $false } + +function Start-ZTracking { + if ($global:_ZMeasuring) { return } + $tp = Join-Path ([System.IO.Path]::GetTempPath()) ("_ztrack_" + [System.Guid]::NewGuid().ToString("N") + ".txt") + $global:_ZTrackPath = $tp + try { Start-Transcript -Path $tp -NoClobber | Out-Null } catch { $global:_ZTrackPath = $null } +} + +function Stop-ZTracking { + if (-not $global:_ZTrackPath) { return } + try { Stop-Transcript | Out-Null } catch {} + $tp = $global:_ZTrackPath + $global:_ZTrackPath = $null + if (-not (Test-Path -LiteralPath $tp)) { return } + try { + $raw = Get-Content -LiteralPath $tp -Raw -Encoding UTF8 + $lines = @($raw -split "`n") + $si = 0 + for ($i = 0; $i -lt $lines.Count; $i++) { + if ($lines[$i] -match '^Transcript started') { $si = $i + 1; break } + } + $ei = $lines.Count + for ($i = $lines.Count - 1; $i -ge 0; $i--) { + if ($lines[$i] -match '^\*{4}') { $ei = $i; break } + } + $body = if ($ei -gt $si) { @($lines[$si..($ei - 1)]) } else { @() } + $text = $body -join "`n" + $lc = ($body | Where-Object { $_.Trim() -ne "" }).Count + $cc = $text.Length + $tok = [math]::Round($cc / 3.5) + Write-Host "" + Write-Host ("--- {0:N0} lines / {1:N0} chars / ~{2:N0} tokens est. (Claude Code) ---" -f $lc, $cc, $tok) -ForegroundColor DarkGray + } catch {} + Remove-Item -LiteralPath $tp -Force -ErrorAction SilentlyContinue +} diff --git a/ZKillOnly.ps1 b/ZKillOnly.ps1 new file mode 100644 index 0000000..bdb8947 --- /dev/null +++ b/ZKillOnly.ps1 @@ -0,0 +1,62 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# ZKillOnly.ps1 — stop local dev server listeners for any project in zconfig.json. +# +# Usage: +# zkill [ ...] [-Port N] [-KillAll] +# +# Examples: +# zkill viteapp +# zkill pyapp nextapp +# zkill nextapp -KillAll # also kill stray node/python processes referencing the project +# +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @(), + [Alias("p")][int]$Port = 0, + [switch]$KillAll +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +if ($Projects.Count -eq 0) { + $keys = (Get-ZProjectKeys) -join ', ' + Write-Host "" + Write-Host "Usage: zkill [ ...] [-Port N] [-KillAll]" -ForegroundColor Yellow + Write-Host " Projects in zconfig.json: $keys" -ForegroundColor Gray + Stop-ZTracking; exit 1 +} + +foreach ($key in $Projects) { + $proj = Get-ZProject -Key $key + $devPort = if ($Port -gt 0) { $Port } elseif ($proj.ports -and $proj.ports.dev) { [int]$proj.ports.dev } else { 0 } + + Write-Host "" + Write-Host "=== zkill ($($proj.label)) ===" -ForegroundColor Cyan + + $killed = 0 + if ($devPort -gt 0) { + Write-Host "Port: $devPort" -ForegroundColor DarkGray + $killed += Stop-ListenersOnPort -Port $devPort + } else { + Write-Host "No dev port configured for '$key' - skipping port kill." -ForegroundColor DarkYellow + } + + if ($KillAll -and $proj.localRoot) { + $killed += Stop-ProjectProcesses -ProjectRoot $proj.localRoot + } + + if ($killed -gt 0) { + Write-Host " $killed process(es) stopped" -ForegroundColor Magenta + } else { + Write-Host " Nothing to kill" -ForegroundColor Green + } +} + +Write-Host "" +Write-Host "Kill complete." -ForegroundColor Cyan +Stop-ZTracking diff --git a/ZKiller.ps1 b/ZKiller.ps1 new file mode 100644 index 0000000..4af1689 --- /dev/null +++ b/ZKiller.ps1 @@ -0,0 +1,65 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# ZKiller.ps1 — kill then restart dev servers for any project in zconfig.json. +# +# Usage: +# zrestart [ ...] [-Port N] [-KillAll] [-NoRestart] [-Detached] [-BindHost host] +# +# Examples: +# zrestart viteapp +# zrestart pyapp -Detached +# +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @(), + [Alias("p")][int]$Port = 0, + [switch]$KillAll, + [switch]$NoRestart, + [switch]$Detached, + [string]$BindHost = "127.0.0.1" +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$KillScript = Join-Path $PSScriptRoot "ZKillOnly.ps1" +$StartScript = Join-Path $PSScriptRoot "zstart.ps1" + +if ($Projects.Count -eq 0) { + $keys = (Get-ZProjectKeys) -join ', ' + Write-Host "" + Write-Host "Usage: zrestart [ ...] [-Port N] [-KillAll] [-NoRestart] [-Detached] [-BindHost host]" -ForegroundColor Yellow + Write-Host " Projects in zconfig.json: $keys" -ForegroundColor Gray + Stop-ZTracking; exit 1 +} + +Write-Host "" +Write-Host "=== zrestart ===" -ForegroundColor Cyan + +foreach ($key in $Projects) { + Write-Host "--- $key ---" -ForegroundColor DarkCyan + + # Splat named params via a hashtable, not an array. The target scripts bind + # $Projects with ValueFromRemainingArguments (greedy), which swallows an + # array-splatted "-Detached"/"-KillAll" string as a literal project value + # instead of binding the switch. Hashtable splatting binds switches reliably. + $killParams = @{} + if ($Port -gt 0) { $killParams.Port = $Port } + if ($KillAll) { $killParams.KillAll = $true } + & $KillScript $key @killParams + if ($LASTEXITCODE -ne 0) { Stop-ZTracking; exit $LASTEXITCODE } + + if (-not $NoRestart) { + Start-Sleep -Seconds 1 + $startParams = @{} + if ($Port -gt 0) { $startParams.Port = $Port } + if ($BindHost -ne "127.0.0.1") { $startParams.BindHost = $BindHost } + if ($Detached) { $startParams.Detached = $true } + & $StartScript $key @startParams + if ($LASTEXITCODE -ne 0) { Stop-ZTracking; exit $LASTEXITCODE } + } +} +Stop-ZTracking diff --git a/md/Z-ScriptTokenData.md b/md/Z-ScriptTokenData.md new file mode 100644 index 0000000..3a4c3eb --- /dev/null +++ b/md/Z-ScriptTokenData.md @@ -0,0 +1,169 @@ +# Z-Scripts Summary: Manual Automation to Reduce Token Usage + +Running these scripts **manually** instead of asking Claude Code to orchestrate them saves significant token usage because you skip the overhead of Claude reasoning about deployment/build/testing orchestration. + +--- + +## **Local Development Control** + +### `zstart.ps1` — Start dev servers +**What:** Starts local dev servers for any project defined in `zconfig.json` +```powershell +zstart viteapp # Start Vite dev server +zstart pyapp -Port 3000 # Start Python app on custom port +zstart nextapp -Detached # Start in background +``` +**Why run manually:** Eliminates Claude's need to track startup, wait for health checks, or validate ports. You start the server once and Claude just edits files—tokens saved on orchestration, ~150-300 tokens per use. + +--- + +### `zkill.ps1` — Stop dev servers +**What:** Kills Node/Python/Docker processes on specified ports +```powershell +zkill viteapp # Kill Vite dev server +zkill pyapp nextapp # Kill multiple projects +``` +**Why run manually:** You control when to stop iteration cycles. Saves Claude from having to reason about process cleanup, ~100-200 tokens. + +--- + +### `zrestart.ps1` — Restart in one command +**What:** Calls `zkill` + `zstart` atomically; useful after major changes +```powershell +zrestart viteapp # Kill + restart Vite app +``` +**Why run manually:** Cleaner than telling Claude "stop the server and start it again"—one script handles the sequence. Saves ~200-300 tokens on orchestration logic. + +--- + +## **Build & Deployment** + +### `zdeploy.ps1` — Deploy to EC2 +**What:** Zips source, SCP to EC2, runs docker compose up, verifies build version +```powershell +zdeploy pyapp -Note "Fix nav alignment" +zdeploy edge # Just reload edge nginx config +zdeploy nextapp # Deploy Next.js app + db +zdeploy all # Deploy all projects (edge first) +``` +**Why run manually:** Deployment is deterministic once code is ready. You test locally, then run the deploy script—Claude never needs to understand EC2 SSH, zip compression, docker compose, or deployment verification. Saves ~800-1200 tokens that would otherwise go to deployment orchestration. + +--- + +### `zstart_docker.ps1` — Ensure Docker daemon is running +**What:** Starts Docker Desktop if not running (Windows convenience) +**Why run manually:** One-time setup; doesn't need Claude involvement. + +--- + +## **Backup & Sync** + +### `zbackup.ps1` — Backup projects locally +**What:** Zips any project defined in `zconfig.json` to the local backups folder with a timestamp +```powershell +zbackup # Backup everything + this scripts folder +zbackup viteapp nextapp # Backup just those two projects +zbackup pyapp -Tag "pre-refactor" +``` +**Why run manually:** You decide when to snapshot. Running this yourself before risky changes means Claude never needs to reason about backup strategy or file compression. Saves ~300-500 tokens per session. + +--- + +### `zsync.ps1` — Sync backups to OneDrive +**What:** Robocopy new files from the local backups folder to OneDrive (incremental) +```powershell +zsync # Sync all new backup files +zsync viteapp # Build + mirror vite dist to $env:ZSYNC_DEST +``` +**Why run manually:** You manage backup cadence independently. Claude doesn't need to reason about incremental sync logic or file enumeration. Saves ~250-400 tokens. + +--- + +### `zbackup_ec2.ps1` — Remote backups on EC2 +**What:** SSH to EC2, tar application and database data, pull to local backups folder +**Why run manually:** Separates database/app backup concerns from code changes. Claude focuses on code; you manage infrastructure snapshots. + +--- + +## **Diagnostics & Troubleshooting** + +### `zec2.ps1` — Check EC2 reachability +**What:** TCP + HTTP connectivity tests + live build version for all projects with a `domain` +```powershell +zec2 viteapp # Check if Vite app is up on EC2 +zec2 # Test all projects +``` +**Why run manually:** When a deploy fails, you run this to verify EC2 is reachable before asking Claude to debug. Eliminates Claude doing network diagnostics blind. Saves ~400-600 tokens of troubleshooting overhead. + +--- + +### `zec2online.ps1` — Deep EC2 health check +**What:** Full health check; auto-starts downed stacks, streams diagnostics +**Why run manually:** Quick check before starting work; Claude doesn't need to validate infrastructure state. + +--- + +### `zrepair.ps1` — Audit & repair container routing +**What:** SSH to EC2, verify nginx proxy routes, check docker compose health, run smoke tests +```powershell +zrepair viteapp # Audit proxy path + smoke test +zrepair all # Audit all projects +``` +**Why run manually:** When pages 502, you run this first to isolate whether it's routing, DNS, or app logic. Saves ~1000-1500 tokens of "try this, check logs, try that" debugging. + +--- + +### `zsetup_mail.ps1` — Email account provisioning +**What:** Automates creation of email accounts on EC2; displays Route 53 DNS requirements +**Why run manually:** One-time setup task; doesn't benefit from Claude guidance. + +--- + +## **Token Usage Impact by Script** + +| Script | Manual Run Saves | Without Script (Claude orchestrates) | +|--------|-----------------|--------------------------------------| +| **zstart** | ~150-300 tokens | Claude tracks startup, validates ports, polls health | +| **zkill** | ~100-200 tokens | Claude enumerates processes, checks exit codes | +| **zrestart** | ~200-300 tokens | Claude chains stop→wait→start with error handling | +| **zdeploy** | **~800-1200 tokens** | Claude manages zip, SSH, SCP, compose, verification | +| **zbackup** | ~300-500 tokens | Claude enumerates, compresses, manages timestamps | +| **zsync** | ~250-400 tokens | Claude tracks file diffs, runs robocopy, verifies copy | +| **zec2** | ~400-600 tokens | Claude does TCP/HTTP tests, parses output | +| **zrepair** | **~1000-1500 tokens** | Claude SSH, grep logs, run smoke tests, interpret failures | + +**Total potential savings per day of active development: 3,000–7,000 tokens** if you run these manually vs. asking Claude to orchestrate. + +--- + +## **Claude Model Token Costs** *(Approximate, May 2026)* + +| Model | Input Cost | Output Cost | Use Case | +|-------|-----------|-----------|----------| +| **Haiku 4.5** | ~$0.80/1M | ~$4/1M | Quick code edits, small changes | +| **Sonnet 4.6** | ~$3/1M | ~$15/1M | Daily coding, medium complexity | +| **Opus 4.8** | ~$15/1M | ~$60/1M | Complex reasoning, multi-file refactors | + +**Token savings example:** +- Running `zdeploy` manually: ~1000 tokens saved × $15/1M (Sonnet input) = ~$0.015 saved +- Running `zrepair` manually: ~1500 tokens saved × $15/1M = ~$0.0225 saved +- Running all scripts daily: ~5000 tokens × $0.015 = ~$0.075 saved per day, ~$22.50/month + +**More importantly:** Manual scripts let Claude focus on *code logic* instead of *infrastructure orchestration*—where Claude adds real value. + +--- + +## **TL;DR** + +Run these scripts manually when: +- ✅ You know the exact deployment/test/backup action needed +- ✅ The script is deterministic (same input = same result) +- ✅ You want to parallelize (run zstart while asking Claude for code) +- ✅ You're troubleshooting and need fast feedback loops + +Ask Claude to *invoke* them only when: +- ❌ You need complex conditional logic (e.g., "if this test fails, try X") +- ❌ You're chaining many operations that depend on each other's output +- ❌ You want Claude to interpret script output and decide next steps + +**Bottom line:** Your z-scripts are optimized for **you** to run directly. Use them. Save tokens. Let Claude focus on coding. diff --git a/setup_backup_schedule.ps1 b/setup_backup_schedule.ps1 new file mode 100644 index 0000000..c6a15b6 --- /dev/null +++ b/setup_backup_schedule.ps1 @@ -0,0 +1,78 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# setup_backup_schedule.ps1 — create a scheduled task for daily backups + OneDrive sync +# +# Usage: +# Right-click PowerShell -> "Run as Administrator" +# Then run: .\setup_backup_schedule.ps1 +# +# This creates a task that runs zbackup_and_sync.ps1 every day at 2:00 AM + +#Requires -RunAsAdministrator + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") + +$cfg = Get-ZConfig +$ScriptPath = Join-Path $cfg.paths.scriptsRoot "zbackup_and_sync.ps1" +$TaskName = "ZScripts-Backup-And-Sync" + +Write-Host "" +Write-Host "Setting up scheduled backup task..." -ForegroundColor Cyan +Write-Host "" + +if (-not (Test-Path -LiteralPath $ScriptPath)) { + Write-Host "ERROR: Script not found: $ScriptPath" -ForegroundColor Red + Write-Host " Check paths.scriptsRoot in zconfig.json" -ForegroundColor DarkGray + exit 1 +} + +$existingTask = Get-ScheduledTask -TaskName $TaskName -ErrorAction SilentlyContinue +if ($existingTask) { + Write-Host "Task already exists: $TaskName" -ForegroundColor Yellow + Write-Host "" + $deleteChoice = Read-Host "Delete and recreate? (y/n)" + if ($deleteChoice -eq "y") { + Unregister-ScheduledTask -TaskName $TaskName -Confirm:$false + Write-Host "Deleted existing task." -ForegroundColor Green + } else { + Write-Host "Keeping existing task. Exiting." -ForegroundColor Yellow + exit 0 + } +} + +$trigger = New-ScheduledTaskTrigger -Daily -At "02:00" +$action = New-ScheduledTaskAction ` + -Execute "powershell.exe" ` + -Argument "-ExecutionPolicy Bypass -NoProfile -File `"$ScriptPath`"" +$settings = New-ScheduledTaskSettingsSet ` + -AllowStartIfOnBatteries ` + -DontStopIfGoingOnBatteries ` + -StartWhenAvailable ` + -Compatibility Win8 + +Write-Host "Creating scheduled task: $TaskName" -ForegroundColor Yellow +Register-ScheduledTask ` + -TaskName $TaskName ` + -Trigger $trigger ` + -Action $action ` + -Settings $settings ` + -Description "Daily backup of all projects + OneDrive sync. Runs at 2:00 AM daily." ` + -Force + +Write-Host "" +Write-Host "Success! Scheduled task created." -ForegroundColor Green +Write-Host "" +Write-Host "Task Details:" -ForegroundColor Cyan +Write-Host " Name: $TaskName" -ForegroundColor Gray +Write-Host " Schedule: Daily at 2:00 AM" -ForegroundColor Gray +Write-Host " Action: $ScriptPath" -ForegroundColor Gray +Write-Host "" +Write-Host "To manage the task:" -ForegroundColor Gray +Write-Host " View: Get-ScheduledTask -TaskName '$TaskName'" -ForegroundColor DarkGray +Write-Host " Run now: Start-ScheduledTask -TaskName '$TaskName'" -ForegroundColor DarkGray +Write-Host " Disable: Disable-ScheduledTask -TaskName '$TaskName'" -ForegroundColor DarkGray +Write-Host " Delete: Unregister-ScheduledTask -TaskName '$TaskName'" -ForegroundColor DarkGray +Write-Host "" diff --git a/token-count.ps1 b/token-count.ps1 new file mode 100644 index 0000000..c6667bc --- /dev/null +++ b/token-count.ps1 @@ -0,0 +1,172 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# token-count.ps1 — measure script output volume to estimate AI agent token costs. +# +# Runs each z-script and captures lines + characters of output. Use the results +# to validate or update the figures in TOKEN_SAVINGS.md. +# +# Usage: +# .\token-count.ps1 # run all scripts against a project key +# .\token-count.ps1 -Skip zbackup_ec2,zrepair # skip slow/destructive ones +# .\token-count.ps1 -Only zdeploy,zec2 # run specific scripts only +# +# Output: a summary table with lines, chars, and estimated tokens per script. +# Estimated tokens = chars / 3.5 (Claude's approximate tokenization rate). + +param( + [Parameter(Position = 0, Mandatory = $true)] + [string]$Project, + [string[]]$Skip = @(), + [string[]]$Only = @() +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") + +$CHARS_PER_TOKEN = 3.5 + +$results = [System.Collections.Generic.List[hashtable]]::new() + +function Invoke-Measured { + param( + [string]$Label, + [string]$ScriptName, + [scriptblock]$Block, + [string]$Note = "" + ) + + if ($Only.Count -gt 0 -and $Only -notcontains $ScriptName) { return } + if ($Skip -contains $ScriptName) { + $results.Add(@{ label = $Label; skipped = $true; note = "skipped" }) + return + } + + Write-Host "Running $Label..." -ForegroundColor DarkGray -NoNewline + $tp = Join-Path ([System.IO.Path]::GetTempPath()) ("_zm_" + [System.Guid]::NewGuid().ToString("N") + ".txt") + $global:_ZMeasuring = $true + try { + # Use Start-Transcript so Write-Host (stream 6) is captured. + # _ZMeasuring tells Start-ZTracking in all called scripts to no-op, + # so no inner script can replace or stop our transcript. + Start-Transcript -Path $tp -NoClobber | Out-Null + try { & $Block } catch {} + try { Stop-Transcript | Out-Null } catch {} + + $raw = Get-Content -LiteralPath $tp -Raw -Encoding UTF8 -ErrorAction SilentlyContinue + if (-not $raw) { $raw = "" } + $allLines = @($raw -split "`n") + $si = 0 + for ($i = 0; $i -lt $allLines.Count; $i++) { + if ($allLines[$i] -match '^Transcript started') { $si = $i + 1; break } + } + $ei = $allLines.Count + for ($i = $allLines.Count - 1; $i -ge 0; $i--) { + if ($allLines[$i] -match '^\*{4}') { $ei = $i; break } + } + $body = if ($ei -gt $si) { @($allLines[$si..($ei - 1)]) } else { @() } + $text = $body -join "`n" + $lc = ($body | Where-Object { $_.Trim() -ne "" }).Count + $cc = $text.Length + $tokens = [math]::Round($cc / $CHARS_PER_TOKEN) + $results.Add(@{ + label = $Label + script = $ScriptName + lines = $lc + chars = $cc + tokens = $tokens + note = $Note + }) + Write-Host " $lc lines / $cc chars / ~$tokens tokens" -ForegroundColor Green + } catch { + $results.Add(@{ label = $Label; script = $ScriptName; error = $_.Exception.Message }) + Write-Host " ERROR: $($_.Exception.Message)" -ForegroundColor Red + } finally { + $global:_ZMeasuring = $false + try { Stop-Transcript | Out-Null } catch {} + Remove-Item -LiteralPath $tp -Force -ErrorAction SilentlyContinue + } +} + +Write-Host "" +Write-Host "=== Token Count - project: $Project ===" -ForegroundColor Cyan +Write-Host "Chars per token: $CHARS_PER_TOKEN (Claude approximate)" -ForegroundColor DarkGray +Write-Host "" + +# ── Local dev ────────────────────────────────────────────────────────────────── +Invoke-Measured "zstart " "zstart" { & (Join-Path $PSScriptRoot "zstart.ps1") $Project -Detached } ` + -Note "detached - output may be minimal" + +Invoke-Measured "zkill " "zkill" { & (Join-Path $PSScriptRoot "ZKillOnly.ps1") $Project } + +Invoke-Measured "zrestart " "zrestart" { & (Join-Path $PSScriptRoot "ZKiller.ps1") $Project -Detached } ` + -Note "detached" + +# Takes no project key; brings up zscripts\docker\docker-compose.yml. Needs Docker +# Desktop running AND that compose file present, or it only emits an error. +Invoke-Measured "zstart_docker" "zstart_docker" { & (Join-Path $PSScriptRoot "zstart_docker.ps1") } ` + -Note "detached docker stack - needs Docker Desktop + docker/docker-compose.yml" + +# ── Deploy ───────────────────────────────────────────────────────────────────── +Invoke-Measured "zdeploy " "zdeploy" { & (Join-Path $PSScriptRoot "zdeploy.ps1") $Project } ` + -Note "includes docker build - run twice; first may be inflated by package installs" + +# ── Diagnostics ──────────────────────────────────────────────────────────────── +Invoke-Measured "zec2 " "zec2" { & (Join-Path $PSScriptRoot "zec2.ps1") $Project } + +Invoke-Measured "zec2online" "zec2online" { & (Join-Path $PSScriptRoot "zec2online.ps1") $Project } + +Invoke-Measured "zrepair " "zrepair" { & (Join-Path $PSScriptRoot "zrepair.ps1") $Project } ` + -Note "may restart containers on EC2" + +# ── Backups ──────────────────────────────────────────────────────────────────── +Invoke-Measured "zbackup " "zbackup" { & (Join-Path $PSScriptRoot "zbackup.ps1") $Project } + +Invoke-Measured "zbackup_ec2" "zbackup_ec2" { & (Join-Path $PSScriptRoot "zbackup_ec2.ps1") $Project } ` + -Note "pulls from EC2 - skippable if slow" + +Invoke-Measured "zsync " "zsync" { & (Join-Path $PSScriptRoot "zsync.ps1") } + +# ── Summary table ────────────────────────────────────────────────────────────── +Write-Host "" +Write-Host "=== Summary ===" -ForegroundColor Cyan +Write-Host ("{0,-14} {1,8} {2,10} {3,10} {4}" -f "Script", "Lines", "Chars", "~Tokens", "Notes") -ForegroundColor Yellow +Write-Host ("-" * 70) -ForegroundColor DarkGray + +$totalTokens = 0 +foreach ($r in $results) { + if ($r.skipped) { + Write-Host ("{0,-14} {1,8}" -f $r.label, "(skipped)") -ForegroundColor DarkGray + } elseif ($r.error) { + Write-Host ("{0,-14} {1}" -f $r.label, "ERROR: $($r.error)") -ForegroundColor Red + } else { + Write-Host ("{0,-14} {1,8} {2,10} {3,10} {4}" -f ` + $r.label, $r.lines, $r.chars, $r.tokens, $r.note) -ForegroundColor White + $totalTokens += $r.tokens + } +} + +Write-Host ("-" * 70) -ForegroundColor DarkGray +Write-Host ("{0,-14} {1,8} {2,10} {3,10}" -f "TOTAL", "", "", $totalTokens) -ForegroundColor Cyan + +# ── Dollar cost at current pricing ───────────────────────────────────────────── +Write-Host "" +Write-Host "=== Estimated cost if Claude orchestrated all of the above ===" -ForegroundColor Cyan +$models = @( + @{ name = "Haiku 4.5"; blended = 2.25 }, + @{ name = "Sonnet 5 "; blended = 9.00 }, + @{ name = "Opus 4.8 "; blended = 15.00 }, + @{ name = "Fable 5 "; blended = 30.00 } +) +foreach ($m in $models) { + $cost = [math]::Round($totalTokens / 1000000 * $m.blended, 6) + Write-Host (" {0} ~{1,8} tokens x `${2}/1M blended = `${3}" -f ` + $m.name, $totalTokens, $m.blended, $cost) -ForegroundColor White +} + +Write-Host "" +Write-Host "Note: zdeploy output varies significantly between runs." -ForegroundColor DarkGray +Write-Host " First run after package updates can be 2-3x larger than a cached run." -ForegroundColor DarkGray +Write-Host " Run zdeploy twice and use the second (cached) figure for TOKEN_SAVINGS.md." -ForegroundColor DarkGray +Write-Host "" diff --git a/zbackup.cmd b/zbackup.cmd new file mode 100644 index 0000000..e29f314 --- /dev/null +++ b/zbackup.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zbackup.ps1" %* diff --git a/zbackup.ps1 b/zbackup.ps1 new file mode 100644 index 0000000..09c9bfb --- /dev/null +++ b/zbackup.ps1 @@ -0,0 +1,200 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zbackup.ps1 — local backups: zip project sources (plus a Postgres dump when the +# project's .env has a DATABASE_URL) into the backups folder. +# +# Usage: +# zbackup # every project in zconfig.json + this scripts folder +# zbackup [ ...] +# zbackup scripts # just this scripts folder ('scripts' is a reserved word) +# zbackup pyapp -Tag "pre-migration" +# +# Output: \\_[_tag].zip + +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @(), + [string]$Tag = "" +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$cfg = Get-ZConfig +$ProjectsBackupRoot = $cfg.paths.backupsLocal + +$includeScripts = $false +if ($Projects.Count -eq 0) { + $Projects = @(Get-ZProjectKeys) + $includeScripts = $true +} elseif ($Projects -contains 'scripts') { + $Projects = @($Projects | Where-Object { $_ -ne 'scripts' }) + $includeScripts = $true +} + +function Get-BackupTimestamp { return Get-Date -Format "yyyyMMdd-HHmmss" } + +function Get-TagSuffix { + if ([string]::IsNullOrWhiteSpace($Tag)) { return "" } + return "_" + ($Tag.Trim() -replace '\s+', '_') +} + +function Ensure-BackupDir { + param([string]$ProjectKey) + $dir = Join-Path $ProjectsBackupRoot $ProjectKey + if (-not (Test-Path -LiteralPath $dir)) { + New-Item -ItemType Directory -Path $dir -Force | Out-Null + } + return $dir +} + +# Dump the project's Postgres database if its .env declares a DATABASE_URL. +# Checks \.env, then \backend\.env (frontend/backend split projects). +function Invoke-LocalPgDump { + param([string]$Root, [string]$OutPath) + $envFile = Join-Path $Root ".env" + if (-not (Test-Path -LiteralPath $envFile)) { $envFile = Join-Path $Root "backend\.env" } + if (-not (Test-Path -LiteralPath $envFile)) { return $false } + $dbLine = @(Get-Content -LiteralPath $envFile | Where-Object { $_ -match "^DATABASE_URL=" } | Select-Object -First 1) + if ($dbLine.Count -eq 0) { return $false } + + $dbUrl = ($dbLine[0] -replace "^DATABASE_URL=", "").Trim() + $dbUrl = $dbUrl -replace '^postgresql\+[^:]+://', 'postgresql://' + $rx = [regex]::Match( + $dbUrl, + '^postgresql://(?[^:]+):(?[^@]+)@(?[^:]+):(?\d+)/(?[^?]+)' + ) + if (-not $rx.Success) { + Write-Host ' Could not parse DATABASE_URL - skipping PG backup' -ForegroundColor Red + return $false + } + $env:PGPASSWORD = [Uri]::UnescapeDataString($rx.Groups['pass'].Value) + $pgUser = $rx.Groups['user'].Value + $pgHost = $rx.Groups['host'].Value + $pgPort = $rx.Groups['port'].Value + $pgDb = $rx.Groups['db'].Value + + $pgDump = "pg_dump" + $pgBinPaths = @( + "C:\Program Files\PostgreSQL\18\bin\pg_dump.exe", + "C:\Program Files\PostgreSQL\17\bin\pg_dump.exe", + "C:\Program Files\PostgreSQL\16\bin\pg_dump.exe", + "C:\Program Files\PostgreSQL\15\bin\pg_dump.exe" + ) + foreach ($candidate in $pgBinPaths) { + if (Test-Path $candidate) { $pgDump = $candidate; break } + } + & $pgDump -h $pgHost -p $pgPort -U $pgUser -d $pgDb -F p -f $OutPath 2>$null + $ok = ($LASTEXITCODE -eq 0) -and (Test-Path -LiteralPath $OutPath) + Remove-Item Env:\PGPASSWORD -ErrorAction SilentlyContinue + if ($ok) { + $size = [math]::Round((Get-Item $OutPath).Length / 1KB, 1) + Write-Host " PostgreSQL dump: ${size} KB" -ForegroundColor Green + } else { + Write-Host " pg_dump failed (exit $LASTEXITCODE)" -ForegroundColor Red + } + return $ok +} + +function Invoke-ProjectBackup { + param([string]$Key) + $proj = Get-ZProject -Key $Key + $root = $proj.localRoot + $ts = Get-BackupTimestamp + $suffix = Get-TagSuffix + $outDir = Ensure-BackupDir -ProjectKey $Key + $zipPath = Join-Path $outDir "${ts}_${Key}${suffix}.zip" + + Write-Host "" + Write-Host "=== [$($Key.ToUpper())] Local backup ($($proj.label)) ===" -ForegroundColor Cyan + Write-Host " Output: $zipPath" -ForegroundColor Gray + + if (-not (Test-Path -LiteralPath $root)) { throw "$($proj.label) root not found: $root" } + + $dumpDir = Join-Path $env:TEMP "zbackup_${Key}_dump_$ts" + if (Test-Path $dumpDir) { Remove-Item $dumpDir -Recurse -Force } + New-Item -ItemType Directory -Path $dumpDir -Force | Out-Null + + try { + $extraFiles = @() + Write-Host " [1/3] Checking for a local database to dump..." -ForegroundColor Yellow + $dbDumpPath = Join-Path $dumpDir "database_pg.sql" + if (Invoke-LocalPgDump -Root $root -OutPath $dbDumpPath) { + $extraFiles += $dbDumpPath + } else { + Write-Host " No local DATABASE_URL - source-only backup." -ForegroundColor DarkGray + } + + Write-Host " [2/3] Archiving source..." -ForegroundColor Yellow + New-ProjectArchive -SourcePath $root -DestinationZip $zipPath ` + -TopLevelExclude (Get-ArchiveExcludes -Project $proj -ForBackup) -ExtraFiles $extraFiles + + Write-Host " [3/3] Done." -ForegroundColor Yellow + $zipSize = [math]::Round((Get-Item $zipPath).Length / 1MB, 2) + return @{ ok = $true; path = $zipPath; sizeMb = $zipSize } + } + finally { + if (Test-Path $dumpDir) { Remove-Item $dumpDir -Recurse -Force -ErrorAction SilentlyContinue } + } +} + +function Invoke-ScriptsBackup { + $ts = Get-BackupTimestamp + $suffix = Get-TagSuffix + $outDir = Ensure-BackupDir -ProjectKey "scripts" + $zipPath = Join-Path $outDir "${ts}_scripts${suffix}.zip" + $scriptsRoot = $cfg.paths.scriptsRoot + + Write-Host "" + Write-Host "=== [SCRIPTS] Local backup (this scripts folder) ===" -ForegroundColor Cyan + Write-Host " Output: $zipPath" -ForegroundColor Gray + + if (-not (Test-Path -LiteralPath $scriptsRoot)) { throw "Scripts root not found: $scriptsRoot" } + + Write-Host " [1/2] Archiving source..." -ForegroundColor Yellow + New-ProjectArchive -SourcePath $scriptsRoot -DestinationZip $zipPath ` + -TopLevelExclude @(".git", "archive", "tmp", "nul") -IncludeScriptFiles + + Write-Host " [2/2] Done." -ForegroundColor Yellow + $zipSize = [math]::Round((Get-Item $zipPath).Length / 1MB, 2) + return @{ ok = $true; path = $zipPath; sizeMb = $zipSize } +} + +$exitCode = 0 +$results = @() + +foreach ($key in $Projects) { + try { + $results += @{ project = $key; result = Invoke-ProjectBackup -Key $key } + } catch { + Write-Host " FAILED [$key]: $($_.Exception.Message)" -ForegroundColor Red + $results += @{ project = $key; failed = $_.Exception.Message } + if ($exitCode -eq 0) { $exitCode = 1 } + } +} + +if ($includeScripts) { + try { + $results += @{ project = "scripts"; result = Invoke-ScriptsBackup } + } catch { + Write-Host " FAILED [scripts]: $($_.Exception.Message)" -ForegroundColor Red + $results += @{ project = "scripts"; failed = $_.Exception.Message } + if ($exitCode -eq 0) { $exitCode = 1 } + } +} + +Write-Host "" +Write-Host "=== Backup summary ===" -ForegroundColor Cyan +foreach ($r in $results) { + if ($r.result) { + Write-Host (' {0} -> {1} ({2} MB)' -f $r.project, $r.result.path, $r.result.sizeMb) -ForegroundColor Green + } else { + Write-Host (' {0} -> FAILED: {1}' -f $r.project, $r.failed) -ForegroundColor Red + } +} +Write-Host "" + +Stop-ZTracking; exit $exitCode diff --git a/zbackup_and_sync.ps1 b/zbackup_and_sync.ps1 new file mode 100644 index 0000000..cb0576b --- /dev/null +++ b/zbackup_and_sync.ps1 @@ -0,0 +1,60 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zbackup_and_sync.ps1 — run backups, then sync the backups folder offsite. +# +# Usage: +# zbackup_and_sync.ps1 # backup everything + sync +# zbackup_and_sync.ps1 [ ...] +# +# Scheduled Task example (see setup_backup_schedule.ps1): +# powershell -ExecutionPolicy Bypass -NoProfile -File "\zbackup_and_sync.ps1" + +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @() +) + +$ErrorActionPreference = "Stop" +$ScriptRoot = Split-Path -Parent $MyInvocation.MyCommand.Definition +. (Join-Path $ScriptRoot "ZHelpers.ps1") +Start-ZTracking + +Write-Host "" +Write-Host "============================================" -ForegroundColor Cyan +Write-Host " Backup & Sync - $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss')" -ForegroundColor Cyan +Write-Host "============================================" -ForegroundColor Cyan +Write-Host "" + +Write-Host "[1/2] Running backups..." -ForegroundColor Yellow +$backupPath = Join-Path $ScriptRoot "zbackup.ps1" +if ($Projects.Count -gt 0) { + & powershell -NoProfile -File $backupPath @Projects +} else { + & powershell -NoProfile -File $backupPath +} +$backupExitCode = $LASTEXITCODE + +if ($backupExitCode -ne 0) { + Write-Host "Backup failed (exit code: $backupExitCode)" -ForegroundColor Red + Stop-ZTracking; exit $backupExitCode +} + +Write-Host "" +Write-Host "[2/2] Syncing offsite..." -ForegroundColor Yellow +$syncPath = Join-Path $ScriptRoot "zsync.ps1" +& powershell -NoProfile -File $syncPath +$syncExitCode = $LASTEXITCODE + +Write-Host "" +Write-Host "============================================" -ForegroundColor Cyan +if ($syncExitCode -eq 0) { + Write-Host " Backup & Sync Complete [OK]" -ForegroundColor Green +} else { + Write-Host " Sync had issues (exit code: $syncExitCode)" -ForegroundColor Yellow +} +Write-Host "============================================" -ForegroundColor Cyan +Write-Host "" + +Stop-ZTracking; exit $syncExitCode diff --git a/zbackup_ec2.cmd b/zbackup_ec2.cmd new file mode 100644 index 0000000..5d01bfa --- /dev/null +++ b/zbackup_ec2.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zbackup_ec2.ps1" %* diff --git a/zbackup_ec2.ps1 b/zbackup_ec2.ps1 new file mode 100644 index 0000000..ef94029 --- /dev/null +++ b/zbackup_ec2.ps1 @@ -0,0 +1,136 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zbackup_ec2.ps1 — pull backups down from the server: a Postgres dump for projects +# with a "db" config block, plus a zip of server-side data dirs (uploads/archive/dist). +# +# Usage: +# zbackup_ec2 # every project with a remote.path +# zbackup_ec2 [ ...] +# +# Output: +# \\__db.sql (projects with a db block) +# \\__files.zip + +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @() +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$cfg = Get-ZConfig +$PEM_KEY = $cfg.ec2.pemKey +$SSH_TARGET = Get-Ec2Target +$RemoteHome = Get-Ec2Home +$Ec2BackupRoot = $cfg.paths.backupsEc2 + +if ($Projects.Count -eq 0) { + $Projects = @(Get-ZProjectKeys | Where-Object { $cfg.projects.$_.remote -and $cfg.projects.$_.remote.path }) +} + +function Get-BackupTimestamp { return Get-Date -Format "yyyyMMdd-HHmmss" } + +function Ensure-Ec2BackupDir { + param([string]$ProjectKey) + $dir = Join-Path $Ec2BackupRoot $ProjectKey + if (-not (Test-Path -LiteralPath $dir)) { + New-Item -ItemType Directory -Path $dir -Force | Out-Null + } + return $dir +} + +function Invoke-ScpFromEc2 { + param([string]$RemotePath, [string]$LocalPath) + scp -o StrictHostKeyChecking=no -i $PEM_KEY "${SSH_TARGET}:$RemotePath" $LocalPath + if ($LASTEXITCODE -ne 0) { throw "SCP download failed: $RemotePath -> $LocalPath (exit $LASTEXITCODE)" } +} + +function Invoke-ProjectEc2Backup { + param([string]$Key) + $proj = Get-ZProject -Key $Key + $remotePath = $proj.remote.path + $composeDir = Get-RemoteComposeDir -Key $Key + $ts = Get-BackupTimestamp + $outDir = Ensure-Ec2BackupDir -ProjectKey $Key + $remoteDb = "$RemoteHome/ec2_${Key}_db_$ts.sql" + $remoteZip = "$RemoteHome/ec2_${Key}_files_$ts.zip" + $localDb = Join-Path $outDir "${ts}_${Key}_db.sql" + $localZip = Join-Path $outDir "${ts}_${Key}_files.zip" + $hasDb = ($proj.db -and $proj.db.user -and $proj.db.name) + + Write-Host "" + Write-Host "=== [$($Key.ToUpper())] Server backup ($($proj.label)) ===" -ForegroundColor Cyan + + if ($hasDb) { + Write-Host " [1/4] PostgreSQL dump on the server..." -ForegroundColor Yellow + $dumpCmd = "cd $composeDir && sudo docker compose exec -T db pg_dump -U $($proj.db.user) -d $($proj.db.name) > $remoteDb" + Invoke-Ec2Step "pg_dump $($proj.db.name)" $dumpCmd + } else { + Write-Host " [1/4] No db config block - skipping database dump." -ForegroundColor DarkGray + } + + Write-Host " [2/4] Zipping server-side data (uploads/archive/dist if present)..." -ForegroundColor Yellow + $zipCmd = @( + "sudo apt-get install -y zip >/dev/null 2>&1", + "FILES=''", + "test -d $remotePath/uploads && FILES=`"`$FILES $remotePath/uploads`"", + "test -d $remotePath/archive && FILES=`"`$FILES $remotePath/archive`"", + "test -d $remotePath/dist && FILES=`"`$FILES $remotePath/dist`"", + "test -f $remotePath/build-version.json && FILES=`"`$FILES $remotePath/build-version.json`"", + "if [ -z `"`$FILES`" ]; then echo 'no data dirs found' | sudo zip $remoteZip - >/dev/null; else sudo zip -r $remoteZip `$FILES; fi" + ) -join '; ' + try { + Invoke-Ec2Step "zip $Key files" $zipCmd + } catch { + Write-Host " WARNING: files zip failed - continuing: $($_.Exception.Message)" -ForegroundColor DarkYellow + Invoke-Ec2Step "placeholder zip" "sudo apt-get install -y zip >/dev/null 2>&1; echo 'zip failed' | sudo zip $remoteZip -" + } + + Write-Host " [3/4] Downloading to local..." -ForegroundColor Yellow + if ($hasDb) { Invoke-ScpFromEc2 -RemotePath $remoteDb -LocalPath $localDb } + Invoke-ScpFromEc2 -RemotePath $remoteZip -LocalPath $localZip + + Write-Host " [4/4] Cleaning up remote..." -ForegroundColor Yellow + Invoke-Ec2Step "remove remote temp files" "rm -f $remoteDb $remoteZip" + + $result = @{ ok = $true; zipPath = $localZip } + $result.zipMb = if (Test-Path $localZip) { [math]::Round((Get-Item $localZip).Length / 1MB, 2) } else { 0 } + if ($hasDb -and (Test-Path $localDb)) { + $result.dbPath = $localDb + $result.dbMb = [math]::Round((Get-Item $localDb).Length / 1MB, 2) + } + return $result +} + +$exitCode = 0 +$results = @() + +foreach ($key in $Projects) { + try { + $results += @{ project = $key; result = Invoke-ProjectEc2Backup -Key $key } + } catch { + Write-Host " FAILED [$key]: $($_.Exception.Message)" -ForegroundColor Red + $results += @{ project = $key; failed = $_.Exception.Message } + if ($exitCode -eq 0) { $exitCode = 1 } + } +} + +Write-Host "" +Write-Host "=== Server backup summary ===" -ForegroundColor Cyan +foreach ($r in $results) { + if ($r.result) { + $line = " $($r.project) ->" + if ($r.result.dbPath) { $line += " $($r.result.dbPath) ($($r.result.dbMb) MB) |" } + if ($r.result.zipPath) { $line += " $($r.result.zipPath) ($($r.result.zipMb) MB)" } + Write-Host $line -ForegroundColor Green + } else { + Write-Host (" {0} -> FAILED: {1}" -f $r.project, $r.failed) -ForegroundColor Red + } +} +Write-Host "" + +Stop-ZTracking; exit $exitCode diff --git a/zconfig.example.json b/zconfig.example.json new file mode 100644 index 0000000..c54b6c1 --- /dev/null +++ b/zconfig.example.json @@ -0,0 +1,92 @@ +{ + "_comment": "Copy this file to zconfig.json and fill in your values. zconfig.json is gitignored — never commit it.", + + "ec2": { + "ip": "YOUR_SERVER_IP", + "user": "YOUR_SSH_USER", + "pemKey": "C:\\Users\\YourUser\\.ssh\\YourKey.pem", + "stackRoot": "/home/YOUR_SSH_USER/stack" + }, + + "paths": { + "temp": "C:\\YourRoot\\temp", + "backupsLocal": "C:\\YourRoot\\backups\\projects", + "backupsEc2": "C:\\YourRoot\\backups\\ec2", + "scriptsRoot": "C:\\YourRoot\\zscripts", + "oneDriveBackups": "" + }, + + "projects": { + "_comment": "Rename these keys to your own project names — the key IS the command argument: zstart pyapp, zdeploy viteapp, zbackup nextapp. Add as many projects as you like. Keys starting with _ are ignored.", + + "pyapp": { + "label": "My Python App", + "kind": "python", + "localRoot": "C:\\YourRoot\\pyapp", + "startModule": "pyapp.main", + "ports": { "dev": 8080 }, + "domain": "pyapp.yourdomain.com", + "start": { + "_comment": "Optional zstart pre-steps: gitPull runs 'git pull --ff-only' first; env sets variables for the server process.", + "gitPull": true, + "env": { "MYAPP_DEBUG": "1" } + }, + "db": { "user": "pyapp_user", "name": "pyapp_db" }, + "remote": { + "path": "/home/YOUR_SSH_USER/stack/pyapp", + "composeDir": "/home/YOUR_SSH_USER/stack/pyapp/docker", + "appService": "app" + }, + "deploy": { "zipName": "PyAppDeploy.zip", "gitPull": true, "exclude": ["docs"] } + }, + + "viteapp": { + "label": "My Vite Site", + "kind": "vite", + "localRoot": "C:\\YourRoot\\viteapp", + "ports": { "dev": 5173 }, + "domain": "www.yourdomain.com", + "remote": { + "path": "/home/YOUR_SSH_USER/stack/viteapp", + "containerName": "viteapp" + }, + "deploy": { "zipName": "ViteAppDeploy.zip" } + }, + + "nextapp": { + "label": "My Next.js App", + "kind": "nextjs", + "localRoot": "C:\\YourRoot\\nextapp", + "ports": { "dev": 4173, "prod": 3000 }, + "domain": "app.yourdomain.com", + "db": { "user": "nextapp_user", "name": "nextapp_db" }, + "migrations": "prisma", + "remote": { + "path": "/home/YOUR_SSH_USER/stack/nextapp", + "appService": "web" + }, + "deploy": { "zipName": "NextAppDeploy.zip" } + }, + + "edge": { + "label": "Edge Nginx Proxy", + "kind": "edge", + "localRoot": "C:\\YourRoot\\edge", + "proxyContainer": "edge_proxy", + "certsSource": "", + "remote": { + "path": "/home/YOUR_SSH_USER/stack/edge" + } + }, + + "analytics": { + "label": "Analytics (any docker compose app)", + "kind": "docker", + "localRoot": "C:\\YourRoot\\analytics", + "domain": "analytics.yourdomain.com", + "remote": { + "path": "/home/YOUR_SSH_USER/stack/analytics" + } + } + } +} diff --git a/zdeploy.cmd b/zdeploy.cmd new file mode 100644 index 0000000..21928fb --- /dev/null +++ b/zdeploy.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zdeploy.ps1" %* diff --git a/zdeploy.ps1 b/zdeploy.ps1 new file mode 100644 index 0000000..b587704 --- /dev/null +++ b/zdeploy.ps1 @@ -0,0 +1,564 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zdeploy.ps1 — deploy any project defined in zconfig.json to the server. +# Each project runs its own docker compose stack; the handler is picked by the +# project's "kind": python | vite | nextjs | edge | docker. +# +# Usage: +# zdeploy [ ...] [-Note "message"] +# zdeploy all # every project (edge kinds first), stop at first failure +# +# Examples: +# zdeploy viteapp +# zdeploy pyapp -Note "fix billing banner" +# zdeploy all -Note "weekly release" +# +# Flow (python/vite/nextjs): zip source -> free server disk space -> scp up -> +# unzip into remote.path (preserving server-side .env) -> docker compose build + up +# -> verify the live site reports the new build version. Zips are always deleted. +# +# Compose service-name conventions (override with remote.appService): +# python kind: app service "app", db service "db" +# nextjs kind: app service "web", db service "db" +# +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @(), + [string]$Note = "Build deployed" +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$cfg = Get-ZConfig +$EC2_IP = $cfg.ec2.ip +$PEM_KEY = $cfg.ec2.pemKey +$STACK_ROOT = $cfg.ec2.stackRoot +$SSH_TARGET = Get-Ec2Target +$RemoteHome = Get-Ec2Home +$Ec2User = $cfg.ec2.user + +$TempRoot = $cfg.paths.temp +if (-not (Test-Path -LiteralPath $TempRoot)) { + New-Item -ItemType Directory -Path $TempRoot -Force | Out-Null +} + +if ($Projects.Count -eq 0) { + $keys = (Get-ZProjectKeys) -join ', ' + Write-Host "" + Write-Host "Usage: zdeploy [ ...] | all [-Note `"message`"]" -ForegroundColor Yellow + Write-Host " Projects in zconfig.json: $keys" -ForegroundColor Gray + Write-Host " 'all' deploys everything (edge kinds first) and stops at the first failure." -ForegroundColor Gray + Stop-ZTracking; exit 1 +} + +# Tolerate switch-style args (zdeploy -myproject) from muscle memory. +$Projects = @($Projects | ForEach-Object { $_.TrimStart('-') }) + +if ($Projects -contains 'all') { + $allKeys = Get-ZProjectKeys + $edgeKeys = @($allKeys | Where-Object { $cfg.projects.$_.kind -eq 'edge' }) + $restKeys = @($allKeys | Where-Object { $cfg.projects.$_.kind -ne 'edge' }) + $Projects = @($edgeKeys + $restKeys) + Write-Host "Deploying all projects: $($Projects -join ', ')" -ForegroundColor Cyan +} + +function Get-DeployZipName { + param([string]$Key, $Proj) + if ($Proj.deploy -and $Proj.deploy.zipName) { return $Proj.deploy.zipName } + return "${Key}Deploy.zip" +} + +# Pre-upload cleanup: remove stale deploy zips, prune docker, truncate big logs, +# fail if under 1.5 GB free. +function Invoke-Ec2PreflightCleanup { + param([string[]]$ExtraZipsToRemove = @()) + Write-Host "`n--- [Preflight] Freeing disk space on the server ---" -ForegroundColor Cyan + $rmZipsClause = if ($ExtraZipsToRemove.Count -gt 0) { "rm -f $($ExtraZipsToRemove -join ' ')" } else { "true" } + $preflightCmd = @( + "echo '--- df / before cleanup ---'", + "df -h /", + "echo '--- removing stale deploy artifacts ---'", + $rmZipsClause, + "echo '--- pruning docker build cache + dangling images + stopped containers ---'", + "sudo docker container prune -f >/dev/null 2>&1 || true", + "sudo docker builder prune -f >/dev/null 2>&1 || true", + "sudo docker image prune -af >/dev/null 2>&1 || true", + "echo '--- truncating large container logs ---'", + "sudo find /var/lib/docker/containers/ -name '*-json.log' -size +50M -exec truncate -s 0 {} + 2>/dev/null || true", + "echo '--- df / after cleanup ---'", + "df -h /", + "avail_mb=`$(df --output=avail -BM / | tail -n 1 | tr -dc 0-9)", + "[ -z `"`$avail_mb`" ] && avail_mb=0", + "echo available_mb=`$avail_mb", + "if [ `"`$avail_mb`" -lt 1500 ]; then echo 'ERROR: less than 1.5 GB free on /. Grow the root volume or run: sudo docker system prune -af' >&2; exit 11; fi" + ) -join '; ' + ssh -o StrictHostKeyChecking=no -i $PEM_KEY $SSH_TARGET $preflightCmd + if ($LASTEXITCODE -ne 0) { + throw "Server pre-flight cleanup failed (exit $LASTEXITCODE). Root volume too full (need ~1.5 GB free, ideally 3+)." + } +} + +# Post-deploy cleanup: prune build cache and dangling images created during this deploy. +# Containers/volumes still in use by the running stack are NOT touched. +function Invoke-Ec2PostDeployCleanup { + param([string]$Label = "post-deploy") + Write-Host "`n--- [Post-deploy] Reclaiming disk space ($Label) ---" -ForegroundColor Cyan + $cmd = @( + "sudo docker container prune -f >/dev/null 2>&1 || true", + "sudo docker builder prune -f >/dev/null 2>&1 || true", + "sudo docker image prune -af >/dev/null 2>&1 || true", + "sudo find /var/lib/docker/containers/ -name '*-json.log' -size +50M -exec truncate -s 0 {} + 2>/dev/null || true", + "df -h /" + ) -join '; ' + ssh -o StrictHostKeyChecking=no -i $PEM_KEY $SSH_TARGET $cmd + if ($LASTEXITCODE -ne 0) { + Write-Host " Post-deploy cleanup returned non-zero exit ($LASTEXITCODE); continuing." -ForegroundColor DarkYellow + } +} + +function Send-DeployZip { + param([string]$LocalZip, [string]$ZipName) + scp -i $PEM_KEY $LocalZip "${SSH_TARGET}:$RemoteHome/" + if ($LASTEXITCODE -ne 0) { + throw "SCP upload failed (exit $LASTEXITCODE). Likely server disk space. Try: rm -f $RemoteHome/$ZipName" + } +} + +function Invoke-RemoteUnzip { + param([string]$ZipName, [string]$DestPath) + $bash = 'test -f {2}/{0} || {{ echo "missing {2}/{0}"; exit 2; }}; unzip -t {2}/{0} || exit 3; unzip -o {2}/{0} -d {1}; uc=$?; if [ $uc -gt 1 ]; then exit $uc; fi; exit 0' -f $ZipName, $DestPath, $RemoteHome + Invoke-Ec2Step "unzip $ZipName" $bash +} + +# ── Deploy verification (build-version match, not just HTTP 200 — a 200 can be +# a stale cached build; the version match proves the new build is live) ───── + +function Wait-VerifyStaticBuild { + param([string]$Key, $Proj, [object]$PreZipBuildState) + if (-not $PreZipBuildState) { + Write-Host "`n--- [$Key version] SKIPPED (no local build-version.json - see 'Enabling deploy verification' in README) ---" -ForegroundColor DarkYellow + return + } + $expBn = [int]$PreZipBuildState.buildNumber + 1 + $pv = [string]$PreZipBuildState.productVersion + $expectedLabel = "v$pv.$expBn" + Write-Host "`n--- [$Key] Live build verification (expect $expectedLabel) ---" -ForegroundColor Cyan + $containerName = $Proj.remote.containerName + $deadline = (Get-Date).AddSeconds(45) + while ((Get-Date) -lt $deadline) { + try { + $r = $null + if ($containerName) { + # build-version.json may be blocked from external requests by the edge + # proxy; read it inside the running container instead. + $raw = ssh -o StrictHostKeyChecking=no -i $PEM_KEY $SSH_TARGET ` + "sudo docker exec $containerName cat /usr/share/nginx/html/build-version.json 2>/dev/null" + if ($raw) { $r = $raw | ConvertFrom-Json -ErrorAction Stop } + } else { + $headers = @{} + if ($Proj.domain) { $headers['Host'] = $Proj.domain } + $r = Invoke-RestMethod -Uri "http://$EC2_IP/build-version.json" -Headers $headers -TimeoutSec 10 -ErrorAction Stop + } + if ($r) { + $remoteLabel = Get-LabelFromBuildJsonObj $r + if ($remoteLabel -eq $expectedLabel) { + Write-Host " PASS - live build $remoteLabel matches expected." -ForegroundColor Green + return + } + Write-Host " Live build is $remoteLabel, expected $expectedLabel - waiting..." -ForegroundColor DarkYellow + } + } catch { + Write-Host " Container not ready yet - waiting..." -ForegroundColor DarkGray + } + Start-Sleep -Seconds 3 + } + Write-Host " WARNING: live build did not match $expectedLabel within 45s (upload or Docker build may have failed, or a stale build is cached)." -ForegroundColor Yellow +} + +function Wait-VerifyApiBuild { + param([string]$Key, $Proj, [string]$ExpectedLabel, [int]$TimeoutSec = 60) + Write-Host "`n--- [$Key] Live build verification (expect $ExpectedLabel) ---" -ForegroundColor Cyan + $headers = @{} + if ($Proj.domain) { $headers['Host'] = $Proj.domain } + $deadline = (Get-Date).AddSeconds($TimeoutSec) + while ((Get-Date) -lt $deadline) { + try { + $r = Invoke-RestMethod -Uri "http://$EC2_IP/api/build-version" -Headers $headers -TimeoutSec 10 -ErrorAction Stop + if ($r -and $r.build_version) { + if ([string]$r.build_version -eq $ExpectedLabel) { + Write-Host " PASS - live build $($r.build_version) matches expected." -ForegroundColor Green + return $true + } + Write-Host " Live build is $($r.build_version), expected $ExpectedLabel - waiting..." -ForegroundColor DarkYellow + } + } catch { + Write-Host " /api/build-version not ready yet - waiting..." -ForegroundColor DarkGray + } + Start-Sleep -Seconds 3 + } + Write-Host " WARNING: live build did not match $ExpectedLabel within ${TimeoutSec}s (a stale build may be cached)." -ForegroundColor Yellow + return $false +} + +# ── Kind handlers ──────────────────────────────────────────────────────────── + +function Invoke-PythonDeploy { + param([string]$Key, $Proj, [string]$ChangeNote) + + $DeployStart = Get-Date + $prevLoc = Get-Location + $root = $Proj.localRoot + $remotePath = $Proj.remote.path + $composeDir = if ($Proj.remote.composeDir) { $Proj.remote.composeDir } else { $remotePath } + $appSvc = if ($Proj.remote.appService) { $Proj.remote.appService } else { "app" } + $zipName = Get-DeployZipName -Key $Key -Proj $Proj + $zipLocal = Join-Path $TempRoot $zipName + $BuildVersion = $null + $versionTool = Join-Path $root "scripts\build_version_tool.py" + $hasVersionTool = Test-Path -LiteralPath $versionTool + + try { + if (-not (Test-Path -LiteralPath $root)) { throw "Project root not found: $root" } + Set-Location -LiteralPath $root + + Write-Host "`n=== $($Proj.label) deploy (python) ===" -ForegroundColor Cyan + Write-Host "Local zip: $zipLocal" -ForegroundColor DarkGray + + Write-Host "`n--- [1] Zipping $($Proj.label) ---" -ForegroundColor Cyan + Get-ChildItem -LiteralPath $root -Directory -Recurse -Filter "__pycache__" -ErrorAction SilentlyContinue | + Remove-Item -Recurse -Force -ErrorAction SilentlyContinue + New-ProjectArchive -SourcePath $root -DestinationZip $zipLocal -TopLevelExclude (Get-ArchiveExcludes -Project $Proj) + + Invoke-Ec2PreflightCleanup -ExtraZipsToRemove @("$RemoteHome/$zipName") + + Write-Host "`n--- [2] Uploading zip ---" -ForegroundColor Cyan + Send-DeployZip -LocalZip $zipLocal -ZipName $zipName + + Write-Host "`n--- [3] Unzipping and rebuilding on the server ---" -ForegroundColor Cyan + Invoke-Ec2Step "apt-get install unzip" "sudo apt-get update -qq && sudo apt-get install -y unzip" + Invoke-Ec2Step "ensure stack root" "sudo mkdir -p $STACK_ROOT && sudo chown ${Ec2User}:${Ec2User} $STACK_ROOT" + Invoke-Ec2Step "ensure shared web network" "sudo docker network create web 2>/dev/null || true" + Invoke-Ec2Step "backup .env if present" "if [ -f $remotePath/.env ]; then cp $remotePath/.env $RemoteHome/.env.${Key}_bak; fi" + Invoke-Ec2Step "replace project directory" "sudo rm -rf $remotePath && sudo mkdir -p $remotePath && sudo chown ${Ec2User}:${Ec2User} $remotePath" + Invoke-RemoteUnzip -ZipName $zipName -DestPath $remotePath + Invoke-Ec2Step "restore .env from backup" "if [ -f $RemoteHome/.env.${Key}_bak ]; then cp $RemoteHome/.env.${Key}_bak $remotePath/.env; fi" + Invoke-Ec2Step "require compose directory" "test -d $composeDir" + Invoke-Ec2Step "docker compose build $appSvc" "cd $composeDir && sudo COMPOSE_BAKE=false docker compose build $appSvc" + Invoke-Ec2Step "docker compose up -d" "cd $composeDir && sudo COMPOSE_BAKE=false docker compose up -d" + Invoke-Ec2Step "record deploy time; remove remote zip" "date -u +'%Y-%m-%d %H:%M:%S UTC' | sudo tee $remotePath/.last_deploy_utc > /dev/null && rm -f $RemoteHome/$zipName" + + if ($hasVersionTool) { + Write-Host "`n--- [4] Incrementing build version ---" -ForegroundColor Cyan + $BumpCmd = "cd $composeDir && sudo docker compose exec -T $appSvc python scripts/build_version_tool.py bump" + for ($attempt = 1; $attempt -le 5; $attempt++) { + $output = ssh -o StrictHostKeyChecking=no -i $PEM_KEY $SSH_TARGET $BumpCmd + if ($LASTEXITCODE -eq 0 -and $output) { + $BuildVersion = ($output | Select-Object -Last 1).ToString().Trim() + break + } + Write-Host " Attempt $attempt failed, retrying in 3s..." -ForegroundColor DarkYellow + Start-Sleep -Seconds 3 + } + if (-not $BuildVersion) { throw "Build version bump failed after 5 attempts" } + + python $versionTool set $BuildVersion | Out-Null + ssh -o StrictHostKeyChecking=no -i $PEM_KEY $SSH_TARGET "echo '$BuildVersion' | sudo tee $remotePath/.build_version > /dev/null" + $changelogTool = Join-Path $root "scripts\build_changelog_tool.py" + if (Test-Path -LiteralPath $changelogTool) { + if ([string]::IsNullOrWhiteSpace($ChangeNote)) { $ChangeNote = "Build deployed" } + python $changelogTool append --version $BuildVersion --note "$ChangeNote" | Out-Null + } + + Write-Host "`n--- [5] Restarting app to pick up new version ---" -ForegroundColor Cyan + ssh -o StrictHostKeyChecking=no -i $PEM_KEY $SSH_TARGET "cd $composeDir && sudo COMPOSE_BAKE=false docker compose restart $appSvc" + if ($LASTEXITCODE -ne 0) { throw "App restart after build bump failed (exit $LASTEXITCODE)" } + + Wait-VerifyApiBuild -Key $Key -Proj $Proj -ExpectedLabel $BuildVersion -TimeoutSec 30 | Out-Null + } else { + Write-Host "`n--- [4] Basic reachability check (no build_version_tool - see 'Enabling deploy verification' in README) ---" -ForegroundColor Cyan + $headers = @{} + if ($Proj.domain) { $headers['Host'] = $Proj.domain } + $deadline = (Get-Date).AddSeconds(30) + $up = $false + while ((Get-Date) -lt $deadline) { + Start-Sleep -Seconds 3 + try { + $resp = Invoke-WebRequest -Uri "http://$EC2_IP/" -Headers $headers -UseBasicParsing -TimeoutSec 8 -ErrorAction Stop + if ($resp.StatusCode -lt 500) { $up = $true; break } + } catch { Write-Host " App not ready yet - waiting..." -ForegroundColor DarkGray } + } + if ($up) { Write-Host " App is responding." -ForegroundColor Green } + else { Write-Host " WARNING: app did not respond within 30s." -ForegroundColor Yellow } + } + + Invoke-Ec2PostDeployCleanup -Label $Key + + $Elapsed = (Get-Date) - $DeployStart + $ElapsedStr = "{0:mm\:ss}" -f $Elapsed + Write-Host "`n--- [Done] $($Proj.label) deployed! ---" -ForegroundColor Green + if ($Proj.domain) { Write-Host "Site: https://$($Proj.domain)" -ForegroundColor Yellow } + if ($BuildVersion) { Write-Host "Build Version: $BuildVersion" -ForegroundColor Magenta } + Write-Host "Change Note: $ChangeNote" -ForegroundColor Cyan + Write-Host "Deploy Time: $ElapsedStr ($([math]::Round($Elapsed.TotalSeconds))s)" -ForegroundColor DarkGray + } + finally { + if (Test-Path -LiteralPath $zipLocal) { + try { Remove-Item -LiteralPath $zipLocal -Force -ErrorAction Stop } catch { } + } + Set-Location -LiteralPath $prevLoc + } +} + +function Invoke-ViteDeploy { + param([string]$Key, $Proj, [string]$ChangeNote) + + $DeployStart = Get-Date + $prevLoc = Get-Location + $root = $Proj.localRoot + $remotePath = $Proj.remote.path + $zipName = Get-DeployZipName -Key $Key -Proj $Proj + $zipLocal = Join-Path $TempRoot $zipName + $preZipBuild = $null + + try { + if (-not (Test-Path -LiteralPath $root)) { throw "Project root not found: $root" } + Set-Location -LiteralPath $root + + Write-Host "`n=== $($Proj.label) deploy (vite/static) ===" -ForegroundColor Cyan + Write-Host "Local zip: $zipLocal" -ForegroundColor DarkGray + + Write-Host "`n--- [1] Zipping site ---" -ForegroundColor Cyan + $preZipBuild = Read-JsonBuildVersion -FilePath (Join-Path $root "build-version.json") + if ($preZipBuild) { + Write-Host " Pre-zip build label: $(Get-LabelFromBuildJsonObj $preZipBuild) (server-side build will bump +1)" -ForegroundColor Gray + } + New-ProjectArchive -SourcePath $root -DestinationZip $zipLocal -TopLevelExclude (Get-ArchiveExcludes -Project $Proj) + + Invoke-Ec2PreflightCleanup -ExtraZipsToRemove @("$RemoteHome/$zipName") + + Write-Host "`n--- [2] Uploading zip ---" -ForegroundColor Cyan + Send-DeployZip -LocalZip $zipLocal -ZipName $zipName + + Write-Host "`n--- [3] Unzipping and rebuilding on the server ---" -ForegroundColor Cyan + Invoke-Ec2Step "apt-get install unzip" "sudo apt-get update -qq && sudo apt-get install -y unzip" + Invoke-Ec2Step "ensure stack root" "sudo mkdir -p $STACK_ROOT && sudo chown ${Ec2User}:${Ec2User} $STACK_ROOT" + Invoke-Ec2Step "ensure shared web network" "sudo docker network create web 2>/dev/null || true" + Invoke-Ec2Step "replace project directory" "sudo rm -rf $remotePath && sudo mkdir -p $remotePath && sudo chown ${Ec2User}:${Ec2User} $remotePath" + Invoke-RemoteUnzip -ZipName $zipName -DestPath $remotePath + Invoke-Ec2Step "require compose file" "test -f $remotePath/docker-compose.yml" + Invoke-Ec2Step "docker compose build" "cd $remotePath && sudo COMPOSE_BAKE=false docker compose build" + Invoke-Ec2Step "docker compose up -d" "cd $remotePath && sudo COMPOSE_BAKE=false docker compose up -d" + + $edgeProj = Get-ZEdgeProject + if ($edgeProj -and $edgeProj.Config.proxyContainer) { + Invoke-Ec2Step "reload edge nginx (flush DNS cache for new container IP)" "sudo docker exec $($edgeProj.Config.proxyContainer) nginx -s reload" + } + Invoke-Ec2Step "record deploy time; remove remote zip" "date -u +'%Y-%m-%d %H:%M:%S UTC' | sudo tee $remotePath/.last_deploy_utc > /dev/null && rm -f $RemoteHome/$zipName" + + Wait-VerifyStaticBuild -Key $Key -Proj $Proj -PreZipBuildState $preZipBuild + Invoke-Ec2PostDeployCleanup -Label $Key + + $Elapsed = (Get-Date) - $DeployStart + $ElapsedStr = "{0:mm\:ss}" -f $Elapsed + Write-Host "`n--- [Done] $($Proj.label) deployed! ---" -ForegroundColor Green + if ($Proj.domain) { Write-Host "Site: https://$($Proj.domain)" -ForegroundColor Yellow } + Write-Host "Change Note: $ChangeNote" -ForegroundColor Cyan + Write-Host "Deploy Time: $ElapsedStr ($([math]::Round($Elapsed.TotalSeconds))s)" -ForegroundColor DarkGray + } + finally { + if (Test-Path -LiteralPath $zipLocal) { + try { Remove-Item -LiteralPath $zipLocal -Force -ErrorAction Stop } catch { } + } + Set-Location -LiteralPath $prevLoc + } +} + +function Invoke-NextDeploy { + param([string]$Key, $Proj, [string]$ChangeNote) + + $DeployStart = Get-Date + $prevLoc = Get-Location + $root = $Proj.localRoot + $remotePath = $Proj.remote.path + $appSvc = if ($Proj.remote.appService) { $Proj.remote.appService } else { "web" } + $zipName = Get-DeployZipName -Key $Key -Proj $Proj + $zipLocal = Join-Path $TempRoot $zipName + $preZipBuild = $null + + try { + if (-not (Test-Path -LiteralPath $root)) { throw "Project root not found: $root" } + Set-Location -LiteralPath $root + + Write-Host "`n=== $($Proj.label) deploy (nextjs) ===" -ForegroundColor Cyan + Write-Host "Local zip: $zipLocal" -ForegroundColor DarkGray + + $preZipBuild = Read-JsonBuildVersion -FilePath (Join-Path $root "public\build-version.json") + if ($preZipBuild) { + Write-Host " Pre-zip build label: $(Get-LabelFromBuildJsonObj $preZipBuild) (server-side build will bump +1)" -ForegroundColor Gray + } + + Write-Host "`n--- [1] Zipping project files ---" -ForegroundColor Cyan + New-ProjectArchive -SourcePath $root -DestinationZip $zipLocal -TopLevelExclude (Get-ArchiveExcludes -Project $Proj) + + Invoke-Ec2PreflightCleanup -ExtraZipsToRemove @("$RemoteHome/$zipName") + + Write-Host "`n--- [2] Uploading zip ---" -ForegroundColor Cyan + Send-DeployZip -LocalZip $zipLocal -ZipName $zipName + + Write-Host "`n--- [3] Unzipping and rebuilding on the server ---" -ForegroundColor Cyan + Invoke-Ec2Step "ensure unzip installed" "sudo apt-get update -qq && sudo apt-get install -y unzip" + Invoke-Ec2Step "ensure stack root" "sudo mkdir -p $STACK_ROOT && sudo chown ${Ec2User}:${Ec2User} $STACK_ROOT" + Invoke-Ec2Step "ensure shared web network" "sudo docker network create web 2>/dev/null || true" + Invoke-Ec2Step "backup .env if present" "if [ -f $remotePath/.env ]; then cp $remotePath/.env $RemoteHome/.env.${Key}_bak; fi" + Invoke-Ec2Step "replace project directory" "sudo rm -rf $remotePath && sudo mkdir -p $remotePath && sudo chown ${Ec2User}:${Ec2User} $remotePath" + Invoke-RemoteUnzip -ZipName $zipName -DestPath $remotePath + Invoke-Ec2Step "restore .env from backup" "if [ -f $RemoteHome/.env.${Key}_bak ]; then cp $RemoteHome/.env.${Key}_bak $remotePath/.env; fi" + + Write-Host "`n--- [4] Docker compose rebuild ---" -ForegroundColor Cyan + Invoke-Ec2Step "docker compose down" "cd $remotePath && sudo COMPOSE_BAKE=false docker compose down" + Invoke-Ec2Step "docker compose build" "cd $remotePath && sudo COMPOSE_BAKE=false docker compose build" + Invoke-Ec2Step "docker compose up -d" "cd $remotePath && sudo COMPOSE_BAKE=false docker compose up -d" + + if ($Proj.db -and $Proj.db.user -and $Proj.db.name) { + $waitDb = "cd $remotePath && for i in `$(seq 1 30); do sudo docker compose exec -T db pg_isready -U $($Proj.db.user) -d $($Proj.db.name) >/dev/null 2>&1 && break; sleep 2; done" + Invoke-Ec2Step "wait for postgres ready" $waitDb + } + if ($Proj.migrations -eq "prisma") { + Invoke-Ec2Step "apply prisma migrations" "cd $remotePath && sudo docker compose exec -T $appSvc npx prisma migrate deploy" + } + Invoke-Ec2Step "record deploy timestamp; remove remote zip" "date -u +'%Y-%m-%d %H:%M:%S UTC' | sudo tee $remotePath/.last_deploy_utc > /dev/null && rm -f $RemoteHome/$zipName" + + Write-Host "`n--- [5] Verifying deployment ---" -ForegroundColor Cyan + if ($Proj.ports -and $Proj.ports.prod) { + $directUrl = "http://${EC2_IP}:$([int]$Proj.ports.prod)/" + $deadline = (Get-Date).AddSeconds(60) + $verified = $false + while ((Get-Date) -lt $deadline) { + Start-Sleep -Seconds 4 + try { + $resp = Invoke-WebRequest -Uri $directUrl -TimeoutSec 8 -ErrorAction Stop -UseBasicParsing + if ($resp.StatusCode -eq 200) { + Write-Host " PASS - app is responding at $directUrl" -ForegroundColor Green + $verified = $true + break + } + } catch { + Write-Host " App not ready yet - waiting..." -ForegroundColor DarkGray + } + } + if (-not $verified) { + Write-Host " WARNING: no response at $directUrl within 60s (is the port open in the security group?)." -ForegroundColor Yellow + } + } + if ($preZipBuild) { + $expBn = [int]$preZipBuild.buildNumber + 1 + $expectedLabel = "v$([string]$preZipBuild.productVersion).$expBn" + Wait-VerifyApiBuild -Key $Key -Proj $Proj -ExpectedLabel $expectedLabel -TimeoutSec 60 | Out-Null + } else { + Write-Host " (No public/build-version.json - version verification skipped. See 'Enabling deploy verification' in README.)" -ForegroundColor DarkYellow + } + + Invoke-Ec2PostDeployCleanup -Label $Key + + $Elapsed = (Get-Date) - $DeployStart + $ElapsedStr = "{0:mm\:ss}" -f $Elapsed + Write-Host "`n--- [Done] $($Proj.label) deployed! ---" -ForegroundColor Green + if ($Proj.domain) { Write-Host "Site: https://$($Proj.domain)" -ForegroundColor Yellow } + Write-Host "Change Note: $ChangeNote" -ForegroundColor Cyan + Write-Host "Deploy Time: $ElapsedStr ($([math]::Round($Elapsed.TotalSeconds))s)" -ForegroundColor DarkGray + } + finally { + if (Test-Path -LiteralPath $zipLocal) { + try { Remove-Item -LiteralPath $zipLocal -Force -ErrorAction Stop } catch { } + } + Set-Location -LiteralPath $prevLoc + } +} + +function Invoke-EdgeDeploy { + param([string]$Key, $Proj) + + $root = $Proj.localRoot + $remotePath = $Proj.remote.path + $pc = $Proj.proxyContainer + + Write-Host "`n=== $($Proj.label) deploy (edge nginx ingress) ===" -ForegroundColor Cyan + if (-not (Test-Path -LiteralPath $root)) { throw "Edge root not found: $root" } + foreach ($required in @('docker-compose.yml', 'nginx.conf')) { + if (-not (Test-Path -LiteralPath (Join-Path $root $required))) { throw "Missing $root\$required" } + } + + Invoke-Ec2Step "ensure shared web network" "sudo docker network create web 2>/dev/null || true" + Invoke-Ec2Step "ensure edge dir" "sudo mkdir -p $remotePath && sudo chown ${Ec2User}:${Ec2User} $remotePath" + + # Ship every top-level file in the edge folder — nginx.conf, compose, css, + # htpasswd, whatever the proxy serves. Subdirectories (logs, certs) stay put. + $files = @(Get-ChildItem -LiteralPath $root -File | Where-Object { $_.Name -ne 'nul' }) + foreach ($f in $files) { + Write-Host " >> uploading $($f.Name)" -ForegroundColor DarkCyan + scp -i $PEM_KEY $f.FullName "${SSH_TARGET}:$remotePath/" + if ($LASTEXITCODE -ne 0) { throw "SCP failed for $($f.Name) (exit $LASTEXITCODE)" } + } + + $certMount = if ($Proj.certsSource) { "-v $($Proj.certsSource):/etc/letsencrypt/:ro " } else { "" } + Invoke-Ec2Step "validate new nginx.conf" "sudo docker run --rm -v $remotePath/nginx.conf:/etc/nginx/nginx.conf:ro ${certMount}nginx:1.27-alpine nginx -t -c /etc/nginx/nginx.conf" + + # A container from an older compose project may still hold the proxy name; + # docker refuses a second create with the same name, so remove it first. + $rmStale = if ($pc) { "; sudo docker rm -f $pc 2>/dev/null || true" } else { "" } + Invoke-Ec2Step "edge: compose down + remove stale proxy" "cd $remotePath && sudo docker compose down 2>/dev/null || true$rmStale" + Invoke-Ec2Step "edge compose up -d" "cd $remotePath && sudo docker compose up -d" + if ($pc) { + Invoke-Ec2Step "edge nginx reload" "sudo docker exec $pc nginx -s reload || true" + } + Invoke-Ec2Step "fix nginx-logs permissions (if present)" "if [ -d $remotePath/nginx-logs ]; then sudo chmod 777 $remotePath/nginx-logs; sudo chmod 666 $remotePath/nginx-logs/*.log 2>/dev/null || true; fi" + Write-Host "--- [Done] Edge proxy deploy finished ---" -ForegroundColor Green +} + +function Invoke-DockerDeploy { + param([string]$Key, $Proj) + + $root = $Proj.localRoot + $remotePath = $Proj.remote.path + + Write-Host "`n=== $($Proj.label) deploy (docker compose) ===" -ForegroundColor Cyan + if (-not (Test-Path -LiteralPath $root)) { throw "Project root not found: $root" } + if (-not (Test-Path -LiteralPath (Join-Path $root "docker-compose.yml"))) { throw "Missing $root\docker-compose.yml" } + + Invoke-Ec2Step "ensure shared web network" "sudo docker network create web 2>/dev/null || true" + Invoke-Ec2Step "ensure project dir" "sudo mkdir -p $remotePath && sudo chown ${Ec2User}:${Ec2User} $remotePath" + + $files = @(Get-ChildItem -LiteralPath $root -File -Force | Where-Object { $_.Name -ne 'nul' }) + foreach ($f in $files) { + Write-Host " >> uploading $($f.Name)" -ForegroundColor DarkCyan + scp -i $PEM_KEY $f.FullName "${SSH_TARGET}:$remotePath/" + if ($LASTEXITCODE -ne 0) { throw "SCP failed for $($f.Name) (exit $LASTEXITCODE)" } + } + + Invoke-Ec2Step "docker compose pull" "cd $remotePath && sudo docker compose pull" + Invoke-Ec2Step "docker compose up -d" "cd $remotePath && sudo docker compose up -d" + + Invoke-Ec2PostDeployCleanup -Label $Key + Write-Host "`n--- [Done] $($Proj.label) deploy finished ---" -ForegroundColor Green + if ($Proj.domain) { Write-Host "Site: https://$($Proj.domain)" -ForegroundColor Yellow } +} + +# ── Dispatch ───────────────────────────────────────────────────────────────── + +foreach ($key in $Projects) { + $proj = Get-ZProject -Key $key + Invoke-DeployGitPull -Proj $proj # no-op unless deploy.gitPull is set + switch ([string]$proj.kind) { + "python" { Invoke-PythonDeploy -Key $key -Proj $proj -ChangeNote $Note } + "vite" { Invoke-ViteDeploy -Key $key -Proj $proj -ChangeNote $Note } + "nextjs" { Invoke-NextDeploy -Key $key -Proj $proj -ChangeNote $Note } + "edge" { Invoke-EdgeDeploy -Key $key -Proj $proj } + "docker" { Invoke-DockerDeploy -Key $key -Proj $proj } + default { throw "No deploy handler for kind '$($proj.kind)' (project '$key'). Add an Invoke-Deploy function in zdeploy.ps1." } + } +} +Stop-ZTracking diff --git a/zec2.cmd b/zec2.cmd new file mode 100644 index 0000000..5e068b7 --- /dev/null +++ b/zec2.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zec2.ps1" %* diff --git a/zec2.ps1 b/zec2.ps1 new file mode 100644 index 0000000..2db49f7 --- /dev/null +++ b/zec2.ps1 @@ -0,0 +1,118 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zec2.ps1 — quick reachability check (TCP + HTTP + live build version) for deployed projects. +# +# Usage: +# zec2 # every project with a "domain" in zconfig.json +# zec2 [ ...] +# zec2 viteapp -HostName 203.0.113.10 +# +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @(), + [string]$HostName = "" +) + +$ErrorActionPreference = "Continue" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$cfg = Get-ZConfig +if (-not $HostName) { $HostName = $cfg.ec2.ip } + +if ($Projects.Count -eq 0) { + $Projects = @(Get-ZProjectKeys | Where-Object { $cfg.projects.$_.domain }) + if ($Projects.Count -eq 0) { + Write-Host "No projects with a 'domain' configured in zconfig.json." -ForegroundColor Yellow + Stop-ZTracking; exit 1 + } +} + +function Test-Zec2Reachability { + param([string]$Label, [int]$Port, [string]$HostHeader) + Write-Host "" + Write-Host "=== zec2: $Label ===" -ForegroundColor Cyan + Write-Host " Target: ${HostName}:${Port}$(if ($HostHeader) { " (Host: $HostHeader)" })" -ForegroundColor Gray + Write-Host "" + + $tcpOk = $false + Write-Host " [1/2] TCP connect to port ${Port}..." -ForegroundColor Yellow + try { + $tn = Test-NetConnection -ComputerName $HostName -Port $Port -WarningAction SilentlyContinue + if ($tn.TcpTestSucceeded) { + Write-Host " OK - port ${Port} is open (TCP succeeded)" -ForegroundColor Green + $tcpOk = $true + } else { + Write-Host " FAIL - port ${Port} did not accept TCP (check firewall/security group + app on the server)" -ForegroundColor Red + } + } catch { + Write-Host " FAIL - $($_.Exception.Message)" -ForegroundColor Red + } + + if (-not $tcpOk) { return 1 } + + $uri = "http://${HostName}:${Port}/" + Write-Host " [2/2] HTTP GET $uri ..." -ForegroundColor Yellow + try { + $headers = @{} + if ($HostHeader) { $headers['Host'] = $HostHeader } + $resp = Invoke-WebRequest -Uri $uri -Headers $headers -UseBasicParsing -TimeoutSec 15 -MaximumRedirection 5 + $code = [int]$resp.StatusCode + Write-Host " OK - HTTP $code" -ForegroundColor Green + if ($resp.Headers["Server"]) { + Write-Host " Server: $($resp.Headers['Server'])" -ForegroundColor Gray + } + } catch { + $status = $_.Exception.Response.StatusCode.value__ + if ($status) { + Write-Host " HTTP response: $status (connection worked; app may redirect or require auth)" -ForegroundColor Yellow + } else { + Write-Host " FAIL - $($_.Exception.Message)" -ForegroundColor Red + return 1 + } + } + + Write-Host "" + Write-Host " Done ($Label)." -ForegroundColor Cyan + return 0 +} + +# Live build version by kind: python/nextjs expose /api/build-version, +# vite/static serves /build-version.json. +function Show-Zec2LiveVersion { + param($Proj) + if (-not $Proj.domain) { return } + $headers = @{ Host = $Proj.domain } + try { + if ($Proj.kind -eq 'vite') { + $r = Invoke-RestMethod -Uri "http://${HostName}/build-version.json" -Headers $headers -TimeoutSec 10 -ErrorAction Stop + if ($r) { Write-Host " Live build: $(Get-LabelFromBuildJsonObj $r)" -ForegroundColor Gray } + } else { + $r = Invoke-RestMethod -Uri "http://${HostName}/api/build-version" -Headers $headers -TimeoutSec 10 -ErrorAction Stop + if ($r -and $r.build_version) { Write-Host " Live build: $($r.build_version)" -ForegroundColor Gray } + } + } catch { + Write-Host " (Could not read live version endpoint - see 'Enabling deploy verification' in README)" -ForegroundColor DarkGray + } +} + +$exitCode = 0 +foreach ($key in $Projects) { + $proj = Get-ZProject -Key $key + if (-not $proj.domain) { + Write-Host "" + Write-Host "Skipping '$key' - no domain configured." -ForegroundColor DarkYellow + continue + } + $rc = Test-Zec2Reachability -Label "$($proj.label)" -Port 80 -HostHeader $proj.domain + if ($rc -eq 0) { Show-Zec2LiveVersion -Proj $proj } + if ($rc -ne 0) { $exitCode = $rc } +} + +Write-Host "" +Write-Host "If TCP fails: open inbound TCP in the server's firewall/security group." -ForegroundColor Cyan +Write-Host "If TCP OK but HTTP fails: SSH in and check docker/nginx (curl -sI http://127.0.0.1/)." -ForegroundColor Cyan +Write-Host "" +Stop-ZTracking; exit $exitCode diff --git a/zec2online.cmd b/zec2online.cmd new file mode 100644 index 0000000..f0afb45 --- /dev/null +++ b/zec2online.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zec2online.ps1" %* diff --git a/zec2online.ps1 b/zec2online.ps1 new file mode 100644 index 0000000..71e2d4f --- /dev/null +++ b/zec2online.ps1 @@ -0,0 +1,261 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zec2online.ps1 — deep health check: verify apps are live AND running the expected +# build; auto-start downed stacks via docker compose and stream diagnostics. +# +# Usage: +# zec2online # every project with a "domain" in zconfig.json +# zec2online [ ...] +# +# Verification compares the LOCAL build version against what the server is actually +# serving. A plain HTTP-200 check passes even when a stale cached build is live — +# the version match is what proves the deployed build is the one running. +# +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @(), + [string]$HostName = "" +) + +$ErrorActionPreference = 'SilentlyContinue' +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$cfg = Get-ZConfig +if (-not $HostName) { $HostName = $cfg.ec2.ip } +$PemKey = $cfg.ec2.pemKey +$SshTarget = Get-Ec2Target +$edgeProj = Get-ZEdgeProject + +if ($Projects.Count -eq 0) { + $Projects = @(Get-ZProjectKeys | Where-Object { $cfg.projects.$_.domain }) + if ($Projects.Count -eq 0) { + Write-Host "No projects with a 'domain' configured in zconfig.json." -ForegroundColor Yellow + Stop-ZTracking; exit 1 + } +} + +# ── Version helpers ────────────────────────────────────────────────────────── + +function Get-LocalVersionLabel { + param($Proj) + switch ([string]$Proj.kind) { + "python" { + $tool = Join-Path $Proj.localRoot "scripts\build_version_tool.py" + if (Test-Path $tool) { + try { + $out = python $tool get 2>$null + if ($LASTEXITCODE -eq 0 -and $out) { + $val = ($out | Select-Object -Last 1).ToString().Trim() + if ($val) { return $val } + } + } catch { } + } + $bvFile = Join-Path $Proj.localRoot ".build_version" + if (Test-Path $bvFile) { + $val = (Get-Content $bvFile -ErrorAction SilentlyContinue | Select-Object -Last 1) + if ($val) { return $val.Trim() } + } + return "unknown" + } + "vite" { + $j = Read-JsonBuildVersion (Join-Path $Proj.localRoot "build-version.json") + if ($j) { return Get-LabelFromBuildJsonObj $j } + return "unknown" + } + "nextjs" { + $j = Read-JsonBuildVersion (Join-Path $Proj.localRoot "public\build-version.json") + if ($j) { return Get-LabelFromBuildJsonObj $j } + return "unknown" + } + default { return "unknown" } + } +} + +function Get-RemoteVersionLabel { + param($Proj) + $headers = @{} + if ($Proj.domain) { $headers['Host'] = $Proj.domain } + try { + if ($Proj.kind -eq 'vite') { + $r = Invoke-RestMethod -Uri "http://${HostName}/build-version.json" -Headers $headers -TimeoutSec 10 -ErrorAction Stop + if ($r) { return Get-LabelFromBuildJsonObj $r } + } else { + $r = Invoke-RestMethod -Uri "http://${HostName}/api/build-version" -Headers $headers -TimeoutSec 10 -ErrorAction Stop + if ($r -and $r.build_version) { return [string]$r.build_version } + } + } catch { } + # python fallback: read the recorded version file over SSH + if ($Proj.kind -eq 'python' -and (Test-Path $PemKey) -and $Proj.remote.path) { + try { + $out = ssh -o StrictHostKeyChecking=no -o ConnectTimeout=15 -i $PemKey $SshTarget "cat $($Proj.remote.path)/.build_version 2>/dev/null || true" + if ($LASTEXITCODE -eq 0 -and $out) { + $val = ($out | Select-Object -Last 1).ToString().Trim() + if ($val) { return $val } + } + } catch { } + } + return "unknown" +} + +function Compare-BuildVersions { + param([string]$Key, [string]$Local, [string]$Server) + if ($Local -eq "unknown" -and $Server -eq "unknown") { + Write-Host " Version: unable to determine local or server version (see 'Enabling deploy verification' in README)" -ForegroundColor DarkGray + } elseif ($Local -eq "unknown") { + Write-Host " Server: $Server" -ForegroundColor Magenta + Write-Host " Local: unknown (could not read local build version)" -ForegroundColor DarkGray + } elseif ($Server -eq "unknown") { + Write-Host " Local: $Local" -ForegroundColor Magenta + Write-Host " Server: unknown (could not read server build version)" -ForegroundColor DarkGray + } elseif ($Local -eq $Server) { + Write-Host " Version: $Server (local and server match)" -ForegroundColor Green + } else { + Write-Host " Local: $Local" -ForegroundColor Magenta + Write-Host " Server: $Server" -ForegroundColor Magenta + Write-Host " WARNING: local and server versions differ - redeploy with: zdeploy $Key" -ForegroundColor Yellow + } +} + +# ── HTTP / recovery helpers ────────────────────────────────────────────────── + +function Test-HttpOnline { + param([int]$Port, [string]$HostHeader) + $tn = Test-NetConnection -ComputerName $HostName -Port $Port -WarningAction SilentlyContinue -InformationLevel Quiet 2>$null + if (-not $tn) { return $false } + try { + $headers = @{} + if ($HostHeader) { $headers['Host'] = $HostHeader } + $resp = Invoke-WebRequest -Uri "http://${HostName}:${Port}/" -Headers $headers -UseBasicParsing -TimeoutSec 8 -MaximumRedirection 5 -ErrorAction Stop + if ($resp.StatusCode -lt 500) { + Write-Host " HTTP $($resp.StatusCode) - service reachable." -ForegroundColor DarkGreen + return $true + } + Write-Host " HTTP $($resp.StatusCode) - unexpected server response." -ForegroundColor DarkYellow + return $false + } catch { + $status = $_.Exception.Response.StatusCode.value__ + if ($status -ge 300 -and $status -lt 500) { + Write-Host " HTTP $status - redirect from app, server is live." -ForegroundColor DarkGreen + return $true + } + return $false + } +} + +function Invoke-DockerStart { + param([string]$StartCmd) + if (-not (Test-Path $PemKey)) { + Write-Host " Cannot start: PEM key not found at: $PemKey" -ForegroundColor Red + return $false + } + ssh -o StrictHostKeyChecking=no -o ConnectTimeout=15 -i $PemKey $SshTarget $StartCmd + if ($LASTEXITCODE -ne 0) { + Write-Host " SSH failed (exit $LASTEXITCODE). Check connectivity and PEM key." -ForegroundColor Red + return $false + } + return $true +} + +function Show-Diagnostics { + param([string]$Key, $Proj, [string]$Reason) + if (-not (Test-Path $PemKey)) { + Write-Host " Diagnostics skipped: PEM key not found at $PemKey" -ForegroundColor DarkGray + return + } + $composeDir = Get-RemoteComposeDir -Key $Key + Write-Host '' + Write-Host " --- $Key diagnostics ($Reason) ---" -ForegroundColor Magenta + $lines = @( + "echo '== services =='; cd $composeDir 2>/dev/null && sudo docker compose ps", + "echo '== uptime / df =='; uptime; df -h /", + "echo '== oom =='; dmesg -T 2>/dev/null | grep -iE 'oom|killed' | tail -n 10", + "echo '== app logs (80) =='; cd $composeDir 2>/dev/null && sudo docker compose logs --tail 80" + ) + if ($edgeProj -and $edgeProj.Config.proxyContainer) { + $pc = $edgeProj.Config.proxyContainer + $lines += "echo '== edge proxy state =='; sudo docker ps --filter name=$pc --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'" + $lines += "echo '== edge proxy logs (40) =='; sudo docker logs --tail 40 $pc 2>&1 | tail -n 40" + } + ssh -o StrictHostKeyChecking=no -o ConnectTimeout=15 -i $PemKey $SshTarget ($lines -join '; ') + if ($LASTEXITCODE -ne 0) { + Write-Host " Diagnostics SSH failed (exit $LASTEXITCODE)." -ForegroundColor DarkYellow + } + Write-Host " --- end $Key diagnostics ---" -ForegroundColor Magenta + Write-Host '' +} + +function Wait-UntilOnline { + param([scriptblock]$TestFn, [int]$Seconds = 30) + $deadline = (Get-Date).AddSeconds($Seconds) + while ((Get-Date) -lt $deadline) { + Start-Sleep -Seconds 3 + Write-Host ' checking...' -ForegroundColor DarkGray + if (& $TestFn) { return $true } + } + return $false +} + +function Invoke-OnlineCheck { + param([string]$Key, $Proj) + + Write-Host '' + Write-Host "=== zec2online: $($Proj.label) ($Key) ===" -ForegroundColor Cyan + Write-Host " Target: http://${HostName}/ (Host: $($Proj.domain))" -ForegroundColor Gray + Write-Host '' + + Write-Host ' [1] Checking TCP + HTTP...' -ForegroundColor Yellow + if (Test-HttpOnline -Port 80 -HostHeader $Proj.domain) { + Compare-BuildVersions -Key $Key -Local (Get-LocalVersionLabel $Proj) -Server (Get-RemoteVersionLabel $Proj) + Write-Host " UP - $($Proj.label) is running." -ForegroundColor Green + Write-Host '' + return 0 + } + + Write-Host ' DOWN - Service not responding.' -ForegroundColor Red + Show-Diagnostics -Key $Key -Proj $Proj -Reason 'DOWN before recovery' + + Write-Host ' [2] Starting stack (and edge proxy if defined)...' -ForegroundColor Yellow + $composeDir = Get-RemoteComposeDir -Key $Key + $startLines = @( + "sudo docker network create web 2>/dev/null || true", + "if [ -f $composeDir/docker-compose.yml ]; then cd $composeDir && sudo docker compose up -d 2>&1 | tail -10; else echo 'compose missing at $composeDir'; exit 2; fi" + ) + if ($edgeProj -and $edgeProj.Config.remote.path) { + $edgeDir = $edgeProj.Config.remote.path + $startLines += "if [ -f $edgeDir/docker-compose.yml ]; then cd $edgeDir && sudo docker compose up -d 2>&1 | tail -10; fi" + } + if (-not (Invoke-DockerStart -StartCmd ($startLines -join '; '))) { return 1 } + + Write-Host '' + Write-Host ' [3] Waiting for service (up to 30s)...' -ForegroundColor Yellow + $domain = $Proj.domain + $online = Wait-UntilOnline -TestFn { Test-HttpOnline -Port 80 -HostHeader $domain }.GetNewClosure() + + Write-Host '' + if ($online) { + Compare-BuildVersions -Key $Key -Local (Get-LocalVersionLabel $Proj) -Server (Get-RemoteVersionLabel $Proj) + Write-Host " UP - $($Proj.label) is now running." -ForegroundColor Green + return 0 + } + + Write-Host ' TIMEOUT - Service did not respond within 30 seconds.' -ForegroundColor Red + Show-Diagnostics -Key $Key -Proj $Proj -Reason 'recovery TIMEOUT' + return 1 +} + +$exitCode = 0 +foreach ($key in $Projects) { + $proj = Get-ZProject -Key $key + if (-not $proj.domain) { + Write-Host "" + Write-Host "Skipping '$key' - no domain configured." -ForegroundColor DarkYellow + continue + } + $rc = Invoke-OnlineCheck -Key $key -Proj $proj + if ($rc -ne 0) { $exitCode = $rc } +} + +Stop-ZTracking; exit $exitCode diff --git a/zkill.cmd b/zkill.cmd new file mode 100644 index 0000000..fd3fb33 --- /dev/null +++ b/zkill.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0ZKillOnly.ps1" %* diff --git a/zkill.ps1 b/zkill.ps1 new file mode 100644 index 0000000..67aded4 --- /dev/null +++ b/zkill.ps1 @@ -0,0 +1,7 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zkill.ps1 — alias for ZKillOnly.ps1 (kept so both names work). All args pass through. +& (Join-Path $PSScriptRoot "ZKillOnly.ps1") @args +exit $LASTEXITCODE diff --git a/zrepair.cmd b/zrepair.cmd new file mode 100644 index 0000000..534c11e --- /dev/null +++ b/zrepair.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zrepair.ps1" %* diff --git a/zrepair.ps1 b/zrepair.ps1 new file mode 100644 index 0000000..23e2a15 --- /dev/null +++ b/zrepair.ps1 @@ -0,0 +1,103 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zrepair.ps1 — audit and repair container/proxy routing on the server, then smoke test. +# +# Usage: +# zrepair [ ...] +# +# For each project: shows compose status, starts the stack if it's down, +# validates the edge proxy's nginx config (once, if an edge project is defined), +# and smoke-tests the live domain. +# +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @() +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$cfg = Get-ZConfig +$EC2_IP = $cfg.ec2.ip + +if ($Projects.Count -eq 0) { + $keys = (Get-ZProjectKeys) -join ', ' + Write-Host "" + Write-Host "Usage: zrepair [ ...]" -ForegroundColor Yellow + Write-Host " Projects in zconfig.json: $keys" -ForegroundColor Gray + Stop-ZTracking; exit 1 +} + +Write-Host "" +Write-Host "=== Routing repair & diagnostics ===" -ForegroundColor Cyan +Write-Host "Target host: $EC2_IP" -ForegroundColor Gray + +# Validate the edge proxy config once up front, if one is defined. +$edgeProj = Get-ZEdgeProject +if ($edgeProj -and $edgeProj.Config.proxyContainer) { + $pc = $edgeProj.Config.proxyContainer + Write-Host "" + Write-Host " [edge] Validating nginx config in proxy container '$pc'..." -ForegroundColor Yellow + Invoke-Ec2Step -Label "nginx -t in $pc" -Bash "sudo docker exec $pc nginx -t 2>&1" +} + +foreach ($key in $Projects) { + $proj = Get-ZProject -Key $key + $composeDir = Get-RemoteComposeDir -Key $key + + Write-Host "" + Write-Host "=========================================" -ForegroundColor Cyan + Write-Host "=== Repairing $($proj.label) ($key) ===" -ForegroundColor Cyan + Write-Host "=========================================" -ForegroundColor Cyan + Write-Host "" + + Write-Host " [1/3] Compose status on the server..." -ForegroundColor Yellow + $statusBash = @" +cd $composeDir || exit 1 +echo "Running containers:" +sudo docker compose ps +"@ + Invoke-Ec2Step -Label "Check compose status" -Bash $statusBash + + Write-Host " [2/3] Ensuring stack is up..." -ForegroundColor Yellow + $ensureBash = @" +cd $composeDir || exit 1 +if ! sudo docker compose ps | grep -q "Up"; then + echo " [Action] Stack is down. Starting..." + sudo docker compose up -d +else + echo " [OK] Stack is active and UP." +fi +"@ + Invoke-Ec2Step -Label "Ensure stack is up" -Bash $ensureBash + + if ($proj.domain) { + Write-Host " [3/3] Smoke test for https://$($proj.domain)..." -ForegroundColor Yellow + try { + $resp = Invoke-WebRequest -Uri "http://$EC2_IP/" -Headers @{ Host = $proj.domain } -UseBasicParsing -TimeoutSec 15 -MaximumRedirection 5 + $status = [int]$resp.StatusCode + if ($status -ge 200 -and $status -lt 400) { + Write-Host " PASS - responded with HTTP $status (Online)" -ForegroundColor Green + } else { + Write-Host " WARNING - unexpected HTTP $status" -ForegroundColor Yellow + } + } catch { + $status = $_.Exception.Response.StatusCode.value__ + if ($status -ge 200 -and $status -lt 500) { + Write-Host " PASS - responded with HTTP $status (Online/Redirect)" -ForegroundColor Green + } else { + Write-Host " FAIL - smoke test failed: $($_.Exception.Message)" -ForegroundColor Red + } + } + } else { + Write-Host " [3/3] No domain configured - skipping smoke test." -ForegroundColor DarkYellow + } +} + +Write-Host "" +Write-Host "=== Repair & diagnostics completed ===" -ForegroundColor Green +Write-Host "" +Stop-ZTracking diff --git a/zrestart.cmd b/zrestart.cmd new file mode 100644 index 0000000..92f7cd3 --- /dev/null +++ b/zrestart.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0ZKiller.ps1" %* diff --git a/zrestart.ps1 b/zrestart.ps1 new file mode 100644 index 0000000..1554cf2 --- /dev/null +++ b/zrestart.ps1 @@ -0,0 +1,7 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zrestart.ps1 — alias for ZKiller.ps1 (kept so both names work). All args pass through. +& (Join-Path $PSScriptRoot "ZKiller.ps1") @args +exit $LASTEXITCODE diff --git a/zrestartd.cmd b/zrestartd.cmd new file mode 100644 index 0000000..fc4f5ab --- /dev/null +++ b/zrestartd.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0ZKiller.ps1" -Detached %* diff --git a/zsetup_mail.ps1 b/zsetup_mail.ps1 new file mode 100644 index 0000000..35c1579 --- /dev/null +++ b/zsetup_mail.ps1 @@ -0,0 +1,117 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zsetup_mail.ps1 — create admin@ and noreply@ mailboxes in a docker-mailserver +# container on the server, and print the DNS records + SMTP/IMAP settings to use. +# +# Usage: +# zsetup_mail.ps1 -domain yourdomain.com [-mailHost mail.yourdomain.com] [-ec2Host ] [-pemKey ] +# +param ( + [string]$domain = "", + [string]$mailHost = "", + [string]$ec2Host = "", + [string]$pemKey = "" +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$cfg = Get-ZConfig +if (-not $ec2Host) { $ec2Host = $cfg.ec2.ip } +if (-not $pemKey) { $pemKey = $cfg.ec2.pemKey } + +if ([string]::IsNullOrWhiteSpace($domain)) { + Write-Host "" + Write-Host "Usage: zsetup_mail.ps1 -domain yourdomain.com [-mailHost mail.yourdomain.com]" -ForegroundColor Yellow + Write-Host " Creates admin@ and noreply@ mailboxes on the server's mailserver container." -ForegroundColor Gray + Stop-ZTracking; exit 1 +} +if (-not $mailHost) { $mailHost = "mail.$domain" } + +Write-Host "=== Mail Setup Utility ===" -ForegroundColor Cyan + +if (-not (Test-Path $pemKey)) { + Write-Error "PEM key not found: $pemKey" + exit 1 +} + +Write-Host "Using PEM Key: $pemKey" -ForegroundColor Gray +Write-Host "Target Host : $ec2Host" -ForegroundColor Gray +Write-Host "Domain Name : $domain" -ForegroundColor Gray + +function Generate-Password { + $chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#%^&*" + $random = New-Object System.Random + $password = "" + for ($i = 0; $i -lt 16; $i++) { + $password += $chars[$random.Next(0, $chars.Length)] + } + return $password +} + +$adminPassword = Generate-Password +$noreplyPassword = Generate-Password + +Write-Host "`n[1/3] Connecting to the mail server to provision mailboxes..." -ForegroundColor Yellow + +$SshOpts = @("-o", "ConnectTimeout=15", "-o", "StrictHostKeyChecking=no", "-i", $pemKey, "$($cfg.ec2.user)@$ec2Host") + +function Exec-Remote ([string]$cmd) { & ssh @SshOpts $cmd } + +Write-Host "Adding email account: admin@$domain..." -ForegroundColor Gray +Exec-Remote "sudo docker exec mailserver setup email add admin@$domain '$adminPassword'" +if ($LASTEXITCODE -eq 0) { + Write-Host "Account admin@$domain added successfully." -ForegroundColor Green +} else { + Write-Warning "Could not add admin account (it might already exist or docker setup failed)." +} + +Write-Host "Adding email account: noreply@$domain..." -ForegroundColor Gray +Exec-Remote "sudo docker exec mailserver setup email add noreply@$domain '$noreplyPassword'" +if ($LASTEXITCODE -eq 0) { + Write-Host "Account noreply@$domain added successfully." -ForegroundColor Green +} else { + Write-Warning "Could not add noreply account (it might already exist or docker setup failed)." +} + +Write-Host "`n[2/3] DNS Configuration Requirements" -ForegroundColor Yellow +Write-Host "Add or update the following records in your DNS zone for ${domain}:" -ForegroundColor Gray +Write-Host "--------------------------------------------------------------------------------" -ForegroundColor Cyan +Write-Host "1. MX Record (Inbound mail routing):" -ForegroundColor White +Write-Host " - Name : (leave blank or @)" -ForegroundColor Green +Write-Host " - Type : MX" -ForegroundColor Green +Write-Host " - Value : 10 $mailHost" -ForegroundColor Yellow +Write-Host "2. TXT Record (SPF authorization):" -ForegroundColor White +Write-Host " - Name : (leave blank or @)" -ForegroundColor Green +Write-Host " - Type : TXT" -ForegroundColor Green +Write-Host " - Value : `"v=spf1 mx ~all`"" -ForegroundColor Yellow +Write-Host "3. A Record for Webmail:" -ForegroundColor White +Write-Host " - Name : webmail" -ForegroundColor Green +Write-Host " - Type : A" -ForegroundColor Green +Write-Host " - Value : $ec2Host" -ForegroundColor Yellow +Write-Host "4. A Record for Mail Admin:" -ForegroundColor White +Write-Host " - Name : mail-admin" -ForegroundColor Green +Write-Host " - Type : A" -ForegroundColor Green +Write-Host " - Value : $ec2Host" -ForegroundColor Yellow +Write-Host "--------------------------------------------------------------------------------" -ForegroundColor Cyan + +Write-Host "`n[3/3] Application Connection Credentials" -ForegroundColor Yellow +Write-Host "Use these connection settings in your app config or .env:" -ForegroundColor Gray +Write-Host "--------------------------------------------------------------------------------" -ForegroundColor Cyan +Write-Host "SMTP Hostname : $mailHost" -ForegroundColor White +Write-Host "SMTP Port (STARTTLS) : 587" -ForegroundColor White +Write-Host "SMTP Port (SSL/TLS) : 465" -ForegroundColor White +Write-Host "IMAP Hostname : $mailHost" -ForegroundColor White +Write-Host "IMAP Port (SSL/TLS) : 993" -ForegroundColor White +Write-Host "--------------------------------------------------------------------------------" -ForegroundColor Cyan +Write-Host "Email User 1 : admin@$domain" -ForegroundColor Green +Write-Host "Password : $adminPassword" -ForegroundColor Yellow +Write-Host "" +Write-Host "Email User 2 : noreply@$domain" -ForegroundColor Green +Write-Host "Password : $noreplyPassword" -ForegroundColor Yellow +Write-Host "--------------------------------------------------------------------------------" -ForegroundColor Cyan +Write-Host "Completed successfully!" -ForegroundColor Green +Stop-ZTracking diff --git a/zstart.cmd b/zstart.cmd new file mode 100644 index 0000000..c5ef5e9 --- /dev/null +++ b/zstart.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0ZStart.ps1" %* diff --git a/zstart.ps1 b/zstart.ps1 new file mode 100644 index 0000000..5a8ff7c --- /dev/null +++ b/zstart.ps1 @@ -0,0 +1,214 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zstart.ps1 — start local dev servers for any project defined in zconfig.json. +# +# Usage: +# zstart [ ...] [-Port N] [-BindHost host] [-Detached] +# +# Examples: +# zstart pyapp +# zstart viteapp -Port 3000 +# zstart pyapp nextapp -Detached +# +# Handlers by kind: python (python -m , prefers .venv), +# vite (npm run dev -- --host --port), nextjs (npm run dev with PORT). +# +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @(), + [Alias("p")][int]$Port = 0, + [string]$BindHost = "127.0.0.1", + [switch]$Detached +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +if ($Projects.Count -eq 0) { + $keys = (Get-ZProjectKeys) -join ', ' + Write-Host "" + Write-Host "Usage: zstart [ ...] [-Port N] [-Detached] [-BindHost host]" -ForegroundColor Yellow + Write-Host " Projects in zconfig.json: $keys" -ForegroundColor Gray + Stop-ZTracking; exit 1 +} + +function Test-PortNeedsAdmin { + param([int]$ListenPort) + if ($ListenPort -ge 1024) { return } + $isAdmin = ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator) + if (-not $isAdmin) { + Write-Host "WARNING: Port $ListenPort requires Administrator privileges to bind. Run this shell as Administrator!" -ForegroundColor Red + } +} + +function Start-PythonProject { + param([string]$Key, $Proj, [int]$ListenPort, [bool]$RunDetached) + $root = $Proj.localRoot + if (-not (Test-Path -LiteralPath $root)) { throw "Project root not found: $root" } + $module = $Proj.startModule + if (-not $module) { + throw "Project '$Key' (kind=python) needs 'startModule' in zconfig.json (e.g. `"startModule`": `"pyapp.main`" runs 'python -m pyapp.main')." + } + Set-Location -LiteralPath $root + $venvPython = Join-Path $root ".venv\Scripts\python.exe" + $exe = if (Test-Path -LiteralPath $venvPython) { $venvPython } else { "python" } + + # Optional convention: if the project ships scripts/build_version_tool.py, + # bump (or at least read) the build version on every dev start. + $buildVersion = "" + $versionTool = Join-Path $root "scripts\build_version_tool.py" + if (Test-Path -LiteralPath $versionTool) { + try { + $bumpOut = & $exe $versionTool bump 2>$null + if ($LASTEXITCODE -eq 0 -and $bumpOut) { + $buildVersion = ($bumpOut | Select-Object -Last 1).ToString().Trim() + } + } catch { } + if ([string]::IsNullOrWhiteSpace($buildVersion)) { + try { + $getOut = & $exe $versionTool get 2>$null + if ($LASTEXITCODE -eq 0 -and $getOut) { + $buildVersion = ($getOut | Select-Object -Last 1).ToString().Trim() + } + } catch { } + } + } + + Write-Host "" + Write-Host "=== zstart ($($Proj.label)) ===" -ForegroundColor Cyan + Write-Host "Starting python -m $module on port $ListenPort..." -ForegroundColor Cyan + if (-not [string]::IsNullOrWhiteSpace($buildVersion)) { + Write-Host "Build Version: $buildVersion" -ForegroundColor Magenta + } + + Show-ProjectMotd -Root $root -BuildVersion $buildVersion + + Write-Host "Press Ctrl+C to stop the server" -ForegroundColor DarkGray + Test-PortNeedsAdmin -ListenPort $ListenPort + Write-Host "" + + if ($RunDetached) { + Start-Process -FilePath $exe -ArgumentList "-m", $module -WorkingDirectory $root | Out-Null + Write-Host "Started in detached mode." -ForegroundColor Green + Write-Host "Use zkill $Key to stop it." -ForegroundColor DarkGray + } else { + & $exe -m $module + } +} + +function Start-ViteProject { + param([string]$Key, $Proj, [int]$ListenPort, [string]$HostBind, [bool]$RunDetached) + $root = $Proj.localRoot + if (-not (Test-Path -LiteralPath $root)) { throw "Project root not found: $root" } + if (-not (Test-Path -LiteralPath (Join-Path $root "package.json"))) { + throw "package.json not found in $root" + } + Set-Location -LiteralPath $root + + Write-Host "" + Write-Host "=== zstart ($($Proj.label) - Vite) ===" -ForegroundColor Cyan + Write-Host "Directory: $root" -ForegroundColor DarkGray + Write-Host "URL: http://${HostBind}:$ListenPort/" -ForegroundColor Cyan + Write-Host "" + + if (-not (Test-Path -LiteralPath (Join-Path $root "node_modules"))) { + Write-Host "node_modules missing - running npm install..." -ForegroundColor Yellow + & npm install + if ($LASTEXITCODE -ne 0) { throw "npm install failed with exit $LASTEXITCODE" } + } + + Show-ProjectMotd -Root $root + + Write-Host "Press Ctrl+C to stop the dev server" -ForegroundColor DarkGray + Test-PortNeedsAdmin -ListenPort $ListenPort + Write-Host "" + + $npmArgs = @("run", "dev", "--", "--host", $HostBind, "--port", "$ListenPort") + if ($RunDetached) { + Start-Process -FilePath "npm" -ArgumentList $npmArgs -WorkingDirectory $root -WindowStyle Normal | Out-Null + Write-Host "Started npm run dev in a new window (detached)." -ForegroundColor Green + Write-Host "Use zkill $Key to free port $ListenPort." -ForegroundColor DarkGray + } else { + & npm @npmArgs + } +} + +function Start-NextProject { + param([string]$Key, $Proj, [int]$ListenPort, [bool]$RunDetached) + $root = $Proj.localRoot + if (-not (Test-Path -LiteralPath $root)) { throw "Project root not found: $root" } + if (-not (Test-Path -LiteralPath (Join-Path $root "package.json"))) { + throw "package.json not found in $root" + } + Set-Location -LiteralPath $root + + Write-Host "" + Write-Host "=== zstart ($($Proj.label) - Next.js) ===" -ForegroundColor Cyan + Write-Host "Starting on port $ListenPort..." -ForegroundColor Cyan + Write-Host "" + + Show-ProjectMotd -Root $root + + Write-Host "Press Ctrl+C to stop the server" -ForegroundColor DarkGray + Test-PortNeedsAdmin -ListenPort $ListenPort + Write-Host "" + + $env:PORT = "$ListenPort" + if ($RunDetached) { + Start-Process -FilePath "npm" -ArgumentList "run", "dev" -WorkingDirectory $root | Out-Null + Write-Host "Started in detached mode on port $ListenPort." -ForegroundColor Green + Write-Host "Use zkill $Key to stop it." -ForegroundColor DarkGray + } else { + npm run dev + } +} + +# Optional per-project pre-start steps from zconfig.json: +# "start": { "gitPull": true, "env": { "SOME_FLAG": "1" } } +# gitPull runs 'git pull --ff-only' in the project root; env sets process +# environment variables before the server starts. +function Invoke-ProjectStartPrep { + param($Proj) + if (-not $Proj.start) { return } + if ($Proj.start.env) { + foreach ($item in $Proj.start.env.PSObject.Properties) { + Set-Item -Path "Env:$($item.Name)" -Value ([string]$item.Value) + Write-Host " env $($item.Name)=$($item.Value)" -ForegroundColor DarkGray + } + } + if ($Proj.start.gitPull -and (Test-Path (Join-Path $Proj.localRoot ".git"))) { + Push-Location -LiteralPath $Proj.localRoot + try { + $pullOut = git pull --ff-only 2>&1 + $last = ($pullOut | Select-Object -Last 1) + Write-Host " git pull: $last" -ForegroundColor DarkGray + if ($LASTEXITCODE -ne 0) { + Write-Host " Auto-pull failed - run 'git pull' manually if needed." -ForegroundColor Yellow + } + } finally { + Pop-Location + } + } +} + +foreach ($key in $Projects) { + $proj = Get-ZProject -Key $key + $devPort = if ($Port -gt 0) { $Port } elseif ($proj.ports -and $proj.ports.dev) { [int]$proj.ports.dev } else { 3000 } + + Invoke-ProjectStartPrep -Proj $proj + + switch ([string]$proj.kind) { + "python" { Start-PythonProject -Key $key -Proj $proj -ListenPort $devPort -RunDetached $Detached.IsPresent } + "vite" { Start-ViteProject -Key $key -Proj $proj -ListenPort $devPort -HostBind $BindHost -RunDetached $Detached.IsPresent } + "nextjs" { Start-NextProject -Key $key -Proj $proj -ListenPort $devPort -RunDetached $Detached.IsPresent } + default { + Write-Host "" + Write-Host "Project '$key' has kind '$($proj.kind)' - no local dev server to start." -ForegroundColor Yellow + Write-Host "Supported kinds for zstart: python, vite, nextjs" -ForegroundColor Gray + } + } +} +Stop-ZTracking diff --git a/zstart_docker.cmd b/zstart_docker.cmd new file mode 100644 index 0000000..b97487f --- /dev/null +++ b/zstart_docker.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zstart_docker.ps1" %* diff --git a/zstart_docker.ps1 b/zstart_docker.ps1 new file mode 100644 index 0000000..7834ec9 --- /dev/null +++ b/zstart_docker.ps1 @@ -0,0 +1,78 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zstart_docker.ps1 — bring up a local docker compose stack from \docker\. +# +# Usage: +# zstart_docker [-Build] [-Attached] [-Solo] +# +# -Build rebuild images before starting +# -Attached stream logs in the foreground (default is detached) +# -Solo use docker-compose.solo.yml (app only, no proxy) +# +param( + [switch]$Build, + [switch]$Attached, + [switch]$Solo +) + +$ErrorActionPreference = "Stop" +Set-Location -Path $PSScriptRoot +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +Write-Host "=== zstart_docker ===" -ForegroundColor Cyan + +$composeFileName = if ($Solo) { "docker-compose.solo.yml" } else { "docker-compose.yml" } + +$dockerPing = Start-Process -FilePath "docker" -ArgumentList "info" -Wait -NoNewWindow -PassThru +if ($dockerPing.ExitCode -ne 0) { + Write-Host "" + Write-Host "ERROR: Docker Engine is not running or this shell cannot reach it." -ForegroundColor Red + Write-Host " (npipe dockerDesktopLinuxEngine missing usually means Docker Desktop is stopped or still starting.)" -ForegroundColor DarkYellow + Write-Host "" + Write-Host "Try:" -ForegroundColor Cyan + Write-Host " 1. Start Docker Desktop from the Start menu; wait until it shows Running." -ForegroundColor Gray + Write-Host " 2. If it hangs: open PowerShell as Administrator, run: wsl --shutdown" -ForegroundColor Gray + Write-Host " then start Docker Desktop again." -ForegroundColor Gray + Write-Host "" + Stop-ZTracking; exit 1 +} + +$composeFile = Join-Path $PSScriptRoot "docker\$composeFileName" +if (-not (Test-Path $composeFile)) { + Write-Host "ERROR: Missing $composeFile" -ForegroundColor Red + Stop-ZTracking; exit 1 +} + +$envFile = Join-Path $PSScriptRoot ".env" +if (-not (Test-Path $envFile)) { + Write-Host "WARNING: .env not found at $envFile - compose may still start if env vars are set elsewhere." -ForegroundColor DarkYellow +} + +Push-Location -Path (Join-Path $PSScriptRoot "docker") +try { + $dcArgs = @("compose", "-f", $composeFileName, "up") + if (-not $Attached) { + $dcArgs += "-d" + } + if ($Build) { + $dcArgs += "--build" + } + Write-Host "Running: docker $($dcArgs -join ' ')" -ForegroundColor DarkGray + & docker @dcArgs + if ($LASTEXITCODE -ne 0) { + Stop-ZTracking; exit $LASTEXITCODE + } +} finally { + Pop-Location +} + +Write-Host "" +Write-Host "Stack: http://localhost/" -ForegroundColor Green +if (-not $Attached) { + Write-Host 'Logs: docker compose -f docker/docker-compose.yml logs -f' -ForegroundColor DarkGray + Write-Host 'Stop: docker compose -f docker/docker-compose.yml down' -ForegroundColor DarkGray +} +Stop-ZTracking diff --git a/zstartd.cmd b/zstartd.cmd new file mode 100644 index 0000000..a4c9b65 --- /dev/null +++ b/zstartd.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0ZStart.ps1" -Detached %* diff --git a/zstop.ps1 b/zstop.ps1 new file mode 100644 index 0000000..891ca44 --- /dev/null +++ b/zstop.ps1 @@ -0,0 +1,50 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zstop.ps1 — stop docker compose stacks on the server without removing data or files. +# +# Usage: +# zstop [ ...] +# +# Examples: +# zstop pyapp +# zstop viteapp nextapp edge +# +# Data volumes are preserved. Run zdeploy to bring a stack back up. +# +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @() +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$cfg = Get-ZConfig + +if ($Projects.Count -eq 0) { + $keys = (Get-ZProjectKeys) -join ', ' + Write-Host "" + Write-Host "Usage: zstop [ ...]" -ForegroundColor Yellow + Write-Host " Projects in zconfig.json: $keys" -ForegroundColor Gray + Stop-ZTracking; exit 1 +} + +foreach ($key in $Projects) { + $proj = Get-ZProject -Key $key + $composeDir = Get-RemoteComposeDir -Key $key + + Write-Host "`n--- Stopping $($proj.label) ---" -ForegroundColor Cyan + $cmd = "cd $composeDir && sudo docker compose down 2>&1 || true" + ssh -o StrictHostKeyChecking=no -i $cfg.ec2.pemKey (Get-Ec2Target) $cmd + if ($LASTEXITCODE -ne 0) { + Write-Host " WARNING: docker compose down returned exit $LASTEXITCODE for $($proj.label)" -ForegroundColor Yellow + } else { + Write-Host " $($proj.label) stopped." -ForegroundColor Green + } +} + +Write-Host "`nDone. Data volumes are intact. Run zdeploy to restart." -ForegroundColor DarkGray +Stop-ZTracking diff --git a/zsync.cmd b/zsync.cmd new file mode 100644 index 0000000..c13b346 --- /dev/null +++ b/zsync.cmd @@ -0,0 +1,6 @@ +REM Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. + +@echo off +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zsync.ps1" %* diff --git a/zsync.ps1 b/zsync.ps1 new file mode 100644 index 0000000..057e6b0 --- /dev/null +++ b/zsync.ps1 @@ -0,0 +1,129 @@ +# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. + +# zsync.ps1 — copy new backup files offsite; or build + mirror a vite project's dist. +# +# Usage: +# zsync # new files: backups folder -> paths.oneDriveBackups +# zsync -Destination 'D:\mirror\dist' +# +# The no-args mode copies only files that don't exist at the destination yet +# (never overwrites, never deletes). The project mode runs npm run build, then +# robocopy /MIR of dist/ to -Destination (or $env:ZSYNC_DEST). +# +param( + [Parameter(Position = 0, ValueFromRemainingArguments = $true)] + [string[]]$Projects = @(), + [string]$Destination = $env:ZSYNC_DEST +) + +$ErrorActionPreference = "Stop" +. (Join-Path $PSScriptRoot "ZHelpers.ps1") +Start-ZTracking + +$cfg = Get-ZConfig +$BackupsRoot = Split-Path -Parent $cfg.paths.backupsLocal +$OneDriveBackupsRoot = $cfg.paths.oneDriveBackups + +# Copy any file under $SourceRoot that does not yet exist at the matching path under $DestinationRoot. +function Sync-NewFilesTree { + param( + [Parameter(Mandatory)] [string]$SourceRoot, + [Parameter(Mandatory)] [string]$DestinationRoot + ) + if (-not (Test-Path -LiteralPath $SourceRoot)) { + Write-Host " Source not found (skipped): $SourceRoot" -ForegroundColor DarkYellow + return @{ copied = 0; skipped = 0 } + } + if (-not (Test-Path -LiteralPath $DestinationRoot)) { + New-Item -ItemType Directory -Path $DestinationRoot -Force | Out-Null + } + + $sourceFull = (Get-Item -LiteralPath $SourceRoot).FullName.TrimEnd('\') + $files = Get-ChildItem -LiteralPath $SourceRoot -Recurse -File -Force -ErrorAction SilentlyContinue + $copied = 0 + $skipped = 0 + + foreach ($file in $files) { + $rel = $file.FullName.Substring($sourceFull.Length).TrimStart('\') + $destFile = Join-Path $DestinationRoot $rel + $destParent = Split-Path -Parent $destFile + if ($destParent -and -not (Test-Path -LiteralPath $destParent)) { + New-Item -ItemType Directory -Path $destParent -Force | Out-Null + } + if (Test-Path -LiteralPath $destFile) { $skipped++; continue } + Copy-Item -LiteralPath $file.FullName -Destination $destFile -Force + $sizeMb = "{0:N2}" -f ($file.Length / 1MB) + Write-Host " Copied: $rel ($sizeMb MB)" -ForegroundColor Green + $copied++ + } + return @{ copied = $copied; skipped = $skipped } +} + +if ($Projects.Count -eq 0) { + Write-Host "" + Write-Host "=== zsync (backups -> offsite) ===" -ForegroundColor Cyan + Write-Host " Source: $BackupsRoot" -ForegroundColor Gray + Write-Host " Destination: $OneDriveBackupsRoot" -ForegroundColor Gray + Write-Host " Mode: new files and folders only" -ForegroundColor Gray + Write-Host "" + + if ([string]::IsNullOrWhiteSpace($OneDriveBackupsRoot)) { + Write-Host " paths.oneDriveBackups is not set in zconfig.json - nothing to do." -ForegroundColor Yellow + Stop-ZTracking; exit 0 + } + + $stats = Sync-NewFilesTree -SourceRoot $BackupsRoot -DestinationRoot $OneDriveBackupsRoot + Write-Host "" + Write-Host " Done: $($stats.copied) copied, $($stats.skipped) already present" -ForegroundColor Cyan + Write-Host "" + Stop-ZTracking; exit 0 +} + +foreach ($key in $Projects) { + $proj = Get-ZProject -Key $key + if ($proj.kind -ne "vite") { + Write-Host "" + Write-Host "zsync project mode only supports vite kind (builds and mirrors dist/). '$key' is kind '$($proj.kind)'." -ForegroundColor Yellow + Stop-ZTracking; exit 1 + } + $root = $proj.localRoot + + if ([string]::IsNullOrWhiteSpace($Destination)) { + Write-Host "" + Write-Host "=== zsync ($($proj.label)) ===" -ForegroundColor Cyan + Write-Host "No destination set." -ForegroundColor Yellow + Write-Host "Set ZSYNC_DEST or run: zsync $key -Destination 'D:\path\to\folder'" -ForegroundColor DarkGray + Write-Host "This runs npm run build, then robocopy dist -> destination." -ForegroundColor DarkGray + Stop-ZTracking; exit 1 + } + + $prevLoc = Get-Location + try { + Set-Location -LiteralPath $root + + Write-Host "" + Write-Host "=== zsync ($($proj.label)) ===" -ForegroundColor Cyan + Write-Host "Build + mirror dist -> $Destination" -ForegroundColor Cyan + Write-Host "" + + & npm run build + if ($LASTEXITCODE -ne 0) { throw "npm run build failed" } + + $dist = Join-Path $root "dist" + if (-not (Test-Path -LiteralPath $dist)) { throw "dist/ missing after build." } + + New-Item -ItemType Directory -Path $Destination -Force | Out-Null + & robocopy $dist $Destination /MIR /NFL /NDL /NJH /NJS /NP + $rc = $LASTEXITCODE + if ($rc -ge 8) { throw "robocopy failed with exit code $rc" } + + Write-Host "" + Write-Host "Sync complete." -ForegroundColor Green + } + finally { + Set-Location -LiteralPath $prevLoc + } +} +Stop-ZTracking