From c20e82e2091c9ea1357305d7b8fe4f58b354612a Mon Sep 17 00:00:00 2001 From: kellymichels Date: Sun, 26 Jul 2026 13:24:23 -0500 Subject: [PATCH] chore: hold the bash port back from the public repo until it is tested (#34) The bash port and its bats suite are removed from the tree while they get more testing. Everything they were referenced from is cleaned up so nothing dangles: - README drops the 'Linux / macOS / WSL' pointer to bash/README.md (a dead link once the folder is gone) and the aside that ZCONFIG is honoured by both ports. - CHANGELOG drops the two bash mentions. - ZHelpers.ps1's ZCONFIG comment no longer cites zhelpers.sh. - .gitattributes drops the now-dead bash/**, tests/bash/** and *.bats rules, keeping the PowerShell CRLF rules. No behaviour change to the PowerShell scripts; the Pester suite is untouched and still passes 169/169. Deliberately NOT a history rewrite: the port stays in this repo's history and in full in the private mirror, so it can be restored with a revert when the testing is done. Nothing here is secret - it is unfinished, not sensitive. --- .gitattributes | 9 - CHANGELOG.md | 9 +- README.md | 4 +- ZHelpers.ps1 | 5 +- bash/README.md | 107 -------- bash/setup_backup_schedule | 51 ---- bash/zbackup | 151 ----------- bash/zbackup_and_sync | 56 ----- bash/zbackup_ec2 | 108 -------- bash/zconfig.example.json | 87 ------- bash/zdeploy | 497 ------------------------------------- bash/zec2 | 100 -------- bash/zec2online | 155 ------------ bash/zhelpers.sh | 359 --------------------------- bash/zkill | 77 ------ bash/zrepair | 80 ------ bash/zrestart | 54 ---- bash/zsetup | 95 ------- bash/zsetup_mail | 107 -------- bash/zstart | 203 --------------- bash/zstart_docker | 66 ----- bash/zstop | 38 --- bash/zsync | 97 -------- tests/bash/args.bats | 232 ----------------- tests/bash/helpers.bats | 237 ------------------ 25 files changed, 7 insertions(+), 2977 deletions(-) delete mode 100644 bash/README.md delete mode 100644 bash/setup_backup_schedule delete mode 100644 bash/zbackup delete mode 100644 bash/zbackup_and_sync delete mode 100644 bash/zbackup_ec2 delete mode 100644 bash/zconfig.example.json delete mode 100644 bash/zdeploy delete mode 100644 bash/zec2 delete mode 100644 bash/zec2online delete mode 100644 bash/zhelpers.sh delete mode 100644 bash/zkill delete mode 100644 bash/zrepair delete mode 100644 bash/zrestart delete mode 100755 bash/zsetup delete mode 100644 bash/zsetup_mail delete mode 100644 bash/zstart delete mode 100644 bash/zstart_docker delete mode 100644 bash/zstop delete mode 100644 bash/zsync delete mode 100644 tests/bash/args.bats delete mode 100644 tests/bash/helpers.bats diff --git a/.gitattributes b/.gitattributes index 7ae3d52..0a2bf76 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,12 +1,3 @@ -# Bash scripts must stay LF — CRLF breaks the shebang on Linux/macOS/WSL. -bash/** text eol=lf - -# The bats suites are bash too, and they live outside bash/ — without this a -# Windows checkout gets CRLF and `bats tests/bash` fails there (including when -# run through WSL against the Windows working copy). -tests/bash/** text eol=lf -*.bats text eol=lf - # PowerShell/cmd are happiest with CRLF on Windows. *.ps1 text eol=crlf *.cmd text eol=crlf diff --git a/CHANGELOG.md b/CHANGELOG.md index cb755d1..70f17d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,9 +28,9 @@ Notable changes to the Evomedia.net Token Savers. lookups, `remote.composeDir` fallback, EC2 target composition, and build-label formatting. Run with `Invoke-Pester .\tests` (Pester 5+). Verified by mutation testing — reintroducing each historical bug turns the suite red. -- **`ZCONFIG` environment variable (PowerShell)** — overrides the path to - `zconfig.json`, matching the bash port, which has always honored it. Closes a - parity gap and gives the test suite a seam for injecting a fixture config. +- **`ZCONFIG` environment variable** — overrides the path to `zconfig.json`, so + a run can target an alternate config. Also gives the test suite a seam for + injecting a fixture. - **`zec2_rotatekeys` — safely rotate/reset server-side secrets** — a new tool for when a secret leaks or a deploy overwrites a production `.env` with dev values. `-Rotate KEY` regenerates a key **on the server** @@ -46,8 +46,7 @@ Notable changes to the Evomedia.net Token Savers. - **`zkill all`** — `zkill` now accepts `all`, stopping the dev server of every project that has a `ports.dev` (edge/docker stacks with no local dev server are skipped). Brings it in line with `zdeploy all` / `zbackup all`; - the one-shot "stop everything I've got running locally". Ported to both the - PowerShell and bash versions. + the one-shot "stop everything I've got running locally". - **`zdeploy` server-side health verification (`verify` block)** — projects not published through the edge proxy can declare `"verify": { "port": ..., "path": "/health", "expect": "..." }` and the diff --git a/README.md b/README.md index c2c6c16..e700831 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,6 @@ Every command is a tiny PowerShell script driven by a single JSON config file. T **Requirements:** Windows, PowerShell 5.1+, OpenSSH client (`ssh`/`scp`, ships with Windows 10/11), and Docker + docker compose on the remote host for the deploy scripts. -**Linux / macOS / WSL:** a native bash port of every command lives in [`bash/`](bash/README.md) — same config schema, same commands, no PowerShell needed. - --- ## Install @@ -49,7 +47,7 @@ notepad zconfig.json The example config ships with sample projects named by their kind — `pyapp`, `viteapp`, `nextapp`, `edge`, `analytics`. **Rename the keys to your own project names**; the key is what you type as the command argument. Add as many projects as you like — no script edits ever needed. -Set the `ZCONFIG` environment variable to point at a config somewhere else (both the PowerShell scripts and the bash port honor it) — handy for a second machine profile, or for running against a scratch config without touching your real one. +Set the `ZCONFIG` environment variable to point at a config somewhere else — handy for a second machine profile, or for running against a scratch config without touching your real one. ### Config reference diff --git a/ZHelpers.ps1 b/ZHelpers.ps1 index d83b566..4295bb9 100644 --- a/ZHelpers.ps1 +++ b/ZHelpers.ps1 @@ -34,9 +34,8 @@ $script:JunkDirNames = @( $script:ZConfigCache = $null # Where zconfig.json lives. Defaults to next to the scripts; override with the -# ZCONFIG environment variable, matching the bash port (zhelpers.sh does the -# same). Useful for pointing a run at an alternate config, and it is the seam -# the test suite uses to inject a fixture. +# ZCONFIG environment variable. Useful for pointing a run at an alternate +# config, and it is the seam the test suite uses to inject a fixture. function Get-ZConfigPath { if ($env:ZCONFIG) { return $env:ZCONFIG } return (Join-Path $PSScriptRoot "zconfig.json") diff --git a/bash/README.md b/bash/README.md deleted file mode 100644 index 6cf73c3..0000000 --- a/bash/README.md +++ /dev/null @@ -1,107 +0,0 @@ - - -# Token Savers — bash port (Linux / macOS / WSL) - -Native bash versions of the z-scripts. Same commands, same `zconfig.json` schema, -same behavior — no PowerShell required. - -## Requirements - -- bash 3.2+ (the stock macOS bash works), `jq`, `curl`, `zip`, OpenSSH (`ssh`/`scp`) -- `lsof` (for `zkill`/`zrestart`), `rsync` (for `zsync ` mirror mode) -- Docker + docker compose on the remote host for the deploy scripts - -Debian/Ubuntu: `sudo apt install jq curl zip lsof rsync` -macOS: `brew install jq` (bash, curl, zip, ssh, lsof and rsync already ship with macOS) - -## Install - -```bash -git clone https://github.com/kellymichels/zscripts-token-savers.git ~/zscripts -cd ~/zscripts/bash -cp zconfig.example.json zconfig.json # then fill in your values -chmod +x z* setup_backup_schedule -echo 'export PATH="$HOME/zscripts/bash:$PATH"' >> ~/.bashrc && source ~/.bashrc -``` - -## Commands - -Same set as the PowerShell versions — the project key in `zconfig.json` IS the -command argument: - -| Command | What it does | -|---|---| -| `zsetup

` | create the project's python venv + install deps (or `npm install`) | -| `zstart

[--detached]` | start local dev server (python/vite/nextjs) | -| `zkill

` | free the project's dev port | -| `zrestart

[--detached]` | kill + start in one command | -| `zstop

` | stop the project's compose stack on the server | -| `zdeploy

[--note "msg"]` | zip → scp → docker compose build/up → verify build version | -| `zec2 [

]` | TCP + HTTP + live-version reachability check | -| `zec2online [

]` | deep health check; auto-starts downed stacks | -| `zrepair

` | audit/repair container + proxy routing, smoke test | -| `zbackup [

] [--tag t]` | local source zip (+ pg_dump when .env has DATABASE_URL) | -| `zbackup_ec2 [

]` | pull db dump + data dirs down from the server | -| `zsync` | copy new backup files offsite (never overwrites) | -| `zbackup_and_sync` | both of the above; cron-friendly | -| `setup_backup_schedule --install` | daily backup cron job | -| `zsetup_mail --domain d` | provision docker-mailserver mailboxes + print DNS records | -| `zstart_docker` | bring up the local compose stack in `bash/docker/` | - -Flags use GNU style (`--port 3000`, `--detached`) instead of PowerShell style -(`-Port`, `-Detached`). Detached dev servers log to `/tmp/zstart-.log`. - -## Usage tracking - -Every run prints a footer with its own output volume: - -``` ---- 1,048 lines / 69,009 chars / ~19,717 tokens est. (Claude Code) --- -``` - -and (when `jq` is available) appends one JSON line per run to `tokens.jsonl`, -so you can see how much infrastructure output you keep out of an AI agent's -context over time. Fields: `ts, script, projects, lines, chars, est, model` -(`est` ≈ `chars / 3.5`). Nested runs (the `zkill`/`zstart` inside `zrestart`) -are counted once, not double. - -Where it's written, first match wins: - -1. `$ZTOKENS_DATA` -2. `ztokens.dataDir` in `zconfig.json` -3. the sibling `../ztokens/data` directory, if present -4. `~/.ztokens/data` (created on first run) - -Set `ZTOKENS_MODEL` to tag records with a specific model; it defaults to -`est. chars/3.5`. - -## Tests - -The bash port has its own [bats](https://github.com/bats-core/bats-core) suite, mirroring the PowerShell Pester tests assertion-for-assertion where the two are meant to agree — the exclude lists, config accessors, `z_path` translation, and argument handling: - -```bash -bats tests/bash -``` - -Install bats without root: - -```bash -git clone --depth 1 https://github.com/bats-core/bats-core.git /tmp/bats-core -/tmp/bats-core/install.sh ~/.local # then ensure ~/.local/bin is on PATH -``` - -A fixture config is injected through `ZCONFIG`, so the suite never reads your real `zconfig.json` and passes on a machine that has never been configured. Tests that run the scripts stay on paths that exit before doing any work; only `zkill` runs with a real target, against deliberately unused ports. - -Why a separate suite: the bash port reimplements the exclude lists and argument parsing, so it can drift from the PowerShell side independently. Both suites assert the same deploy-vs-backup rule — **deploys drop `.env*` and `uploads/`, backups keep them** — because that one has broken in production. - -## Differences from the PowerShell versions - -- `zsync ` mirrors `dist/` with `rsync -a --delete` (robocopy /MIR equivalent). -- Scheduling uses cron (`setup_backup_schedule`) instead of Windows Task Scheduler. -- The MOTD banner picks a random `motd/*.txt` instead of tracking a shuffle rotation. -- Windows-only helpers (`.cmd` launchers) don't exist — scripts are directly executable. -- **A leading dash on a project key is only tolerated by `zdeploy`.** PowerShell accepts `-myapp` anywhere; in bash every other script treats `-myapp` as an unknown option and exits 1. Tracked as a parity gap — use bare keys in the bash port. diff --git a/bash/setup_backup_schedule b/bash/setup_backup_schedule deleted file mode 100644 index 48a0e72..0000000 --- a/bash/setup_backup_schedule +++ /dev/null @@ -1,51 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# setup_backup_schedule — schedule daily zbackup_and_sync via cron. -# (The PowerShell version creates a Windows Scheduled Task; this is the cron equivalent.) -# -# Usage: -# setup_backup_schedule # print the crontab line to add -# setup_backup_schedule --install # append it to your crontab (2:00 AM daily) -# setup_backup_schedule --time 03:30 --install - -set -uo pipefail -_HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -source "$_HERE/zhelpers.sh" -z_track_start "$@" - -install=0; at_time="02:00" -while [ $# -gt 0 ]; do - case "$1" in - --install) install=1; shift ;; - --time) at_time="${2:-02:00}"; shift 2 ;; - *) err "Unknown option: $1"; exit 1 ;; - esac -done - -hh="${at_time%%:*}"; mm="${at_time##*:}" -if ! [[ "$hh" =~ ^[0-9]{1,2}$ && "$mm" =~ ^[0-9]{2}$ ]] || [ "$hh" -gt 23 ] || [ "$mm" -gt 59 ]; then - err "Invalid --time '$at_time' (expected HH:MM, 24h)"; exit 1 -fi - -cron_line="${mm#0} ${hh#0} * * * $_HERE/zbackup_and_sync all >> \$HOME/zbackup_and_sync.log 2>&1" - -info "=== Backup schedule (cron) ===" -dim "Daily at $at_time:" -printf '\n %s\n\n' "$cron_line" - -if [ "$install" -eq 1 ]; then - z_require crontab - if crontab -l 2>/dev/null | grep -qF "$_HERE/zbackup_and_sync"; then - warn "A crontab entry for zbackup_and_sync already exists - not adding a duplicate." - dim "Edit it with: crontab -e" - exit 0 - fi - ( crontab -l 2>/dev/null; printf '%s\n' "$cron_line" ) | crontab - \ - || { err "Failed to update crontab."; exit 1; } - ok "Installed. Verify with: crontab -l" -else - dim "Run with --install to add it to your crontab, or add it manually with: crontab -e" -fi diff --git a/bash/zbackup b/bash/zbackup deleted file mode 100644 index 31c5cc2..0000000 --- a/bash/zbackup +++ /dev/null @@ -1,151 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zbackup — local backups: zip project sources (plus a Postgres dump when the -# project's .env has a DATABASE_URL) into the backups folder. -# -# Usage: -# zbackup [ ...] -# zbackup all # every project in zconfig.json + this scripts folder -# zbackup scripts # just this scripts folder ('scripts' is a reserved word) -# zbackup pyapp --tag "pre-migration" -# -# Output: //_[_tag].zip - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config -z_require zip - -projects=(); tag="" -while [ $# -gt 0 ]; do - case "$1" in - --tag) tag="${2:-}"; shift 2 ;; - -*) err "Unknown option: $1"; exit 1 ;; - *) projects+=("$1"); shift ;; - esac -done - -BACKUP_ROOT="$(z_path "$(zq '.paths.backupsLocal')")" - -# Require an explicit target: bare invocation shows usage instead of quietly -# backing up everything - 'all' is explicit, matching zdeploy. -if [ "${#projects[@]}" -eq 0 ]; then - printf '\n'; warn "Usage: zbackup [ ...] | all | scripts [--tag \"label\"]" - dim " Projects in zconfig.json: $(zproj_csv)" - dim " 'all' backs up every project plus this scripts folder; 'scripts' just this folder." - exit 1 -fi - -include_scripts=0 -if printf '%s\n' "${projects[@]}" | grep -qx 'all'; then - projects=(); while IFS= read -r _zl || [ -n "$_zl" ]; do projects+=("$_zl"); done < <(zproj_keys) - include_scripts=1 -else - filtered=() - for p in "${projects[@]}"; do - if [ "$p" = "scripts" ]; then include_scripts=1; else filtered+=("$p"); fi - done - projects=("${filtered[@]+"${filtered[@]}"}") -fi - -ts() { date +%Y%m%d-%H%M%S; } -tag_suffix() { [ -n "$tag" ] && printf '_%s' "$(printf '%s' "$tag" | tr -s '[:space:]' '_')"; } - -# Dump the project's Postgres database if its .env declares a DATABASE_URL. -# Checks /.env, then /backend/.env. Prints its own status line for -# every outcome; returns 0 (dumped, $2 written) or 1 (nothing dumped). -# A not-running database (connection refused) is a calm, expected skip; a real -# failure (version mismatch, auth, missing db) prints the loud pg_dump error. -local_pg_dump() { # - local root="$1" out="$2" env_file db_url err_out rc - env_file="$root/.env"; [ -f "$env_file" ] || env_file="$root/backend/.env" - if [ ! -f "$env_file" ]; then dim " No local DATABASE_URL - source-only backup."; return 1; fi - db_url="$(grep -m1 '^DATABASE_URL=' "$env_file" | cut -d= -f2- | tr -d '[:space:]')" - if [ -z "$db_url" ]; then dim " No local DATABASE_URL - source-only backup."; return 1; fi - db_url="$(printf '%s' "$db_url" | sed -E 's|^postgresql\+[^:]+://|postgresql://|')" - if [[ "$db_url" =~ ^postgresql://([^:]+):([^@]+)@([^:]+):([0-9]+)/([^?]+) ]]; then - local user="${BASH_REMATCH[1]}" pass="${BASH_REMATCH[2]}" host="${BASH_REMATCH[3]}" - local port="${BASH_REMATCH[4]}" db="${BASH_REMATCH[5]}" - command -v pg_dump >/dev/null 2>&1 || { err ' pg_dump not installed - skipping PG backup'; return 1; } - err_out="$(PGPASSWORD="$(z_urldecode "$pass")" pg_dump -h "$host" -p "$port" -U "$user" -d "$db" -F p -f "$out" 2>&1)"; rc=$? - if [ "$rc" -eq 0 ] && [ -f "$out" ]; then - ok " PostgreSQL dump: $(awk -v b="$(wc -c < "$out")" 'BEGIN{printf "%.1f", b/1024}') KB" - return 0 - fi - # Not reachable (usually just not running locally) is expected - stay calm. - if printf '%s' "$err_out" | grep -qiE 'connection refused|could not connect|no route to host|could not translate host|timeout expired'; then - dim " Local database not running at $host:$port - source-only backup." - return 1 - fi - # A real failure (version mismatch, auth, missing db) - show the reason. - err " pg_dump failed ($host:$port/$db as $user):" - printf '%s\n' "$err_out" | grep -v '^[[:space:]]*$' | sed 's/^/ /' >&2 - return 1 - fi - err ' Could not parse DATABASE_URL - skipping PG backup' - return 1 -} - -declare -a summary -exit_code=0 - -backup_project() { # - local key="$1" root out_dir zip_path dump_dir db_dump extra="-" - zproj_require "$key" - root="$(zproj_root "$key")" - out_dir="$BACKUP_ROOT/$key"; mkdir -p "$out_dir" - zip_path="$out_dir/$(ts)_${key}$(tag_suffix).zip" - - printf '\n'; info "=== [$(printf '%s' "$key" | tr '[:lower:]' '[:upper:]')] Local backup ($(zproj "$key" .label)) ===" - dim " Output: $zip_path" - [ -d "$root" ] || { err " Root not found: $root"; return 1; } - - dump_dir="$(mktemp -d "${TMPDIR:-/tmp}/zbackup_${key}_XXXXXX")" - db_dump="$dump_dir/database_pg.sql" - warn " [1/3] Checking for a local database to dump..." - if local_pg_dump "$root" "$db_dump"; then extra="$db_dump"; fi - - warn " [2/3] Archiving source..." - local excl=() - excl=(); while IFS= read -r _zl || [ -n "$_zl" ]; do excl+=("$_zl"); done < <(z_archive_excludes "$key" 1) - if ! z_archive "$root" "$zip_path" 0 "$extra" "${excl[@]}"; then - rm -rf "$dump_dir"; return 1 - fi - rm -rf "$dump_dir" - warn " [3/3] Done." - summary+=("$key -> $zip_path ($(z_size_mb "$zip_path") MB)") -} - -backup_scripts() { - local out_dir zip_path scripts_root - scripts_root="$(z_path "$(zq '.paths.scriptsRoot')")" - out_dir="$BACKUP_ROOT/scripts"; mkdir -p "$out_dir" - zip_path="$out_dir/$(ts)_scripts$(tag_suffix).zip" - - printf '\n'; info "=== [SCRIPTS] Local backup (this scripts folder) ===" - dim " Output: $zip_path" - [ -d "$scripts_root" ] || { err " Scripts root not found: $scripts_root"; return 1; } - - warn " [1/2] Archiving source..." - z_archive "$scripts_root" "$zip_path" 1 "-" .git archive tmp nul || return 1 - warn " [2/2] Done." - summary+=("scripts -> $zip_path ($(z_size_mb "$zip_path") MB)") -} - -for key in "${projects[@]+"${projects[@]}"}"; do - backup_project "$key" || { summary+=("$key -> FAILED"); exit_code=1; } -done -if [ "$include_scripts" -eq 1 ]; then - backup_scripts || { summary+=("scripts -> FAILED"); exit_code=1; } -fi - -printf '\n'; info "=== Backup summary ===" -for line in "${summary[@]+"${summary[@]}"}"; do - case "$line" in *FAILED*) err " $line" ;; *) ok " $line" ;; esac -done -printf '\n' -exit "$exit_code" diff --git a/bash/zbackup_and_sync b/bash/zbackup_and_sync deleted file mode 100644 index 06bbcec..0000000 --- a/bash/zbackup_and_sync +++ /dev/null @@ -1,56 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zbackup_and_sync — run backups, then sync the backups folder offsite. -# -# Usage: -# zbackup_and_sync [ ...] -# zbackup_and_sync all # backup everything + sync -# -# Cron example (see setup_backup_schedule): -# 0 2 * * * /path/to/zscripts/bash/zbackup_and_sync all >> ~/zbackup.log 2>&1 - -set -uo pipefail -_HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -source "$_HERE/zhelpers.sh" -z_track_start "$@" -z_need_config - -# Require an explicit target (matching zbackup/zdeploy); 'all' backs up everything. -if [ $# -eq 0 ]; then - printf '\n'; warn "Usage: zbackup_and_sync [ ...] | all" - dim " Projects in zconfig.json: $(zproj_csv)" - dim " 'all' backs up every project plus the scripts folder, then syncs offsite." - exit 1 -fi - -printf '\n' -info "============================================" -info " Backup & Sync - $(date '+%Y-%m-%d %H:%M:%S')" -info "============================================" -printf '\n' - -warn "[1/2] Running backups..." -"$_HERE/zbackup" "$@" -backup_rc=$? -if [ $backup_rc -ne 0 ]; then - err "Backup failed (exit code: $backup_rc)" - exit $backup_rc -fi - -printf '\n'; warn "[2/2] Syncing offsite..." -"$_HERE/zsync" -sync_rc=$? - -printf '\n' -info "============================================" -if [ $sync_rc -eq 0 ]; then - ok " Backup & Sync Complete [OK]" -else - warn " Sync had issues (exit code: $sync_rc)" -fi -info "============================================" -printf '\n' -exit $sync_rc diff --git a/bash/zbackup_ec2 b/bash/zbackup_ec2 deleted file mode 100644 index 56bc23d..0000000 --- a/bash/zbackup_ec2 +++ /dev/null @@ -1,108 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zbackup_ec2 — pull backups down from the server: a Postgres dump for projects -# with a "db" config block, plus a zip of server-side data dirs (uploads/archive/dist). -# -# Usage: -# zbackup_ec2 [ ...] -# zbackup_ec2 all # every project with a remote.path -# -# Output: -# //__db.sql (projects with a db block) -# //__files.zip - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config -z_require ssh scp - -projects=("$@") -# Require an explicit target; 'all' pulls every project with a remote.path. -if [ "${#projects[@]}" -eq 0 ]; then - printf '\n'; warn "Usage: zbackup_ec2 [ ...] | all" - dim " Projects in zconfig.json: $(zproj_csv)" - dim " 'all' pulls a server backup of every project with a remote.path." - exit 1 -fi -if printf '%s\n' "${projects[@]}" | grep -qx 'all'; then - projects=(); while IFS= read -r _zl || [ -n "$_zl" ]; do projects+=("$_zl"); done \ - < <(jq -r '.projects | to_entries[] | select((.key | startswith("_") | not) and .value.remote.path != null) | .key' "$ZCONFIG") -fi - -EC2_BACKUP_ROOT="$(z_path "$(zq '.paths.backupsEc2')")" -REMOTE_HOME="/home/$(zec2_user)" - -scp_down() { # - scp -o StrictHostKeyChecking=no -i "$(zec2_pem)" "$(zec2_target):$1" "$2" \ - || { err "SCP download failed: $1 -> $2"; return 1; } -} - -declare -a summary -exit_code=0 - -ec2_backup() { # - local key="$1" remote_path compose_dir ts out_dir remote_db remote_zip local_db local_zip - local db_user db_name has_db=0 line - zproj_require "$key" - remote_path="$(zproj "$key" .remote.path)" - compose_dir="$(zremote_compose_dir "$key")" - ts="$(date +%Y%m%d-%H%M%S)" - out_dir="$EC2_BACKUP_ROOT/$key"; mkdir -p "$out_dir" - remote_db="$REMOTE_HOME/ec2_${key}_db_$ts.sql" - remote_zip="$REMOTE_HOME/ec2_${key}_files_$ts.zip" - local_db="$out_dir/${ts}_${key}_db.sql" - local_zip="$out_dir/${ts}_${key}_files.zip" - db_user="$(zproj "$key" .db.user)"; db_name="$(zproj "$key" .db.name)" - [ -n "$db_user" ] && [ -n "$db_name" ] && has_db=1 - - printf '\n'; info "=== [$(printf '%s' "$key" | tr '[:lower:]' '[:upper:]')] Server backup ($(zproj "$key" .label)) ===" - - if [ "$has_db" -eq 1 ]; then - warn " [1/4] PostgreSQL dump on the server..." - zec2_step "pg_dump $db_name" \ - "cd $compose_dir && sudo docker compose exec -T db pg_dump -U $db_user -d $db_name > $remote_db" || return 1 - else - dim " [1/4] No db config block - skipping database dump." - fi - - warn " [2/4] Zipping server-side data (uploads/archive/dist if present)..." - local zip_cmd - zip_cmd="sudo apt-get install -y zip >/dev/null 2>&1" - zip_cmd+="; FILES=''" - zip_cmd+="; test -d $remote_path/uploads && FILES=\"\$FILES $remote_path/uploads\"" - zip_cmd+="; test -d $remote_path/archive && FILES=\"\$FILES $remote_path/archive\"" - zip_cmd+="; test -d $remote_path/dist && FILES=\"\$FILES $remote_path/dist\"" - zip_cmd+="; test -f $remote_path/build-version.json && FILES=\"\$FILES $remote_path/build-version.json\"" - zip_cmd+="; if [ -z \"\$FILES\" ]; then echo 'no data dirs found' | sudo zip $remote_zip - >/dev/null; else sudo zip -r $remote_zip \$FILES; fi" - if ! zec2_step "zip $key files" "$zip_cmd"; then - warn " WARNING: files zip failed - writing placeholder and continuing." - zec2_step "placeholder zip" "echo 'zip failed' | sudo zip $remote_zip - >/dev/null" || true - fi - - warn " [3/4] Downloading to local..." - [ "$has_db" -eq 1 ] && { scp_down "$remote_db" "$local_db" || return 1; } - scp_down "$remote_zip" "$local_zip" || return 1 - - warn " [4/4] Cleaning up remote..." - zec2_step "remove remote temp files" "rm -f $remote_db $remote_zip" || true - - line="$key ->" - [ "$has_db" -eq 1 ] && [ -f "$local_db" ] && line+=" $local_db ($(z_size_mb "$local_db") MB) |" - line+=" $local_zip ($(z_size_mb "$local_zip") MB)" - summary+=("$line") -} - -for key in "${projects[@]+"${projects[@]}"}"; do - ec2_backup "$key" || { summary+=("$key -> FAILED"); exit_code=1; } -done - -printf '\n'; info "=== Server backup summary ===" -for line in "${summary[@]+"${summary[@]}"}"; do - case "$line" in *FAILED*) err " $line" ;; *) ok " $line" ;; esac -done -printf '\n' -exit "$exit_code" diff --git a/bash/zconfig.example.json b/bash/zconfig.example.json deleted file mode 100644 index 6d054e0..0000000 --- a/bash/zconfig.example.json +++ /dev/null @@ -1,87 +0,0 @@ -{ - "_comment": "Copy this file to zconfig.json and fill in your values. zconfig.json is gitignored — never commit it.", - - "ec2": { - "ip": "YOUR_SERVER_IP", - "user": "YOUR_SSH_USER", - "pemKey": "/home/youruser/.ssh/YourKey.pem", - "stackRoot": "/home/YOUR_SSH_USER/stack" - }, - - "paths": { - "temp": "/tmp/zscripts", - "backupsLocal": "/home/youruser/backups/projects", - "backupsEc2": "/home/youruser/backups/ec2", - "scriptsRoot": "/home/youruser/zscripts/bash", - "oneDriveBackups": "" - }, - - "ztokens": { - "_comment": "Optional. Where each run's token-usage stats (tokens.jsonl) are recorded. Omit to use ~/.ztokens/data.", - "dataDir": "/home/youruser/.ztokens/data" - }, - - "projects": { - "_comment": "Rename these keys to your own project names — the key IS the command argument: zstart pyapp, zdeploy viteapp, zbackup nextapp. Add as many projects as you like. Keys starting with _ are ignored.", - - "pyapp": { - "label": "My Python App", - "kind": "python", - "localRoot": "/home/youruser/code/pyapp", - "startModule": "pyapp.main", - "_startApp_note": "ASGI/FastAPI app? Use \"startApp\": \"app.main:app\" instead of startModule (runs uvicorn --port --reload).", - "install": "-e .", - "ports": { "dev": 8080 }, - "domain": "pyapp.yourdomain.com", - "start": { - "_comment": "Optional zstart pre-steps: gitPull runs 'git pull --ff-only' first; env sets variables for the server process.", - "gitPull": true, - "env": { "MYAPP_DEBUG": "1" } - }, - "db": { "user": "pyapp_user", "name": "pyapp_db" }, - "remote": { - "path": "/home/YOUR_SSH_USER/stack/pyapp", - "composeDir": "/home/YOUR_SSH_USER/stack/pyapp/docker", - "appService": "app" - }, - "deploy": { "zipName": "PyAppDeploy.zip", "gitPull": true, "exclude": ["docs"] } - }, - - "viteapp": { - "label": "My Vite Site", - "kind": "vite", - "localRoot": "/home/youruser/code/viteapp", - "ports": { "dev": 5173 }, - "domain": "www.yourdomain.com", - "remote": { - "path": "/home/YOUR_SSH_USER/stack/viteapp", - "containerName": "viteapp" - }, - "deploy": { "zipName": "ViteAppDeploy.zip" } - }, - - "nextapp": { - "label": "My Next.js App", - "kind": "nextjs", - "localRoot": "/home/youruser/code/nextapp", - "ports": { "dev": 4173, "prod": 3000 }, - "domain": "app.yourdomain.com", - "migrations": "prisma", - "db": { "user": "nextapp_user", "name": "nextapp_db" }, - "remote": { - "path": "/home/YOUR_SSH_USER/stack/nextapp", - "appService": "web" - }, - "deploy": { "zipName": "NextAppDeploy.zip" } - }, - - "edge": { - "label": "Edge Proxy", - "kind": "edge", - "localRoot": "/home/youruser/code/edge", - "proxyContainer": "edge-proxy", - "certsSource": "/etc/letsencrypt", - "remote": { "path": "/home/YOUR_SSH_USER/stack/edge" } - } - } -} diff --git a/bash/zdeploy b/bash/zdeploy deleted file mode 100644 index 7bd0c33..0000000 --- a/bash/zdeploy +++ /dev/null @@ -1,497 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zdeploy — deploy any project defined in zconfig.json to the server. -# Each project runs its own docker compose stack; the handler is picked by the -# project's "kind": python | vite | nextjs | edge | docker. -# -# Usage: -# zdeploy [ ...] [--note "message"] -# zdeploy all # every project (edge kinds first), stop at first failure -# -# Flow (python/vite/nextjs): zip source -> free server disk space -> scp up -> -# unzip into remote.path (preserving server-side .env) -> docker compose build + up -# -> verify the live site reports the new build version. Zips are always deleted. - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config -z_require ssh scp zip curl jq - -projects=(); NOTE="Build deployed" -while [ $# -gt 0 ]; do - case "$1" in - --note) NOTE="${2:-Build deployed}"; shift 2 ;; - *) projects+=("${1#-}"); shift ;; # tolerate switch-style keys (zdeploy -myproject) - esac -done - -if [ "${#projects[@]}" -eq 0 ]; then - printf '\n'; warn "Usage: zdeploy [ ...] | all [--note \"message\"]" - dim " Projects in zconfig.json: $(zproj_csv)" - dim " 'all' deploys everything (edge kinds first) and stops at the first failure." - exit 1 -fi - -EC2_IP="$(zec2_ip)" -STACK_ROOT="$(zq '.ec2.stackRoot')" -REMOTE_HOME="/home/$(zec2_user)" -EC2_USER="$(zec2_user)" -TEMP_ROOT="$(z_path "$(zq '.paths.temp')")" -{ [ -z "$TEMP_ROOT" ] || [ "$TEMP_ROOT" = "null" ]; } && TEMP_ROOT="${TMPDIR:-/tmp}" -mkdir -p "$TEMP_ROOT" - -# 'all' -> every project, edge kinds first (the proxy must route before apps ship). -if printf '%s\n' "${projects[@]}" | grep -qx 'all'; then - edge_keys=(); while IFS= read -r _zl || [ -n "$_zl" ]; do edge_keys+=("$_zl"); done \ - < <(jq -r '.projects | to_entries[] | select((.key | startswith("_") | not) and .value.kind == "edge") | .key' "$ZCONFIG") - rest_keys=(); while IFS= read -r _zl || [ -n "$_zl" ]; do rest_keys+=("$_zl"); done \ - < <(jq -r '.projects | to_entries[] | select((.key | startswith("_") | not) and .value.kind != "edge") | .key' "$ZCONFIG") - projects=("${edge_keys[@]+"${edge_keys[@]}"}" "${rest_keys[@]+"${rest_keys[@]}"}") - info "Deploying all projects: $(printf '%s ' "${projects[@]}")" -fi - -deploy_zip_name() { # - local zn; zn="$(zproj "$1" .deploy.zipName)" - [ -n "$zn" ] && printf '%s' "$zn" || printf '%sDeploy.zip' "$1" -} - -# Pre-upload cleanup: remove stale deploy zips, prune docker, truncate big logs, -# fail if under 1.5 GB free. -preflight_cleanup() { # - printf '\n'; info "--- [Preflight] Freeing disk space on the server ---" - local rm_clause="true" - [ $# -gt 0 ] && rm_clause="rm -f $*" - local cmd - cmd="echo '--- df / before cleanup ---'; df -h /" - cmd+="; echo '--- removing stale deploy artifacts ---'; $rm_clause" - cmd+="; echo '--- pruning docker build cache + dangling images + stopped containers ---'" - cmd+="; sudo docker container prune -f >/dev/null 2>&1 || true" - cmd+="; sudo docker builder prune -f >/dev/null 2>&1 || true" - cmd+="; sudo docker image prune -af >/dev/null 2>&1 || true" - cmd+="; echo '--- truncating large container logs ---'" - cmd+="; sudo find /var/lib/docker/containers/ -name '*-json.log' -size +50M -exec truncate -s 0 {} + 2>/dev/null || true" - cmd+="; echo '--- df / after cleanup ---'; df -h /" - cmd+='; avail_mb=$(df --output=avail -BM / | tail -n 1 | tr -dc 0-9)' - cmd+='; [ -z "$avail_mb" ] && avail_mb=0' - cmd+='; echo available_mb=$avail_mb' - cmd+='; if [ "$avail_mb" -lt 1500 ]; then echo "ERROR: less than 1.5 GB free on /. Grow the root volume or run: sudo docker system prune -af" >&2; exit 11; fi' - zssh "$cmd" || { err "Server pre-flight cleanup failed. Root volume too full (need ~1.5 GB free, ideally 3+)."; return 1; } -} - -# Post-deploy cleanup: prune what this deploy created; running stacks untouched. -post_cleanup() { # "$log" 2>&1 3>&- 4>&-; exec nohup "$@" ) & - disown 2>/dev/null || true - ok "Started in detached mode (logs: $log)." - dim "Use zkill $key to stop it." -} - -# Optional per-project pre-start steps from zconfig.json: -# "start": { "gitPull": true, "env": { "SOME_FLAG": "1" } } -start_prep() { # - local key="$1" root kv name val - root="$(zproj_root "$key")" - while IFS=$'\t' read -r name val; do - [ -n "$name" ] || continue - export "$name=$val" - dim " env $name=$val" - done < <(jq -r --arg k "$key" '.projects[$k].start.env // {} | to_entries[] | "\(.key)\t\(.value)"' "$ZCONFIG") - if [ "$(zproj "$key" .start.gitPull)" = "true" ] && [ -d "$root/.git" ]; then - # GIT_TERMINAL_PROMPT=0 so a repo that needs credentials fails fast instead - # of blocking the server start on an interactive "Username for ..." prompt. - local out rc - out="$( (cd "$root" && GIT_TERMINAL_PROMPT=0 git pull --ff-only 2>&1) )"; rc=$? - dim " git pull: $(printf '%s\n' "$out" | tail -1)" - [ "$rc" -eq 0 ] || warn " Auto-pull skipped (exit $rc) - starting with the current checkout. (git needs credentials here, or set start.gitPull=false)" - fi -} - -start_python() { # - local key="$1" port="$2" root module app exe bv="" run_cmd=() what using_venv=1 - root="$(zproj_root "$key")" - module="$(zproj "$key" .startModule)" - app="$(zproj "$key" .startApp)" - [ -d "$root" ] || { err "Project root not found: $root"; return 1; } - [ -n "$module" ] || [ -n "$app" ] || { err "Project '$key' (kind=python) needs 'startModule' (python -m ...) or 'startApp' (uvicorn app:app) in zconfig.json."; return 1; } - - # Prefer the project venv. The Scripts/python.exe branch is a *Windows* venv - # layout — only runnable from a Windows-family shell (Git Bash/MSYS/Cygwin). - # On WSL/Linux/macOS a Windows-built venv may sit on a mounted drive and look - # executable but can't exec, so guard that branch to fall through to python3. - if [ -x "$root/.venv/bin/python" ]; then exe="$root/.venv/bin/python" - elif [ -x "$root/.venv/Scripts/python.exe" ] && [[ "$OSTYPE" == msys* || "$OSTYPE" == cygwin* ]]; then exe="$root/.venv/Scripts/python.exe" - elif command -v python3 >/dev/null 2>&1; then exe="python3"; using_venv=0 - else exe="python"; using_venv=0; fi - - # Optional convention: scripts/build_version_tool.py bumps the version on dev start. - if [ -f "$root/scripts/build_version_tool.py" ]; then - bv="$(cd "$root" && "$exe" scripts/build_version_tool.py bump 2>/dev/null | tail -1)" - [ -n "$bv" ] || bv="$(cd "$root" && "$exe" scripts/build_version_tool.py get 2>/dev/null | tail -1)" - fi - - # startApp (uvicorn ASGI target, e.g. app.main:app) takes precedence over - # startModule (python -m ...). uvicorn is run via the venv python's -m so no - # PATH juggling is needed, and it gets zstart's bind-host + dev port. - if [ -n "$app" ]; then - run_cmd=("$exe" -m uvicorn "$app" --host "$bind_host" --port "$port" --reload) - what="uvicorn $app" - else - run_cmd=("$exe" -m "$module") - what="python -m $module" - fi - - printf '\n'; info "=== zstart ($(zproj "$key" .label)) ===" - # No project venv found - warn, since the system interpreter usually lacks the - # app's deps (the failure would otherwise be a cryptic ModuleNotFoundError). - if [ "$using_venv" -eq 0 ]; then - warn "No project venv at $root/.venv - using system '$exe' (its deps may be missing)." - dim " Create one: (cd \"$root\" && python3 -m venv .venv && .venv/bin/pip install -e .)" - fi - info "Starting $what on port $port..." - [ -n "$bv" ] && note "Build Version: $bv" - show_project_motd "$root" - dim "Press Ctrl+C to stop the server" - warn_if_privileged_port "$port" - printf '\n' - - if [ "$detached" -eq 1 ]; then - detach "$key" "$root" "${run_cmd[@]}" - else - ( cd "$root" && exec "${run_cmd[@]}" ) - fi -} - -start_vite() { # - local key="$1" port="$2" root - root="$(zproj_root "$key")" - [ -d "$root" ] || { err "Project root not found: $root"; return 1; } - [ -f "$root/package.json" ] || { err "package.json not found in $root"; return 1; } - - printf '\n'; info "=== zstart ($(zproj "$key" .label) - Vite) ===" - dim "Directory: $root" - info "URL: http://${bind_host}:${port}/" - printf '\n' - - if [ ! -d "$root/node_modules" ]; then - warn "node_modules missing - running npm install..." - (cd "$root" && npm install) || { err "npm install failed"; return 1; } - fi - - show_project_motd "$root" - dim "Press Ctrl+C to stop the dev server" - warn_if_privileged_port "$port" - printf '\n' - - if [ "$detached" -eq 1 ]; then - detach "$key" "$root" npm run dev -- --host "$bind_host" --port "$port" - else - ( cd "$root" && exec npm run dev -- --host "$bind_host" --port "$port" ) - fi -} - -start_next() { # - local key="$1" port="$2" root - root="$(zproj_root "$key")" - [ -d "$root" ] || { err "Project root not found: $root"; return 1; } - [ -f "$root/package.json" ] || { err "package.json not found in $root"; return 1; } - - printf '\n'; info "=== zstart ($(zproj "$key" .label) - Next.js) ===" - info "Starting on port $port..." - printf '\n' - show_project_motd "$root" - dim "Press Ctrl+C to stop the server" - warn_if_privileged_port "$port" - printf '\n' - - export PORT="$port" - if [ "$detached" -eq 1 ]; then - detach "$key" "$root" npm run dev - else - ( cd "$root" && exec npm run dev ) - fi -} - -rc=0 -for key in "${projects[@]}"; do - zproj_require "$key" - kind="$(zproj "$key" .kind)" - if [ "$port_override" -gt 0 ] 2>/dev/null; then port="$port_override" - else port="$(zproj "$key" .ports.dev)"; [[ "$port" =~ ^[0-9]+$ ]] || port=3000; fi - - start_prep "$key" - - case "$kind" in - python) start_python "$key" "$port" || rc=1 ;; - vite) start_vite "$key" "$port" || rc=1 ;; - nextjs) start_next "$key" "$port" || rc=1 ;; - *) - printf '\n' - warn "Project '$key' has kind '$kind' - no local dev server to start." - dim "Supported kinds for zstart: python, vite, nextjs" - ;; - esac -done -exit "$rc" diff --git a/bash/zstart_docker b/bash/zstart_docker deleted file mode 100644 index 86f2903..0000000 --- a/bash/zstart_docker +++ /dev/null @@ -1,66 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zstart_docker — bring up a local docker compose stack from /docker/. -# -# Usage: -# zstart_docker [--build] [--attached] [--solo] -# -# --build rebuild images before starting -# --attached stream logs in the foreground (default is detached) -# --solo use docker-compose.solo.yml (app only, no proxy) - -set -uo pipefail -_HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -source "$_HERE/zhelpers.sh" -z_track_start "$@" -z_require docker - -build=0; attached=0; solo=0 -while [ $# -gt 0 ]; do - case "$1" in - --build) build=1; shift ;; - --attached) attached=1; shift ;; - --solo) solo=1; shift ;; - *) err "Unknown option: $1"; exit 1 ;; - esac -done - -info "=== zstart_docker ===" - -compose_file="docker-compose.yml" -[ "$solo" -eq 1 ] && compose_file="docker-compose.solo.yml" - -if ! docker info >/dev/null 2>&1; then - printf '\n' - err "ERROR: Docker Engine is not running or this shell cannot reach it." - printf '\n' - info "Try:" - dim " 1. Start the Docker daemon (systemctl start docker / open Docker Desktop)." - dim " 2. Check your user is in the docker group: groups | grep docker" - printf '\n' - exit 1 -fi - -compose_path="$_HERE/docker/$compose_file" -if [ ! -f "$compose_path" ]; then - err "ERROR: Missing $compose_path" - exit 1 -fi - -[ -f "$_HERE/.env" ] || warn "WARNING: .env not found at $_HERE/.env - compose may still start if env vars are set elsewhere." - -dc_args=(compose -f "$compose_file" up) -[ "$attached" -eq 0 ] && dc_args+=(-d) -[ "$build" -eq 1 ] && dc_args+=(--build) - -dim "Running: docker ${dc_args[*]}" -(cd "$_HERE/docker" && docker "${dc_args[@]}") || exit $? - -printf '\n'; ok "Stack: http://localhost/" -if [ "$attached" -eq 0 ]; then - dim 'Logs: docker compose -f docker/docker-compose.yml logs -f' - dim 'Stop: docker compose -f docker/docker-compose.yml down' -fi diff --git a/bash/zstop b/bash/zstop deleted file mode 100644 index ccb5bd8..0000000 --- a/bash/zstop +++ /dev/null @@ -1,38 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zstop — stop docker compose stacks on the server without removing data or files. -# -# Usage: -# zstop [ ...] -# -# Data volumes are preserved. Run zdeploy to bring a stack back up. - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config -z_require ssh - -if [ $# -eq 0 ]; then - printf '\n'; warn "Usage: zstop [ ...]" - dim " Projects in zconfig.json: $(zproj_csv)" - exit 1 -fi - -for key in "$@"; do - zproj_require "$key" - label="$(zproj "$key" .label)"; [ -n "$label" ] || label="$key" - compose_dir="$(zremote_compose_dir "$key")" - - printf '\n'; info "--- Stopping $label ---" - if zssh "cd $compose_dir && sudo docker compose down 2>&1 || true"; then - ok " $label stopped." - else - warn " WARNING: docker compose down returned non-zero for $label" - fi -done - -printf '\n'; dim "Done. Data volumes are intact. Run zdeploy to restart." diff --git a/bash/zsync b/bash/zsync deleted file mode 100644 index b96b6bc..0000000 --- a/bash/zsync +++ /dev/null @@ -1,97 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zsync — copy new backup files offsite; or build + mirror a vite project's dist. -# -# Usage: -# zsync # new files: backups folder -> paths.oneDriveBackups -# zsync --dest /mnt/mirror/dist -# -# The no-args mode copies only files that don't exist at the destination yet -# (never overwrites, never deletes). The project mode runs npm run build, then -# mirrors dist/ to --dest (or $ZSYNC_DEST) with rsync --delete. - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config - -projects=(); dest="${ZSYNC_DEST:-}" -while [ $# -gt 0 ]; do - case "$1" in - --dest) dest="${2:-}"; shift 2 ;; - -*) err "Unknown option: $1"; exit 1 ;; - *) projects+=("$1"); shift ;; - esac -done - -# ---- no-args mode: backups -> offsite, new files only ------------------------ -if [ "${#projects[@]}" -eq 0 ]; then - BACKUPS_ROOT="$(dirname "$(z_path "$(zq '.paths.backupsLocal')")")" - OFFSITE_ROOT="$(z_path "$(zq '.paths.oneDriveBackups')")" - - printf '\n'; info "=== zsync (backups -> offsite) ===" - dim " Source: $BACKUPS_ROOT" - dim " Destination: $OFFSITE_ROOT" - dim " Mode: new files and folders only" - printf '\n' - - if [ -z "$OFFSITE_ROOT" ]; then - warn " paths.oneDriveBackups is not set in zconfig.json - nothing to do." - exit 0 - fi - if [ ! -d "$BACKUPS_ROOT" ]; then - warn " Source not found (skipped): $BACKUPS_ROOT" - exit 0 - fi - mkdir -p "$OFFSITE_ROOT" - - copied=0; skipped=0 - while IFS= read -r -d '' f; do - rel="${f#"$BACKUPS_ROOT"/}" - target="$OFFSITE_ROOT/$rel" - if [ -e "$target" ]; then skipped=$((skipped + 1)); continue; fi - mkdir -p "$(dirname "$target")" - cp -p "$f" "$target" - ok " Copied: $rel ($(z_size_mb "$f") MB)" - copied=$((copied + 1)) - done < <(find "$BACKUPS_ROOT" -type f -print0 2>/dev/null) - - printf '\n'; info " Done: $copied copied, $skipped already present" - printf '\n' - exit 0 -fi - -# ---- project mode: build + mirror a vite project's dist ---------------------- -z_require rsync -for key in "${projects[@]}"; do - zproj_require "$key" - kind="$(zproj "$key" .kind)" - if [ "$kind" != "vite" ]; then - printf '\n'; warn "zsync project mode only supports vite kind (builds and mirrors dist/). '$key' is kind '$kind'." - exit 1 - fi - root="$(zproj_root "$key")" - - if [ -z "$dest" ]; then - printf '\n'; info "=== zsync ($(zproj "$key" .label)) ===" - warn "No destination set." - dim "Set ZSYNC_DEST or run: zsync $key --dest /path/to/folder" - dim "This runs npm run build, then mirrors dist -> destination (rsync --delete)." - exit 1 - fi - - printf '\n'; info "=== zsync ($(zproj "$key" .label)) ===" - info "Build + mirror dist -> $dest" - printf '\n' - - (cd "$root" && npm run build) || { err "npm run build failed"; exit 1; } - [ -d "$root/dist" ] || { err "dist/ missing after build."; exit 1; } - - mkdir -p "$dest" - rsync -a --delete "$root/dist/" "$dest/" || { err "rsync failed"; exit 1; } - - printf '\n'; ok "Sync complete." -done diff --git a/tests/bash/args.bats b/tests/bash/args.bats deleted file mode 100644 index 0f0c8a9..0000000 --- a/tests/bash/args.bats +++ /dev/null @@ -1,232 +0,0 @@ -#!/usr/bin/env bats -# Evomedia.net Token Savers - https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels - dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# args.bats - how the bash scripts read their target arguments. -# -# bats tests/bash -# -# Mirrors tests/ArgumentParsing.Tests.ps1. Same safety rule as the PowerShell -# side: these run the real scripts, so they stay on code paths that exit BEFORE -# doing any work (no args, unknown key). Only zkill runs with a valid target, -# and only because the fixture's dev ports are deliberately unused and its -# localRoots do not exist - it finds no listeners and kills nothing. zdeploy, -# zbackup_ec2, zec2, zec2online, zrepair, zstop, zstart and zbackup are never -# invoked with a real target; that needs a disposable server. - -setup() { - REPO="$(cd "$(dirname "$BATS_TEST_FILENAME")/../.." && pwd)" - BASHDIR="$REPO/bash" - export ZCONFIG="$BATS_TEST_TMPDIR/zconfig.json" - cat > "$ZCONFIG" <<'JSON' -{ - "ec2": { "ip": "203.0.113.10", "user": "testuser", "pemKey": "/fixtures/test.pem", "stackRoot": "/home/testuser/stack" }, - "paths": { "temp": "/fixtures/temp", "backupsLocal": "/fixtures/backups", "backupsEc2": "/fixtures/backups-ec2", "scriptsRoot": "/fixtures/zscripts", "oneDriveBackups": "" }, - "projects": { - "_note": "comment key - must never be treated as a project", - "pyapp": { - "label": "Fixture Python App", "kind": "python", "localRoot": "/fixtures/nonexistent-pyapp", - "startModule": "pyapp.main", "ports": { "dev": 59990 }, - "remote": { "path": "/home/testuser/stack/pyapp" } - }, - "viteapp": { - "label": "Fixture Vite Site", "kind": "vite", "localRoot": "/fixtures/nonexistent-viteapp", - "ports": { "dev": 59991 }, "remote": { "path": "/home/testuser/stack/viteapp" } - }, - "edgeproxy": { - "label": "Fixture Edge", "kind": "edge", "localRoot": "/fixtures/nonexistent-edge", - "remote": { "path": "/home/testuser/stack/edge" } - } - } -} -JSON -} - -# ---- no arguments ----------------------------------------------------------- -# Running bare must never mean "do it to everything". - -@test "zkill with no args shows usage and exits 1" { - run "$BASHDIR/zkill" - [ "$status" -eq 1 ] - [[ "$output" == *"Usage"* ]] -} - -@test "zstart with no args shows usage and exits non-zero" { - run "$BASHDIR/zstart" - [ "$status" -ne 0 ] - [[ "$output" == *"Usage"* ]] -} - -@test "zdeploy with no args shows usage and exits non-zero" { - run "$BASHDIR/zdeploy" - [ "$status" -ne 0 ] - [[ "$output" == *"Usage"* ]] -} - -@test "zbackup with no args shows usage and exits non-zero" { - run "$BASHDIR/zbackup" - [ "$status" -ne 0 ] - [[ "$output" == *"Usage"* ]] -} - -@test "usage lists the configured project keys" { - run "$BASHDIR/zkill" - [[ "$output" == *"pyapp"* ]] - [[ "$output" == *"viteapp"* ]] -} - -@test "usage never advertises the underscore comment key" { - run "$BASHDIR/zkill" - [[ "$output" != *"_note"* ]] -} - -# ---- unknown key ------------------------------------------------------------ -# Exiting 0 having selected nothing is the dangerous outcome. - -@test "zkill rejects an unknown project key" { - run "$BASHDIR/zkill" definitelynotaproject - [ "$status" -ne 0 ] - [[ "$output" == *"Unknown project key"* ]] -} - -@test "zstart rejects an unknown project key" { - run "$BASHDIR/zstart" definitelynotaproject - [ "$status" -ne 0 ] -} - -@test "zdeploy rejects an unknown project key" { - run "$BASHDIR/zdeploy" definitelynotaproject - [ "$status" -ne 0 ] -} - -@test "the unknown-key error names both the bad key and the valid ones" { - run "$BASHDIR/zkill" definitelynotaproject - [[ "$output" == *"definitelynotaproject"* ]] - [[ "$output" == *"pyapp"* ]] -} - -@test "the underscore comment key is not addressable as a project" { - run "$BASHDIR/zkill" _note - [ "$status" -ne 0 ] -} - -# The convention is that ANY "_"-prefixed key is a comment, not a project - -# PowerShell's Get-ZProject refuses them outright. A string-valued comment is -# also caught by the "is it an object?" check, so this uses an OBJECT-valued -# underscore key (e.g. someone keeping a "_template" block to copy from) to -# prove the prefix rule itself is enforced. -@test "an object-valued underscore key is still not addressable" { - cat > "$BATS_TEST_TMPDIR/objkey.json" <<'JSON' -{ - "ec2": { "ip": "203.0.113.10", "user": "testuser", "pemKey": "/f/t.pem", "stackRoot": "/s" }, - "paths": { "temp": "/f/t", "backupsLocal": "/f/b", "backupsEc2": "/f/be", "scriptsRoot": "/f/z", "oneDriveBackups": "" }, - "projects": { - "_template": { - "label": "Template To Copy", "kind": "python", "localRoot": "/fixtures/none", - "ports": { "dev": 59993 }, "remote": { "path": "/s/t" } - }, - "pyapp": { - "label": "Fixture Python App", "kind": "python", "localRoot": "/fixtures/nonexistent-pyapp", - "ports": { "dev": 59990 }, "remote": { "path": "/s/py" } - } - } -} -JSON - ZCONFIG="$BATS_TEST_TMPDIR/objkey.json" run "$BASHDIR/zkill" _template - [ "$status" -ne 0 ] - [[ "$output" == *"Unknown project key"* ]] -} - -@test "an object-valued underscore key is excluded from 'all'" { - cat > "$BATS_TEST_TMPDIR/objkey2.json" <<'JSON' -{ - "ec2": { "ip": "203.0.113.10", "user": "testuser", "pemKey": "/f/t.pem", "stackRoot": "/s" }, - "paths": { "temp": "/f/t", "backupsLocal": "/f/b", "backupsEc2": "/f/be", "scriptsRoot": "/f/z", "oneDriveBackups": "" }, - "projects": { - "_template": { - "label": "Template To Copy", "kind": "python", "localRoot": "/fixtures/none", - "ports": { "dev": 59993 }, "remote": { "path": "/s/t" } - }, - "pyapp": { - "label": "Fixture Python App", "kind": "python", "localRoot": "/fixtures/nonexistent-pyapp", - "ports": { "dev": 59990 }, "remote": { "path": "/s/py" } - } - } -} -JSON - ZCONFIG="$BATS_TEST_TMPDIR/objkey2.json" run "$BASHDIR/zkill" all - [ "$status" -eq 0 ] - [[ "$output" == *"Fixture Python App"* ]] - [[ "$output" != *"Template To Copy"* ]] -} - -# ---- leading dash: CURRENT bash behaviour ------------------------------------ -# NOTE: this is a known divergence from the PowerShell port, pinned here as it -# actually behaves rather than as it ideally would. PowerShell tolerates a -# leading dash on any project key (zdeploy -myapp == zdeploy myapp). In bash, -# only zdeploy strips it ("${1#-}"); every other script treats -myapp as an -# unknown option and exits 1. These tests document today's behaviour so a fix -# is a deliberate change with a failing test to update, not a silent surprise. - -@test "bash zdeploy strips a leading dash from a project key" { - # Unknown key keeps this on the no-work path; the error must name the key - # without its dash, proving the strip happened. - run "$BASHDIR/zdeploy" -definitelynotaproject - [ "$status" -ne 0 ] - [[ "$output" == *"definitelynotaproject"* ]] - [[ "$output" != *"-definitelynotaproject"* ]] -} - -@test "bash zkill rejects a dashed key as an unknown option (diverges from PowerShell)" { - run "$BASHDIR/zkill" -pyapp - [ "$status" -eq 1 ] - [[ "$output" == *"Unknown option"* ]] -} - -@test "bash zstart rejects a dashed key as an unknown option (diverges from PowerShell)" { - run "$BASHDIR/zstart" -pyapp - [ "$status" -ne 0 ] - [[ "$output" == *"Unknown option"* ]] -} - -# ---- zkill target resolution (safe: fixture ports are unused) ---------------- - -@test "zkill accepts a bare project key" { - run "$BASHDIR/zkill" pyapp - [ "$status" -eq 0 ] - [[ "$output" == *"Fixture Python App"* ]] - [[ "$output" == *"59990"* ]] -} - -@test "zkill accepts several keys at once" { - run "$BASHDIR/zkill" pyapp viteapp - [ "$status" -eq 0 ] - [[ "$output" == *"Fixture Python App"* ]] - [[ "$output" == *"Fixture Vite Site"* ]] -} - -@test "zkill 'all' expands to every project with a dev port" { - run "$BASHDIR/zkill" all - [ "$status" -eq 0 ] - [[ "$output" == *"Fixture Python App"* ]] - [[ "$output" == *"Fixture Vite Site"* ]] -} - -@test "zkill 'all' skips projects with no dev port (edge/docker stacks)" { - run "$BASHDIR/zkill" all - [[ "$output" != *"Fixture Edge"* ]] -} - -@test "zkill --port overrides the configured dev port" { - run "$BASHDIR/zkill" pyapp --port 59999 - [ "$status" -eq 0 ] - [[ "$output" == *"59999"* ]] - [[ "$output" != *"59990"* ]] -} - -@test "zkill announces that --kill-all is not implemented rather than silently ignoring it" { - run "$BASHDIR/zkill" pyapp --kill-all - [ "$status" -eq 0 ] - [[ "$output" == *"kill-all"* ]] -} diff --git a/tests/bash/helpers.bats b/tests/bash/helpers.bats deleted file mode 100644 index 85d999b..0000000 --- a/tests/bash/helpers.bats +++ /dev/null @@ -1,237 +0,0 @@ -#!/usr/bin/env bats -# Evomedia.net Token Savers - https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels - dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# helpers.bats - coverage for the pure/config functions in bash/zhelpers.sh. -# -# bats tests/bash -# -# The bash port has its own implementation of the exclude lists and config -# accessors, so it can drift from the PowerShell side independently. These -# mirror tests/ZHelpers.Tests.ps1 assertion-for-assertion where the two are -# meant to agree. -# -# A fixture config is injected through ZCONFIG (zhelpers.sh has always honored -# it), so no real zconfig.json is read. - -setup() { - REPO="$(cd "$(dirname "$BATS_TEST_FILENAME")/../.." && pwd)" - export ZCONFIG="$BATS_TEST_TMPDIR/zconfig.json" - cat > "$ZCONFIG" <<'JSON' -{ - "ec2": { "ip": "203.0.113.10", "user": "testuser", "pemKey": "/fixtures/test.pem", "stackRoot": "/home/testuser/stack" }, - "paths": { "temp": "/fixtures/temp", "backupsLocal": "/fixtures/backups", "backupsEc2": "/fixtures/backups-ec2", "scriptsRoot": "/fixtures/zscripts", "oneDriveBackups": "" }, - "projects": { - "_note": "comment key - must never be treated as a project", - "pyapp": { - "label": "Fixture Python App", "kind": "python", "localRoot": "/fixtures/pyapp", - "startModule": "pyapp.main", "ports": { "dev": 59990 }, "domain": "pyapp.example.com", - "remote": { "path": "/home/testuser/stack/pyapp", "composeDir": "/home/testuser/stack/pyapp/docker", "appService": "app" }, - "deploy": { "zipName": "PyAppDeploy.zip", "exclude": ["docs", "fixtures-extra"] } - }, - "viteapp": { - "label": "Fixture Vite Site", "kind": "vite", "localRoot": "/fixtures/viteapp", - "ports": { "dev": 59991 }, "domain": "www.example.com", - "remote": { "path": "/home/testuser/stack/viteapp" } - }, - "nextapp": { - "label": "Fixture Next App", "kind": "nextjs", "localRoot": "/fixtures/nextapp", - "ports": { "dev": 59992 }, "remote": { "path": "/home/testuser/stack/nextapp" } - }, - "edgeproxy": { - "label": "Fixture Edge", "kind": "edge", "localRoot": "/fixtures/edge", - "proxyContainer": "edge_proxy", "remote": { "path": "/home/testuser/stack/edge" } - }, - "noports": { - "label": "Fixture No Dev Port", "kind": "python", "localRoot": "/fixtures/noports", - "remote": { "path": "/home/testuser/stack/noports" } - } - } -} -JSON - # shellcheck disable=SC1090 - source "$REPO/bash/zhelpers.sh" -} - -# Emit the exclude list for a project; $2=1 means "for backup". -excludes() { z_archive_excludes "$1" "${2:-0}"; } - -# Fixed-string, whole-line match. -F matters: without it grep treats the needle -# as a regex, and '.env' would match 'venv' (leading '.' = any char) - which -# silently turns every ".env is excluded" assertion into a false pass. -has() { printf '%s\n' "$1" | grep -qxF "$2"; } - -# ---- config accessors ------------------------------------------------------- - -@test "zproj_keys returns every project in config order" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj_keys | paste -sd, -" - [ "$status" -eq 0 ] - [ "$output" = "pyapp,viteapp,nextapp,edgeproxy,noports" ] -} - -@test "zproj_keys omits underscore comment keys" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj_keys" - [ "$status" -eq 0 ] - ! has "$output" "_note" -} - -@test "zproj reads a project field" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj pyapp .label" - [ "$output" = "Fixture Python App" ] -} - -@test "zproj returns empty for a missing field rather than the string null" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj viteapp .startModule" - [ "$output" = "" ] -} - -@test "zec2_target composes user@ip" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zec2_target" - [ "$output" = "testuser@203.0.113.10" ] -} - -@test "zremote_compose_dir prefers composeDir" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zremote_compose_dir pyapp" - [ "$output" = "/home/testuser/stack/pyapp/docker" ] -} - -@test "zremote_compose_dir falls back to remote.path" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zremote_compose_dir viteapp" - [ "$output" = "/home/testuser/stack/viteapp" ] -} - -@test "zedge_key finds the first edge-kind project" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zedge_key" - [ "$output" = "edgeproxy" ] -} - -@test "zproj_keys_with_domain lists only projects that have a domain" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj_keys_with_domain | paste -sd, -" - [ "$output" = "pyapp,viteapp" ] -} - -# ---- z_path (Windows -> host path translation) ------------------------------ -# A shared zconfig.json may carry Windows paths when the same file is used from -# a Windows checkout and from WSL. - -@test "z_path translates a Windows drive path to /mnt form" { - run bash -c "source '$REPO/bash/zhelpers.sh'; z_path 'F:\\evomedia.net\\app'" - [ "$output" = "/mnt/f/evomedia.net/app" ] -} - -@test "z_path lowercases the drive letter" { - run bash -c "source '$REPO/bash/zhelpers.sh'; z_path 'C:\\Users\\Someone'" - [ "$output" = "/mnt/c/Users/Someone" ] -} - -@test "z_path leaves a unix path untouched (no-op on native Linux/macOS)" { - run bash -c "source '$REPO/bash/zhelpers.sh'; z_path '/home/kelly/project'" - [ "$output" = "/home/kelly/project" ] -} - -@test "z_path leaves an empty string empty" { - run bash -c "source '$REPO/bash/zhelpers.sh'; z_path ''" - [ "$output" = "" ] -} - -# ---- z_archive_excludes ----------------------------------------------------- -# Parity target: tests/ZHelpers.Tests.ps1 "Get-ArchiveExcludes". - -@test "excludes: common junk applies to every kind" { - run excludes pyapp - for n in .git .idea .vscode .claude tmp nul .DS_Store backups; do - has "$output" "$n" || { echo "missing common exclude: $n"; return 1; } - done -} - -@test "excludes: python build/venv cruft" { - run excludes pyapp - for n in .venv venv __pycache__ .pytest_cache .nicegui archive dist build htmlcov; do - has "$output" "$n" || { echo "missing python exclude: $n"; return 1; } - done -} - -@test "excludes: vite drops node_modules and dist" { - run excludes viteapp - has "$output" "node_modules" - has "$output" "dist" -} - -@test "excludes: nextjs drops .next and .vercel" { - run excludes nextapp - has "$output" ".next" - has "$output" ".vercel" - has "$output" "next-env.d.ts" -} - -# The regression this suite exists for: a dev .env shipped to production in a -# deploy zip. Deploys drop secrets and uploads; backups must keep them or the -# backup is not a restore point. -@test "excludes: python DEPLOY drops .env, .env.local, .env.production and uploads" { - run excludes pyapp 0 - for n in .env .env.local .env.production uploads; do - has "$output" "$n" || { echo "deploy should exclude: $n"; return 1; } - done -} - -@test "excludes: python BACKUP keeps .env, .env.local, .env.production and uploads" { - run excludes pyapp 1 - for n in .env .env.local .env.production uploads; do - if has "$output" "$n"; then echo "backup must NOT exclude: $n"; return 1; fi - done -} - -@test "excludes: python BACKUP still drops the virtualenv (bulk, not a secret)" { - run excludes pyapp 1 - has "$output" ".venv" -} - -@test "excludes: vite DEPLOY drops .env files" { - run excludes viteapp 0 - for n in .env .env.local .env.production; do - has "$output" "$n" || { echo "deploy should exclude: $n"; return 1; } - done -} - -@test "excludes: vite BACKUP keeps .env files" { - run excludes viteapp 1 - for n in .env .env.local .env.production; do - if has "$output" "$n"; then echo "backup must NOT exclude: $n"; return 1; fi - done -} - -@test "excludes: vite BACKUP still drops node_modules" { - run excludes viteapp 1 - has "$output" "node_modules" -} - -# nextjs has no backup gate on the PowerShell side either - both agree. -@test "excludes: nextjs drops .env in a backup too (no gate, matches PowerShell)" { - run excludes nextapp 1 - has "$output" ".env" -} - -@test "excludes: the project's own deploy.exclude entries are merged in" { - run excludes pyapp - has "$output" "docs" - has "$output" "fixtures-extra" -} - -@test "excludes: a kind with no specific list still gets the common entries" { - run excludes edgeproxy - has "$output" ".git" -} - -# ---- json_build_label ------------------------------------------------------- - -@test "json_build_label formats v." { - printf '{ "productVersion": "1.0", "buildNumber": 42 }' > "$BATS_TEST_TMPDIR/bv.json" - run bash -c "source '$REPO/bash/zhelpers.sh'; json_build_label '$BATS_TEST_TMPDIR/bv.json'" - [ "$output" = "v1.0.42" ] -} - -@test "json_build_label reports unknown for a missing file" { - run bash -c "source '$REPO/bash/zhelpers.sh'; json_build_label '$BATS_TEST_TMPDIR/nope.json'" - [ "$output" = "unknown" ] -}