From ddece471c92977168e2599018874bc422500b6c6 Mon Sep 17 00:00:00 2001 From: KellyMichels Date: Sun, 26 Jul 2026 13:23:33 -0500 Subject: [PATCH] chore: hold the bash port back from the public repo until it is tested The bash port and its bats suite are removed from the tree while they get more testing. Everything they were referenced from is cleaned up so nothing dangles: - README drops the 'Linux / macOS / WSL' pointer to bash/README.md (a dead link once the folder is gone) and the aside that ZCONFIG is honoured by both ports. - CHANGELOG drops the two bash mentions. - ZHelpers.ps1's ZCONFIG comment no longer cites zhelpers.sh. - .gitattributes drops the now-dead bash/**, tests/bash/** and *.bats rules, keeping the PowerShell CRLF rules. No behaviour change to the PowerShell scripts; the Pester suite is untouched and still passes 169/169. Deliberately NOT a history rewrite: the port stays in this repo's history and in full in the private mirror, so it can be restored with a revert when the testing is done. Nothing here is secret - it is unfinished, not sensitive. --- .gitattributes | 9 - CHANGELOG.md | 9 +- README.md | 4 +- ZHelpers.ps1 | 5 +- bash/README.md | 107 -------- bash/setup_backup_schedule | 51 ---- bash/zbackup | 151 ----------- bash/zbackup_and_sync | 56 ----- bash/zbackup_ec2 | 108 -------- bash/zconfig.example.json | 87 ------- bash/zdeploy | 497 ------------------------------------- bash/zec2 | 100 -------- bash/zec2online | 155 ------------ bash/zhelpers.sh | 359 --------------------------- bash/zkill | 77 ------ bash/zrepair | 80 ------ bash/zrestart | 54 ---- bash/zsetup | 95 ------- bash/zsetup_mail | 107 -------- bash/zstart | 203 --------------- bash/zstart_docker | 66 ----- bash/zstop | 38 --- bash/zsync | 97 -------- tests/bash/args.bats | 232 ----------------- tests/bash/helpers.bats | 237 ------------------ 25 files changed, 7 insertions(+), 2977 deletions(-) delete mode 100644 bash/README.md delete mode 100644 bash/setup_backup_schedule delete mode 100644 bash/zbackup delete mode 100644 bash/zbackup_and_sync delete mode 100644 bash/zbackup_ec2 delete mode 100644 bash/zconfig.example.json delete mode 100644 bash/zdeploy delete mode 100644 bash/zec2 delete mode 100644 bash/zec2online delete mode 100644 bash/zhelpers.sh delete mode 100644 bash/zkill delete mode 100644 bash/zrepair delete mode 100644 bash/zrestart delete mode 100755 bash/zsetup delete mode 100644 bash/zsetup_mail delete mode 100644 bash/zstart delete mode 100644 bash/zstart_docker delete mode 100644 bash/zstop delete mode 100644 bash/zsync delete mode 100644 tests/bash/args.bats delete mode 100644 tests/bash/helpers.bats diff --git a/.gitattributes b/.gitattributes index 7ae3d52..0a2bf76 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,12 +1,3 @@ -# Bash scripts must stay LF — CRLF breaks the shebang on Linux/macOS/WSL. -bash/** text eol=lf - -# The bats suites are bash too, and they live outside bash/ — without this a -# Windows checkout gets CRLF and `bats tests/bash` fails there (including when -# run through WSL against the Windows working copy). -tests/bash/** text eol=lf -*.bats text eol=lf - # PowerShell/cmd are happiest with CRLF on Windows. *.ps1 text eol=crlf *.cmd text eol=crlf diff --git a/CHANGELOG.md b/CHANGELOG.md index cb755d1..70f17d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,9 +28,9 @@ Notable changes to the Evomedia.net Token Savers. lookups, `remote.composeDir` fallback, EC2 target composition, and build-label formatting. Run with `Invoke-Pester .\tests` (Pester 5+). Verified by mutation testing — reintroducing each historical bug turns the suite red. -- **`ZCONFIG` environment variable (PowerShell)** — overrides the path to - `zconfig.json`, matching the bash port, which has always honored it. Closes a - parity gap and gives the test suite a seam for injecting a fixture config. +- **`ZCONFIG` environment variable** — overrides the path to `zconfig.json`, so + a run can target an alternate config. Also gives the test suite a seam for + injecting a fixture. - **`zec2_rotatekeys` — safely rotate/reset server-side secrets** — a new tool for when a secret leaks or a deploy overwrites a production `.env` with dev values. `-Rotate KEY` regenerates a key **on the server** @@ -46,8 +46,7 @@ Notable changes to the Evomedia.net Token Savers. - **`zkill all`** — `zkill` now accepts `all`, stopping the dev server of every project that has a `ports.dev` (edge/docker stacks with no local dev server are skipped). Brings it in line with `zdeploy all` / `zbackup all`; - the one-shot "stop everything I've got running locally". Ported to both the - PowerShell and bash versions. + the one-shot "stop everything I've got running locally". - **`zdeploy` server-side health verification (`verify` block)** — projects not published through the edge proxy can declare `"verify": { "port": ..., "path": "/health", "expect": "..." }` and the diff --git a/README.md b/README.md index c2c6c16..e700831 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,6 @@ Every command is a tiny PowerShell script driven by a single JSON config file. T **Requirements:** Windows, PowerShell 5.1+, OpenSSH client (`ssh`/`scp`, ships with Windows 10/11), and Docker + docker compose on the remote host for the deploy scripts. -**Linux / macOS / WSL:** a native bash port of every command lives in [`bash/`](bash/README.md) — same config schema, same commands, no PowerShell needed. - --- ## Install @@ -49,7 +47,7 @@ notepad zconfig.json The example config ships with sample projects named by their kind — `pyapp`, `viteapp`, `nextapp`, `edge`, `analytics`. **Rename the keys to your own project names**; the key is what you type as the command argument. Add as many projects as you like — no script edits ever needed. -Set the `ZCONFIG` environment variable to point at a config somewhere else (both the PowerShell scripts and the bash port honor it) — handy for a second machine profile, or for running against a scratch config without touching your real one. +Set the `ZCONFIG` environment variable to point at a config somewhere else — handy for a second machine profile, or for running against a scratch config without touching your real one. ### Config reference diff --git a/ZHelpers.ps1 b/ZHelpers.ps1 index d83b566..4295bb9 100644 --- a/ZHelpers.ps1 +++ b/ZHelpers.ps1 @@ -34,9 +34,8 @@ $script:JunkDirNames = @( $script:ZConfigCache = $null # Where zconfig.json lives. Defaults to next to the scripts; override with the -# ZCONFIG environment variable, matching the bash port (zhelpers.sh does the -# same). Useful for pointing a run at an alternate config, and it is the seam -# the test suite uses to inject a fixture. +# ZCONFIG environment variable. Useful for pointing a run at an alternate +# config, and it is the seam the test suite uses to inject a fixture. function Get-ZConfigPath { if ($env:ZCONFIG) { return $env:ZCONFIG } return (Join-Path $PSScriptRoot "zconfig.json") diff --git a/bash/README.md b/bash/README.md deleted file mode 100644 index 6cf73c3..0000000 --- a/bash/README.md +++ /dev/null @@ -1,107 +0,0 @@ - - -# Token Savers — bash port (Linux / macOS / WSL) - -Native bash versions of the z-scripts. Same commands, same `zconfig.json` schema, -same behavior — no PowerShell required. - -## Requirements - -- bash 3.2+ (the stock macOS bash works), `jq`, `curl`, `zip`, OpenSSH (`ssh`/`scp`) -- `lsof` (for `zkill`/`zrestart`), `rsync` (for `zsync ` mirror mode) -- Docker + docker compose on the remote host for the deploy scripts - -Debian/Ubuntu: `sudo apt install jq curl zip lsof rsync` -macOS: `brew install jq` (bash, curl, zip, ssh, lsof and rsync already ship with macOS) - -## Install - -```bash -git clone https://github.com/kellymichels/zscripts-token-savers.git ~/zscripts -cd ~/zscripts/bash -cp zconfig.example.json zconfig.json # then fill in your values -chmod +x z* setup_backup_schedule -echo 'export PATH="$HOME/zscripts/bash:$PATH"' >> ~/.bashrc && source ~/.bashrc -``` - -## Commands - -Same set as the PowerShell versions — the project key in `zconfig.json` IS the -command argument: - -| Command | What it does | -|---|---| -| `zsetup

` | create the project's python venv + install deps (or `npm install`) | -| `zstart

[--detached]` | start local dev server (python/vite/nextjs) | -| `zkill

` | free the project's dev port | -| `zrestart

[--detached]` | kill + start in one command | -| `zstop

` | stop the project's compose stack on the server | -| `zdeploy

[--note "msg"]` | zip → scp → docker compose build/up → verify build version | -| `zec2 [

]` | TCP + HTTP + live-version reachability check | -| `zec2online [

]` | deep health check; auto-starts downed stacks | -| `zrepair

` | audit/repair container + proxy routing, smoke test | -| `zbackup [

] [--tag t]` | local source zip (+ pg_dump when .env has DATABASE_URL) | -| `zbackup_ec2 [

]` | pull db dump + data dirs down from the server | -| `zsync` | copy new backup files offsite (never overwrites) | -| `zbackup_and_sync` | both of the above; cron-friendly | -| `setup_backup_schedule --install` | daily backup cron job | -| `zsetup_mail --domain d` | provision docker-mailserver mailboxes + print DNS records | -| `zstart_docker` | bring up the local compose stack in `bash/docker/` | - -Flags use GNU style (`--port 3000`, `--detached`) instead of PowerShell style -(`-Port`, `-Detached`). Detached dev servers log to `/tmp/zstart-.log`. - -## Usage tracking - -Every run prints a footer with its own output volume: - -``` ---- 1,048 lines / 69,009 chars / ~19,717 tokens est. (Claude Code) --- -``` - -and (when `jq` is available) appends one JSON line per run to `tokens.jsonl`, -so you can see how much infrastructure output you keep out of an AI agent's -context over time. Fields: `ts, script, projects, lines, chars, est, model` -(`est` ≈ `chars / 3.5`). Nested runs (the `zkill`/`zstart` inside `zrestart`) -are counted once, not double. - -Where it's written, first match wins: - -1. `$ZTOKENS_DATA` -2. `ztokens.dataDir` in `zconfig.json` -3. the sibling `../ztokens/data` directory, if present -4. `~/.ztokens/data` (created on first run) - -Set `ZTOKENS_MODEL` to tag records with a specific model; it defaults to -`est. chars/3.5`. - -## Tests - -The bash port has its own [bats](https://github.com/bats-core/bats-core) suite, mirroring the PowerShell Pester tests assertion-for-assertion where the two are meant to agree — the exclude lists, config accessors, `z_path` translation, and argument handling: - -```bash -bats tests/bash -``` - -Install bats without root: - -```bash -git clone --depth 1 https://github.com/bats-core/bats-core.git /tmp/bats-core -/tmp/bats-core/install.sh ~/.local # then ensure ~/.local/bin is on PATH -``` - -A fixture config is injected through `ZCONFIG`, so the suite never reads your real `zconfig.json` and passes on a machine that has never been configured. Tests that run the scripts stay on paths that exit before doing any work; only `zkill` runs with a real target, against deliberately unused ports. - -Why a separate suite: the bash port reimplements the exclude lists and argument parsing, so it can drift from the PowerShell side independently. Both suites assert the same deploy-vs-backup rule — **deploys drop `.env*` and `uploads/`, backups keep them** — because that one has broken in production. - -## Differences from the PowerShell versions - -- `zsync ` mirrors `dist/` with `rsync -a --delete` (robocopy /MIR equivalent). -- Scheduling uses cron (`setup_backup_schedule`) instead of Windows Task Scheduler. -- The MOTD banner picks a random `motd/*.txt` instead of tracking a shuffle rotation. -- Windows-only helpers (`.cmd` launchers) don't exist — scripts are directly executable. -- **A leading dash on a project key is only tolerated by `zdeploy`.** PowerShell accepts `-myapp` anywhere; in bash every other script treats `-myapp` as an unknown option and exits 1. Tracked as a parity gap — use bare keys in the bash port. diff --git a/bash/setup_backup_schedule b/bash/setup_backup_schedule deleted file mode 100644 index 48a0e72..0000000 --- a/bash/setup_backup_schedule +++ /dev/null @@ -1,51 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# setup_backup_schedule — schedule daily zbackup_and_sync via cron. -# (The PowerShell version creates a Windows Scheduled Task; this is the cron equivalent.) -# -# Usage: -# setup_backup_schedule # print the crontab line to add -# setup_backup_schedule --install # append it to your crontab (2:00 AM daily) -# setup_backup_schedule --time 03:30 --install - -set -uo pipefail -_HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -source "$_HERE/zhelpers.sh" -z_track_start "$@" - -install=0; at_time="02:00" -while [ $# -gt 0 ]; do - case "$1" in - --install) install=1; shift ;; - --time) at_time="${2:-02:00}"; shift 2 ;; - *) err "Unknown option: $1"; exit 1 ;; - esac -done - -hh="${at_time%%:*}"; mm="${at_time##*:}" -if ! [[ "$hh" =~ ^[0-9]{1,2}$ && "$mm" =~ ^[0-9]{2}$ ]] || [ "$hh" -gt 23 ] || [ "$mm" -gt 59 ]; then - err "Invalid --time '$at_time' (expected HH:MM, 24h)"; exit 1 -fi - -cron_line="${mm#0} ${hh#0} * * * $_HERE/zbackup_and_sync all >> \$HOME/zbackup_and_sync.log 2>&1" - -info "=== Backup schedule (cron) ===" -dim "Daily at $at_time:" -printf '\n %s\n\n' "$cron_line" - -if [ "$install" -eq 1 ]; then - z_require crontab - if crontab -l 2>/dev/null | grep -qF "$_HERE/zbackup_and_sync"; then - warn "A crontab entry for zbackup_and_sync already exists - not adding a duplicate." - dim "Edit it with: crontab -e" - exit 0 - fi - ( crontab -l 2>/dev/null; printf '%s\n' "$cron_line" ) | crontab - \ - || { err "Failed to update crontab."; exit 1; } - ok "Installed. Verify with: crontab -l" -else - dim "Run with --install to add it to your crontab, or add it manually with: crontab -e" -fi diff --git a/bash/zbackup b/bash/zbackup deleted file mode 100644 index 31c5cc2..0000000 --- a/bash/zbackup +++ /dev/null @@ -1,151 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zbackup — local backups: zip project sources (plus a Postgres dump when the -# project's .env has a DATABASE_URL) into the backups folder. -# -# Usage: -# zbackup [ ...] -# zbackup all # every project in zconfig.json + this scripts folder -# zbackup scripts # just this scripts folder ('scripts' is a reserved word) -# zbackup pyapp --tag "pre-migration" -# -# Output: //_[_tag].zip - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config -z_require zip - -projects=(); tag="" -while [ $# -gt 0 ]; do - case "$1" in - --tag) tag="${2:-}"; shift 2 ;; - -*) err "Unknown option: $1"; exit 1 ;; - *) projects+=("$1"); shift ;; - esac -done - -BACKUP_ROOT="$(z_path "$(zq '.paths.backupsLocal')")" - -# Require an explicit target: bare invocation shows usage instead of quietly -# backing up everything - 'all' is explicit, matching zdeploy. -if [ "${#projects[@]}" -eq 0 ]; then - printf '\n'; warn "Usage: zbackup [ ...] | all | scripts [--tag \"label\"]" - dim " Projects in zconfig.json: $(zproj_csv)" - dim " 'all' backs up every project plus this scripts folder; 'scripts' just this folder." - exit 1 -fi - -include_scripts=0 -if printf '%s\n' "${projects[@]}" | grep -qx 'all'; then - projects=(); while IFS= read -r _zl || [ -n "$_zl" ]; do projects+=("$_zl"); done < <(zproj_keys) - include_scripts=1 -else - filtered=() - for p in "${projects[@]}"; do - if [ "$p" = "scripts" ]; then include_scripts=1; else filtered+=("$p"); fi - done - projects=("${filtered[@]+"${filtered[@]}"}") -fi - -ts() { date +%Y%m%d-%H%M%S; } -tag_suffix() { [ -n "$tag" ] && printf '_%s' "$(printf '%s' "$tag" | tr -s '[:space:]' '_')"; } - -# Dump the project's Postgres database if its .env declares a DATABASE_URL. -# Checks /.env, then /backend/.env. Prints its own status line for -# every outcome; returns 0 (dumped, $2 written) or 1 (nothing dumped). -# A not-running database (connection refused) is a calm, expected skip; a real -# failure (version mismatch, auth, missing db) prints the loud pg_dump error. -local_pg_dump() { # - local root="$1" out="$2" env_file db_url err_out rc - env_file="$root/.env"; [ -f "$env_file" ] || env_file="$root/backend/.env" - if [ ! -f "$env_file" ]; then dim " No local DATABASE_URL - source-only backup."; return 1; fi - db_url="$(grep -m1 '^DATABASE_URL=' "$env_file" | cut -d= -f2- | tr -d '[:space:]')" - if [ -z "$db_url" ]; then dim " No local DATABASE_URL - source-only backup."; return 1; fi - db_url="$(printf '%s' "$db_url" | sed -E 's|^postgresql\+[^:]+://|postgresql://|')" - if [[ "$db_url" =~ ^postgresql://([^:]+):([^@]+)@([^:]+):([0-9]+)/([^?]+) ]]; then - local user="${BASH_REMATCH[1]}" pass="${BASH_REMATCH[2]}" host="${BASH_REMATCH[3]}" - local port="${BASH_REMATCH[4]}" db="${BASH_REMATCH[5]}" - command -v pg_dump >/dev/null 2>&1 || { err ' pg_dump not installed - skipping PG backup'; return 1; } - err_out="$(PGPASSWORD="$(z_urldecode "$pass")" pg_dump -h "$host" -p "$port" -U "$user" -d "$db" -F p -f "$out" 2>&1)"; rc=$? - if [ "$rc" -eq 0 ] && [ -f "$out" ]; then - ok " PostgreSQL dump: $(awk -v b="$(wc -c < "$out")" 'BEGIN{printf "%.1f", b/1024}') KB" - return 0 - fi - # Not reachable (usually just not running locally) is expected - stay calm. - if printf '%s' "$err_out" | grep -qiE 'connection refused|could not connect|no route to host|could not translate host|timeout expired'; then - dim " Local database not running at $host:$port - source-only backup." - return 1 - fi - # A real failure (version mismatch, auth, missing db) - show the reason. - err " pg_dump failed ($host:$port/$db as $user):" - printf '%s\n' "$err_out" | grep -v '^[[:space:]]*$' | sed 's/^/ /' >&2 - return 1 - fi - err ' Could not parse DATABASE_URL - skipping PG backup' - return 1 -} - -declare -a summary -exit_code=0 - -backup_project() { # - local key="$1" root out_dir zip_path dump_dir db_dump extra="-" - zproj_require "$key" - root="$(zproj_root "$key")" - out_dir="$BACKUP_ROOT/$key"; mkdir -p "$out_dir" - zip_path="$out_dir/$(ts)_${key}$(tag_suffix).zip" - - printf '\n'; info "=== [$(printf '%s' "$key" | tr '[:lower:]' '[:upper:]')] Local backup ($(zproj "$key" .label)) ===" - dim " Output: $zip_path" - [ -d "$root" ] || { err " Root not found: $root"; return 1; } - - dump_dir="$(mktemp -d "${TMPDIR:-/tmp}/zbackup_${key}_XXXXXX")" - db_dump="$dump_dir/database_pg.sql" - warn " [1/3] Checking for a local database to dump..." - if local_pg_dump "$root" "$db_dump"; then extra="$db_dump"; fi - - warn " [2/3] Archiving source..." - local excl=() - excl=(); while IFS= read -r _zl || [ -n "$_zl" ]; do excl+=("$_zl"); done < <(z_archive_excludes "$key" 1) - if ! z_archive "$root" "$zip_path" 0 "$extra" "${excl[@]}"; then - rm -rf "$dump_dir"; return 1 - fi - rm -rf "$dump_dir" - warn " [3/3] Done." - summary+=("$key -> $zip_path ($(z_size_mb "$zip_path") MB)") -} - -backup_scripts() { - local out_dir zip_path scripts_root - scripts_root="$(z_path "$(zq '.paths.scriptsRoot')")" - out_dir="$BACKUP_ROOT/scripts"; mkdir -p "$out_dir" - zip_path="$out_dir/$(ts)_scripts$(tag_suffix).zip" - - printf '\n'; info "=== [SCRIPTS] Local backup (this scripts folder) ===" - dim " Output: $zip_path" - [ -d "$scripts_root" ] || { err " Scripts root not found: $scripts_root"; return 1; } - - warn " [1/2] Archiving source..." - z_archive "$scripts_root" "$zip_path" 1 "-" .git archive tmp nul || return 1 - warn " [2/2] Done." - summary+=("scripts -> $zip_path ($(z_size_mb "$zip_path") MB)") -} - -for key in "${projects[@]+"${projects[@]}"}"; do - backup_project "$key" || { summary+=("$key -> FAILED"); exit_code=1; } -done -if [ "$include_scripts" -eq 1 ]; then - backup_scripts || { summary+=("scripts -> FAILED"); exit_code=1; } -fi - -printf '\n'; info "=== Backup summary ===" -for line in "${summary[@]+"${summary[@]}"}"; do - case "$line" in *FAILED*) err " $line" ;; *) ok " $line" ;; esac -done -printf '\n' -exit "$exit_code" diff --git a/bash/zbackup_and_sync b/bash/zbackup_and_sync deleted file mode 100644 index 06bbcec..0000000 --- a/bash/zbackup_and_sync +++ /dev/null @@ -1,56 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zbackup_and_sync — run backups, then sync the backups folder offsite. -# -# Usage: -# zbackup_and_sync [ ...] -# zbackup_and_sync all # backup everything + sync -# -# Cron example (see setup_backup_schedule): -# 0 2 * * * /path/to/zscripts/bash/zbackup_and_sync all >> ~/zbackup.log 2>&1 - -set -uo pipefail -_HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -source "$_HERE/zhelpers.sh" -z_track_start "$@" -z_need_config - -# Require an explicit target (matching zbackup/zdeploy); 'all' backs up everything. -if [ $# -eq 0 ]; then - printf '\n'; warn "Usage: zbackup_and_sync [ ...] | all" - dim " Projects in zconfig.json: $(zproj_csv)" - dim " 'all' backs up every project plus the scripts folder, then syncs offsite." - exit 1 -fi - -printf '\n' -info "============================================" -info " Backup & Sync - $(date '+%Y-%m-%d %H:%M:%S')" -info "============================================" -printf '\n' - -warn "[1/2] Running backups..." -"$_HERE/zbackup" "$@" -backup_rc=$? -if [ $backup_rc -ne 0 ]; then - err "Backup failed (exit code: $backup_rc)" - exit $backup_rc -fi - -printf '\n'; warn "[2/2] Syncing offsite..." -"$_HERE/zsync" -sync_rc=$? - -printf '\n' -info "============================================" -if [ $sync_rc -eq 0 ]; then - ok " Backup & Sync Complete [OK]" -else - warn " Sync had issues (exit code: $sync_rc)" -fi -info "============================================" -printf '\n' -exit $sync_rc diff --git a/bash/zbackup_ec2 b/bash/zbackup_ec2 deleted file mode 100644 index 56bc23d..0000000 --- a/bash/zbackup_ec2 +++ /dev/null @@ -1,108 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zbackup_ec2 — pull backups down from the server: a Postgres dump for projects -# with a "db" config block, plus a zip of server-side data dirs (uploads/archive/dist). -# -# Usage: -# zbackup_ec2 [ ...] -# zbackup_ec2 all # every project with a remote.path -# -# Output: -# //__db.sql (projects with a db block) -# //__files.zip - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config -z_require ssh scp - -projects=("$@") -# Require an explicit target; 'all' pulls every project with a remote.path. -if [ "${#projects[@]}" -eq 0 ]; then - printf '\n'; warn "Usage: zbackup_ec2 [ ...] | all" - dim " Projects in zconfig.json: $(zproj_csv)" - dim " 'all' pulls a server backup of every project with a remote.path." - exit 1 -fi -if printf '%s\n' "${projects[@]}" | grep -qx 'all'; then - projects=(); while IFS= read -r _zl || [ -n "$_zl" ]; do projects+=("$_zl"); done \ - < <(jq -r '.projects | to_entries[] | select((.key | startswith("_") | not) and .value.remote.path != null) | .key' "$ZCONFIG") -fi - -EC2_BACKUP_ROOT="$(z_path "$(zq '.paths.backupsEc2')")" -REMOTE_HOME="/home/$(zec2_user)" - -scp_down() { # - scp -o StrictHostKeyChecking=no -i "$(zec2_pem)" "$(zec2_target):$1" "$2" \ - || { err "SCP download failed: $1 -> $2"; return 1; } -} - -declare -a summary -exit_code=0 - -ec2_backup() { # - local key="$1" remote_path compose_dir ts out_dir remote_db remote_zip local_db local_zip - local db_user db_name has_db=0 line - zproj_require "$key" - remote_path="$(zproj "$key" .remote.path)" - compose_dir="$(zremote_compose_dir "$key")" - ts="$(date +%Y%m%d-%H%M%S)" - out_dir="$EC2_BACKUP_ROOT/$key"; mkdir -p "$out_dir" - remote_db="$REMOTE_HOME/ec2_${key}_db_$ts.sql" - remote_zip="$REMOTE_HOME/ec2_${key}_files_$ts.zip" - local_db="$out_dir/${ts}_${key}_db.sql" - local_zip="$out_dir/${ts}_${key}_files.zip" - db_user="$(zproj "$key" .db.user)"; db_name="$(zproj "$key" .db.name)" - [ -n "$db_user" ] && [ -n "$db_name" ] && has_db=1 - - printf '\n'; info "=== [$(printf '%s' "$key" | tr '[:lower:]' '[:upper:]')] Server backup ($(zproj "$key" .label)) ===" - - if [ "$has_db" -eq 1 ]; then - warn " [1/4] PostgreSQL dump on the server..." - zec2_step "pg_dump $db_name" \ - "cd $compose_dir && sudo docker compose exec -T db pg_dump -U $db_user -d $db_name > $remote_db" || return 1 - else - dim " [1/4] No db config block - skipping database dump." - fi - - warn " [2/4] Zipping server-side data (uploads/archive/dist if present)..." - local zip_cmd - zip_cmd="sudo apt-get install -y zip >/dev/null 2>&1" - zip_cmd+="; FILES=''" - zip_cmd+="; test -d $remote_path/uploads && FILES=\"\$FILES $remote_path/uploads\"" - zip_cmd+="; test -d $remote_path/archive && FILES=\"\$FILES $remote_path/archive\"" - zip_cmd+="; test -d $remote_path/dist && FILES=\"\$FILES $remote_path/dist\"" - zip_cmd+="; test -f $remote_path/build-version.json && FILES=\"\$FILES $remote_path/build-version.json\"" - zip_cmd+="; if [ -z \"\$FILES\" ]; then echo 'no data dirs found' | sudo zip $remote_zip - >/dev/null; else sudo zip -r $remote_zip \$FILES; fi" - if ! zec2_step "zip $key files" "$zip_cmd"; then - warn " WARNING: files zip failed - writing placeholder and continuing." - zec2_step "placeholder zip" "echo 'zip failed' | sudo zip $remote_zip - >/dev/null" || true - fi - - warn " [3/4] Downloading to local..." - [ "$has_db" -eq 1 ] && { scp_down "$remote_db" "$local_db" || return 1; } - scp_down "$remote_zip" "$local_zip" || return 1 - - warn " [4/4] Cleaning up remote..." - zec2_step "remove remote temp files" "rm -f $remote_db $remote_zip" || true - - line="$key ->" - [ "$has_db" -eq 1 ] && [ -f "$local_db" ] && line+=" $local_db ($(z_size_mb "$local_db") MB) |" - line+=" $local_zip ($(z_size_mb "$local_zip") MB)" - summary+=("$line") -} - -for key in "${projects[@]+"${projects[@]}"}"; do - ec2_backup "$key" || { summary+=("$key -> FAILED"); exit_code=1; } -done - -printf '\n'; info "=== Server backup summary ===" -for line in "${summary[@]+"${summary[@]}"}"; do - case "$line" in *FAILED*) err " $line" ;; *) ok " $line" ;; esac -done -printf '\n' -exit "$exit_code" diff --git a/bash/zconfig.example.json b/bash/zconfig.example.json deleted file mode 100644 index 6d054e0..0000000 --- a/bash/zconfig.example.json +++ /dev/null @@ -1,87 +0,0 @@ -{ - "_comment": "Copy this file to zconfig.json and fill in your values. zconfig.json is gitignored — never commit it.", - - "ec2": { - "ip": "YOUR_SERVER_IP", - "user": "YOUR_SSH_USER", - "pemKey": "/home/youruser/.ssh/YourKey.pem", - "stackRoot": "/home/YOUR_SSH_USER/stack" - }, - - "paths": { - "temp": "/tmp/zscripts", - "backupsLocal": "/home/youruser/backups/projects", - "backupsEc2": "/home/youruser/backups/ec2", - "scriptsRoot": "/home/youruser/zscripts/bash", - "oneDriveBackups": "" - }, - - "ztokens": { - "_comment": "Optional. Where each run's token-usage stats (tokens.jsonl) are recorded. Omit to use ~/.ztokens/data.", - "dataDir": "/home/youruser/.ztokens/data" - }, - - "projects": { - "_comment": "Rename these keys to your own project names — the key IS the command argument: zstart pyapp, zdeploy viteapp, zbackup nextapp. Add as many projects as you like. Keys starting with _ are ignored.", - - "pyapp": { - "label": "My Python App", - "kind": "python", - "localRoot": "/home/youruser/code/pyapp", - "startModule": "pyapp.main", - "_startApp_note": "ASGI/FastAPI app? Use \"startApp\": \"app.main:app\" instead of startModule (runs uvicorn --port --reload).", - "install": "-e .", - "ports": { "dev": 8080 }, - "domain": "pyapp.yourdomain.com", - "start": { - "_comment": "Optional zstart pre-steps: gitPull runs 'git pull --ff-only' first; env sets variables for the server process.", - "gitPull": true, - "env": { "MYAPP_DEBUG": "1" } - }, - "db": { "user": "pyapp_user", "name": "pyapp_db" }, - "remote": { - "path": "/home/YOUR_SSH_USER/stack/pyapp", - "composeDir": "/home/YOUR_SSH_USER/stack/pyapp/docker", - "appService": "app" - }, - "deploy": { "zipName": "PyAppDeploy.zip", "gitPull": true, "exclude": ["docs"] } - }, - - "viteapp": { - "label": "My Vite Site", - "kind": "vite", - "localRoot": "/home/youruser/code/viteapp", - "ports": { "dev": 5173 }, - "domain": "www.yourdomain.com", - "remote": { - "path": "/home/YOUR_SSH_USER/stack/viteapp", - "containerName": "viteapp" - }, - "deploy": { "zipName": "ViteAppDeploy.zip" } - }, - - "nextapp": { - "label": "My Next.js App", - "kind": "nextjs", - "localRoot": "/home/youruser/code/nextapp", - "ports": { "dev": 4173, "prod": 3000 }, - "domain": "app.yourdomain.com", - "migrations": "prisma", - "db": { "user": "nextapp_user", "name": "nextapp_db" }, - "remote": { - "path": "/home/YOUR_SSH_USER/stack/nextapp", - "appService": "web" - }, - "deploy": { "zipName": "NextAppDeploy.zip" } - }, - - "edge": { - "label": "Edge Proxy", - "kind": "edge", - "localRoot": "/home/youruser/code/edge", - "proxyContainer": "edge-proxy", - "certsSource": "/etc/letsencrypt", - "remote": { "path": "/home/YOUR_SSH_USER/stack/edge" } - } - } -} diff --git a/bash/zdeploy b/bash/zdeploy deleted file mode 100644 index 7bd0c33..0000000 --- a/bash/zdeploy +++ /dev/null @@ -1,497 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zdeploy — deploy any project defined in zconfig.json to the server. -# Each project runs its own docker compose stack; the handler is picked by the -# project's "kind": python | vite | nextjs | edge | docker. -# -# Usage: -# zdeploy [ ...] [--note "message"] -# zdeploy all # every project (edge kinds first), stop at first failure -# -# Flow (python/vite/nextjs): zip source -> free server disk space -> scp up -> -# unzip into remote.path (preserving server-side .env) -> docker compose build + up -# -> verify the live site reports the new build version. Zips are always deleted. - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config -z_require ssh scp zip curl jq - -projects=(); NOTE="Build deployed" -while [ $# -gt 0 ]; do - case "$1" in - --note) NOTE="${2:-Build deployed}"; shift 2 ;; - *) projects+=("${1#-}"); shift ;; # tolerate switch-style keys (zdeploy -myproject) - esac -done - -if [ "${#projects[@]}" -eq 0 ]; then - printf '\n'; warn "Usage: zdeploy [ ...] | all [--note \"message\"]" - dim " Projects in zconfig.json: $(zproj_csv)" - dim " 'all' deploys everything (edge kinds first) and stops at the first failure." - exit 1 -fi - -EC2_IP="$(zec2_ip)" -STACK_ROOT="$(zq '.ec2.stackRoot')" -REMOTE_HOME="/home/$(zec2_user)" -EC2_USER="$(zec2_user)" -TEMP_ROOT="$(z_path "$(zq '.paths.temp')")" -{ [ -z "$TEMP_ROOT" ] || [ "$TEMP_ROOT" = "null" ]; } && TEMP_ROOT="${TMPDIR:-/tmp}" -mkdir -p "$TEMP_ROOT" - -# 'all' -> every project, edge kinds first (the proxy must route before apps ship). -if printf '%s\n' "${projects[@]}" | grep -qx 'all'; then - edge_keys=(); while IFS= read -r _zl || [ -n "$_zl" ]; do edge_keys+=("$_zl"); done \ - < <(jq -r '.projects | to_entries[] | select((.key | startswith("_") | not) and .value.kind == "edge") | .key' "$ZCONFIG") - rest_keys=(); while IFS= read -r _zl || [ -n "$_zl" ]; do rest_keys+=("$_zl"); done \ - < <(jq -r '.projects | to_entries[] | select((.key | startswith("_") | not) and .value.kind != "edge") | .key' "$ZCONFIG") - projects=("${edge_keys[@]+"${edge_keys[@]}"}" "${rest_keys[@]+"${rest_keys[@]}"}") - info "Deploying all projects: $(printf '%s ' "${projects[@]}")" -fi - -deploy_zip_name() { # - local zn; zn="$(zproj "$1" .deploy.zipName)" - [ -n "$zn" ] && printf '%s' "$zn" || printf '%sDeploy.zip' "$1" -} - -# Pre-upload cleanup: remove stale deploy zips, prune docker, truncate big logs, -# fail if under 1.5 GB free. -preflight_cleanup() { # - printf '\n'; info "--- [Preflight] Freeing disk space on the server ---" - local rm_clause="true" - [ $# -gt 0 ] && rm_clause="rm -f $*" - local cmd - cmd="echo '--- df / before cleanup ---'; df -h /" - cmd+="; echo '--- removing stale deploy artifacts ---'; $rm_clause" - cmd+="; echo '--- pruning docker build cache + dangling images + stopped containers ---'" - cmd+="; sudo docker container prune -f >/dev/null 2>&1 || true" - cmd+="; sudo docker builder prune -f >/dev/null 2>&1 || true" - cmd+="; sudo docker image prune -af >/dev/null 2>&1 || true" - cmd+="; echo '--- truncating large container logs ---'" - cmd+="; sudo find /var/lib/docker/containers/ -name '*-json.log' -size +50M -exec truncate -s 0 {} + 2>/dev/null || true" - cmd+="; echo '--- df / after cleanup ---'; df -h /" - cmd+='; avail_mb=$(df --output=avail -BM / | tail -n 1 | tr -dc 0-9)' - cmd+='; [ -z "$avail_mb" ] && avail_mb=0' - cmd+='; echo available_mb=$avail_mb' - cmd+='; if [ "$avail_mb" -lt 1500 ]; then echo "ERROR: less than 1.5 GB free on /. Grow the root volume or run: sudo docker system prune -af" >&2; exit 11; fi' - zssh "$cmd" || { err "Server pre-flight cleanup failed. Root volume too full (need ~1.5 GB free, ideally 3+)."; return 1; } -} - -# Post-deploy cleanup: prune what this deploy created; running stacks untouched. -post_cleanup() { # "$log" 2>&1 3>&- 4>&-; exec nohup "$@" ) & - disown 2>/dev/null || true - ok "Started in detached mode (logs: $log)." - dim "Use zkill $key to stop it." -} - -# Optional per-project pre-start steps from zconfig.json: -# "start": { "gitPull": true, "env": { "SOME_FLAG": "1" } } -start_prep() { # - local key="$1" root kv name val - root="$(zproj_root "$key")" - while IFS=$'\t' read -r name val; do - [ -n "$name" ] || continue - export "$name=$val" - dim " env $name=$val" - done < <(jq -r --arg k "$key" '.projects[$k].start.env // {} | to_entries[] | "\(.key)\t\(.value)"' "$ZCONFIG") - if [ "$(zproj "$key" .start.gitPull)" = "true" ] && [ -d "$root/.git" ]; then - # GIT_TERMINAL_PROMPT=0 so a repo that needs credentials fails fast instead - # of blocking the server start on an interactive "Username for ..." prompt. - local out rc - out="$( (cd "$root" && GIT_TERMINAL_PROMPT=0 git pull --ff-only 2>&1) )"; rc=$? - dim " git pull: $(printf '%s\n' "$out" | tail -1)" - [ "$rc" -eq 0 ] || warn " Auto-pull skipped (exit $rc) - starting with the current checkout. (git needs credentials here, or set start.gitPull=false)" - fi -} - -start_python() { # - local key="$1" port="$2" root module app exe bv="" run_cmd=() what using_venv=1 - root="$(zproj_root "$key")" - module="$(zproj "$key" .startModule)" - app="$(zproj "$key" .startApp)" - [ -d "$root" ] || { err "Project root not found: $root"; return 1; } - [ -n "$module" ] || [ -n "$app" ] || { err "Project '$key' (kind=python) needs 'startModule' (python -m ...) or 'startApp' (uvicorn app:app) in zconfig.json."; return 1; } - - # Prefer the project venv. The Scripts/python.exe branch is a *Windows* venv - # layout — only runnable from a Windows-family shell (Git Bash/MSYS/Cygwin). - # On WSL/Linux/macOS a Windows-built venv may sit on a mounted drive and look - # executable but can't exec, so guard that branch to fall through to python3. - if [ -x "$root/.venv/bin/python" ]; then exe="$root/.venv/bin/python" - elif [ -x "$root/.venv/Scripts/python.exe" ] && [[ "$OSTYPE" == msys* || "$OSTYPE" == cygwin* ]]; then exe="$root/.venv/Scripts/python.exe" - elif command -v python3 >/dev/null 2>&1; then exe="python3"; using_venv=0 - else exe="python"; using_venv=0; fi - - # Optional convention: scripts/build_version_tool.py bumps the version on dev start. - if [ -f "$root/scripts/build_version_tool.py" ]; then - bv="$(cd "$root" && "$exe" scripts/build_version_tool.py bump 2>/dev/null | tail -1)" - [ -n "$bv" ] || bv="$(cd "$root" && "$exe" scripts/build_version_tool.py get 2>/dev/null | tail -1)" - fi - - # startApp (uvicorn ASGI target, e.g. app.main:app) takes precedence over - # startModule (python -m ...). uvicorn is run via the venv python's -m so no - # PATH juggling is needed, and it gets zstart's bind-host + dev port. - if [ -n "$app" ]; then - run_cmd=("$exe" -m uvicorn "$app" --host "$bind_host" --port "$port" --reload) - what="uvicorn $app" - else - run_cmd=("$exe" -m "$module") - what="python -m $module" - fi - - printf '\n'; info "=== zstart ($(zproj "$key" .label)) ===" - # No project venv found - warn, since the system interpreter usually lacks the - # app's deps (the failure would otherwise be a cryptic ModuleNotFoundError). - if [ "$using_venv" -eq 0 ]; then - warn "No project venv at $root/.venv - using system '$exe' (its deps may be missing)." - dim " Create one: (cd \"$root\" && python3 -m venv .venv && .venv/bin/pip install -e .)" - fi - info "Starting $what on port $port..." - [ -n "$bv" ] && note "Build Version: $bv" - show_project_motd "$root" - dim "Press Ctrl+C to stop the server" - warn_if_privileged_port "$port" - printf '\n' - - if [ "$detached" -eq 1 ]; then - detach "$key" "$root" "${run_cmd[@]}" - else - ( cd "$root" && exec "${run_cmd[@]}" ) - fi -} - -start_vite() { # - local key="$1" port="$2" root - root="$(zproj_root "$key")" - [ -d "$root" ] || { err "Project root not found: $root"; return 1; } - [ -f "$root/package.json" ] || { err "package.json not found in $root"; return 1; } - - printf '\n'; info "=== zstart ($(zproj "$key" .label) - Vite) ===" - dim "Directory: $root" - info "URL: http://${bind_host}:${port}/" - printf '\n' - - if [ ! -d "$root/node_modules" ]; then - warn "node_modules missing - running npm install..." - (cd "$root" && npm install) || { err "npm install failed"; return 1; } - fi - - show_project_motd "$root" - dim "Press Ctrl+C to stop the dev server" - warn_if_privileged_port "$port" - printf '\n' - - if [ "$detached" -eq 1 ]; then - detach "$key" "$root" npm run dev -- --host "$bind_host" --port "$port" - else - ( cd "$root" && exec npm run dev -- --host "$bind_host" --port "$port" ) - fi -} - -start_next() { # - local key="$1" port="$2" root - root="$(zproj_root "$key")" - [ -d "$root" ] || { err "Project root not found: $root"; return 1; } - [ -f "$root/package.json" ] || { err "package.json not found in $root"; return 1; } - - printf '\n'; info "=== zstart ($(zproj "$key" .label) - Next.js) ===" - info "Starting on port $port..." - printf '\n' - show_project_motd "$root" - dim "Press Ctrl+C to stop the server" - warn_if_privileged_port "$port" - printf '\n' - - export PORT="$port" - if [ "$detached" -eq 1 ]; then - detach "$key" "$root" npm run dev - else - ( cd "$root" && exec npm run dev ) - fi -} - -rc=0 -for key in "${projects[@]}"; do - zproj_require "$key" - kind="$(zproj "$key" .kind)" - if [ "$port_override" -gt 0 ] 2>/dev/null; then port="$port_override" - else port="$(zproj "$key" .ports.dev)"; [[ "$port" =~ ^[0-9]+$ ]] || port=3000; fi - - start_prep "$key" - - case "$kind" in - python) start_python "$key" "$port" || rc=1 ;; - vite) start_vite "$key" "$port" || rc=1 ;; - nextjs) start_next "$key" "$port" || rc=1 ;; - *) - printf '\n' - warn "Project '$key' has kind '$kind' - no local dev server to start." - dim "Supported kinds for zstart: python, vite, nextjs" - ;; - esac -done -exit "$rc" diff --git a/bash/zstart_docker b/bash/zstart_docker deleted file mode 100644 index 86f2903..0000000 --- a/bash/zstart_docker +++ /dev/null @@ -1,66 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zstart_docker — bring up a local docker compose stack from /docker/. -# -# Usage: -# zstart_docker [--build] [--attached] [--solo] -# -# --build rebuild images before starting -# --attached stream logs in the foreground (default is detached) -# --solo use docker-compose.solo.yml (app only, no proxy) - -set -uo pipefail -_HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -source "$_HERE/zhelpers.sh" -z_track_start "$@" -z_require docker - -build=0; attached=0; solo=0 -while [ $# -gt 0 ]; do - case "$1" in - --build) build=1; shift ;; - --attached) attached=1; shift ;; - --solo) solo=1; shift ;; - *) err "Unknown option: $1"; exit 1 ;; - esac -done - -info "=== zstart_docker ===" - -compose_file="docker-compose.yml" -[ "$solo" -eq 1 ] && compose_file="docker-compose.solo.yml" - -if ! docker info >/dev/null 2>&1; then - printf '\n' - err "ERROR: Docker Engine is not running or this shell cannot reach it." - printf '\n' - info "Try:" - dim " 1. Start the Docker daemon (systemctl start docker / open Docker Desktop)." - dim " 2. Check your user is in the docker group: groups | grep docker" - printf '\n' - exit 1 -fi - -compose_path="$_HERE/docker/$compose_file" -if [ ! -f "$compose_path" ]; then - err "ERROR: Missing $compose_path" - exit 1 -fi - -[ -f "$_HERE/.env" ] || warn "WARNING: .env not found at $_HERE/.env - compose may still start if env vars are set elsewhere." - -dc_args=(compose -f "$compose_file" up) -[ "$attached" -eq 0 ] && dc_args+=(-d) -[ "$build" -eq 1 ] && dc_args+=(--build) - -dim "Running: docker ${dc_args[*]}" -(cd "$_HERE/docker" && docker "${dc_args[@]}") || exit $? - -printf '\n'; ok "Stack: http://localhost/" -if [ "$attached" -eq 0 ]; then - dim 'Logs: docker compose -f docker/docker-compose.yml logs -f' - dim 'Stop: docker compose -f docker/docker-compose.yml down' -fi diff --git a/bash/zstop b/bash/zstop deleted file mode 100644 index ccb5bd8..0000000 --- a/bash/zstop +++ /dev/null @@ -1,38 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zstop — stop docker compose stacks on the server without removing data or files. -# -# Usage: -# zstop [ ...] -# -# Data volumes are preserved. Run zdeploy to bring a stack back up. - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config -z_require ssh - -if [ $# -eq 0 ]; then - printf '\n'; warn "Usage: zstop [ ...]" - dim " Projects in zconfig.json: $(zproj_csv)" - exit 1 -fi - -for key in "$@"; do - zproj_require "$key" - label="$(zproj "$key" .label)"; [ -n "$label" ] || label="$key" - compose_dir="$(zremote_compose_dir "$key")" - - printf '\n'; info "--- Stopping $label ---" - if zssh "cd $compose_dir && sudo docker compose down 2>&1 || true"; then - ok " $label stopped." - else - warn " WARNING: docker compose down returned non-zero for $label" - fi -done - -printf '\n'; dim "Done. Data volumes are intact. Run zdeploy to restart." diff --git a/bash/zsync b/bash/zsync deleted file mode 100644 index b96b6bc..0000000 --- a/bash/zsync +++ /dev/null @@ -1,97 +0,0 @@ -#!/usr/bin/env bash -# Evomedia.net Token Savers — https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels · dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# zsync — copy new backup files offsite; or build + mirror a vite project's dist. -# -# Usage: -# zsync # new files: backups folder -> paths.oneDriveBackups -# zsync --dest /mnt/mirror/dist -# -# The no-args mode copies only files that don't exist at the destination yet -# (never overwrites, never deletes). The project mode runs npm run build, then -# mirrors dist/ to --dest (or $ZSYNC_DEST) with rsync --delete. - -set -uo pipefail -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/zhelpers.sh" -z_track_start "$@" -z_need_config - -projects=(); dest="${ZSYNC_DEST:-}" -while [ $# -gt 0 ]; do - case "$1" in - --dest) dest="${2:-}"; shift 2 ;; - -*) err "Unknown option: $1"; exit 1 ;; - *) projects+=("$1"); shift ;; - esac -done - -# ---- no-args mode: backups -> offsite, new files only ------------------------ -if [ "${#projects[@]}" -eq 0 ]; then - BACKUPS_ROOT="$(dirname "$(z_path "$(zq '.paths.backupsLocal')")")" - OFFSITE_ROOT="$(z_path "$(zq '.paths.oneDriveBackups')")" - - printf '\n'; info "=== zsync (backups -> offsite) ===" - dim " Source: $BACKUPS_ROOT" - dim " Destination: $OFFSITE_ROOT" - dim " Mode: new files and folders only" - printf '\n' - - if [ -z "$OFFSITE_ROOT" ]; then - warn " paths.oneDriveBackups is not set in zconfig.json - nothing to do." - exit 0 - fi - if [ ! -d "$BACKUPS_ROOT" ]; then - warn " Source not found (skipped): $BACKUPS_ROOT" - exit 0 - fi - mkdir -p "$OFFSITE_ROOT" - - copied=0; skipped=0 - while IFS= read -r -d '' f; do - rel="${f#"$BACKUPS_ROOT"/}" - target="$OFFSITE_ROOT/$rel" - if [ -e "$target" ]; then skipped=$((skipped + 1)); continue; fi - mkdir -p "$(dirname "$target")" - cp -p "$f" "$target" - ok " Copied: $rel ($(z_size_mb "$f") MB)" - copied=$((copied + 1)) - done < <(find "$BACKUPS_ROOT" -type f -print0 2>/dev/null) - - printf '\n'; info " Done: $copied copied, $skipped already present" - printf '\n' - exit 0 -fi - -# ---- project mode: build + mirror a vite project's dist ---------------------- -z_require rsync -for key in "${projects[@]}"; do - zproj_require "$key" - kind="$(zproj "$key" .kind)" - if [ "$kind" != "vite" ]; then - printf '\n'; warn "zsync project mode only supports vite kind (builds and mirrors dist/). '$key' is kind '$kind'." - exit 1 - fi - root="$(zproj_root "$key")" - - if [ -z "$dest" ]; then - printf '\n'; info "=== zsync ($(zproj "$key" .label)) ===" - warn "No destination set." - dim "Set ZSYNC_DEST or run: zsync $key --dest /path/to/folder" - dim "This runs npm run build, then mirrors dist -> destination (rsync --delete)." - exit 1 - fi - - printf '\n'; info "=== zsync ($(zproj "$key" .label)) ===" - info "Build + mirror dist -> $dest" - printf '\n' - - (cd "$root" && npm run build) || { err "npm run build failed"; exit 1; } - [ -d "$root/dist" ] || { err "dist/ missing after build."; exit 1; } - - mkdir -p "$dest" - rsync -a --delete "$root/dist/" "$dest/" || { err "rsync failed"; exit 1; } - - printf '\n'; ok "Sync complete." -done diff --git a/tests/bash/args.bats b/tests/bash/args.bats deleted file mode 100644 index 0f0c8a9..0000000 --- a/tests/bash/args.bats +++ /dev/null @@ -1,232 +0,0 @@ -#!/usr/bin/env bats -# Evomedia.net Token Savers - https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels - dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# args.bats - how the bash scripts read their target arguments. -# -# bats tests/bash -# -# Mirrors tests/ArgumentParsing.Tests.ps1. Same safety rule as the PowerShell -# side: these run the real scripts, so they stay on code paths that exit BEFORE -# doing any work (no args, unknown key). Only zkill runs with a valid target, -# and only because the fixture's dev ports are deliberately unused and its -# localRoots do not exist - it finds no listeners and kills nothing. zdeploy, -# zbackup_ec2, zec2, zec2online, zrepair, zstop, zstart and zbackup are never -# invoked with a real target; that needs a disposable server. - -setup() { - REPO="$(cd "$(dirname "$BATS_TEST_FILENAME")/../.." && pwd)" - BASHDIR="$REPO/bash" - export ZCONFIG="$BATS_TEST_TMPDIR/zconfig.json" - cat > "$ZCONFIG" <<'JSON' -{ - "ec2": { "ip": "203.0.113.10", "user": "testuser", "pemKey": "/fixtures/test.pem", "stackRoot": "/home/testuser/stack" }, - "paths": { "temp": "/fixtures/temp", "backupsLocal": "/fixtures/backups", "backupsEc2": "/fixtures/backups-ec2", "scriptsRoot": "/fixtures/zscripts", "oneDriveBackups": "" }, - "projects": { - "_note": "comment key - must never be treated as a project", - "pyapp": { - "label": "Fixture Python App", "kind": "python", "localRoot": "/fixtures/nonexistent-pyapp", - "startModule": "pyapp.main", "ports": { "dev": 59990 }, - "remote": { "path": "/home/testuser/stack/pyapp" } - }, - "viteapp": { - "label": "Fixture Vite Site", "kind": "vite", "localRoot": "/fixtures/nonexistent-viteapp", - "ports": { "dev": 59991 }, "remote": { "path": "/home/testuser/stack/viteapp" } - }, - "edgeproxy": { - "label": "Fixture Edge", "kind": "edge", "localRoot": "/fixtures/nonexistent-edge", - "remote": { "path": "/home/testuser/stack/edge" } - } - } -} -JSON -} - -# ---- no arguments ----------------------------------------------------------- -# Running bare must never mean "do it to everything". - -@test "zkill with no args shows usage and exits 1" { - run "$BASHDIR/zkill" - [ "$status" -eq 1 ] - [[ "$output" == *"Usage"* ]] -} - -@test "zstart with no args shows usage and exits non-zero" { - run "$BASHDIR/zstart" - [ "$status" -ne 0 ] - [[ "$output" == *"Usage"* ]] -} - -@test "zdeploy with no args shows usage and exits non-zero" { - run "$BASHDIR/zdeploy" - [ "$status" -ne 0 ] - [[ "$output" == *"Usage"* ]] -} - -@test "zbackup with no args shows usage and exits non-zero" { - run "$BASHDIR/zbackup" - [ "$status" -ne 0 ] - [[ "$output" == *"Usage"* ]] -} - -@test "usage lists the configured project keys" { - run "$BASHDIR/zkill" - [[ "$output" == *"pyapp"* ]] - [[ "$output" == *"viteapp"* ]] -} - -@test "usage never advertises the underscore comment key" { - run "$BASHDIR/zkill" - [[ "$output" != *"_note"* ]] -} - -# ---- unknown key ------------------------------------------------------------ -# Exiting 0 having selected nothing is the dangerous outcome. - -@test "zkill rejects an unknown project key" { - run "$BASHDIR/zkill" definitelynotaproject - [ "$status" -ne 0 ] - [[ "$output" == *"Unknown project key"* ]] -} - -@test "zstart rejects an unknown project key" { - run "$BASHDIR/zstart" definitelynotaproject - [ "$status" -ne 0 ] -} - -@test "zdeploy rejects an unknown project key" { - run "$BASHDIR/zdeploy" definitelynotaproject - [ "$status" -ne 0 ] -} - -@test "the unknown-key error names both the bad key and the valid ones" { - run "$BASHDIR/zkill" definitelynotaproject - [[ "$output" == *"definitelynotaproject"* ]] - [[ "$output" == *"pyapp"* ]] -} - -@test "the underscore comment key is not addressable as a project" { - run "$BASHDIR/zkill" _note - [ "$status" -ne 0 ] -} - -# The convention is that ANY "_"-prefixed key is a comment, not a project - -# PowerShell's Get-ZProject refuses them outright. A string-valued comment is -# also caught by the "is it an object?" check, so this uses an OBJECT-valued -# underscore key (e.g. someone keeping a "_template" block to copy from) to -# prove the prefix rule itself is enforced. -@test "an object-valued underscore key is still not addressable" { - cat > "$BATS_TEST_TMPDIR/objkey.json" <<'JSON' -{ - "ec2": { "ip": "203.0.113.10", "user": "testuser", "pemKey": "/f/t.pem", "stackRoot": "/s" }, - "paths": { "temp": "/f/t", "backupsLocal": "/f/b", "backupsEc2": "/f/be", "scriptsRoot": "/f/z", "oneDriveBackups": "" }, - "projects": { - "_template": { - "label": "Template To Copy", "kind": "python", "localRoot": "/fixtures/none", - "ports": { "dev": 59993 }, "remote": { "path": "/s/t" } - }, - "pyapp": { - "label": "Fixture Python App", "kind": "python", "localRoot": "/fixtures/nonexistent-pyapp", - "ports": { "dev": 59990 }, "remote": { "path": "/s/py" } - } - } -} -JSON - ZCONFIG="$BATS_TEST_TMPDIR/objkey.json" run "$BASHDIR/zkill" _template - [ "$status" -ne 0 ] - [[ "$output" == *"Unknown project key"* ]] -} - -@test "an object-valued underscore key is excluded from 'all'" { - cat > "$BATS_TEST_TMPDIR/objkey2.json" <<'JSON' -{ - "ec2": { "ip": "203.0.113.10", "user": "testuser", "pemKey": "/f/t.pem", "stackRoot": "/s" }, - "paths": { "temp": "/f/t", "backupsLocal": "/f/b", "backupsEc2": "/f/be", "scriptsRoot": "/f/z", "oneDriveBackups": "" }, - "projects": { - "_template": { - "label": "Template To Copy", "kind": "python", "localRoot": "/fixtures/none", - "ports": { "dev": 59993 }, "remote": { "path": "/s/t" } - }, - "pyapp": { - "label": "Fixture Python App", "kind": "python", "localRoot": "/fixtures/nonexistent-pyapp", - "ports": { "dev": 59990 }, "remote": { "path": "/s/py" } - } - } -} -JSON - ZCONFIG="$BATS_TEST_TMPDIR/objkey2.json" run "$BASHDIR/zkill" all - [ "$status" -eq 0 ] - [[ "$output" == *"Fixture Python App"* ]] - [[ "$output" != *"Template To Copy"* ]] -} - -# ---- leading dash: CURRENT bash behaviour ------------------------------------ -# NOTE: this is a known divergence from the PowerShell port, pinned here as it -# actually behaves rather than as it ideally would. PowerShell tolerates a -# leading dash on any project key (zdeploy -myapp == zdeploy myapp). In bash, -# only zdeploy strips it ("${1#-}"); every other script treats -myapp as an -# unknown option and exits 1. These tests document today's behaviour so a fix -# is a deliberate change with a failing test to update, not a silent surprise. - -@test "bash zdeploy strips a leading dash from a project key" { - # Unknown key keeps this on the no-work path; the error must name the key - # without its dash, proving the strip happened. - run "$BASHDIR/zdeploy" -definitelynotaproject - [ "$status" -ne 0 ] - [[ "$output" == *"definitelynotaproject"* ]] - [[ "$output" != *"-definitelynotaproject"* ]] -} - -@test "bash zkill rejects a dashed key as an unknown option (diverges from PowerShell)" { - run "$BASHDIR/zkill" -pyapp - [ "$status" -eq 1 ] - [[ "$output" == *"Unknown option"* ]] -} - -@test "bash zstart rejects a dashed key as an unknown option (diverges from PowerShell)" { - run "$BASHDIR/zstart" -pyapp - [ "$status" -ne 0 ] - [[ "$output" == *"Unknown option"* ]] -} - -# ---- zkill target resolution (safe: fixture ports are unused) ---------------- - -@test "zkill accepts a bare project key" { - run "$BASHDIR/zkill" pyapp - [ "$status" -eq 0 ] - [[ "$output" == *"Fixture Python App"* ]] - [[ "$output" == *"59990"* ]] -} - -@test "zkill accepts several keys at once" { - run "$BASHDIR/zkill" pyapp viteapp - [ "$status" -eq 0 ] - [[ "$output" == *"Fixture Python App"* ]] - [[ "$output" == *"Fixture Vite Site"* ]] -} - -@test "zkill 'all' expands to every project with a dev port" { - run "$BASHDIR/zkill" all - [ "$status" -eq 0 ] - [[ "$output" == *"Fixture Python App"* ]] - [[ "$output" == *"Fixture Vite Site"* ]] -} - -@test "zkill 'all' skips projects with no dev port (edge/docker stacks)" { - run "$BASHDIR/zkill" all - [[ "$output" != *"Fixture Edge"* ]] -} - -@test "zkill --port overrides the configured dev port" { - run "$BASHDIR/zkill" pyapp --port 59999 - [ "$status" -eq 0 ] - [[ "$output" == *"59999"* ]] - [[ "$output" != *"59990"* ]] -} - -@test "zkill announces that --kill-all is not implemented rather than silently ignoring it" { - run "$BASHDIR/zkill" pyapp --kill-all - [ "$status" -eq 0 ] - [[ "$output" == *"kill-all"* ]] -} diff --git a/tests/bash/helpers.bats b/tests/bash/helpers.bats deleted file mode 100644 index 85d999b..0000000 --- a/tests/bash/helpers.bats +++ /dev/null @@ -1,237 +0,0 @@ -#!/usr/bin/env bats -# Evomedia.net Token Savers - https://github.com/kellymichels/zscripts-token-savers -# Created by Kelly Michels - dev@evomedia.net -# Licensed under the MIT License. See LICENSE. -# -# helpers.bats - coverage for the pure/config functions in bash/zhelpers.sh. -# -# bats tests/bash -# -# The bash port has its own implementation of the exclude lists and config -# accessors, so it can drift from the PowerShell side independently. These -# mirror tests/ZHelpers.Tests.ps1 assertion-for-assertion where the two are -# meant to agree. -# -# A fixture config is injected through ZCONFIG (zhelpers.sh has always honored -# it), so no real zconfig.json is read. - -setup() { - REPO="$(cd "$(dirname "$BATS_TEST_FILENAME")/../.." && pwd)" - export ZCONFIG="$BATS_TEST_TMPDIR/zconfig.json" - cat > "$ZCONFIG" <<'JSON' -{ - "ec2": { "ip": "203.0.113.10", "user": "testuser", "pemKey": "/fixtures/test.pem", "stackRoot": "/home/testuser/stack" }, - "paths": { "temp": "/fixtures/temp", "backupsLocal": "/fixtures/backups", "backupsEc2": "/fixtures/backups-ec2", "scriptsRoot": "/fixtures/zscripts", "oneDriveBackups": "" }, - "projects": { - "_note": "comment key - must never be treated as a project", - "pyapp": { - "label": "Fixture Python App", "kind": "python", "localRoot": "/fixtures/pyapp", - "startModule": "pyapp.main", "ports": { "dev": 59990 }, "domain": "pyapp.example.com", - "remote": { "path": "/home/testuser/stack/pyapp", "composeDir": "/home/testuser/stack/pyapp/docker", "appService": "app" }, - "deploy": { "zipName": "PyAppDeploy.zip", "exclude": ["docs", "fixtures-extra"] } - }, - "viteapp": { - "label": "Fixture Vite Site", "kind": "vite", "localRoot": "/fixtures/viteapp", - "ports": { "dev": 59991 }, "domain": "www.example.com", - "remote": { "path": "/home/testuser/stack/viteapp" } - }, - "nextapp": { - "label": "Fixture Next App", "kind": "nextjs", "localRoot": "/fixtures/nextapp", - "ports": { "dev": 59992 }, "remote": { "path": "/home/testuser/stack/nextapp" } - }, - "edgeproxy": { - "label": "Fixture Edge", "kind": "edge", "localRoot": "/fixtures/edge", - "proxyContainer": "edge_proxy", "remote": { "path": "/home/testuser/stack/edge" } - }, - "noports": { - "label": "Fixture No Dev Port", "kind": "python", "localRoot": "/fixtures/noports", - "remote": { "path": "/home/testuser/stack/noports" } - } - } -} -JSON - # shellcheck disable=SC1090 - source "$REPO/bash/zhelpers.sh" -} - -# Emit the exclude list for a project; $2=1 means "for backup". -excludes() { z_archive_excludes "$1" "${2:-0}"; } - -# Fixed-string, whole-line match. -F matters: without it grep treats the needle -# as a regex, and '.env' would match 'venv' (leading '.' = any char) - which -# silently turns every ".env is excluded" assertion into a false pass. -has() { printf '%s\n' "$1" | grep -qxF "$2"; } - -# ---- config accessors ------------------------------------------------------- - -@test "zproj_keys returns every project in config order" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj_keys | paste -sd, -" - [ "$status" -eq 0 ] - [ "$output" = "pyapp,viteapp,nextapp,edgeproxy,noports" ] -} - -@test "zproj_keys omits underscore comment keys" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj_keys" - [ "$status" -eq 0 ] - ! has "$output" "_note" -} - -@test "zproj reads a project field" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj pyapp .label" - [ "$output" = "Fixture Python App" ] -} - -@test "zproj returns empty for a missing field rather than the string null" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj viteapp .startModule" - [ "$output" = "" ] -} - -@test "zec2_target composes user@ip" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zec2_target" - [ "$output" = "testuser@203.0.113.10" ] -} - -@test "zremote_compose_dir prefers composeDir" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zremote_compose_dir pyapp" - [ "$output" = "/home/testuser/stack/pyapp/docker" ] -} - -@test "zremote_compose_dir falls back to remote.path" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zremote_compose_dir viteapp" - [ "$output" = "/home/testuser/stack/viteapp" ] -} - -@test "zedge_key finds the first edge-kind project" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zedge_key" - [ "$output" = "edgeproxy" ] -} - -@test "zproj_keys_with_domain lists only projects that have a domain" { - run bash -c "source '$REPO/bash/zhelpers.sh'; zproj_keys_with_domain | paste -sd, -" - [ "$output" = "pyapp,viteapp" ] -} - -# ---- z_path (Windows -> host path translation) ------------------------------ -# A shared zconfig.json may carry Windows paths when the same file is used from -# a Windows checkout and from WSL. - -@test "z_path translates a Windows drive path to /mnt form" { - run bash -c "source '$REPO/bash/zhelpers.sh'; z_path 'F:\\evomedia.net\\app'" - [ "$output" = "/mnt/f/evomedia.net/app" ] -} - -@test "z_path lowercases the drive letter" { - run bash -c "source '$REPO/bash/zhelpers.sh'; z_path 'C:\\Users\\Someone'" - [ "$output" = "/mnt/c/Users/Someone" ] -} - -@test "z_path leaves a unix path untouched (no-op on native Linux/macOS)" { - run bash -c "source '$REPO/bash/zhelpers.sh'; z_path '/home/kelly/project'" - [ "$output" = "/home/kelly/project" ] -} - -@test "z_path leaves an empty string empty" { - run bash -c "source '$REPO/bash/zhelpers.sh'; z_path ''" - [ "$output" = "" ] -} - -# ---- z_archive_excludes ----------------------------------------------------- -# Parity target: tests/ZHelpers.Tests.ps1 "Get-ArchiveExcludes". - -@test "excludes: common junk applies to every kind" { - run excludes pyapp - for n in .git .idea .vscode .claude tmp nul .DS_Store backups; do - has "$output" "$n" || { echo "missing common exclude: $n"; return 1; } - done -} - -@test "excludes: python build/venv cruft" { - run excludes pyapp - for n in .venv venv __pycache__ .pytest_cache .nicegui archive dist build htmlcov; do - has "$output" "$n" || { echo "missing python exclude: $n"; return 1; } - done -} - -@test "excludes: vite drops node_modules and dist" { - run excludes viteapp - has "$output" "node_modules" - has "$output" "dist" -} - -@test "excludes: nextjs drops .next and .vercel" { - run excludes nextapp - has "$output" ".next" - has "$output" ".vercel" - has "$output" "next-env.d.ts" -} - -# The regression this suite exists for: a dev .env shipped to production in a -# deploy zip. Deploys drop secrets and uploads; backups must keep them or the -# backup is not a restore point. -@test "excludes: python DEPLOY drops .env, .env.local, .env.production and uploads" { - run excludes pyapp 0 - for n in .env .env.local .env.production uploads; do - has "$output" "$n" || { echo "deploy should exclude: $n"; return 1; } - done -} - -@test "excludes: python BACKUP keeps .env, .env.local, .env.production and uploads" { - run excludes pyapp 1 - for n in .env .env.local .env.production uploads; do - if has "$output" "$n"; then echo "backup must NOT exclude: $n"; return 1; fi - done -} - -@test "excludes: python BACKUP still drops the virtualenv (bulk, not a secret)" { - run excludes pyapp 1 - has "$output" ".venv" -} - -@test "excludes: vite DEPLOY drops .env files" { - run excludes viteapp 0 - for n in .env .env.local .env.production; do - has "$output" "$n" || { echo "deploy should exclude: $n"; return 1; } - done -} - -@test "excludes: vite BACKUP keeps .env files" { - run excludes viteapp 1 - for n in .env .env.local .env.production; do - if has "$output" "$n"; then echo "backup must NOT exclude: $n"; return 1; fi - done -} - -@test "excludes: vite BACKUP still drops node_modules" { - run excludes viteapp 1 - has "$output" "node_modules" -} - -# nextjs has no backup gate on the PowerShell side either - both agree. -@test "excludes: nextjs drops .env in a backup too (no gate, matches PowerShell)" { - run excludes nextapp 1 - has "$output" ".env" -} - -@test "excludes: the project's own deploy.exclude entries are merged in" { - run excludes pyapp - has "$output" "docs" - has "$output" "fixtures-extra" -} - -@test "excludes: a kind with no specific list still gets the common entries" { - run excludes edgeproxy - has "$output" ".git" -} - -# ---- json_build_label ------------------------------------------------------- - -@test "json_build_label formats v." { - printf '{ "productVersion": "1.0", "buildNumber": 42 }' > "$BATS_TEST_TMPDIR/bv.json" - run bash -c "source '$REPO/bash/zhelpers.sh'; json_build_label '$BATS_TEST_TMPDIR/bv.json'" - [ "$output" = "v1.0.42" ] -} - -@test "json_build_label reports unknown for a missing file" { - run bash -c "source '$REPO/bash/zhelpers.sh'; json_build_label '$BATS_TEST_TMPDIR/nope.json'" - [ "$output" = "unknown" ] -}