From 732a448be5d2d8c8ca9311c6934db8235c5b7051 Mon Sep 17 00:00:00 2001 From: Kelly Michels Date: Sat, 12 Sep 2026 15:52:18 -0500 Subject: [PATCH] =?UTF-8?q?feat:=20add=20zmerge=20and=20zpull=20=E2=80=94?= =?UTF-8?q?=20fleet-wide=20PR=20merging=20and=20checkout=20sync=20(#79)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat: add zmerge and zpull Two commands that were private-only until now. They turned out to be useful beyond the fleet they were written for, so they are manifested for publication and removed from the sanitization denylist's private-only list. zmerge merge every pull request across the org that is genuinely ready - MERGEABLE/CLEAN and not a draft - re-checking each one immediately before and after every merge, because merging into a default branch can conflict a sibling PR in the same repository. Dry run by default; -Execute or -e merges. zpull zmerge, then git pull --ff-only in every checkout the merges affected. Skips a checkout that is dirty or is not on its default branch rather than guessing at it. WHY THEY COULD BE PUBLISHED NOW. zmerge carried a hardcoded list of sixteen repository names, which was both the reason it could not be published and a bug: the org has thirty active repositories, so it scanned about half and reported "Nothing open to merge" while a ready pull request sat in one it had never heard of. It asks GitHub now, and the names went with the list. Get-FleetRepos throws rather than returning an empty list when gh fails, because a tool that quietly scans nothing prints the same reassuring line as one that scanned everything and found nothing. -e is an alias for -Execute on both, the way -s already works for -Scan. Also: __pycache__/ is gitignored. scripts/plaintext_twins.py creates it on every run and it was showing up as untracked work. Co-Authored-By: Claude Opus 5 * fix: CRLF the new scripts, as .gitattributes pins them zmerge.ps1, zpull.ps1 and zpull.cmd went in with LF endings. .gitattributes pins *.ps1 and *.cmd to eol=crlf precisely so CHECKSUMS.txt can hold one hash per file rather than one per platform - so git handed CI a CRLF checkout while the manifest carried hashes taken from my LF copies, and the three new files were the only ones that failed. Local verification passed and CI did not, which is the tell: the manifest was generated against bytes that only existed on this machine. Co-Authored-By: Claude Opus 5 --------- Co-authored-by: Claude Opus 5 --- .gitignore | 3 + CHANGELOG.md | 23 ++ CHANGELOG.txt | 25 +++ CHECKSUMS.txt | 3 + README.md | 2 + README.txt | 2 + tests/sanitization-patterns.psd1 | 2 +- zmerge.ps1 | 251 +++++++++++++++++++++ zpull.cmd | 6 + zpull.ps1 | 359 +++++++++++++++++++++++++++++++ 10 files changed, 675 insertions(+), 1 deletion(-) create mode 100644 zmerge.ps1 create mode 100644 zpull.cmd create mode 100644 zpull.ps1 diff --git a/.gitignore b/.gitignore index e22a1d2..f34b514 100644 --- a/.gitignore +++ b/.gitignore @@ -19,3 +19,6 @@ md/ # Pester coverage output (regenerated; never committed) coverage/ + +# Generated by scripts/plaintext_twins.py +__pycache__/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a7fec2..50fbc39 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,29 @@ Notable changes to the Evomedia.net Token Savers. ## Unreleased +### Added + +- **`zmerge`** — merge every pull request across the org that is genuinely + ready (`MERGEABLE` / `CLEAN`, not a draft), re-checking each one immediately + before and after every merge, because merging into a default branch can + conflict a sibling PR in the same repository. Dry run by default; + `-Execute` (or `-e`) merges. +- **`zpull`** — `zmerge`, then `git pull --ff-only` in every checkout the + merges affected. Skips a checkout that is dirty or is not on its default + branch rather than guessing. + +### Changed + +- **`-e` is an alias for `-Execute`** on both of the above, the way `-s` + already works for `-Scan`. +- **`zmerge` discovers repositories instead of listing them.** It asked a + hand-kept list, which had fallen well behind the org - so a scan covered + about half of it and reported "Nothing open to merge" while a ready pull + request sat in a repository the list had never heard of. It now asks GitHub, + and throws rather than returning an empty list if that fails: a tool that + quietly scans nothing prints the same reassuring line as one that scanned + everything, and the two must not be confusable. + ## v1.0.0.0.24 - 2026-09-08 ### Added diff --git a/CHANGELOG.txt b/CHANGELOG.txt index 72860b7..7070510 100644 --- a/CHANGELOG.txt +++ b/CHANGELOG.txt @@ -10,6 +10,31 @@ Notable changes to the Evomedia.net Token Savers. Unreleased ---------- +Added +----- + +- zmerge — merge every pull request across the org that is genuinely + ready (MERGEABLE / CLEAN, not a draft), re-checking each one immediately + before and after every merge, because merging into a default branch can + conflict a sibling PR in the same repository. Dry run by default; + -Execute (or -e) merges. +- zpull — zmerge, then git pull --ff-only in every checkout the + merges affected. Skips a checkout that is dirty or is not on its default + branch rather than guessing. + +Changed +------- + +- -e is an alias for -Execute on both of the above, the way -s + already works for -Scan. +- zmerge discovers repositories instead of listing them. It asked a + hand-kept list, which had fallen well behind the org - so a scan covered + about half of it and reported "Nothing open to merge" while a ready pull + request sat in a repository the list had never heard of. It now asks GitHub, + and throws rather than returning an empty list if that fails: a tool that + quietly scans nothing prints the same reassuring line as one that scanned + everything, and the two must not be confusable. + v1.0.0.0.24 - 2026-09-08 ------------------------ diff --git a/CHECKSUMS.txt b/CHECKSUMS.txt index 910794f..0ef03f6 100644 --- a/CHECKSUMS.txt +++ b/CHECKSUMS.txt @@ -20,6 +20,9 @@ e904b06017619f0ea13a79c34f56c70e642a5f0aae7dfab55159885aac2ac384 ZHelpers.ps1 1eb4c23bdc0ed1a82dc495c623b49e5dcd4e342f026b4d896e32c79d592667db zkill.ps1 1c908b69fb9200610e080ac6bb8f4c15d3d7edf402d2e119c2405158e1986a52 ZKiller.ps1 558dfdfc7b4d12231e476c14a00c678e2e536140c4b7abbc05364bf214562291 ZKillOnly.ps1 +b42899ffd20c0b287af0072199d52a945b82123c86b9969cbf68b78b9f81f3d5 zmerge.ps1 +b8f89795ccb1f31e9a4feb8abdaa04e764f361b2d2db5e3e13d9ca839afdbf75 zpull.cmd +628d26a64a4765667627db199fee7a0ca80aadebd3299350fa742e66bcd83580 zpull.ps1 36b01f2cd1d5967dc3ec1313fc4f82ada2ea669d0529a0d313300e21a5b38d04 zrelease.cmd e48b994605b1ad9f793f95d653f50df41db628a6b2c6976415b7d03e3900a373 zrelease.ps1 02635351847f84d0f644ffa9c0073804e480d360fcabcd6a8b793cffd6f0026f zrepair.cmd diff --git a/README.md b/README.md index cc12b5c..53fdeac 100644 --- a/README.md +++ b/README.md @@ -136,6 +136,8 @@ The `.cmd` wrappers are the everyday interface. Every command takes one or more | `zchecksums [-Update]` | Verify every script against `CHECKSUMS.txt` (SHA-256) | | `zversion [bump \| bump-stage \| set ]` | Show or advance the toolkit version (stamps every header) | | `zrelease [-Verify]` | Package the current version as `releases/zscripts-.zip` + `.sha256` | +| `zmerge [-Execute\|-e]` | Merge every pull request across the org that is genuinely ready | +| `zpull [-Execute\|-e]` | `zmerge`, then bring every affected local checkout current | ### Local development diff --git a/README.txt b/README.txt index f6d0740..9f85b98 100644 --- a/README.txt +++ b/README.txt @@ -133,6 +133,8 @@ The .cmd wrappers are the everyday interface. Every command takes one or more pr | zchecksums [-Update] | Verify every script against CHECKSUMS.txt (SHA-256) | | zversion [bump \| bump-stage \| set ] | Show or advance the toolkit version (stamps every header) | | zrelease [-Verify] | Package the current version as releases/zscripts-.zip + .sha256 | +| zmerge [-Execute\|-e] | Merge every pull request across the org that is genuinely ready | +| zpull [-Execute\|-e] | zmerge, then bring every affected local checkout current | Local development ----------------- diff --git a/tests/sanitization-patterns.psd1 b/tests/sanitization-patterns.psd1 index 50a191f..744b33d 100644 --- a/tests/sanitization-patterns.psd1 +++ b/tests/sanitization-patterns.psd1 @@ -31,7 +31,7 @@ @{ Name = 'current product name'; Pattern = '(?i)\bevo[.-](ehs|ai|edge|locate|proven|platform|civilcode|scripts)\b|\bevoehs' } @{ Name = 'internal issue reference'; Pattern = 'evo\.scripts#\d+' } @{ Name = 'private product domain'; Pattern = '\b(smartplantehs\.com|provensheet\.com|evoehs\.com|civilcode\.evomedia\.net|dashboard\.evomedia\.net|webmail\.evomedia\.net|mail-admin\.evomedia\.net|docketmail\.evomedia\.net|cardiff\.evomedia\.net|platform\.evomedia\.net|ai\.evomedia\.net|git\.evomedia\.net|analytics\.evomedia\.net)\b' } - @{ Name = 'private-only script'; Pattern = '\b(register_civilcode|register_docketmail|sp_seed_demo_prod|zpublish_stats|zcoverage|zmerge|zpull|zresume|swag_set_owner|provision_demo|apply_platform_config_fixes)\b' } + @{ Name = 'private-only script'; Pattern = '\b(register_civilcode|register_docketmail|sp_seed_demo_prod|zpublish_stats|zcoverage|zresume|swag_set_owner|provision_demo|apply_platform_config_fixes)\b' } @{ Name = 'local drive path'; Pattern = '[A-Za-z]:\\\\?evomedia\.net' } @{ Name = 'operator home path'; Pattern = '/home/ubuntu/' } @{ Name = 'real pem key name'; Pattern = 'evomedia-prod\.pem' } diff --git a/zmerge.ps1 b/zmerge.ps1 new file mode 100644 index 0000000..a06f5af --- /dev/null +++ b/zmerge.ps1 @@ -0,0 +1,251 @@ +# Evomedia.net Token Savers — https://github.com/evomedia-net/evo.zscripts +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. +# Version: v1.0.0.0.24 + +# zmerge.ps1 - merge the fleet's ready pull requests in one pass. +# +# Usage: +# zmerge dry run: list every open PR and its verdict +# zmerge -Execute merge everything that is genuinely ready +# zmerge -e the same; -e is an alias, as -s is for -Scan +# zmerge -Exclude 431 skip PRs by number (repeatable) +# zmerge -Repo limit to one repo +# zmerge -Only 68,67 merge just these +# zmerge -Execute -Yes skip the confirmation prompt +# +# WHY THIS EXISTS +# --------------- +# Eleven ready PRs across three repos is eleven trips through the GitHub UI, and +# the failure mode is not the clicking - it is that `gh pr create` and the merge +# button will both happily accept a PR that cannot actually merge. Mergeability +# is computed asynchronously, so a PR reports UNKNOWN for a few seconds after any +# push and CONFLICTING only later. Merging by hand, the tenth PR is the one that +# gets rubber-stamped. +# +# So this refuses to merge anything it has not just re-checked, and it re-checks +# after every merge, because merging one PR can conflict another in the same +# repo. +# +# WHAT IT WILL NOT DO +# ------------------- +# * merge a PR that is not MERGEABLE/CLEAN at the moment it is reached +# * merge a draft, or one with a failing required check +# * bump versions - each repo stamps differently (zbump for zscripts, +# bump_build_version.mjs for www), and a wrong stamp is worse than none. +# The follow-up commands are printed instead. +# * deploy anything. Deploys are run by hand, deliberately. +# +# ON UNKNOWN +# ---------- +# GitHub returns mergeable=UNKNOWN while it computes, which is indistinguishable +# from trouble if you only look once. Each PR is polled up to $PollTries times +# before being treated as not ready, so a slow answer does not read as a failure +# and a real CONFLICTING never reads as "probably fine". + +param( + [Alias('e')][switch]$Execute, + [switch]$Yes, + [int[]]$Exclude = @(), + [int[]]$Only = @(), + [string]$Repo, + [int]$PollTries = 6, + [int]$PollDelaySeconds = 4 +) + +$ErrorActionPreference = "Stop" + +# Two blank lines at the end of a run, matching every other z-script, so output +# is separated from the next prompt. Local copy rather than ZHelpers: this +# script does not dot-source it. +function Write-ZTrailer { Write-Host ""; Write-Host "" } + +# Every repository in the org, asked of GitHub rather than remembered here. +# +# Local to this script for the same reason Write-ZTrailer is: zmerge needs gh +# and nothing else, and dot-sourcing 1,200 lines of deploy helpers for one +# function would trade that away. +# +# THROWS rather than returning an empty list when gh fails. A merge tool that +# quietly scans nothing prints exactly the same reassuring line as one that +# scanned everything and found nothing, and those two must never be +# confusable - which is precisely how the list this replaced hid its own rot. +function Get-FleetRepos { + param([Parameter(Mandatory)][string]$Org) + $raw = & gh repo list $Org --limit 200 --json name,isArchived 2>&1 + if ($LASTEXITCODE -ne 0) { + throw "gh repo list $Org failed ($LASTEXITCODE): $($raw -join ' ')" + } + try { $all = $raw | ConvertFrom-Json } catch { + throw "gh repo list $Org did not return JSON: $($raw -join ' ')" + } + if (-not $all) { throw "gh repo list $Org returned no repositories" } + $names = @($all | Where-Object { -not $_.isArchived } | + ForEach-Object { $_.name } | Sort-Object) + if ($names.Count -eq 0) { throw "every repository in $Org is archived?" } + return $names +} + + + +$ORG = "evomedia-net" +# Discovered, never listed. The list this replaced had fallen fourteen +# repositories behind: a scan covered sixteen of thirty and said "Nothing open +# to merge" while a ready PR sat in one of the fourteen it could not see. +$REPOS = Get-FleetRepos -Org $ORG + +# How each repo advances its build stamp after a merge. Printed as follow-up, +# never run: see the header. +$BUMP = @{ + "evo.zscripts" = "zbump" + "evo.www" = "node scripts/bump_build_version.mjs bump (on main, then push)" +} + +function Invoke-Gh { + param([string[]]$GhArgs, [switch]$AllowFail) + $prev = $ErrorActionPreference + $ErrorActionPreference = "Continue" + try { + $out = & gh @GhArgs 2>&1 | ForEach-Object { "$_" } + $code = $LASTEXITCODE + } finally { $ErrorActionPreference = $prev } + if ($code -ne 0 -and -not $AllowFail) { + throw "gh $($GhArgs -join ' ') failed ($code): $($out -join "`n")" + } + return [pscustomobject]@{ Output = ($out -join "`n"); Code = $code } +} + +function Get-OpenPrs { + param([string]$RepoName) + $r = Invoke-Gh @("pr", "list", "-R", "$ORG/$RepoName", "--state", "open", + "--limit", "100", "--json", "number,title,isDraft,headRefName") -AllowFail + if ($r.Code -ne 0 -or -not $r.Output) { return @() } + return @($r.Output | ConvertFrom-Json) +} + +# Re-checked immediately before every merge, and again after each one, because +# merging into the default branch can conflict a sibling PR in the same repo. +function Get-Readiness { + param([string]$RepoName, [int]$Number) + for ($i = 1; $i -le $PollTries; $i++) { + $r = Invoke-Gh @("pr", "view", "$Number", "-R", "$ORG/$RepoName", + "--json", "mergeable,mergeStateStatus,state,isDraft") -AllowFail + if ($r.Code -ne 0) { return [pscustomobject]@{ Ready = $false; Why = "cannot read PR" } } + $j = $r.Output | ConvertFrom-Json + if ($j.state -ne "OPEN") { return [pscustomobject]@{ Ready = $false; Why = "state is $($j.state)" } } + if ($j.isDraft) { return [pscustomobject]@{ Ready = $false; Why = "draft" } } + if ($j.mergeable -eq "MERGEABLE" -and $j.mergeStateStatus -eq "CLEAN") { + return [pscustomobject]@{ Ready = $true; Why = "MERGEABLE/CLEAN" } + } + if ($j.mergeable -eq "CONFLICTING") { + return [pscustomobject]@{ Ready = $false; Why = "CONFLICTING - rebase it" } + } + # UNKNOWN, or a non-CLEAN state such as BLOCKED/BEHIND: give GitHub a + # moment, since it computes mergeability asynchronously. + if ($j.mergeable -ne "UNKNOWN" -and $j.mergeStateStatus -ne "UNKNOWN") { + return [pscustomobject]@{ Ready = $false; Why = "$($j.mergeable)/$($j.mergeStateStatus)" } + } + Start-Sleep -Seconds $PollDelaySeconds + } + return [pscustomobject]@{ Ready = $false; Why = "still UNKNOWN after $PollTries tries" } +} + +$targets = if ($Repo) { @($Repo) } else { $REPOS } + +Write-Host "" +Write-Host "Scanning $($targets.Count) repo(s) for open pull requests..." -ForegroundColor Cyan + +$queue = @() +foreach ($r in $targets) { + foreach ($pr in (Get-OpenPrs -RepoName $r)) { + if ($Exclude -contains $pr.number) { continue } + if ($Only.Count -gt 0 -and $Only -notcontains $pr.number) { continue } + $queue += [pscustomobject]@{ Repo = $r; Number = $pr.number; Title = $pr.title; Draft = $pr.isDraft } + } +} + +if ($queue.Count -eq 0) { Write-Host "Nothing open to merge." -ForegroundColor Yellow; Write-ZTrailer; exit 0 } + +Write-Host "" +foreach ($p in $queue) { + $v = Get-Readiness -RepoName $p.Repo -Number $p.Number + $p | Add-Member -NotePropertyName Ready -NotePropertyValue $v.Ready -Force + $p | Add-Member -NotePropertyName Why -NotePropertyValue $v.Why -Force + $mark = if ($v.Ready) { "OK " } else { "SKIP" } + $col = if ($v.Ready) { "Green" } else { "Yellow" } + Write-Host (" {0} {1,-14} #{2,-4} {3}" -f $mark, $p.Repo, $p.Number, $p.Title) -ForegroundColor $col + if (-not $v.Ready) { Write-Host (" -> {0}" -f $v.Why) -ForegroundColor DarkYellow } +} + +$ready = @($queue | Where-Object { $_.Ready }) +Write-Host "" +Write-Host "$($ready.Count) of $($queue.Count) ready to merge." -ForegroundColor Cyan + +if (-not $Execute) { + Write-Host "" + Write-Host "Dry run. Re-run with -Execute (or -e) to merge." -ForegroundColor Yellow + Write-ZTrailer + exit 0 +} +if ($ready.Count -eq 0) { Write-ZTrailer; exit 1 } + +if (-not $Yes) { + Write-Host "" + $answer = Read-Host "Squash-merge these $($ready.Count) PRs and delete their branches? (y/N)" + if ($answer -notmatch '^(y|yes)$') { Write-Host "Aborted." -ForegroundColor Yellow; Write-ZTrailer; exit 1 } +} + +$merged = @(); $failed = @() +foreach ($p in $ready) { + # Re-check: an earlier merge in this same repo may have conflicted this one. + $v = Get-Readiness -RepoName $p.Repo -Number $p.Number + if (-not $v.Ready) { + Write-Host (" SKIP {0} #{1} - {2}" -f $p.Repo, $p.Number, $v.Why) -ForegroundColor Yellow + $failed += [pscustomobject]@{ Repo = $p.Repo; Number = $p.Number; Why = $v.Why } + continue + } + $r = Invoke-Gh @("pr", "merge", "$($p.Number)", "-R", "$ORG/$($p.Repo)", + "--squash", "--delete-branch") -AllowFail + if ($r.Code -eq 0) { + Write-Host (" MERGED {0} #{1}" -f $p.Repo, $p.Number) -ForegroundColor Green + $merged += $p + } else { + Write-Host (" FAILED {0} #{1}" -f $p.Repo, $p.Number) -ForegroundColor Red + Write-Host (" {0}" -f $r.Output) -ForegroundColor DarkRed + $failed += [pscustomobject]@{ Repo = $p.Repo; Number = $p.Number; Why = $r.Output } + } +} + +Write-Host "" +Write-Host "merged $($merged.Count), failed/skipped $($failed.Count)" -ForegroundColor Cyan + +# Verify rather than trust the exit codes - a merge can report success and leave +# the PR in an unexpected state. +if ($merged.Count -gt 0) { + Write-Host "" + Write-Host "Verifying:" -ForegroundColor Cyan + foreach ($p in $merged) { + $r = Invoke-Gh @("pr", "view", "$($p.Number)", "-R", "$ORG/$($p.Repo)", + "--json", "state,mergedAt") -AllowFail + $j = if ($r.Code -eq 0) { $r.Output | ConvertFrom-Json } else { $null } + $state = if ($j) { $j.state } else { "unreadable" } + $col = if ($state -eq "MERGED") { "Green" } else { "Red" } + Write-Host (" {0,-14} #{1,-4} {2}" -f $p.Repo, $p.Number, $state) -ForegroundColor $col + } + + # Follow-up, printed not run: ONE build bump per release - not one per + # merged PR - on the default branch, and then a deploy. Both deliberately + # by hand. This used to print one bump per PR, which is how a single + # release came to be stamped as two builds. + Write-Host "" + Write-Host "Follow-up (not run):" -ForegroundColor Cyan + foreach ($grp in ($merged | Group-Object Repo)) { + $how = if ($BUMP.ContainsKey($grp.Name)) { $BUMP[$grp.Name] } else { "bump this repo's build stamp" } + Write-Host (" {0,-14} {1} merged -> 1 build bump for the release: {2}" -f $grp.Name, $grp.Count, $how) + } + Write-Host " then deploy each project you want live (zdeploy, by hand)" +} + +if ($failed.Count -gt 0) { Write-ZTrailer; exit 1 } + +Write-ZTrailer diff --git a/zpull.cmd b/zpull.cmd new file mode 100644 index 0000000..0fd9553 --- /dev/null +++ b/zpull.cmd @@ -0,0 +1,6 @@ +@echo off +REM Evomedia.net Token Savers — https://github.com/evomedia-net/evo.zscripts +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. +REM Version: v1.0.0.0.24 +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zpull.ps1" %* diff --git a/zpull.ps1 b/zpull.ps1 new file mode 100644 index 0000000..ff615dd --- /dev/null +++ b/zpull.ps1 @@ -0,0 +1,359 @@ +# Evomedia.net Token Savers — https://github.com/evomedia-net/evo.zscripts +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. +# Version: v1.0.0.0.24 + +# zpull.ps1 - merge the fleet's ready PRs, then bring the local checkouts current. +# +# Usage: +# zpull dry run: what would merge, what would pull +# zpull -Execute merge ready PRs, then pull every affected checkout +# zpull -e the same; -e is an alias, as -s is for -Scan +# zpull -Repo limit to one repo +# zpull -Only 65 merge just these PR numbers +# zpull -PullOnly skip merging; only bring checkouts up to date +# zpull -Execute -Yes skip zmerge's confirmation prompt +# +# WHY THIS EXISTS +# --------------- +# zmerge stops at the merge, deliberately - deploys are run by hand. But the +# tooling repos are not deployed anywhere at all: they run +# from the local checkout. For those, "deployed" just means "pulled". Merging a +# zdeploy.ps1 fix and then forgetting the pull leaves you running the old file +# while GitHub says the bug is fixed - which is its own kind of lie. +# +# So: merge (via zmerge, which owns all the mergeability safety), then pull. +# +# WHAT IT WILL NOT DO +# ------------------- +# * pull over uncommitted work. It reports and skips. Twice this month a +# checkout sat on a feature branch or held unstaged edits, and anything that +# "helpfully" resolved that would have destroyed real work. +# * pull anything but a fast-forward. A diverged local main is a decision, +# not something a sync script should guess at. +# * deploy to a server. Still by hand. This only touches local checkouts. +# +# HOW CHECKOUTS ARE FOUND +# ----------------------- +# By reading each candidate directory's `origin` remote and matching the repo +# name, not from a hardcoded table - a table drifts the moment a directory is +# renamed, and this fleet renames directories. + +[CmdletBinding(PositionalBinding = $false)] +param( + [Alias('e')][switch]$Execute, + [switch]$Yes, + [switch]$PullOnly, + # Sweep only the repos with no zdeploy target - the ones where a pull is + # the whole job. Tooling, archives, libraries. + [switch]$ReposOnly, + [string]$Repo, + [int[]]$Only = @(), + [int[]]$Exclude = @(), + # PowerShell binds --help to -Help on its own (it tolerates the extra + # dash), so this one switch answers --help, -help and -h. The bare words + # land in $Rest below and are handled there. + [Alias('h')][switch]$Help, + # Catches anything unmatched. Without it, PositionalBinding=$false makes an + # unknown argument a raw PowerShell binding error - a wall of red that does + # not say what the valid arguments are. Owning the message means a typo + # gets the usage block instead. + [Parameter(ValueFromRemainingArguments = $true)][string[]]$Rest = @() +) + +$ErrorActionPreference = "Stop" + +# Two blank lines at the end of a run, matching every other z-script, so output +# is separated from the next prompt. Local copy rather than ZHelpers: this +# script does not dot-source it. +function Write-ZTrailer { Write-Host ""; Write-Host "" } + +$FLEET_ROOT = Split-Path -Parent $PSScriptRoot +$ORG = "evomedia-net" + +function Show-ZPullUsage { + Write-Host "" + Write-Host "zpull - merge the fleet's ready PRs, then bring local checkouts current." -ForegroundColor Cyan + Write-Host "" + Write-Host "Usage: zpull [-Execute|-e] [-Yes] [-PullOnly] [-ReposOnly] [-Repo ] [-Only ] [-Exclude ]" -ForegroundColor Yellow + Write-Host "" + Write-Host " (no args) dry run - what would merge, what would pull. Changes nothing." -ForegroundColor Gray + Write-Host " -Execute, -e actually merge ready PRs, then pull every affected checkout" -ForegroundColor Gray + Write-Host " -PullOnly skip merging entirely; only bring checkouts up to date" -ForegroundColor Gray + Write-Host " -Repo limit to one repo, by its GitHub name" -ForegroundColor Gray + Write-Host " -Only merge just these PR numbers" -ForegroundColor Gray + Write-Host " -Exclude merge everything ready except these PR numbers" -ForegroundColor Gray + Write-Host " -ReposOnly only repos with no deploy target (pull = done)" -ForegroundColor Gray + Write-Host " -Yes skip zmerge's confirmation prompt (needs -Execute)" -ForegroundColor Gray + Write-Host " --help, -h this text" -ForegroundColor Gray + Write-Host "" + Write-Host "What each result line means:" -ForegroundColor Yellow + Write-Host " ok already current - nothing to do" -ForegroundColor Gray + Write-Host " PULLED fast-forwarded to the new tip" -ForegroundColor Gray + Write-Host " SKIP deliberately left alone: uncommitted work, not on the default" -ForegroundColor Gray + Write-Host " branch, or diverged. Never resolved automatically." -ForegroundColor Gray + Write-Host " FAIL the repo could not be read or fetched. The sweep continues;" -ForegroundColor Gray + Write-Host " that one repo is simply not current." -ForegroundColor Gray + Write-Host "" + Write-Host "Each line is marked with what the repo still owes:" -ForegroundColor Yellow + Write-Host " [repo] nothing runs from a server - the pull is the whole job" -ForegroundColor Gray + Write-Host " [zdeploy ] a pull leaves the server on the old build" -ForegroundColor Gray + Write-Host "" + Write-Host "It will not pull over uncommitted work, will not do anything but a" -ForegroundColor DarkGray + Write-Host "fast-forward, and will not deploy. Deploys stay manual." -ForegroundColor DarkGray + Write-Host "" + Write-Host "Checkouts are found by reading each directory's origin remote under" -ForegroundColor DarkGray + Write-Host "$FLEET_ROOT, not from a hardcoded list." -ForegroundColor DarkGray +} + +# Bare-word help too, matching the rest of the toolkit (zdeploy myapp, zkill all). +$helpWords = @('help', '?', '/?', '--help', '-help') +if ($Help -or @($Rest | Where-Object { $helpWords -contains $_.ToLowerInvariant() }).Count -gt 0) { + Show-ZPullUsage + Write-ZTrailer + exit 0 +} +if ($Rest.Count -gt 0) { + Write-Host "" + Write-Host "ERROR: unrecognised argument(s): $($Rest -join ', ')" -ForegroundColor Red + Show-ZPullUsage + Write-ZTrailer + exit 1 +} + +function Get-DeployTargetsByPath { + # Checkout path -> the zdeploy keys that ship from it. + # + # Read from zconfig rather than listed here: a second table would drift the + # first time a target is added, and drift in THIS table is the failure it + # exists to prevent - a repo quietly reported as "done at the pull" while a + # server runs the old build. + # + # Matched by containment, not equality, because a target's localRoot is + # often a subdirectory of its checkout (a service may ship from + # \), and one checkout can carry several targets + # (vidplayer ships cardiff, opensesame and kelly). + $map = @{} + # Read here rather than via ZHelpers' Get-ZConfig: this script is + # standalone by design, and that helper exits the process when the config + # is missing - which would turn "no zconfig" into a dead sweep instead of + # a sweep that simply knows of no deploy targets. + $configPath = if ($env:ZCONFIG) { $env:ZCONFIG } else { Join-Path $PSScriptRoot "zconfig.json" } + if (-not (Test-Path -LiteralPath $configPath)) { return $map } + try { + $cfg = Get-Content -LiteralPath $configPath -Raw | ConvertFrom-Json + } catch { + Write-Host " (zconfig.json unreadable - every repo will report as [repo])" -ForegroundColor Yellow + return $map + } + if (-not $cfg.projects) { return $map } + foreach ($key in $cfg.projects.PSObject.Properties.Name) { + if ($key -like '_*') { continue } # underscore keys are comments + $root = $cfg.projects.$key.localRoot + if (-not $root) { continue } + try { $map[$key] = [System.IO.Path]::GetFullPath($root).TrimEnd('\') } catch { } + } + return $map +} + +function Get-DeployKeysFor { + param([string]$Path, [hashtable]$Targets) + $full = [System.IO.Path]::GetFullPath($Path).TrimEnd('\') + $hits = @() + foreach ($key in $Targets.Keys) { + $t = $Targets[$key] + if ($t -eq $full -or $t.StartsWith($full + '\', [StringComparison]::OrdinalIgnoreCase)) { + $hits += $key + } + } + return @($hits | Sort-Object) +} + +function Get-LocalCheckouts { + # repo name -> local path, discovered from origin remotes. + $map = @{} + $candidates = @(Get-ChildItem -LiteralPath $FLEET_ROOT -Directory -ErrorAction SilentlyContinue) + # One level deeper too: some projects keep theirs nested. + foreach ($d in @($candidates)) { + $candidates += @(Get-ChildItem -LiteralPath $d.FullName -Directory -ErrorAction SilentlyContinue) + } + foreach ($d in $candidates) { + if (-not (Test-Path (Join-Path $d.FullName ".git"))) { continue } + # A pruned worktree leaves a .git FILE pointing at an admin dir that no + # longer exists, so Test-Path above passes and git then fails. Same + # redirect trap as everywhere else, so keep this on Continue and judge + # by exit code. + $prev = $ErrorActionPreference + $ErrorActionPreference = "Continue" + $url = (git -C $d.FullName remote get-url origin 2>$null) + $ok = ($LASTEXITCODE -eq 0) + $ErrorActionPreference = $prev + if (-not $ok -or -not $url) { continue } + if ($url -match "[:/]$ORG/([^/]+?)(\.git)?$") { + $name = $Matches[1] + if (-not $map.ContainsKey($name)) { $map[$name] = $d.FullName } + } + } + return $map +} + +function Get-DefaultBranch { + # origin/HEAD is a LOCAL cache of the remote's default branch. It is written + # at clone time, and repos created some other way (git init + remote add, + # which is how the *-stack and hostops checkouts here were made) simply do + # not have it. `git symbolic-ref` then fails with + # fatal: ref refs/remotes/origin/HEAD is not a symbolic ref + # and - because this script runs under ErrorActionPreference='Stop' - PS 5.1 + # turns that redirected stderr into a TERMINATING NativeCommandError. The + # 2>$null does not prevent it; it is the redirect itself that wraps each + # stderr line in an ErrorRecord. So drop to Continue for the native calls. + param([string]$Path) + $prev = $ErrorActionPreference + $ErrorActionPreference = "Continue" + try { + $d = (git -C $Path symbolic-ref --short refs/remotes/origin/HEAD 2>$null) -replace '^origin/', '' + if (-not $d) { + # Repair the cache from the remote, then re-ask. Costs one network + # round-trip on first run per repo and is permanent afterwards. + git -C $Path remote set-head origin --auto 2>$null | Out-Null + $d = (git -C $Path symbolic-ref --short refs/remotes/origin/HEAD 2>$null) -replace '^origin/', '' + } + if (-not $d) { + # Offline, or no such remote. Believe the remote-tracking refs that + # exist rather than assuming "main" - zscripts is on master, and + # guessing wrong makes this script skip the repo with a misleading + # "on 'master', not 'main'". + foreach ($c in @('main', 'master')) { + git -C $Path rev-parse --verify --quiet "refs/remotes/origin/$c" 2>$null | Out-Null + if ($LASTEXITCODE -eq 0) { $d = $c; break } + } + } + if (-not $d) { $d = (git -C $Path rev-parse --abbrev-ref HEAD 2>$null) } + if (-not $d) { $d = "main" } + return $d + } + finally { $ErrorActionPreference = $prev } +} + +function Sync-Checkout { + param([string]$Name, [string]$Path, [bool]$DoIt, [string[]]$DeployKeys = @()) + + # Everything below judges git by $LASTEXITCODE, so drop to Continue for the + # whole function (scoped, auto-reverts on exit). + # + # This is not tidiness. Under the script's ErrorActionPreference='Stop', a + # stderr REDIRECT on a native command makes PS 5.1 wrap each stderr line in + # a terminating ErrorRecord - so `git fetch origin 2>$null` against one + # repo with an unreachable remote killed the ENTIRE sweep mid-list, leaving + # every repo after it unvisited and unreported. A fleet sweep must survive + # one bad repo; that repo gets a FAIL row and the run continues. + $prev = $ErrorActionPreference + $ErrorActionPreference = "Continue" + try { + Sync-CheckoutCore -Name $Name -Path $Path -DoIt $DoIt -DeployKeys $DeployKeys + } + catch { + Write-Host (" {0,-18} FAIL {1}" -f $Name, $_.Exception.Message) -ForegroundColor Red + } + finally { $ErrorActionPreference = $prev } +} + +function Sync-CheckoutCore { + param([string]$Name, [string]$Path, [bool]$DoIt, [string[]]$DeployKeys = @()) + + # Appended to every line: the point is that you never have to remember + # which kind of repo you are looking at. + $mark = if ($DeployKeys.Count -gt 0) { " [zdeploy $($DeployKeys -join ', ')]" } else { " [repo]" } + + $branch = (git -C $Path rev-parse --abbrev-ref HEAD 2>$null) + if ($LASTEXITCODE -ne 0 -or -not $branch) { + Write-Host (" {0,-18} FAIL not a usable git checkout: {1}{2}" -f $Name, $Path, $mark) -ForegroundColor Red + return + } + $dirty = @(git -C $Path status --porcelain --untracked-files=no 2>$null) + $default = Get-DefaultBranch -Path $Path + + if ($dirty) { + Write-Host (" {0,-18} SKIP uncommitted changes ({1} file(s)) - commit or stash first{2}" -f $Name, $dirty.Count, $mark) -ForegroundColor Yellow + return + } + if ($branch -ne $default) { + Write-Host (" {0,-18} SKIP on '{1}', not '{2}'{3}" -f $Name, $branch, $default, $mark) -ForegroundColor Yellow + return + } + + git -C $Path fetch origin --quiet 2>$null + if ($LASTEXITCODE -ne 0) { + # Unreachable remote, renamed repo, dead credential. Say so and move on + # - continuing would compare against stale remote-tracking refs and + # report "already current" about a repo we could not actually reach. + Write-Host (" {0,-18} FAIL cannot fetch origin - check the remote{1}" -f $Name, $mark) -ForegroundColor Red + return + } + $behind = (git -C $Path rev-list --count "HEAD..origin/$default" 2>$null) + $ahead = (git -C $Path rev-list --count "origin/$default..HEAD" 2>$null) + + if ([int]$ahead -gt 0) { + Write-Host (" {0,-18} SKIP local '{1}' is {2} commit(s) ahead - diverged, resolve by hand{3}" -f $Name, $default, $ahead, $mark) -ForegroundColor Yellow + return + } + if ([int]$behind -eq 0) { + Write-Host (" {0,-18} ok already current{1}" -f $Name, $mark) -ForegroundColor DarkGray + return + } + if (-not $DoIt) { + Write-Host (" {0,-18} would pull {1} commit(s){2}" -f $Name, $behind, $mark) -ForegroundColor Cyan + return + } + git -C $Path merge --ff-only "origin/$default" --quiet 2>$null + if ($LASTEXITCODE -eq 0) { + Write-Host (" {0,-18} PULLED {1} commit(s) -> {2}{3}" -f $Name, $behind, (git -C $Path rev-parse --short HEAD), $mark) -ForegroundColor Green + # The whole reason the marker exists. A tooling repo is finished here; + # a deployable one now has a checkout ahead of its own server, which is + # the state that gets forgotten. + foreach ($k in $DeployKeys) { + Write-Host (" {0,-18} still on the old build - run: zdeploy {1}" -f "", $k) -ForegroundColor Yellow + } + } else { + Write-Host (" {0,-18} FAILED to fast-forward{1}" -f $Name, $mark) -ForegroundColor Red + } +} + +# ── 1. Merge ───────────────────────────────────────────────────── +if (-not $PullOnly) { + Write-Host "`n=== Merging ready PRs (via zmerge) ===" -ForegroundColor Cyan + # Hashtable splatting, not an array. Array splatting passes elements + # positionally, so @("-Repo","") fed "-Repo" into zmerge's + # [int[]]$Exclude and died on the type conversion. + $zm = @{} + if ($Execute) { $zm.Execute = $true } + if ($Yes) { $zm.Yes = $true } + if ($Repo) { $zm.Repo = $Repo } + if ($Only) { $zm.Only = $Only } + if ($Exclude) { $zm.Exclude = $Exclude } + & (Join-Path $PSScriptRoot "zmerge.ps1") @zm +} + +# ── 2. Pull ────────────────────────────────────────────────────── +Write-Host "`n=== Bringing local checkouts current ===" -ForegroundColor Cyan +if (-not $Execute) { + Write-Host " (dry run - nothing will be pulled; add -Execute or -e)" -ForegroundColor DarkGray +} +$checkouts = Get-LocalCheckouts +if ($Repo) { + if ($checkouts.ContainsKey($Repo)) { $checkouts = @{ $Repo = $checkouts[$Repo] } } + else { Write-Host " no local checkout found for '$Repo'" -ForegroundColor Yellow; $checkouts = @{} } +} +$targets = Get-DeployTargetsByPath +if ($ReposOnly) { + Write-Host " (-ReposOnly: repos with a zdeploy target are not listed)" -ForegroundColor DarkGray +} +$shown = 0 +foreach ($name in ($checkouts.Keys | Sort-Object)) { + $keys = Get-DeployKeysFor -Path $checkouts[$name] -Targets $targets + if ($ReposOnly -and $keys.Count -gt 0) { continue } + $shown++ + Sync-Checkout -Name $name -Path $checkouts[$name] -DoIt:$Execute -DeployKeys $keys +} +if ($shown -eq 0) { Write-Host " nothing matched" -ForegroundColor DarkGray } +Write-ZTrailer