diff --git a/.gitignore b/.gitignore index e22a1d2..f34b514 100644 --- a/.gitignore +++ b/.gitignore @@ -19,3 +19,6 @@ md/ # Pester coverage output (regenerated; never committed) coverage/ + +# Generated by scripts/plaintext_twins.py +__pycache__/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a7fec2..50fbc39 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,29 @@ Notable changes to the Evomedia.net Token Savers. ## Unreleased +### Added + +- **`zmerge`** — merge every pull request across the org that is genuinely + ready (`MERGEABLE` / `CLEAN`, not a draft), re-checking each one immediately + before and after every merge, because merging into a default branch can + conflict a sibling PR in the same repository. Dry run by default; + `-Execute` (or `-e`) merges. +- **`zpull`** — `zmerge`, then `git pull --ff-only` in every checkout the + merges affected. Skips a checkout that is dirty or is not on its default + branch rather than guessing. + +### Changed + +- **`-e` is an alias for `-Execute`** on both of the above, the way `-s` + already works for `-Scan`. +- **`zmerge` discovers repositories instead of listing them.** It asked a + hand-kept list, which had fallen well behind the org - so a scan covered + about half of it and reported "Nothing open to merge" while a ready pull + request sat in a repository the list had never heard of. It now asks GitHub, + and throws rather than returning an empty list if that fails: a tool that + quietly scans nothing prints the same reassuring line as one that scanned + everything, and the two must not be confusable. + ## v1.0.0.0.24 - 2026-09-08 ### Added diff --git a/CHANGELOG.txt b/CHANGELOG.txt index 72860b7..7070510 100644 --- a/CHANGELOG.txt +++ b/CHANGELOG.txt @@ -10,6 +10,31 @@ Notable changes to the Evomedia.net Token Savers. Unreleased ---------- +Added +----- + +- zmerge — merge every pull request across the org that is genuinely + ready (MERGEABLE / CLEAN, not a draft), re-checking each one immediately + before and after every merge, because merging into a default branch can + conflict a sibling PR in the same repository. Dry run by default; + -Execute (or -e) merges. +- zpull — zmerge, then git pull --ff-only in every checkout the + merges affected. Skips a checkout that is dirty or is not on its default + branch rather than guessing. + +Changed +------- + +- -e is an alias for -Execute on both of the above, the way -s + already works for -Scan. +- zmerge discovers repositories instead of listing them. It asked a + hand-kept list, which had fallen well behind the org - so a scan covered + about half of it and reported "Nothing open to merge" while a ready pull + request sat in a repository the list had never heard of. It now asks GitHub, + and throws rather than returning an empty list if that fails: a tool that + quietly scans nothing prints the same reassuring line as one that scanned + everything, and the two must not be confusable. + v1.0.0.0.24 - 2026-09-08 ------------------------ diff --git a/CHECKSUMS.txt b/CHECKSUMS.txt index 910794f..f39f13c 100644 --- a/CHECKSUMS.txt +++ b/CHECKSUMS.txt @@ -20,6 +20,9 @@ e904b06017619f0ea13a79c34f56c70e642a5f0aae7dfab55159885aac2ac384 ZHelpers.ps1 1eb4c23bdc0ed1a82dc495c623b49e5dcd4e342f026b4d896e32c79d592667db zkill.ps1 1c908b69fb9200610e080ac6bb8f4c15d3d7edf402d2e119c2405158e1986a52 ZKiller.ps1 558dfdfc7b4d12231e476c14a00c678e2e536140c4b7abbc05364bf214562291 ZKillOnly.ps1 +ac391726eead5008ab343e096fa6395d0dd2e3bdc6d556518e7233af7bb60382 zmerge.ps1 +5434f004042635622e66f17e096ee7e322e7350a5f6c93c4e51e5b5afeb01df2 zpull.cmd +66029cfecb1af3600061a9b4c4201b1b103bc93b8af7aee3de2c5691ea68b3ca zpull.ps1 36b01f2cd1d5967dc3ec1313fc4f82ada2ea669d0529a0d313300e21a5b38d04 zrelease.cmd e48b994605b1ad9f793f95d653f50df41db628a6b2c6976415b7d03e3900a373 zrelease.ps1 02635351847f84d0f644ffa9c0073804e480d360fcabcd6a8b793cffd6f0026f zrepair.cmd diff --git a/README.md b/README.md index cc12b5c..53fdeac 100644 --- a/README.md +++ b/README.md @@ -136,6 +136,8 @@ The `.cmd` wrappers are the everyday interface. Every command takes one or more | `zchecksums [-Update]` | Verify every script against `CHECKSUMS.txt` (SHA-256) | | `zversion [bump \| bump-stage \| set ]` | Show or advance the toolkit version (stamps every header) | | `zrelease [-Verify]` | Package the current version as `releases/zscripts-.zip` + `.sha256` | +| `zmerge [-Execute\|-e]` | Merge every pull request across the org that is genuinely ready | +| `zpull [-Execute\|-e]` | `zmerge`, then bring every affected local checkout current | ### Local development diff --git a/README.txt b/README.txt index f6d0740..9f85b98 100644 --- a/README.txt +++ b/README.txt @@ -133,6 +133,8 @@ The .cmd wrappers are the everyday interface. Every command takes one or more pr | zchecksums [-Update] | Verify every script against CHECKSUMS.txt (SHA-256) | | zversion [bump \| bump-stage \| set ] | Show or advance the toolkit version (stamps every header) | | zrelease [-Verify] | Package the current version as releases/zscripts-.zip + .sha256 | +| zmerge [-Execute\|-e] | Merge every pull request across the org that is genuinely ready | +| zpull [-Execute\|-e] | zmerge, then bring every affected local checkout current | Local development ----------------- diff --git a/tests/sanitization-patterns.psd1 b/tests/sanitization-patterns.psd1 index 50a191f..744b33d 100644 --- a/tests/sanitization-patterns.psd1 +++ b/tests/sanitization-patterns.psd1 @@ -31,7 +31,7 @@ @{ Name = 'current product name'; Pattern = '(?i)\bevo[.-](ehs|ai|edge|locate|proven|platform|civilcode|scripts)\b|\bevoehs' } @{ Name = 'internal issue reference'; Pattern = 'evo\.scripts#\d+' } @{ Name = 'private product domain'; Pattern = '\b(smartplantehs\.com|provensheet\.com|evoehs\.com|civilcode\.evomedia\.net|dashboard\.evomedia\.net|webmail\.evomedia\.net|mail-admin\.evomedia\.net|docketmail\.evomedia\.net|cardiff\.evomedia\.net|platform\.evomedia\.net|ai\.evomedia\.net|git\.evomedia\.net|analytics\.evomedia\.net)\b' } - @{ Name = 'private-only script'; Pattern = '\b(register_civilcode|register_docketmail|sp_seed_demo_prod|zpublish_stats|zcoverage|zmerge|zpull|zresume|swag_set_owner|provision_demo|apply_platform_config_fixes)\b' } + @{ Name = 'private-only script'; Pattern = '\b(register_civilcode|register_docketmail|sp_seed_demo_prod|zpublish_stats|zcoverage|zresume|swag_set_owner|provision_demo|apply_platform_config_fixes)\b' } @{ Name = 'local drive path'; Pattern = '[A-Za-z]:\\\\?evomedia\.net' } @{ Name = 'operator home path'; Pattern = '/home/ubuntu/' } @{ Name = 'real pem key name'; Pattern = 'evomedia-prod\.pem' } diff --git a/zmerge.ps1 b/zmerge.ps1 new file mode 100644 index 0000000..a06f5af --- /dev/null +++ b/zmerge.ps1 @@ -0,0 +1,251 @@ +# Evomedia.net Token Savers — https://github.com/evomedia-net/evo.zscripts +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. +# Version: v1.0.0.0.24 + +# zmerge.ps1 - merge the fleet's ready pull requests in one pass. +# +# Usage: +# zmerge dry run: list every open PR and its verdict +# zmerge -Execute merge everything that is genuinely ready +# zmerge -e the same; -e is an alias, as -s is for -Scan +# zmerge -Exclude 431 skip PRs by number (repeatable) +# zmerge -Repo limit to one repo +# zmerge -Only 68,67 merge just these +# zmerge -Execute -Yes skip the confirmation prompt +# +# WHY THIS EXISTS +# --------------- +# Eleven ready PRs across three repos is eleven trips through the GitHub UI, and +# the failure mode is not the clicking - it is that `gh pr create` and the merge +# button will both happily accept a PR that cannot actually merge. Mergeability +# is computed asynchronously, so a PR reports UNKNOWN for a few seconds after any +# push and CONFLICTING only later. Merging by hand, the tenth PR is the one that +# gets rubber-stamped. +# +# So this refuses to merge anything it has not just re-checked, and it re-checks +# after every merge, because merging one PR can conflict another in the same +# repo. +# +# WHAT IT WILL NOT DO +# ------------------- +# * merge a PR that is not MERGEABLE/CLEAN at the moment it is reached +# * merge a draft, or one with a failing required check +# * bump versions - each repo stamps differently (zbump for zscripts, +# bump_build_version.mjs for www), and a wrong stamp is worse than none. +# The follow-up commands are printed instead. +# * deploy anything. Deploys are run by hand, deliberately. +# +# ON UNKNOWN +# ---------- +# GitHub returns mergeable=UNKNOWN while it computes, which is indistinguishable +# from trouble if you only look once. Each PR is polled up to $PollTries times +# before being treated as not ready, so a slow answer does not read as a failure +# and a real CONFLICTING never reads as "probably fine". + +param( + [Alias('e')][switch]$Execute, + [switch]$Yes, + [int[]]$Exclude = @(), + [int[]]$Only = @(), + [string]$Repo, + [int]$PollTries = 6, + [int]$PollDelaySeconds = 4 +) + +$ErrorActionPreference = "Stop" + +# Two blank lines at the end of a run, matching every other z-script, so output +# is separated from the next prompt. Local copy rather than ZHelpers: this +# script does not dot-source it. +function Write-ZTrailer { Write-Host ""; Write-Host "" } + +# Every repository in the org, asked of GitHub rather than remembered here. +# +# Local to this script for the same reason Write-ZTrailer is: zmerge needs gh +# and nothing else, and dot-sourcing 1,200 lines of deploy helpers for one +# function would trade that away. +# +# THROWS rather than returning an empty list when gh fails. A merge tool that +# quietly scans nothing prints exactly the same reassuring line as one that +# scanned everything and found nothing, and those two must never be +# confusable - which is precisely how the list this replaced hid its own rot. +function Get-FleetRepos { + param([Parameter(Mandatory)][string]$Org) + $raw = & gh repo list $Org --limit 200 --json name,isArchived 2>&1 + if ($LASTEXITCODE -ne 0) { + throw "gh repo list $Org failed ($LASTEXITCODE): $($raw -join ' ')" + } + try { $all = $raw | ConvertFrom-Json } catch { + throw "gh repo list $Org did not return JSON: $($raw -join ' ')" + } + if (-not $all) { throw "gh repo list $Org returned no repositories" } + $names = @($all | Where-Object { -not $_.isArchived } | + ForEach-Object { $_.name } | Sort-Object) + if ($names.Count -eq 0) { throw "every repository in $Org is archived?" } + return $names +} + + + +$ORG = "evomedia-net" +# Discovered, never listed. The list this replaced had fallen fourteen +# repositories behind: a scan covered sixteen of thirty and said "Nothing open +# to merge" while a ready PR sat in one of the fourteen it could not see. +$REPOS = Get-FleetRepos -Org $ORG + +# How each repo advances its build stamp after a merge. Printed as follow-up, +# never run: see the header. +$BUMP = @{ + "evo.zscripts" = "zbump" + "evo.www" = "node scripts/bump_build_version.mjs bump (on main, then push)" +} + +function Invoke-Gh { + param([string[]]$GhArgs, [switch]$AllowFail) + $prev = $ErrorActionPreference + $ErrorActionPreference = "Continue" + try { + $out = & gh @GhArgs 2>&1 | ForEach-Object { "$_" } + $code = $LASTEXITCODE + } finally { $ErrorActionPreference = $prev } + if ($code -ne 0 -and -not $AllowFail) { + throw "gh $($GhArgs -join ' ') failed ($code): $($out -join "`n")" + } + return [pscustomobject]@{ Output = ($out -join "`n"); Code = $code } +} + +function Get-OpenPrs { + param([string]$RepoName) + $r = Invoke-Gh @("pr", "list", "-R", "$ORG/$RepoName", "--state", "open", + "--limit", "100", "--json", "number,title,isDraft,headRefName") -AllowFail + if ($r.Code -ne 0 -or -not $r.Output) { return @() } + return @($r.Output | ConvertFrom-Json) +} + +# Re-checked immediately before every merge, and again after each one, because +# merging into the default branch can conflict a sibling PR in the same repo. +function Get-Readiness { + param([string]$RepoName, [int]$Number) + for ($i = 1; $i -le $PollTries; $i++) { + $r = Invoke-Gh @("pr", "view", "$Number", "-R", "$ORG/$RepoName", + "--json", "mergeable,mergeStateStatus,state,isDraft") -AllowFail + if ($r.Code -ne 0) { return [pscustomobject]@{ Ready = $false; Why = "cannot read PR" } } + $j = $r.Output | ConvertFrom-Json + if ($j.state -ne "OPEN") { return [pscustomobject]@{ Ready = $false; Why = "state is $($j.state)" } } + if ($j.isDraft) { return [pscustomobject]@{ Ready = $false; Why = "draft" } } + if ($j.mergeable -eq "MERGEABLE" -and $j.mergeStateStatus -eq "CLEAN") { + return [pscustomobject]@{ Ready = $true; Why = "MERGEABLE/CLEAN" } + } + if ($j.mergeable -eq "CONFLICTING") { + return [pscustomobject]@{ Ready = $false; Why = "CONFLICTING - rebase it" } + } + # UNKNOWN, or a non-CLEAN state such as BLOCKED/BEHIND: give GitHub a + # moment, since it computes mergeability asynchronously. + if ($j.mergeable -ne "UNKNOWN" -and $j.mergeStateStatus -ne "UNKNOWN") { + return [pscustomobject]@{ Ready = $false; Why = "$($j.mergeable)/$($j.mergeStateStatus)" } + } + Start-Sleep -Seconds $PollDelaySeconds + } + return [pscustomobject]@{ Ready = $false; Why = "still UNKNOWN after $PollTries tries" } +} + +$targets = if ($Repo) { @($Repo) } else { $REPOS } + +Write-Host "" +Write-Host "Scanning $($targets.Count) repo(s) for open pull requests..." -ForegroundColor Cyan + +$queue = @() +foreach ($r in $targets) { + foreach ($pr in (Get-OpenPrs -RepoName $r)) { + if ($Exclude -contains $pr.number) { continue } + if ($Only.Count -gt 0 -and $Only -notcontains $pr.number) { continue } + $queue += [pscustomobject]@{ Repo = $r; Number = $pr.number; Title = $pr.title; Draft = $pr.isDraft } + } +} + +if ($queue.Count -eq 0) { Write-Host "Nothing open to merge." -ForegroundColor Yellow; Write-ZTrailer; exit 0 } + +Write-Host "" +foreach ($p in $queue) { + $v = Get-Readiness -RepoName $p.Repo -Number $p.Number + $p | Add-Member -NotePropertyName Ready -NotePropertyValue $v.Ready -Force + $p | Add-Member -NotePropertyName Why -NotePropertyValue $v.Why -Force + $mark = if ($v.Ready) { "OK " } else { "SKIP" } + $col = if ($v.Ready) { "Green" } else { "Yellow" } + Write-Host (" {0} {1,-14} #{2,-4} {3}" -f $mark, $p.Repo, $p.Number, $p.Title) -ForegroundColor $col + if (-not $v.Ready) { Write-Host (" -> {0}" -f $v.Why) -ForegroundColor DarkYellow } +} + +$ready = @($queue | Where-Object { $_.Ready }) +Write-Host "" +Write-Host "$($ready.Count) of $($queue.Count) ready to merge." -ForegroundColor Cyan + +if (-not $Execute) { + Write-Host "" + Write-Host "Dry run. Re-run with -Execute (or -e) to merge." -ForegroundColor Yellow + Write-ZTrailer + exit 0 +} +if ($ready.Count -eq 0) { Write-ZTrailer; exit 1 } + +if (-not $Yes) { + Write-Host "" + $answer = Read-Host "Squash-merge these $($ready.Count) PRs and delete their branches? (y/N)" + if ($answer -notmatch '^(y|yes)$') { Write-Host "Aborted." -ForegroundColor Yellow; Write-ZTrailer; exit 1 } +} + +$merged = @(); $failed = @() +foreach ($p in $ready) { + # Re-check: an earlier merge in this same repo may have conflicted this one. + $v = Get-Readiness -RepoName $p.Repo -Number $p.Number + if (-not $v.Ready) { + Write-Host (" SKIP {0} #{1} - {2}" -f $p.Repo, $p.Number, $v.Why) -ForegroundColor Yellow + $failed += [pscustomobject]@{ Repo = $p.Repo; Number = $p.Number; Why = $v.Why } + continue + } + $r = Invoke-Gh @("pr", "merge", "$($p.Number)", "-R", "$ORG/$($p.Repo)", + "--squash", "--delete-branch") -AllowFail + if ($r.Code -eq 0) { + Write-Host (" MERGED {0} #{1}" -f $p.Repo, $p.Number) -ForegroundColor Green + $merged += $p + } else { + Write-Host (" FAILED {0} #{1}" -f $p.Repo, $p.Number) -ForegroundColor Red + Write-Host (" {0}" -f $r.Output) -ForegroundColor DarkRed + $failed += [pscustomobject]@{ Repo = $p.Repo; Number = $p.Number; Why = $r.Output } + } +} + +Write-Host "" +Write-Host "merged $($merged.Count), failed/skipped $($failed.Count)" -ForegroundColor Cyan + +# Verify rather than trust the exit codes - a merge can report success and leave +# the PR in an unexpected state. +if ($merged.Count -gt 0) { + Write-Host "" + Write-Host "Verifying:" -ForegroundColor Cyan + foreach ($p in $merged) { + $r = Invoke-Gh @("pr", "view", "$($p.Number)", "-R", "$ORG/$($p.Repo)", + "--json", "state,mergedAt") -AllowFail + $j = if ($r.Code -eq 0) { $r.Output | ConvertFrom-Json } else { $null } + $state = if ($j) { $j.state } else { "unreadable" } + $col = if ($state -eq "MERGED") { "Green" } else { "Red" } + Write-Host (" {0,-14} #{1,-4} {2}" -f $p.Repo, $p.Number, $state) -ForegroundColor $col + } + + # Follow-up, printed not run: ONE build bump per release - not one per + # merged PR - on the default branch, and then a deploy. Both deliberately + # by hand. This used to print one bump per PR, which is how a single + # release came to be stamped as two builds. + Write-Host "" + Write-Host "Follow-up (not run):" -ForegroundColor Cyan + foreach ($grp in ($merged | Group-Object Repo)) { + $how = if ($BUMP.ContainsKey($grp.Name)) { $BUMP[$grp.Name] } else { "bump this repo's build stamp" } + Write-Host (" {0,-14} {1} merged -> 1 build bump for the release: {2}" -f $grp.Name, $grp.Count, $how) + } + Write-Host " then deploy each project you want live (zdeploy, by hand)" +} + +if ($failed.Count -gt 0) { Write-ZTrailer; exit 1 } + +Write-ZTrailer diff --git a/zpull.cmd b/zpull.cmd new file mode 100644 index 0000000..0fd9553 --- /dev/null +++ b/zpull.cmd @@ -0,0 +1,6 @@ +@echo off +REM Evomedia.net Token Savers — https://github.com/evomedia-net/evo.zscripts +REM Created by Kelly Michels · dev@evomedia.net +REM Licensed under the MIT License. See LICENSE. +REM Version: v1.0.0.0.24 +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0zpull.ps1" %* diff --git a/zpull.ps1 b/zpull.ps1 new file mode 100644 index 0000000..ff615dd --- /dev/null +++ b/zpull.ps1 @@ -0,0 +1,359 @@ +# Evomedia.net Token Savers — https://github.com/evomedia-net/evo.zscripts +# Created by Kelly Michels · dev@evomedia.net +# Licensed under the MIT License. See LICENSE. +# Version: v1.0.0.0.24 + +# zpull.ps1 - merge the fleet's ready PRs, then bring the local checkouts current. +# +# Usage: +# zpull dry run: what would merge, what would pull +# zpull -Execute merge ready PRs, then pull every affected checkout +# zpull -e the same; -e is an alias, as -s is for -Scan +# zpull -Repo limit to one repo +# zpull -Only 65 merge just these PR numbers +# zpull -PullOnly skip merging; only bring checkouts up to date +# zpull -Execute -Yes skip zmerge's confirmation prompt +# +# WHY THIS EXISTS +# --------------- +# zmerge stops at the merge, deliberately - deploys are run by hand. But the +# tooling repos are not deployed anywhere at all: they run +# from the local checkout. For those, "deployed" just means "pulled". Merging a +# zdeploy.ps1 fix and then forgetting the pull leaves you running the old file +# while GitHub says the bug is fixed - which is its own kind of lie. +# +# So: merge (via zmerge, which owns all the mergeability safety), then pull. +# +# WHAT IT WILL NOT DO +# ------------------- +# * pull over uncommitted work. It reports and skips. Twice this month a +# checkout sat on a feature branch or held unstaged edits, and anything that +# "helpfully" resolved that would have destroyed real work. +# * pull anything but a fast-forward. A diverged local main is a decision, +# not something a sync script should guess at. +# * deploy to a server. Still by hand. This only touches local checkouts. +# +# HOW CHECKOUTS ARE FOUND +# ----------------------- +# By reading each candidate directory's `origin` remote and matching the repo +# name, not from a hardcoded table - a table drifts the moment a directory is +# renamed, and this fleet renames directories. + +[CmdletBinding(PositionalBinding = $false)] +param( + [Alias('e')][switch]$Execute, + [switch]$Yes, + [switch]$PullOnly, + # Sweep only the repos with no zdeploy target - the ones where a pull is + # the whole job. Tooling, archives, libraries. + [switch]$ReposOnly, + [string]$Repo, + [int[]]$Only = @(), + [int[]]$Exclude = @(), + # PowerShell binds --help to -Help on its own (it tolerates the extra + # dash), so this one switch answers --help, -help and -h. The bare words + # land in $Rest below and are handled there. + [Alias('h')][switch]$Help, + # Catches anything unmatched. Without it, PositionalBinding=$false makes an + # unknown argument a raw PowerShell binding error - a wall of red that does + # not say what the valid arguments are. Owning the message means a typo + # gets the usage block instead. + [Parameter(ValueFromRemainingArguments = $true)][string[]]$Rest = @() +) + +$ErrorActionPreference = "Stop" + +# Two blank lines at the end of a run, matching every other z-script, so output +# is separated from the next prompt. Local copy rather than ZHelpers: this +# script does not dot-source it. +function Write-ZTrailer { Write-Host ""; Write-Host "" } + +$FLEET_ROOT = Split-Path -Parent $PSScriptRoot +$ORG = "evomedia-net" + +function Show-ZPullUsage { + Write-Host "" + Write-Host "zpull - merge the fleet's ready PRs, then bring local checkouts current." -ForegroundColor Cyan + Write-Host "" + Write-Host "Usage: zpull [-Execute|-e] [-Yes] [-PullOnly] [-ReposOnly] [-Repo ] [-Only ] [-Exclude ]" -ForegroundColor Yellow + Write-Host "" + Write-Host " (no args) dry run - what would merge, what would pull. Changes nothing." -ForegroundColor Gray + Write-Host " -Execute, -e actually merge ready PRs, then pull every affected checkout" -ForegroundColor Gray + Write-Host " -PullOnly skip merging entirely; only bring checkouts up to date" -ForegroundColor Gray + Write-Host " -Repo limit to one repo, by its GitHub name" -ForegroundColor Gray + Write-Host " -Only merge just these PR numbers" -ForegroundColor Gray + Write-Host " -Exclude merge everything ready except these PR numbers" -ForegroundColor Gray + Write-Host " -ReposOnly only repos with no deploy target (pull = done)" -ForegroundColor Gray + Write-Host " -Yes skip zmerge's confirmation prompt (needs -Execute)" -ForegroundColor Gray + Write-Host " --help, -h this text" -ForegroundColor Gray + Write-Host "" + Write-Host "What each result line means:" -ForegroundColor Yellow + Write-Host " ok already current - nothing to do" -ForegroundColor Gray + Write-Host " PULLED fast-forwarded to the new tip" -ForegroundColor Gray + Write-Host " SKIP deliberately left alone: uncommitted work, not on the default" -ForegroundColor Gray + Write-Host " branch, or diverged. Never resolved automatically." -ForegroundColor Gray + Write-Host " FAIL the repo could not be read or fetched. The sweep continues;" -ForegroundColor Gray + Write-Host " that one repo is simply not current." -ForegroundColor Gray + Write-Host "" + Write-Host "Each line is marked with what the repo still owes:" -ForegroundColor Yellow + Write-Host " [repo] nothing runs from a server - the pull is the whole job" -ForegroundColor Gray + Write-Host " [zdeploy ] a pull leaves the server on the old build" -ForegroundColor Gray + Write-Host "" + Write-Host "It will not pull over uncommitted work, will not do anything but a" -ForegroundColor DarkGray + Write-Host "fast-forward, and will not deploy. Deploys stay manual." -ForegroundColor DarkGray + Write-Host "" + Write-Host "Checkouts are found by reading each directory's origin remote under" -ForegroundColor DarkGray + Write-Host "$FLEET_ROOT, not from a hardcoded list." -ForegroundColor DarkGray +} + +# Bare-word help too, matching the rest of the toolkit (zdeploy myapp, zkill all). +$helpWords = @('help', '?', '/?', '--help', '-help') +if ($Help -or @($Rest | Where-Object { $helpWords -contains $_.ToLowerInvariant() }).Count -gt 0) { + Show-ZPullUsage + Write-ZTrailer + exit 0 +} +if ($Rest.Count -gt 0) { + Write-Host "" + Write-Host "ERROR: unrecognised argument(s): $($Rest -join ', ')" -ForegroundColor Red + Show-ZPullUsage + Write-ZTrailer + exit 1 +} + +function Get-DeployTargetsByPath { + # Checkout path -> the zdeploy keys that ship from it. + # + # Read from zconfig rather than listed here: a second table would drift the + # first time a target is added, and drift in THIS table is the failure it + # exists to prevent - a repo quietly reported as "done at the pull" while a + # server runs the old build. + # + # Matched by containment, not equality, because a target's localRoot is + # often a subdirectory of its checkout (a service may ship from + # \), and one checkout can carry several targets + # (vidplayer ships cardiff, opensesame and kelly). + $map = @{} + # Read here rather than via ZHelpers' Get-ZConfig: this script is + # standalone by design, and that helper exits the process when the config + # is missing - which would turn "no zconfig" into a dead sweep instead of + # a sweep that simply knows of no deploy targets. + $configPath = if ($env:ZCONFIG) { $env:ZCONFIG } else { Join-Path $PSScriptRoot "zconfig.json" } + if (-not (Test-Path -LiteralPath $configPath)) { return $map } + try { + $cfg = Get-Content -LiteralPath $configPath -Raw | ConvertFrom-Json + } catch { + Write-Host " (zconfig.json unreadable - every repo will report as [repo])" -ForegroundColor Yellow + return $map + } + if (-not $cfg.projects) { return $map } + foreach ($key in $cfg.projects.PSObject.Properties.Name) { + if ($key -like '_*') { continue } # underscore keys are comments + $root = $cfg.projects.$key.localRoot + if (-not $root) { continue } + try { $map[$key] = [System.IO.Path]::GetFullPath($root).TrimEnd('\') } catch { } + } + return $map +} + +function Get-DeployKeysFor { + param([string]$Path, [hashtable]$Targets) + $full = [System.IO.Path]::GetFullPath($Path).TrimEnd('\') + $hits = @() + foreach ($key in $Targets.Keys) { + $t = $Targets[$key] + if ($t -eq $full -or $t.StartsWith($full + '\', [StringComparison]::OrdinalIgnoreCase)) { + $hits += $key + } + } + return @($hits | Sort-Object) +} + +function Get-LocalCheckouts { + # repo name -> local path, discovered from origin remotes. + $map = @{} + $candidates = @(Get-ChildItem -LiteralPath $FLEET_ROOT -Directory -ErrorAction SilentlyContinue) + # One level deeper too: some projects keep theirs nested. + foreach ($d in @($candidates)) { + $candidates += @(Get-ChildItem -LiteralPath $d.FullName -Directory -ErrorAction SilentlyContinue) + } + foreach ($d in $candidates) { + if (-not (Test-Path (Join-Path $d.FullName ".git"))) { continue } + # A pruned worktree leaves a .git FILE pointing at an admin dir that no + # longer exists, so Test-Path above passes and git then fails. Same + # redirect trap as everywhere else, so keep this on Continue and judge + # by exit code. + $prev = $ErrorActionPreference + $ErrorActionPreference = "Continue" + $url = (git -C $d.FullName remote get-url origin 2>$null) + $ok = ($LASTEXITCODE -eq 0) + $ErrorActionPreference = $prev + if (-not $ok -or -not $url) { continue } + if ($url -match "[:/]$ORG/([^/]+?)(\.git)?$") { + $name = $Matches[1] + if (-not $map.ContainsKey($name)) { $map[$name] = $d.FullName } + } + } + return $map +} + +function Get-DefaultBranch { + # origin/HEAD is a LOCAL cache of the remote's default branch. It is written + # at clone time, and repos created some other way (git init + remote add, + # which is how the *-stack and hostops checkouts here were made) simply do + # not have it. `git symbolic-ref` then fails with + # fatal: ref refs/remotes/origin/HEAD is not a symbolic ref + # and - because this script runs under ErrorActionPreference='Stop' - PS 5.1 + # turns that redirected stderr into a TERMINATING NativeCommandError. The + # 2>$null does not prevent it; it is the redirect itself that wraps each + # stderr line in an ErrorRecord. So drop to Continue for the native calls. + param([string]$Path) + $prev = $ErrorActionPreference + $ErrorActionPreference = "Continue" + try { + $d = (git -C $Path symbolic-ref --short refs/remotes/origin/HEAD 2>$null) -replace '^origin/', '' + if (-not $d) { + # Repair the cache from the remote, then re-ask. Costs one network + # round-trip on first run per repo and is permanent afterwards. + git -C $Path remote set-head origin --auto 2>$null | Out-Null + $d = (git -C $Path symbolic-ref --short refs/remotes/origin/HEAD 2>$null) -replace '^origin/', '' + } + if (-not $d) { + # Offline, or no such remote. Believe the remote-tracking refs that + # exist rather than assuming "main" - zscripts is on master, and + # guessing wrong makes this script skip the repo with a misleading + # "on 'master', not 'main'". + foreach ($c in @('main', 'master')) { + git -C $Path rev-parse --verify --quiet "refs/remotes/origin/$c" 2>$null | Out-Null + if ($LASTEXITCODE -eq 0) { $d = $c; break } + } + } + if (-not $d) { $d = (git -C $Path rev-parse --abbrev-ref HEAD 2>$null) } + if (-not $d) { $d = "main" } + return $d + } + finally { $ErrorActionPreference = $prev } +} + +function Sync-Checkout { + param([string]$Name, [string]$Path, [bool]$DoIt, [string[]]$DeployKeys = @()) + + # Everything below judges git by $LASTEXITCODE, so drop to Continue for the + # whole function (scoped, auto-reverts on exit). + # + # This is not tidiness. Under the script's ErrorActionPreference='Stop', a + # stderr REDIRECT on a native command makes PS 5.1 wrap each stderr line in + # a terminating ErrorRecord - so `git fetch origin 2>$null` against one + # repo with an unreachable remote killed the ENTIRE sweep mid-list, leaving + # every repo after it unvisited and unreported. A fleet sweep must survive + # one bad repo; that repo gets a FAIL row and the run continues. + $prev = $ErrorActionPreference + $ErrorActionPreference = "Continue" + try { + Sync-CheckoutCore -Name $Name -Path $Path -DoIt $DoIt -DeployKeys $DeployKeys + } + catch { + Write-Host (" {0,-18} FAIL {1}" -f $Name, $_.Exception.Message) -ForegroundColor Red + } + finally { $ErrorActionPreference = $prev } +} + +function Sync-CheckoutCore { + param([string]$Name, [string]$Path, [bool]$DoIt, [string[]]$DeployKeys = @()) + + # Appended to every line: the point is that you never have to remember + # which kind of repo you are looking at. + $mark = if ($DeployKeys.Count -gt 0) { " [zdeploy $($DeployKeys -join ', ')]" } else { " [repo]" } + + $branch = (git -C $Path rev-parse --abbrev-ref HEAD 2>$null) + if ($LASTEXITCODE -ne 0 -or -not $branch) { + Write-Host (" {0,-18} FAIL not a usable git checkout: {1}{2}" -f $Name, $Path, $mark) -ForegroundColor Red + return + } + $dirty = @(git -C $Path status --porcelain --untracked-files=no 2>$null) + $default = Get-DefaultBranch -Path $Path + + if ($dirty) { + Write-Host (" {0,-18} SKIP uncommitted changes ({1} file(s)) - commit or stash first{2}" -f $Name, $dirty.Count, $mark) -ForegroundColor Yellow + return + } + if ($branch -ne $default) { + Write-Host (" {0,-18} SKIP on '{1}', not '{2}'{3}" -f $Name, $branch, $default, $mark) -ForegroundColor Yellow + return + } + + git -C $Path fetch origin --quiet 2>$null + if ($LASTEXITCODE -ne 0) { + # Unreachable remote, renamed repo, dead credential. Say so and move on + # - continuing would compare against stale remote-tracking refs and + # report "already current" about a repo we could not actually reach. + Write-Host (" {0,-18} FAIL cannot fetch origin - check the remote{1}" -f $Name, $mark) -ForegroundColor Red + return + } + $behind = (git -C $Path rev-list --count "HEAD..origin/$default" 2>$null) + $ahead = (git -C $Path rev-list --count "origin/$default..HEAD" 2>$null) + + if ([int]$ahead -gt 0) { + Write-Host (" {0,-18} SKIP local '{1}' is {2} commit(s) ahead - diverged, resolve by hand{3}" -f $Name, $default, $ahead, $mark) -ForegroundColor Yellow + return + } + if ([int]$behind -eq 0) { + Write-Host (" {0,-18} ok already current{1}" -f $Name, $mark) -ForegroundColor DarkGray + return + } + if (-not $DoIt) { + Write-Host (" {0,-18} would pull {1} commit(s){2}" -f $Name, $behind, $mark) -ForegroundColor Cyan + return + } + git -C $Path merge --ff-only "origin/$default" --quiet 2>$null + if ($LASTEXITCODE -eq 0) { + Write-Host (" {0,-18} PULLED {1} commit(s) -> {2}{3}" -f $Name, $behind, (git -C $Path rev-parse --short HEAD), $mark) -ForegroundColor Green + # The whole reason the marker exists. A tooling repo is finished here; + # a deployable one now has a checkout ahead of its own server, which is + # the state that gets forgotten. + foreach ($k in $DeployKeys) { + Write-Host (" {0,-18} still on the old build - run: zdeploy {1}" -f "", $k) -ForegroundColor Yellow + } + } else { + Write-Host (" {0,-18} FAILED to fast-forward{1}" -f $Name, $mark) -ForegroundColor Red + } +} + +# ── 1. Merge ───────────────────────────────────────────────────── +if (-not $PullOnly) { + Write-Host "`n=== Merging ready PRs (via zmerge) ===" -ForegroundColor Cyan + # Hashtable splatting, not an array. Array splatting passes elements + # positionally, so @("-Repo","") fed "-Repo" into zmerge's + # [int[]]$Exclude and died on the type conversion. + $zm = @{} + if ($Execute) { $zm.Execute = $true } + if ($Yes) { $zm.Yes = $true } + if ($Repo) { $zm.Repo = $Repo } + if ($Only) { $zm.Only = $Only } + if ($Exclude) { $zm.Exclude = $Exclude } + & (Join-Path $PSScriptRoot "zmerge.ps1") @zm +} + +# ── 2. Pull ────────────────────────────────────────────────────── +Write-Host "`n=== Bringing local checkouts current ===" -ForegroundColor Cyan +if (-not $Execute) { + Write-Host " (dry run - nothing will be pulled; add -Execute or -e)" -ForegroundColor DarkGray +} +$checkouts = Get-LocalCheckouts +if ($Repo) { + if ($checkouts.ContainsKey($Repo)) { $checkouts = @{ $Repo = $checkouts[$Repo] } } + else { Write-Host " no local checkout found for '$Repo'" -ForegroundColor Yellow; $checkouts = @{} } +} +$targets = Get-DeployTargetsByPath +if ($ReposOnly) { + Write-Host " (-ReposOnly: repos with a zdeploy target are not listed)" -ForegroundColor DarkGray +} +$shown = 0 +foreach ($name in ($checkouts.Keys | Sort-Object)) { + $keys = Get-DeployKeysFor -Path $checkouts[$name] -Targets $targets + if ($ReposOnly -and $keys.Count -gt 0) { continue } + $shown++ + Sync-Checkout -Name $name -Path $checkouts[$name] -DoIt:$Execute -DeployKeys $keys +} +if ($shown -eq 0) { Write-Host " nothing matched" -ForegroundColor DarkGray } +Write-ZTrailer