Files
pipdock/.github/workflows/ci-integration.yml
T
poli0981andClaude Opus 5.5 12bf723daa ci: failures reach someone, audits run on what is merged, releases are attested
- notify-ci-failure.yml lists its workflows by name. `workflows: ['*']` is not a wildcard to
  workflow_run: it matched nothing, so `main` was red from 2026-09-23 with no one told.
- cargo audit runs a prebuilt cargo-audit (taiki-e/install-action, pinned by commit) with
  `--deny warnings`, replacing rustsec/audit-check@v2 - Node 20, a check-run write token, and on a
  pull request a failure only for "critical", which is how September's advisories passed every PR.
  .cargo/audit.toml's contract (a new informational advisory fails CI) is now what runs.
- security-scan.yml: both audits daily against `main`, so an advisory against something already
  merged is reported the day it is published, not at the next PR.
- cargo deny (licences, sources, bans) with deny.toml: the licence allowlist makes "every bundled
  licence is GPL-3.0-compatible" a test. The workspace declares `publish = false`, which it never
  had - and without which cargo-deny judged the app crates as published ones.
- CodeQL analyses rust and actions as well as javascript-typescript. The comment saying CodeQL had
  no Rust support stopped being true in 2025.
- ci-node.yml: Pages and id-token write move from the workflow to the one job that needs them
  (Sonar S8233); the audit installs with --ignore-scripts (S6505); a new `tauri lockstep` job
  fails when a Tauri crate and its npm package drift apart on Cargo.lock and package-lock.json.
- release.yml: build (read-only token, no persisted credentials) and publish (attest + draft) are
  separate jobs; the tag must match all five version fields; SHA256SUMS.txt is LF, which
  `sha256sum -c` elsewhere needs; actions/attest-build-provenance attests the .exe and .msi;
  softprops/action-gh-release is pinned by commit; `workflow_dispatch` with `dry_run` builds any
  ref without attesting or drafting.
- ci-integration.yml also runs for the CLI, exec, audit, release metadata, errors, store, index,
  Cargo.lock and rust-toolchain.toml - every one of which its end-to-end steps run through.
- dependabot.yml: cooldowns, a group per ecosystem for Tauri packages and for actions and pip,
  TypeScript held below 6.1 (typescript-eslint's peer range), and the rust-toolchain ecosystem.
  A multi-ecosystem group would put both Tauri halves in one PR, but what it does to dependencies
  outside its patterns is undocumented; the lockstep job covers the gap instead.
- Sonar: NOSONAR with reasons on the dev scripts' argv and PATH lookups; bundle-cli.mjs no longer
  claims to run from beforeBuildCommand. The dev CSP moves out of index.html into a Vite plugin
  that reads devCsp from tauri.conf.json, so a release build carries one policy, the strict one
  (S7039); checked in the dev server, where React Refresh's preamble still runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 00:55:27 +07:00

459 lines
23 KiB
YAML

# TESTING L2 — engine integration against disposable venvs.
#
# Project-local and Windows-only: the whole point is exercising the real pip and uv against real
# environments on the platform PipDock ships for. It is heavier than L1, so it runs only on PRs
# touching the code it covers, plus nightly.
#
# The weekly job is the early-warning system RELEASE-CI.md §2 and TESTING L1 describe: it captures
# fixtures against the *latest* pip and uv, so an output-shape change surfaces as a failed job
# rather than as PD-ENG-003 on a user's machine.
name: CI / Integration
on:
pull_request:
paths:
- 'crates/pipdock-core/src/engine/**'
- 'crates/pipdock-core/src/flow/**'
- 'crates/pipdock-core/src/plan/**'
- 'crates/pipdock-core/src/snapshot/**'
- 'crates/pipdock-core/src/graph/**'
- 'crates/pipdock-core/src/envs/**'
# Not under envs/, and the only tests it has parse a canned document — so its real behaviour
# against a real interpreter is only ever exercised here.
- 'crates/pipdock-core/src/probe.py'
- 'crates/pipdock-core/src/health/**'
# Every end-to-end step drives the CLI, and these are what it runs through. A change to
# any of them reached `main` without this job ever seeing it.
- 'crates/pipdock-cli/**'
- 'crates/pipdock-core/src/exec.rs'
- 'crates/pipdock-core/src/audit.rs'
- 'crates/pipdock-core/src/release.rs'
- 'crates/pipdock-core/src/errors/**'
- 'crates/pipdock-core/src/store/**'
- 'crates/pipdock-core/src/index/**'
- 'crates/pipdock-core/tests/**'
# A dependency or toolchain bump is a change to all of the above at once.
- 'Cargo.lock'
- 'rust-toolchain.toml'
- 'spikes/capture.py'
# A Dependabot bump to the Code Health pins has to re-run the bootstrap: whether the new
# pin set still resolves to wheels on 3.10 and 3.14 is exactly what this job can answer
# and a unit test cannot.
- 'tools/**'
- '.github/workflows/ci-integration.yml'
schedule:
# Nightly: the disposable-venv end-to-end pass.
- cron: '17 3 * * *'
# Mondays: the fixture-drift re-capture. It needs its own cron rather than a day-of-week
# condition on the nightly one — `github.event.schedule` is the only thing a job can match
# on, so two behaviours need two entries or the "weekly" job silently runs every night.
- cron: '17 4 * * 1'
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ci-integration-${{ github.ref }}
cancel-in-progress: true
env:
CARGO_TERM_COLOR: always
# Spike SP-2: pip's --report crashes with UnicodeEncodeError on Windows under cp1252, and it is
# data-dependent. The core sets this per child process; setting it here too means a bare `pip`
# in a workflow step cannot reintroduce the failure.
PYTHONIOENCODING: utf-8
PYTHONUTF8: '1'
jobs:
disposable-venvs:
name: Real engines, throwaway venvs
# The Monday cron exists for the drift job below. Without this the whole suite ran twice every
# Monday morning, an hour apart, against the same commit.
if: github.event_name != 'schedule' || github.event.schedule != '17 4 * * 1'
runs-on: windows-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
# The version floor PipDock supports and the newest it claims to (ARCHITECTURE §10).
python: ['3.10', '3.14']
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Setup Python ${{ matrix.python }}
uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python }}
- name: Install uv
# Into its own venv rather than onto PATH globally, mirroring how the spikes pin several
# uv versions side by side.
run: |
python -m venv .uvtools
.uvtools\Scripts\python.exe -m pip install --quiet uv
echo "$PWD\.uvtools\Scripts" >> $env:GITHUB_PATH
shell: pwsh
- name: Cache cargo
uses: actions/cache@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-integ-${{ hashFiles('**/Cargo.lock', 'rust-toolchain.toml') }}
restore-keys: ${{ runner.os }}-cargo-integ-
- name: Show versions
run: |
python --version
python -m pip --version
uv --version
cargo --version
- name: Unit and fixture tests
run: cargo test --workspace --locked
- name: Build the CLI
run: cargo build --locked -p pipdock-cli
# An end-to-end pass over the real product, in the order a user would hit it. Each step is
# a claim the docs make; a failure here means the claim stopped being true.
- name: End-to-end — discovery and inspection
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
python -m venv .itest
.\target\debug\pipdock.exe env list
.\target\debug\pipdock.exe doctor --env .itest
.\target\debug\pipdock.exe list --env .itest --json
# The tools venv is the only thing PipDock installs into an environment of its own, and the
# one whose failure mode is silent: a bootstrap that resolves to an sdist on a new CPython
# would ship PD-BLD-001 to anyone who clicks Health. Running it on the floor and the ceiling
# of the supported range is what a unit test cannot do.
- name: End-to-end — the Code Health tools venv bootstraps
shell: pwsh
env:
# Redirects the whole app-data root (`store::default_app_data` reads LOCALAPPDATA with a
# temp_dir fallback), so this never touches a runner's real profile and needs no
# PipDock-side configuration knob. Verified locally before being relied on here.
LOCALAPPDATA: ${{ github.workspace }}\.itest-appdata
run: |
$ErrorActionPreference = 'Stop'
.\target\debug\pipdock.exe tools sync
if ($LASTEXITCODE -ne 0) { throw "the tools venv did not bootstrap" }
$status = .\target\debug\pipdock.exe tools status --json | ConvertFrom-Json
Write-Host ($status | ConvertTo-Json -Depth 5)
if ($status.need.state -ne 'fresh') {
throw "a just-synced tools venv reports '$($status.need.state)'"
}
foreach ($t in @('deptry', 'vulture', 'ruff')) {
if (-not $status.manifest.tools.$t) { throw "$t is missing from the manifest" }
}
# CODE-HEALTH-SPEC §1 is three tools. pip-audit is in the shipped ledger for the
# post-1.0 Security tab and must not reach the venv — its closure carries msgpack,
# which has no py3-none-any fallback to degrade to.
if ($status.manifest.pins -match 'pip-audit') {
throw "pip-audit reached the tools venv; see health::HEALTH_TOOLS"
}
# The manifest hash is the whole point: a second sync must do nothing.
$t0 = Get-Date
.\target\debug\pipdock.exe tools sync
$warm = ((Get-Date) - $t0).TotalSeconds
Write-Host "warm re-sync took ${warm}s"
if ($warm -gt 10) { throw "a fresh manifest did not short-circuit (${warm}s)" }
# A re-sync must repair the state PD-HLT-001 tells the user to re-sync out of. This
# shipped broken: `pip install -r` over satisfied pins is a no-op, so the sync installed
# nothing and then failed its own verification.
Remove-Item "$env:LOCALAPPDATA\PipDock\data\tools\.venv\Scripts\ruff.exe"
$need = (.\target\debug\pipdock.exe tools status --json | ConvertFrom-Json).need
if ($need.state -ne 'toolMissing' -or $need.tool -ne 'ruff') {
throw "a quarantined ruff.exe reported '$($need.state)', not toolMissing"
}
.\target\debug\pipdock.exe tools sync
if ($LASTEXITCODE -ne 0) { throw "a re-sync could not repair a missing tool" }
# Offline: PD-NET-011, exit 6, and no manifest left claiming the venv is good.
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\PipDock\data\tools"
$env:PIP_NO_INDEX = '1'
$out = .\target\debug\pipdock.exe tools sync 2>&1 | Out-String
$code = $LASTEXITCODE
Remove-Item Env:\PIP_NO_INDEX
Write-Host $out
if ($code -ne 6) { throw "an unreachable index exited $code, not 6" }
if ($out -notmatch 'PD-NET-011') { throw "an unreachable index did not report PD-NET-011" }
if (Test-Path "$env:LOCALAPPDATA\PipDock\data\tools\manifest.json") {
throw "a failed bootstrap left a manifest behind; the next run would think it was fresh"
}
# Required, not tidiness. The runner appends `exit $LASTEXITCODE` to every pwsh step,
# and the last command here *deliberately* exited 6 — so without this the step fails
# with every assertion having passed. Caught by running the script before trusting it.
exit 0
# P0-10, and the two ways it goes wrong. On 3.10 an old pip runs and upgrades normally; on
# 3.14 *every* pip below the 22.2 planner floor is broken — distutils and pkgutil.ImpImporter
# are both gone — so `install -U pip` fails for the same reason the upgrade was offered, and
# only the ensurepip fallback gets out of it. One matrix, both paths, neither reachable from
# a unit test.
- name: End-to-end — pip upkeep, including a pip too broken to upgrade itself
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
python -m venv .oldpip
.\.oldpip\Scripts\python.exe -m pip install --quiet --disable-pip-version-check "pip==22.1.2"
# `list --env`, not `env list`: discovery only finds venvs named .venv/venv/env/.env
# (envs::venv_scan), so a suffixed one is invisible to it. Same probe either way — this
# is the very list EnvRow.pipVersion is read out of.
$before = (.\target\debug\pipdock.exe list --env .oldpip --json | ConvertFrom-Json |
Where-Object { $_.name -eq 'pip' }).version
if ($before -ne '22.1.2') { throw "the probe reported pip '$before', not 22.1.2" }
# Runs pip whatever engine is configured — uv is on PATH in this job, which is what makes
# this leg meaningful rather than incidental.
.\target\debug\pipdock.exe pip-upgrade --env .oldpip --engine uv
if ($LASTEXITCODE -ne 0) { throw "pip-upgrade failed (exit $LASTEXITCODE)" }
$after = (.\target\debug\pipdock.exe list --env .oldpip --json | ConvertFrom-Json |
Where-Object { $_.name -eq 'pip' }).version
if ($after -eq '22.1.2') { throw "pip was not upgraded; still $after" }
Write-Host "pip $before -> $after"
# It must actually run afterwards — on 3.14 it could not before.
.\.oldpip\Scripts\python.exe -m pip --version
if ($LASTEXITCODE -ne 0) { throw "pip still cannot start after the upgrade" }
# PEP 668. A venv is never externally managed, and a venv has no stdlib of its own — so
# `sysconfig.get_path('stdlib')` resolves to the *base*, and every venv built from a Python
# that ships the marker (uv-managed, Debian, Homebrew, Fedora) reported itself managed and
# PipDock refused to mutate it. The marker is planted here rather than hoped for, because the
# runner's own Python does not carry one.
- name: End-to-end — a venv is never externally managed, but its base still is
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
# setup-python's interpreter by its own path, NOT `Get-Command python`. The "Install uv"
# step puts .uvtools\Scripts on PATH and a venv's Scripts holds a python.exe, so a PATH
# lookup here resolves to *that venv* — which is correctly not externally managed, so the
# assertion below failed against a fix that was working. Naming the interpreter removes
# the ambiguity in both directions: which one gets the marker, and which one is asked.
$basePython = Join-Path $env:pythonLocation 'python.exe'
if (-not (Test-Path $basePython)) { throw "no interpreter at $basePython" }
$marker = Join-Path (& $basePython -c "import sysconfig;print(sysconfig.get_path('stdlib'))") 'EXTERNALLY-MANAGED'
Set-Content -Path $marker -Value "[externally-managed]`nError=planted by CI"
try {
# From the base, so this is a venv rather than a venv of a venv.
& $basePython -m venv .pep668
# `doctor --env`, not `env list`: an explicit path needs no discovery, so this does
# not depend on the venv being named one of the four names venv_scan knows.
$venv = .\target\debug\pipdock.exe doctor --env .pep668 --json | ConvertFrom-Json
if ($venv.externallyManaged) {
throw "a venv reported externallyManaged; PipDock would refuse to touch it"
}
# The other direction matters just as much: the base genuinely is managed, and a fix
# that simply stopped looking would pass the assertion above while breaking PD-ENV-002
# for the environments it exists to protect.
$sys = .\target\debug\pipdock.exe doctor --env $basePython --json | ConvertFrom-Json
if (-not $sys.externallyManaged) {
throw "the base did not report the planted marker; the check stopped looking"
}
} finally {
Remove-Item -LiteralPath $marker -ErrorAction SilentlyContinue
}
exit 0
- name: End-to-end — the SP-1 conflict is still reported
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
.itest\Scripts\python.exe -m pip install --quiet "httpx==0.23.0"
# httpx 0.23.0 requires httpcore<0.16, so httpcore must NOT reach its 1.x latest.
$out = .\target\debug\pipdock.exe update httpcore --env .itest --dry-run --yes 2>&1 | Out-String
Write-Host $out
if ($out -notmatch 'held back') {
throw "the held-back conflict was not reported — see spikes/README.md SP-1"
}
if ($out -notmatch 'httpx') {
throw "the blocker was not attributed to httpx"
}
- name: End-to-end — update, snapshot, rollback
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
.\target\debug\pipdock.exe snapshot create --env .itest
# Pin the id now. `latest` is not stable across this flow: `update --all` writes a
# Trigger::Plan snapshot before it mutates, and `snapshot rollback` writes a
# Trigger::Rollback one before it restores (DATA-FLOW §8 — rollback is itself
# reversible). Asserting against `latest` afterwards compares the restored
# environment to the *pre-rollback* state and always reports a full diff.
$id = @(.\target\debug\pipdock.exe snapshot list --env .itest --json | ConvertFrom-Json)[0].id
if (-not $id) { throw "snapshot create left nothing for snapshot list to report" }
Write-Host "pinned snapshot $id"
.\target\debug\pipdock.exe update --all --env .itest --yes
if ($LASTEXITCODE -ne 0) { throw "update --all failed" }
.\target\debug\pipdock.exe snapshot rollback $id --env .itest --yes
if ($LASTEXITCODE -ne 0) { throw "rollback failed" }
# PRD §6's rollback metric: the environment must match the snapshot exactly.
# Assert on the structure, not the prose — a reworded success line must not be able
# to pass this, and an unrelated string must not be able to fail it.
$diff = .\target\debug\pipdock.exe snapshot diff $id --env .itest --json | ConvertFrom-Json
Write-Host ($diff | ConvertTo-Json -Depth 5)
# @($null).Count is 1 in PowerShell, so filter rather than count directly — an omitted
# field must read as "no drift", not as one phantom package.
$drift = @(@($diff.added) + @($diff.removed) + @($diff.changed) |
Where-Object { $null -ne $_ })
if ($drift.Count -ne 0) {
throw "rollback left $($drift.Count) package(s) different from snapshot $id"
}
- name: End-to-end — the uninstall guard refuses to break things
shell: pwsh
run: |
$ErrorActionPreference = 'Continue'
.itest\Scripts\python.exe -m pip install --quiet requests
.\target\debug\pipdock.exe uninstall urllib3 --env .itest
# CLI-SPEC §7: a guard trip exits non-zero so `||` works in scripts.
$guard = $LASTEXITCODE
if ($guard -eq 0) { throw "the guard did not refuse to break requests" }
Write-Host "guard tripped as expected (exit $guard)"
# This step succeeds *because* the command failed, so the non-zero code must not leak:
# the runner ends a pwsh step with `exit $LASTEXITCODE`, which would fail the step on
# the very outcome it is asserting.
exit 0
- name: End-to-end — exit codes match CLI-SPEC §5
shell: pwsh
run: |
$ErrorActionPreference = 'Continue'
.\target\debug\pipdock.exe list --env C:\definitely\not\here 2>$null
if ($LASTEXITCODE -ne 3) { throw "a missing environment should exit 3, got $LASTEXITCODE" }
.\target\debug\pipdock.exe doctor --env .itest > $null
if ($LASTEXITCODE -ne 0) { throw "doctor on a healthy env should exit 0, got $LASTEXITCODE" }
- name: End-to-end — the JSON contracts are real
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
foreach ($t in @('ResolutionReport', 'ExecutionSummary', 'Dist', 'Code')) {
.\target\debug\pipdock.exe schema $t | ConvertFrom-Json | Out-Null
}
.\target\debug\pipdock.exe list --env .itest --json | ConvertFrom-Json | Out-Null
# Two probe claims that only a real editable install can test: it must be listed once
# (its own .egg-info joins the venv's .dist-info on sys.path), and it must report no
# size (its RECORD lists the import shim, not the sources — a few hundred bytes for a
# project of any size, which is worse than reporting nothing).
- name: End-to-end — an editable install is listed once, and without a size
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
New-Item -ItemType Directory -Force .itest-editable\src\edmod | Out-Null
# An array rather than a here-string: a here-string's closing '@ must sit at column 0
# after YAML strips this block's indentation, which makes it break on a re-indent.
Set-Content .itest-editable\pyproject.toml @(
'[build-system]',
'requires = ["setuptools>=64"]',
'build-backend = "setuptools.build_meta"',
'[project]',
'name = "edmod"',
'version = "1.0.0"'
)
Set-Content .itest-editable\src\edmod\__init__.py '"""edmod."""'
# 2 MiB of real content, so a wrong size would be unmistakable.
Set-Content .itest-editable\src\edmod\payload.py ('DATA = "' + ('y' * 2MB) + '"')
.itest\Scripts\python.exe -m pip install --quiet -e .itest-editable
$dists = .\target\debug\pipdock.exe list --env .itest --json | ConvertFrom-Json
$ed = @($dists | Where-Object { $_.name -eq 'edmod' })
if ($ed.Count -ne 1) {
throw "edmod should be listed once, got $($ed.Count) — probe.py is not deduplicating"
}
if ($null -ne $ed[0].sizeBytes) {
throw "an editable install must report no size, got $($ed[0].sizeBytes) bytes"
}
$sized = @($dists | Where-Object { $null -ne $_.sizeBytes })
if ($sized.Count -lt 1) { throw "no distribution reported a size at all" }
Write-Host "$($dists.Count) distributions, $($sized.Count) with a size, edmod correctly without"
fixture-drift:
name: Re-capture against the latest engines
# Weekly rather than nightly: it hits PyPI hard enough that daily would be rude, and engine
# releases are not a daily event.
# Not gated on run_attempt: re-running the workflow is how you reproduce a drift failure,
# so skipping this job on a retry would defeat the point.
if: (github.event_name == 'schedule' && github.event.schedule == '17 4 * * 1') || github.event_name == 'workflow_dispatch'
runs-on: windows-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v7
# An exact patch, on purpose. pip's report records `python_full_version` and uv prints "Using
# Python 3.12.x environment", so a floating `3.12` turns every runner-image Python bump into
# a fixture diff that says nothing about pip or uv. Bump it deliberately, and re-capture.
- name: Setup Python
uses: actions/setup-python@v7
with:
python-version: '3.12.10'
- name: Install the newest uv
run: |
python -m venv .uvtools
.uvtools\Scripts\python.exe -m pip install --quiet --upgrade pip uv
shell: pwsh
- name: Re-capture fixtures
shell: pwsh
run: |
python spikes\capture.py --engine pip
python spikes\capture.py --engine uv --uv-exe .uvtools\Scripts\uv.exe
# Uploaded whether or not the gate below passes: when it fails, the fix is to review and
# commit exactly these bytes, and re-running the capture elsewhere produces different ones
# (another interpreter, another network, another day's index).
- name: Upload the captured fixtures
if: always()
uses: actions/upload-artifact@v7
with:
name: captured-fixtures
path: |
crates/pipdock-core/tests/fixtures/pip
crates/pipdock-core/tests/fixtures/uv
if-no-files-found: error
# A diff means an engine changed its output. That is exactly the PD-ENG-003 situation the
# adapters degrade into, and catching it here is what stops a user meeting it first.
#
# capture-provenance.json is excluded: it records the engine and interpreter versions and
# the argv used, all of which move on every engine release and every setup-python bump.
# Gating on it would fire this job constantly for reasons that have nothing to do with
# output shape. It is still rewritten and still committable, so `git diff` after a run
# always tells you which engine produced the current bytes.
- name: Fail if the captured shapes moved
shell: pwsh
run: |
$paths = @(
'crates/pipdock-core/tests/fixtures',
':!crates/pipdock-core/tests/fixtures/**/capture-provenance.json'
)
git diff --stat -- $paths
if (git status --porcelain -- $paths) {
git diff --text -- $paths
throw "engine output changed — review the diff, update the adapters, commit the new fixtures"
}
- name: Parsers still agree with the new captures
run: cargo test --workspace --locked