feat(core): the shipped pin ledger, parsed once and hashed by meaning
CODE-HEALTH-SPEC §2 says the tools venv is built from exact pins shipped with the app. `tools/tools-requirements.txt` has existed since the first scaffold commit and is already Dependabot-wired at /tools; nothing read it. This reads it. Two things are deliberate and easy to get wrong later. The ledger carries four pins and Code Health installs three. `pip-audit` belongs to the post-1.0 Security tab (PRD P1-1), so installing it would make every user's first Health click download a feature they cannot reach — and its closure contains `msgpack`, which publishes no `py3-none-any` fallback and would hard-fail under `--only-binary=:all:` on a new CPython. It stays in the ledger so Dependabot keeps bumping it. `pins()` fails the suite if any HEALTH_TOOLS name stops resolving, so a rename is a cargo test failure rather than a PD-HLT-001 on a user's machine. The hash is over the parsed pin set, never over the file's bytes. `include_str!` bakes in the build machine's working tree and `.gitattributes`' `* text=auto` makes that core.autocrlf-dependent, so a byte hash would differ between two builds of the same commit and every user of one of them would re-sync on first run — a bug that passes every test. `line_endings_do_not_change_the_hash` is that trap turned into a gate. Also settles the M3 "top unknown": there is no TOOLS_PYTHON_MAX. deptry ships cp310-abi3 and the same wheel installs on 3.12 and 3.14, verified by running both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
4f15936f5a
commit
3ff1d58830
1 file changed
+281
-1
@@ -4,7 +4,10 @@
|
||||
//! **report-only**, and the sole write path is `ruff --fix` / `ruff format` behind an explicit
|
||||
//! confirm. PipDock never edits `pyproject.toml` or `requirements.txt` for the user.
|
||||
|
||||
use crate::errors::Result;
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use crate::errors::{Code, PdError, Result};
|
||||
use crate::model::{PinnedSpec, PkgName, Version};
|
||||
|
||||
/// CODE-HEALTH-SPEC §4: per-tool watchdog; exceeding it yields a partial report (`PD-HLT-003`).
|
||||
pub const TOOL_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(120);
|
||||
@@ -15,6 +18,180 @@ pub const DEFAULT_MIN_CONFIDENCE: u8 = 80;
|
||||
/// CODE-HEALTH-SPEC §4: directories excluded from every tool, plus user globs from Settings.
|
||||
pub const DEFAULT_EXCLUDES: &[&str] = &[".venv", "venv", "node_modules", "build", "dist", ".git"];
|
||||
|
||||
/// The release-time pin ledger, baked into the binary (CODE-HEALTH-SPEC §2).
|
||||
///
|
||||
/// Reached through `CARGO_MANIFEST_DIR` rather than a relative `include_str!` so the path does not
|
||||
/// silently depend on how deeply this module is nested. The file lives outside the crate, which
|
||||
/// would break `cargo package` — irrelevant while nothing is published to crates.io, but the
|
||||
/// reason to keep it a single well-marked constant rather than several.
|
||||
///
|
||||
/// Dependabot bumps it in its own `pip` ecosystem at `/tools` (RELEASE-CI §2), so a tool upgrade is
|
||||
/// never bundled with application dependencies.
|
||||
pub const TOOLS_REQUIREMENTS: &str = include_str!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/../../tools/tools-requirements.txt"
|
||||
));
|
||||
|
||||
/// The tools actually installed into the venv, out of the shipped ledger.
|
||||
///
|
||||
/// The ledger is the **release-time pin set** and also carries `pip-audit` for the post-1.0
|
||||
/// Security tab (PRD P1-1). Code Health is three tools (CODE-HEALTH-SPEC §1), and installing a
|
||||
/// fourth would make every user's first Health click download a feature they cannot reach.
|
||||
///
|
||||
/// It is also the one pin that would import a real CPython-ABI risk. Verified 2026-08-12: the
|
||||
/// three tools here resolve to `deptry-*-cp310-abi3-win_amd64` (stable ABI) and `py3-none-*`
|
||||
/// wheels, and the only version-tagged member of their closure — `tomli`, required by deptry under
|
||||
/// `python_full_version < "3.15"` — also publishes a `py3-none-any` fallback, so a CPython with no
|
||||
/// `cpXXX` wheel degrades to pure Python rather than to an sdist build. `pip-audit`'s closure
|
||||
/// contains `msgpack`, which publishes **no** universal fallback at all and would therefore hard-
|
||||
/// fail under `--only-binary=:all:` on a new CPython.
|
||||
///
|
||||
/// Kept in the ledger so Dependabot keeps bumping it; excluded here. `pins` fails the build's test
|
||||
/// suite if a name below stops resolving, so a Dependabot rename is a `cargo test` failure rather
|
||||
/// than a `PD-HLT-001` on a user's machine.
|
||||
pub const HEALTH_TOOLS: &[&str] = &["deptry", "vulture", "ruff"];
|
||||
|
||||
/// The oldest interpreter that may host the tools venv.
|
||||
///
|
||||
/// The binding floor is `deptry`'s and `pip-audit`'s `requires_python` of `>=3.10`, which is
|
||||
/// already the floor everywhere else in PipDock. **There is deliberately no ceiling** — see
|
||||
/// [`HEALTH_TOOLS`] for the wheel-tag evidence, and `--only-binary=:all:` for what happens if that
|
||||
/// ever stops being true.
|
||||
pub const MIN_TOOLS_PYTHON: (u32, u32) = crate::envs::MIN_PROBE_PYTHON;
|
||||
|
||||
/// The pin set to install, parsed out of the shipped ledger and sorted by normalized name.
|
||||
///
|
||||
/// # Errors
|
||||
/// `PD-PKG-002` when the ledger is malformed, names a version that is not shaped like one, or does
|
||||
/// not contain every entry of [`HEALTH_TOOLS`]. All three are release-time mistakes, which is why
|
||||
/// they surface as a failing test rather than as a runtime error path.
|
||||
pub fn pins() -> Result<Vec<PinnedSpec>> {
|
||||
let ledger = parse_ledger(TOOLS_REQUIREMENTS)?;
|
||||
HEALTH_TOOLS
|
||||
.iter()
|
||||
.map(|tool| {
|
||||
let name = PkgName::parse(tool)?;
|
||||
let version = ledger.get(&name).cloned().ok_or_else(|| {
|
||||
PdError::new(
|
||||
Code::PkgNotFound,
|
||||
format!(
|
||||
"tools-requirements.txt has no pin for {tool:?}; \
|
||||
Code Health cannot be built without one"
|
||||
),
|
||||
)
|
||||
})?;
|
||||
Ok(PinnedSpec { name, version })
|
||||
})
|
||||
.collect::<Result<Vec<_>>>()
|
||||
.map(sorted_by_name)
|
||||
}
|
||||
|
||||
/// SHA-256, hex, over the normalized pin set.
|
||||
///
|
||||
/// Over the **parsed** pins, never over the ledger's bytes. `include_str!` bakes in the working
|
||||
/// tree, and `.gitattributes`' `* text=auto` makes that `core.autocrlf`-dependent — so a byte hash
|
||||
/// would differ between two builds of the same commit, and every user of one of them would re-sync
|
||||
/// on first run. A bug that passes every test.
|
||||
///
|
||||
/// Invariant under line endings, comments, blank lines, ordering, a trailing newline and name
|
||||
/// casing. Changes exactly when a pin's name or version does.
|
||||
///
|
||||
/// Same construction as `envs::env_hash`, so there is one hashing idiom in the codebase.
|
||||
#[must_use]
|
||||
pub fn pins_hash(pins: &[PinnedSpec]) -> String {
|
||||
use std::fmt::Write as _;
|
||||
|
||||
use sha2::{Digest as _, Sha256};
|
||||
|
||||
let digest = Sha256::digest(requirements_body(pins).as_bytes());
|
||||
digest
|
||||
.iter()
|
||||
.fold(String::with_capacity(64), |mut acc, byte| {
|
||||
// Infallible: writing to a String cannot fail, and `?` here would need a Result return
|
||||
// for a function that has no other failure mode.
|
||||
let _ = write!(acc, "{byte:02x}");
|
||||
acc
|
||||
})
|
||||
}
|
||||
|
||||
/// The requirements file written into the tools directory, and the bytes the hash covers.
|
||||
///
|
||||
/// A **rendering**, not a copy of [`TOOLS_REQUIREMENTS`]. That is what keeps the excluded `pip-audit`
|
||||
/// pin, the ledger's comments and whatever line endings the build machine checked out from reaching
|
||||
/// either the hash or the user's disk. Always LF-terminated.
|
||||
#[must_use]
|
||||
pub fn requirements_body(pins: &[PinnedSpec]) -> String {
|
||||
pins.iter().fold(String::new(), |mut acc, pin| {
|
||||
acc.push_str(&pin.to_requirement());
|
||||
acc.push('\n');
|
||||
acc
|
||||
})
|
||||
}
|
||||
|
||||
/// Sort by normalized name, so a reordered ledger is not a re-sync.
|
||||
fn sorted_by_name(mut pins: Vec<PinnedSpec>) -> Vec<PinnedSpec> {
|
||||
pins.sort_by(|a, b| a.name.cmp(&b.name));
|
||||
pins
|
||||
}
|
||||
|
||||
/// Parse every `name==version` line of a pin ledger.
|
||||
///
|
||||
/// A `BTreeMap` rather than a `Vec` so a duplicated entry collapses instead of being installed
|
||||
/// twice, and so ordering cannot reach the hash.
|
||||
///
|
||||
/// **Only `==` is accepted.** CODE-HEALTH-SPEC §2 promises "no floating versions at runtime"; a
|
||||
/// `>=` line is a release-time mistake and must fail here rather than pin to a moving target on a
|
||||
/// user's machine.
|
||||
fn parse_ledger(text: &str) -> Result<BTreeMap<PkgName, Version>> {
|
||||
let mut out = BTreeMap::new();
|
||||
for raw in text.lines() {
|
||||
let line = raw.split('#').next().unwrap_or("").trim();
|
||||
if line.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let Some((name, version)) = line.split_once("==") else {
|
||||
return Err(PdError::new(
|
||||
Code::PkgNotFound,
|
||||
format!(
|
||||
"tools-requirements.txt line {line:?} is not an exact `name==version` pin; \
|
||||
CODE-HEALTH-SPEC §2 forbids floating versions at runtime"
|
||||
),
|
||||
));
|
||||
};
|
||||
out.insert(
|
||||
PkgName::parse(name.trim())?,
|
||||
validated_version(version.trim())?,
|
||||
);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Refuse a ledger version that is not shaped like one.
|
||||
///
|
||||
/// The same character-class check `pins::validated_hold` applies, and for the same reason: these
|
||||
/// strings reach argv as `name==version`, so SECURITY §2's "validated before argv" claim has to
|
||||
/// hold here too. Deliberately not a full PEP 440 parse — the job is to refuse whitespace, quotes,
|
||||
/// path separators and control characters, not to second-guess an epoch or a local-version segment.
|
||||
fn validated_version(raw: &str) -> Result<Version> {
|
||||
let invalid = || {
|
||||
PdError::new(
|
||||
Code::PkgNotFound,
|
||||
format!("invalid version in tools-requirements.txt: {raw:?}"),
|
||||
)
|
||||
};
|
||||
let bytes = raw.as_bytes();
|
||||
if bytes.is_empty() || !bytes[0].is_ascii_alphanumeric() {
|
||||
return Err(invalid());
|
||||
}
|
||||
if !bytes
|
||||
.iter()
|
||||
.all(|&b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_' | b'+' | b'!'))
|
||||
{
|
||||
return Err(invalid());
|
||||
}
|
||||
Ok(Version(raw.to_owned()))
|
||||
}
|
||||
|
||||
/// Create or re-sync the tools venv from the shipped `tools-requirements.txt`.
|
||||
///
|
||||
/// # Errors
|
||||
@@ -23,3 +200,106 @@ pub const DEFAULT_EXCLUDES: &[&str] = &[".venv", "venv", "node_modules", "build"
|
||||
pub fn sync_tools_venv() -> Result<()> {
|
||||
todo!(r"M3: create %LOCALAPPDATA%\PipDock\tools\.venv from exact pins")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The gate that turns a Dependabot rename into a `cargo test` failure rather than a
|
||||
/// `PD-HLT-001` on a user's machine.
|
||||
#[test]
|
||||
fn the_ledger_parses_to_exactly_the_three_health_tools() {
|
||||
let pins = pins().expect("the shipped ledger must parse");
|
||||
|
||||
let names: Vec<_> = pins.iter().map(|p| p.name.to_string()).collect();
|
||||
assert_eq!(names, ["deptry", "ruff", "vulture"], "sorted by name");
|
||||
|
||||
// The ledger carries pip-audit for the post-1.0 Security tab. It must not be installed.
|
||||
assert!(
|
||||
!names.iter().any(|n| n == "pip-audit"),
|
||||
"pip-audit is in the ledger but must not reach the tools venv — see HEALTH_TOOLS"
|
||||
);
|
||||
assert!(
|
||||
parse_ledger(TOOLS_REQUIREMENTS)
|
||||
.expect("ledger parses")
|
||||
.contains_key(&PkgName::parse("pip-audit").expect("valid name")),
|
||||
"pip-audit must stay in the ledger so Dependabot keeps bumping it"
|
||||
);
|
||||
}
|
||||
|
||||
/// The CRLF trap, made into a gate.
|
||||
///
|
||||
/// `include_str!` bakes in the build machine's working tree, so with `core.autocrlf=true` this
|
||||
/// constant arrives with CRLF. A hash over bytes would then differ between two builds of the
|
||||
/// same commit, and every user of one of them would re-sync on first run.
|
||||
#[test]
|
||||
fn line_endings_do_not_change_the_hash() {
|
||||
let lf = "deptry==0.25.1\nvulture==2.16\nruff==0.16.0\n";
|
||||
let crlf = lf.replace('\n', "\r\n");
|
||||
|
||||
assert_eq!(hash_of(lf), hash_of(&crlf));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reordering_the_file_does_not_change_the_hash() {
|
||||
let a = "deptry==0.25.1\nvulture==2.16\nruff==0.16.0\n";
|
||||
let b = "ruff==0.16.0\ndeptry==0.25.1\nvulture==2.16\n";
|
||||
|
||||
assert_eq!(hash_of(a), hash_of(b));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn comments_and_blank_lines_do_not_change_the_hash() {
|
||||
let bare = "deptry==0.25.1\nvulture==2.16\nruff==0.16.0\n";
|
||||
let noisy = "# a comment\n\ndeptry==0.25.1 # inline\n\nvulture==2.16\nruff==0.16.0\n\n";
|
||||
|
||||
assert_eq!(hash_of(bare), hash_of(noisy));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn changing_a_version_does_change_the_hash() {
|
||||
let before = "deptry==0.25.1\nvulture==2.16\nruff==0.16.0\n";
|
||||
let after = "deptry==0.25.1\nvulture==2.16\nruff==0.16.2\n";
|
||||
|
||||
assert_ne!(hash_of(before), hash_of(after));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_floating_version_is_refused() {
|
||||
let err = parse_ledger("ruff>=0.16.0\n").expect_err("`>=` must not parse");
|
||||
assert_eq!(err.code, Code::PkgNotFound);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_version_that_could_reach_argv_as_something_else_is_refused() {
|
||||
for bad in ["0.16.0 --break-system-packages", "../evil", "\"1.0\""] {
|
||||
assert!(
|
||||
parse_ledger(&format!("ruff=={bad}\n")).is_err(),
|
||||
"{bad:?} must not parse"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_written_requirements_are_lf_and_carry_only_the_health_tools() {
|
||||
let body = requirements_body(&pins().expect("ledger parses"));
|
||||
|
||||
assert!(!body.contains('\r'), "always LF on disk");
|
||||
assert!(!body.contains('#'), "a rendering, not a copy");
|
||||
assert!(!body.contains("pip-audit"));
|
||||
assert_eq!(body.lines().count(), HEALTH_TOOLS.len());
|
||||
assert!(body.ends_with('\n'));
|
||||
}
|
||||
|
||||
/// Parse a ledger fragment and hash it, the way `pins`/`pins_hash` do for the real one.
|
||||
fn hash_of(ledger: &str) -> String {
|
||||
let parsed = parse_ledger(ledger).expect("fragment parses");
|
||||
let pins = sorted_by_name(
|
||||
parsed
|
||||
.into_iter()
|
||||
.map(|(name, version)| PinnedSpec { name, version })
|
||||
.collect(),
|
||||
);
|
||||
pins_hash(&pins)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user