Skip to content

Pin pyproject.toml PyPI dependencies to exact pixi.lock versions - #417

Open
mkitti wants to merge 3 commits into
mainfrom
mkitti-sync-pypi-versions
Open

Pin pyproject.toml PyPI dependencies to exact pixi.lock versions#417
mkitti wants to merge 3 commits into
mainfrom
mkitti-sync-pypi-versions

Conversation

@mkitti

@mkitti mkitti commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds scripts/sync_pyproject_versions.py, which reads pixi.lock and rewrites [project.dependencies], [project.optional-dependencies], and [dependency-groups] in pyproject.toml to the exact versions pixi resolved, so pip install fileglancer matches the pixi/conda environment as closely as possible.
  • Where a package resolves to different versions across pixi environments/platforms (e.g. a compiled test-only dependency built differently per platform), the entry is split into multiple PEP 508 lines qualified with python_version/sys_platform/platform_machine markers instead of a single pin, so both pip install and pixi install stay satisfiable.
  • Adds a sync-pypi-versions pixi task (pixi run sync-pypi-versions) so the script is easy to run locally.
  • Adds .github/workflows/sync-pypi-versions.yml, which runs on push to main (or manual workflow_dispatch), checks whether pixi.lock changed, runs the script, and opens a PR with the resulting pyproject.toml diff.
  • --unpin mode: because the exact pins become part of the editable fileglancer package's own metadata, pixi feeds them back into its own solve on every pixi update -- an exact pin can't move, and can even make pixi update fail outright if a conda-resolved version has since diverged from it. pixi run unpin-pypi-versions restores each dependency to a semver range (from the matching [tool.pixi.*] table, or the new [tool.sync-pypi-versions.ranges] table for the three PyPI-only packages -- x2s3, py-cluster-api, build -- that have no such mirror), so pixi update has room to move again. The intended cycle is pixi run unpin-pypi-versions && pixi update && pixi run sync-pypi-versions, or the single pixi run bump-pypi-versions task.
  • Build hook (hatch_build.py): registers a Hatchling build hook that runs sync_pyproject_versions.py --check before a standard wheel build and fails loudly if pyproject.toml doesn't match pixi.lock -- catching a forgotten re-pin before it gets published. It explicitly skips editable builds, since pixi resolves editable-install metadata constantly (including while intentionally unpinned mid-bump-pypi-versions), and that must not fail. Also adds pyyaml to [build-system].requires so the check script can import it inside the isolated build environment pip/build creates.
  • Regenerates pixi.lock to reflect the new exact pins (only the fileglancer package's own requires_dist metadata changes, plus an earlier pixi update that bumped unrelated transitive/build tooling).

Test plan

  • pixi run sync-pypi-versions / pixi run unpin-pypi-versions are each idempotent when run twice in a row
  • Full cycle (unpin -> pixi update -> sync) round-trips back to identical exact versions
  • pixi install -e default|test|release|test-py312 all succeed after the pins are applied
  • Verified the coverage package (which builds differently on macOS x86_64 + Python 3.12 vs. everywhere else) correctly splits into two marker-qualified entries, and that pixi.lock re-locks it as the equivalent python_full_version/platform_machine/sys_platform marker
  • pixi run -e release pypi-build succeeds when pinned (and the built wheel's METADATA carries the exact pins/markers) and fails with a clear error when deliberately left unpinned
  • CI (build.yml) passes on this branch
  • Manually trigger sync-pypi-versions.yml via workflow_dispatch to confirm the PR-opening step works end-to-end

🤖 Generated with Claude Code

Adds scripts/sync_pyproject_versions.py to sync [project.dependencies],
[project.optional-dependencies], and [dependency-groups] to the exact
versions pixi resolved, so `pip install fileglancer` matches the pixi/conda
environment. Where a package resolves differently across pixi environments
or platforms, the entry is split into multiple python_version/sys_platform/
platform_machine-qualified requirements instead of a single pin.

Adds .github/workflows/sync-pypi-versions.yml to run the script and open a
PR whenever pixi.lock changes on main (or on manual workflow_dispatch).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mkitti and others added 2 commits July 29, 2026 17:01
Exact-pinning [project.dependencies] etc. feeds directly back into pixi's
own solve (via the editable fileglancer package's own metadata), so once
pinned, `pixi update` had no room to move a package forward and could even
fail outright if a conda-resolved version diverged from the pin.

scripts/sync_pyproject_versions.py --unpin restores each dependency to a
semver range instead of an exact pin -- from the matching [tool.pixi.*]
table where one exists, or from the new [tool.sync-pypi-versions.ranges]
table for the three PyPI-only packages (x2s3, py-cluster-api, build) that
have no such mirror. The intended maintenance cycle is now:

    pixi run unpin-pypi-versions && pixi update && pixi run sync-pypi-versions

or the equivalent single `pixi run bump-pypi-versions` task.

Also includes the pixi.lock update from `pixi update` (transitive/build
tooling only -- libgcc, uv, prompt-toolkit, etc. -- since re-running the
full unpin/update/re-pin cycle landed back on identical exact versions for
every package sync_pyproject_versions.py manages).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
hatch_build.py registers a Hatchling build hook that runs
`scripts/sync_pyproject_versions.py --check` before a *standard* wheel
build and fails loudly if pyproject.toml's PyPI dependency versions don't
match pixi.lock -- catching a forgotten `pixi run sync-pypi-versions` after
`pixi update` before it gets published.

The hook explicitly skips editable builds (version == "editable"), since
pixi resolves editable-install metadata constantly -- including while
pyproject.toml is intentionally unpinned mid `pixi run bump-pypi-versions`
-- and that must not fail. Verified against a real `pixi run -e release
pypi-build`: succeeds and prints the check result when pinned, fails with a
clear error when not, and the built wheel's METADATA carries the exact pins.

Adds pyyaml to [build-system].requires so the check script can actually
import it inside the isolated build environment pip/build creates (which
only contains the declared build-system requirements, not pixi's runtime
deps).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant