feat: add PyPI resource and pyproject autodiscovery plugins by loispostula · Pull Request #8155 · updatecli/updatecli

loispostula · 2026-03-25T21:34:23Z

Add Python ecosystem support to updatecli with two new plugins: a pypi resource and pyproject autodiscovery.

PyPI Resource (`pkg/plugins/resources/pypi/`)

Queries the PyPI JSON API for Python package versions.

Source: returns the latest version matching a version filter
Condition: checks whether a specific version exists
Changelog: extracts GitHub release notes from project_urls, falls back to PyPI release page links
Private registries: configurable URL + Bearer token
PEP 440 → semver normalization: converts Python pre-release versions (0.51b0 → 0.51.0-beta.0, 1.0a1 → 1.0.0-alpha.1, 2.0rc1 → 2.0.0-rc.1) so that semver version filtering works with the Python ecosystem
Yanked versions are excluded

Pyproject Autodiscovery (`pkg/plugins/autodiscovery/pyproject/`)

Discovers pyproject.toml files and generates updatecli manifests for dependency updates.

Detects package managers via lock files (uv supported, architecture ready for poetry/pdm)
Follows the npm multi-PM pattern with detectLockFileSupport()
Parses PEP 508 dependency strings (name, extras, version constraints, environment markers)
Handles [project.dependencies] and [project.optional-dependencies]
Generated manifests use pypi source + uv add shell target (updates pyproject.toml + uv.lock atomically)
Registered as pyproject with alias python/uv
Supports ignore/only matching rules, custom versionfilter, and custom indexurl

Example configuration

autodiscovery:
   scmid: default
   crawlers:
     pyproject:
       only:
         - packages:
             "requests": ""
       versionfilter:
         kind: semver
         pattern: minoronly

Test

go test ./pkg/plugins/resources/pypi/... ./pkg/plugins/autodiscovery/pyproject/... -v

e2e tests under e2e/updatecli.d/success.d/autodiscovery/pyproject/ using astral-sh/uv-fastapi-example

Additional Information

Checklist

I have updated the documentation via pull request in website repository: docs: add PyPI resource and pyproject autodiscovery documentation website#2769

Tradeoff

uv add does not support --dry-run, following the same approach as pnpm in the npm autodiscovery. The changedif: file/checksum mechanism handles idempotency.
PEP 440 → semver normalization is best-effort: covers a/b/rc pre-releases, .dev releases are excluded, .post releases map to build metadata. Calendar versioning (e.g., 2024.1) is padded to 3-part semver. User can avoid this by using only/ignore

Potential improvement

Add poetry/pdm support by extending detectLockFileSupport() and adding corresponding target templates
Add [dependency-groups] support (PEP 735)

olblak · 2026-03-30T15:24:47Z

PEP 440 → semver normalization is best-effort: covers a/b/rc pre-releases, .dev releases are excluded, .post releases map to build metadata. Calendar versioning (e.g., 2024.1) is padded to 3-part semver. User can avoid this by using only/ignore

Should we leverage a library such as https://github.com/aquasecurity/go-pep440-version

olblak · 2026-03-30T15:25:03Z

That being said we are starting to have a lot of devependencies

loispostula · 2026-03-31T20:12:29Z

@olblak now that #8245 has been merged, i've rebased this pr and incorporated the new versionfilter

Copilot

Pull request overview

Adds Python ecosystem support to Updatecli by introducing a new pypi resource (PyPI JSON API-backed version discovery/conditions/changelogs) and a new pyproject autodiscovery crawler (pyproject.toml dependency discovery with uv/uv.lock support), plus CI/config updates to support running uv-based e2e coverage.

Changes:

Add pypi resource plugin (source/condition/changelog) and register it in the resource factory.
Add pyproject autodiscovery crawler (uv lock detection, PEP 508 parsing, only/ignore rules, manifest template generation) and register it with alias python/uv.
Add uv installation to CI and add an Updatecli policy to keep the workflow uv version pinned/up to date; add e2e autodiscovery configs.

Reviewed changes

Copilot reviewed 29 out of 33 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
updatecli/updatecli.d/uv.yaml	Updatecli policy to bump `setup-uv` version in GitHub workflows
pkg/plugins/resources/pypi/main.go	Implements PyPI JSON API fetch + version filtering/normalization + report config
pkg/plugins/resources/pypi/source.go	`pypi` source implementation
pkg/plugins/resources/pypi/source_test.go	Source unit tests (filtering, yanked handling, auth cases)
pkg/plugins/resources/pypi/condition.go	`pypi` condition implementation (version existence)
pkg/plugins/resources/pypi/condition_test.go	Condition unit tests
pkg/plugins/resources/pypi/changelog.go	Changelog extraction (GitHub releases fallback to PyPI links)
pkg/plugins/resources/pypi/changelog_test.go	Changelog unit tests
pkg/plugins/resources/pypi/target.go	Explicitly marks target as unsupported
pkg/plugins/resources/pypi/main_test.go	`New()` constructor validation tests
pkg/plugins/resources/pypi/test_utils.go	HTTP mock utilities + fixture JSON payloads
pkg/plugins/autodiscovery/pyproject/main.go	Crawler spec/config + initialization (defaults, rootdir handling, uv presence)
pkg/plugins/autodiscovery/pyproject/dependencies.go	TOML parsing + dependency discovery + manifest generation
pkg/plugins/autodiscovery/pyproject/manifestTemplate.go	Template used to generate Updatecli manifests for discovered deps
pkg/plugins/autodiscovery/pyproject/utils.go	File walking, uv/lock detection, PEP 508 parsing helpers
pkg/plugins/autodiscovery/pyproject/utils_test.go	Unit tests for PEP 508 parsing
pkg/plugins/autodiscovery/pyproject/matchingRule.go	Only/Ignore matching rules (path + package constraints)
pkg/plugins/autodiscovery/pyproject/matchingRule_test.go	Unit tests for matching rules logic
pkg/plugins/autodiscovery/pyproject/main_test.go	End-to-end-ish manifest generation tests from testdata projects
pkg/plugins/autodiscovery/pyproject/testdata/simple_project/pyproject.toml	Test fixture: basic deps
pkg/plugins/autodiscovery/pyproject/testdata/simple_project/uv.lock	Test fixture: uv lock presence
pkg/plugins/autodiscovery/pyproject/testdata/optional_deps/pyproject.toml	Test fixture: optional-dependencies groups
pkg/plugins/autodiscovery/pyproject/testdata/optional_deps/uv.lock	Test fixture: uv lock presence
pkg/plugins/autodiscovery/pyproject/testdata/no_version/pyproject.toml	Test fixture: dependency without constraint
pkg/plugins/autodiscovery/pyproject/testdata/no_version/uv.lock	Test fixture: uv lock presence
pkg/plugins/autodiscovery/pyproject/testdata/no_lockfile/pyproject.toml	Test fixture: source-only mode (no uv.lock)
pkg/plugins/autodiscovery/pyproject/testdata/markers/pyproject.toml	Test fixture: env markers in deps
pkg/plugins/autodiscovery/pyproject/testdata/markers/uv.lock	Test fixture: uv lock presence
pkg/core/pipeline/resource/main.go	Registers new `pypi` resource kind
pkg/core/pipeline/autodiscovery/main.go	Registers new `pyproject` crawler + alias `python/uv`
e2e/updatecli.d/success.d/autodiscovery/pyproject/pyproject.yaml	e2e config for pyproject autodiscovery (only rule)
e2e/updatecli.d/success.d/autodiscovery/pyproject/pyproject.versionfilter.yaml	e2e config for pyproject autodiscovery with versionfilter override
.github/workflows/go.yaml	Installs `uv` in CI to support pyproject/uv e2e coverage

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

olblak · 2026-04-01T06:38:46Z

@loispostula Thanks for the pullrequest, that's a big beast 😄 could you review copilot message?
Feel free to ignore nit pick comment

Add Python ecosystem support to updatecli with two new plugins: - pypi resource: queries PyPI JSON API for package versions, with PEP 440 to semver normalization (a/b/rc pre-releases), private registry support (Bearer token), and yanked version filtering. - pyproject autodiscovery: discovers pyproject.toml + uv.lock pairs, parses PEP 508 dependencies, generates manifests using pypi source and uv add shell target. Named pyproject (alias python/uv) for multi-PM extensibility following the npm pattern. Signed-off-by: Loïs Postula <lois@postu.la>

…i#8155) * feat: add PyPI resource and pyproject autodiscovery plugins Add Python ecosystem support to updatecli with two new plugins: - pypi resource: queries PyPI JSON API for package versions, with PEP 440 to semver normalization (a/b/rc pre-releases), private registry support (Bearer token), and yanked version filtering. - pyproject autodiscovery: discovers pyproject.toml + uv.lock pairs, parses PEP 508 dependencies, generates manifests using pypi source and uv add shell target. Named pyproject (alias python/uv) for multi-PM extensibility following the npm pattern. Signed-off-by: Loïs Postula <lois@postu.la> * fix: cursor comments --------- Signed-off-by: Loïs Postula <lois@postu.la>

* feat(github): add client-side pull request merge option (#2312) Add a new 'clientsidemerge' option to the GitHub action spec that allows updatecli to merge a pull request directly via the GitHub GraphQL API (mergePullRequest mutation), bypassing the need for server-side auto-merge which requires a paid plan or a public repository. When 'clientsidemerge: true' is set, updatecli will attempt to merge the pull request immediately after it is created or updated. The merge will fail if the PR is not in a mergeable state (e.g. has conflicts or failing required checks). The merge method is controlled by the existing 'mergemethod' field and defaults to GitHub's default ('MERGE') if not specified. The 'usetitleforautomerge' flag is also respected to set a custom commit headline for squash and rebase merges. Fixes #2312 * feat(github): restructure merge spec and deprecate automerge field - Replace ClientSideMerge bool with nested Merge.Strategy/After fields - Deprecate AutoMerge bool -> *bool with migration warning to merge.strategy: auto - Add merge.after support to delay client-side merge by a configured duration - Minimize mutationMergePullRequest response struct (Copilot suggestion) - Guard nil rateLimit before calling Pause() in MergePullRequest - Add CreatedAt field to PullRequestApi * feat: add PyPI resource and pyproject autodiscovery plugins (#8155) * feat: add PyPI resource and pyproject autodiscovery plugins Add Python ecosystem support to updatecli with two new plugins: - pypi resource: queries PyPI JSON API for package versions, with PEP 440 to semver normalization (a/b/rc pre-releases), private registry support (Bearer token), and yanked version filtering. - pyproject autodiscovery: discovers pyproject.toml + uv.lock pairs, parses PEP 508 dependencies, generates manifests using pypi source and uv add shell target. Named pyproject (alias python/uv) for multi-PM extensibility following the npm pattern. Signed-off-by: Loïs Postula <lois@postu.la> * fix: cursor comments --------- Signed-off-by: Loïs Postula <lois@postu.la> * fix: various fixes and improvements * use constants for merge client strategy * add closing pullrequest in cleanAction * set default value to 24h for auto merge client * correctly fallback from automerge setting to either manual or auto Signed-off-by: Olblak <me@olblak.com> * fix: ignore gosec Signed-off-by: Olblak <me@olblak.com> --------- Signed-off-by: Loïs Postula <lois@postu.la> Signed-off-by: Olblak <me@olblak.com> Co-authored-by: Olivier Vernin <olivier@vernin.me> Co-authored-by: Loïs Postula <lois@postu.la> Co-authored-by: Olblak <me@olblak.com>

loispostula force-pushed the push-ultwmqxxtlvu branch 2 times, most recently from 34c38d7 to 0918799 Compare March 26, 2026 11:35

loispostula added enhancement New feature or request autodiscovery All things related to the autodiscovery feature resource-pypi labels Mar 26, 2026

loispostula mentioned this pull request Mar 31, 2026

Feature Request: Support new versionfilter kind pep440 #8244

Closed

loispostula force-pushed the push-ultwmqxxtlvu branch from f0a216c to df8765b Compare March 31, 2026 20:05

olblak requested a review from Copilot April 1, 2026 06:27

Copilot started reviewing on behalf of olblak April 1, 2026 06:28 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

Comment thread pkg/plugins/resources/pypi/main.go Outdated

Comment thread pkg/plugins/resources/pypi/main.go Outdated

Comment thread pkg/plugins/resources/pypi/condition.go

Comment thread pkg/plugins/resources/pypi/main.go Outdated

Comment thread pkg/plugins/autodiscovery/pyproject/utils.go

loispostula force-pushed the push-ultwmqxxtlvu branch from 83d1c74 to e06458b Compare April 1, 2026 18:54

loispostula force-pushed the push-ultwmqxxtlvu branch 2 times, most recently from 3f57dc8 to 24ad585 Compare April 1, 2026 21:32

fix: cursor comments

a6f721d

loispostula force-pushed the push-ultwmqxxtlvu branch from 24ad585 to a6f721d Compare April 3, 2026 17:33

olblak merged commit ab9569c into updatecli:main Apr 3, 2026
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add PyPI resource and pyproject autodiscovery plugins#8155

feat: add PyPI resource and pyproject autodiscovery plugins#8155
olblak merged 2 commits into
updatecli:mainfrom
loispostula:push-ultwmqxxtlvu

loispostula commented Mar 25, 2026 •

edited

Loading

Uh oh!

olblak commented Mar 30, 2026

Uh oh!

olblak commented Mar 30, 2026

Uh oh!

loispostula commented Mar 31, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

olblak commented Apr 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

loispostula commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PyPI Resource (pkg/plugins/resources/pypi/)

Pyproject Autodiscovery (pkg/plugins/autodiscovery/pyproject/)

Example configuration

Test

Additional Information

Checklist

Tradeoff

Potential improvement

Uh oh!

olblak commented Mar 30, 2026

Uh oh!

olblak commented Mar 30, 2026

Uh oh!

loispostula commented Mar 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

olblak commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

loispostula commented Mar 25, 2026 •

edited

Loading

PyPI Resource (`pkg/plugins/resources/pypi/`)

Pyproject Autodiscovery (`pkg/plugins/autodiscovery/pyproject/`)

olblak commented Apr 1, 2026 •

edited

Loading