Skip to content

feat(analyzer): add PlatformIO package manager - #12442

Draft
rtzoeller wants to merge 2 commits into
oss-review-toolkit:mainfrom
rtzoeller:platformio
Draft

rtzoeller wants to merge 2 commits into
oss-review-toolkit:mainfrom
rtzoeller:platformio

Conversation

@rtzoeller

@rtzoeller rtzoeller commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Add support for PlatformIO embedded C/C++ projects, detected via platformio.ini.

PlatformIO does not provide a unified interface for querying all dependencies; libraries declared via lib_deps (and their own transitive dependencies) are parsed from the .pio/libdeps/<env> directory populated by pio pkg install as well as the local lib/ directory, while platform and framework dependencies are parsed from the output of pio pkg list.

The output of pio pkg list is a human-readable tree rather than JSON or another machine-readable format, and includes every package the platform's manifest declares as usable. For packages like ststm32 this lists all possible sub-packages, e.g. both framework-stm32cubef1 and framework-stm32cubeh7, even though only one might be used by the actual build. For framework dependencies we must determine the actually-used packages by invoking pio project metadata --json-output, which we can use to filter the overall list.

Comment thread Dockerfile
COPY --from=dart-build --chown=$USER:$USER $DART_SDK $DART_SDK

# PlatformIO
RUN pip install --no-cache-dir -U platformio=="$PLATFORMIO_VERSION"
@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.14%. Comparing base (a7efc0f) to head (3c0b6d9).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##               main   #12442   +/-   ##
=========================================
  Coverage     59.13%   59.14%           
+ Complexity     1881     1879    -2     
=========================================
  Files           365      365           
  Lines         13813    13814    +1     
  Branches       1460     1460           
=========================================
+ Hits           8169     8170    +1     
  Misses         5114     5114           
  Partials        530      530           
Flag Coverage Δ
funTest-no-external-tools 29.40% <0.00%> (-0.09%) ⬇️
test-ubuntu-26.04 42.70% <100.00%> (+<0.01%) ⬆️
test-windows-2025 42.68% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@rtzoeller
rtzoeller marked this pull request as draft September 11, 2026 15:08
@rtzoeller
rtzoeller force-pushed the platformio branch 2 times, most recently from 0d8d4cf to 8e3d177 Compare September 11, 2026 18:22
Add support for PlatformIO (https://platformio.org/) embedded C/C++
projects, detected via `platformio.ini`.

PlatformIO does not provide a unified interface for querying all
dependencies; libraries declared via `lib_deps` (and their own transitive
dependencies) are parsed from the `.pio/libdeps/<env>` directory populated
by `pio pkg install` as well as the local `lib/` directory, while platform
and framework dependencies are parsed from the output of `pio pkg list`.

The output of `pio pkg list` is a human-readable tree rather than JSON
or another machine-readable format, and includes every package the
platform's manifest declares as usable. For packages like `ststm32` this
lists all possible sub-packages, e.g. both `framework-stm32cubef1` and
`framework-stm32cubeh7`, even though only one might be used by the actual
build. For framework dependencies we must determine the actually-used
packages by invoking `pio project metadata --json-output`, which we can
use to filter the overall list.

Signed-off-by: Ryan Zoeller <ryan.zoeller@aliaro.com>
Signed-off-by: Ryan Zoeller <ryan.zoeller@aliaro.com>
@tsteenbe

Copy link
Copy Markdown
Member

Thanks for this @rtzoeller, good to see you back after #9727 and #9729. Implementing a whole new package manager is a major contribution - like the direction you are going.

CI was failing and as likely none of us maintainers know PlatformIO, so I asked Claude to help track it down. Root cause: PlatformIO Core 6.2.0 installs the platform_packages pin tool-scons@~4.40801.0, then installs its own required ~4.41101.0 and removes the pinned version again - SCons is a Core tool rather than an atmelavr platform package, so it then drops out of pio pkg list --only-platforms entirely and the expected output can't be produced. ef7b13b fixes it by pinning the platform, its packages and all libraries (including transitive ones) to exact versions, and moving the platform_packages override onto packages the platform actually manages. Feel free to cherry-pick it, or say the word and I'll push it to your branch.

I also had it review the PR. You know PlatformIO far better than we do, so please push back on anything where it's misread the tooling - it's a mix of real bugs, open questions and nits, and it hasn't been validated against real-world projects beyond the synthetic one.

Open issues - PR #12442 (PlatformIO package manager)

Correctness

1. library.properties depends version constraints are not strippedModel.kt:141-146

The comment asserts the field is "a comma-separated list of library names, without any owner or version
specification", but the Arduino library specification says otherwise:

"Version constraints for the dependency may be specified in parentheses after the name:
depends=ArduinoHttpClient (>=1.0.0)"

So a real entry yields LibraryDependency(name = "ArduinoHttpClient (>=1.0.0)"), which can never match a manifest
name. Two silent consequences: the edge is dropped in resolve(), and since the name never enters transitiveNames,
the real dependency is misclassified as a root, inflating the top of the scope. substringBefore('(').trim()
fixes it; worth a ModelTest case next to the existing depends tests.

2. java.util.Properties is the wrong parser for library.propertiesModel.kt:122

The spec only says "a key=value properties list… UTF-8 encoded" — it is not a Java .properties file, and
Properties.load() layers Java escape processing on top: \ becomes an escape character (a literal \n in a
paragraph turns into a newline, a lone \ is swallowed, a trailing \ merges the next line into the value),
: is accepted as an alternative separator, and #/! start comments. Result: silently corrupted
description/author/license, not a failure. Using StringReader over readText() correctly sidesteps the
ISO-8859-1 trap — it's only the escaping that's wrong. A split('=', limit = 2) line parser would match the spec
exactly.

3. Libraries in a dependency cycle disappear entirelyPlatformIo.kt:214-217

val transitiveNames = rawLibraries.values.flatMapTo(mutableSetOf()) { raw -> raw.dependencies.map { it.name } }
val rootNames = rawLibraries.keys - transitiveNames
return rootNames.mapNotNull { resolve(it) }

The pre-registration guard in resolve() correctly prevents infinite recursion, but if A depends on B and B on A
(and nothing else references them), both land in transitiveNames, so neither is a root and neither reaches the
graph — with no issue reported. DependencyGraphBuilder.addDependencyToGraph threads a processed set precisely for
cycles, so keeping them is safe.

4. Unresolvable dependency edges are dropped silentlyPlatformIo.kt:209

raw.dependencies.mapNotNull { resolve(it.name) } — when a declared dependency isn't installed or its name doesn't
match, the edge vanishes with no onIssue(...). Every other failure path reports an issue; this is the one silent
one, and given #1 it's the most likely to fire.

5. A missing name yields a degenerate Identifier and collapses libraries togetherModel.kt:131,
PlatformIo.kt:198

name = property("name").orEmpty() produces Identifier("PlatformIO", "", "", version), and because readLibraries
does associateBy { it.manifest.name }, every such library collides on the "" key and all but the last are
dropped. Failing the parse when name is absent would match the library.json path, where LibraryManifest.name is
non-nullable. Same class of thing for version: metadata?.version ?: manifest.version.orEmpty()
(PlatformIo.kt:247, PlatformResolver.kt:194) can leave the version component empty.

6. Dependencies are matched by bare name, ignoring ownerPlatformIo.kt:198,209

LibraryDependency carries owner and toIdentifier() uses it as the namespace
(PlatformIoDependencyHandler.kt:57), but both the index and the lookup ignore it, so two libraries with the same
name from different owners collide. PlatformIO's registry is owner/name, so the information is available.

Design questions (not blockers)

7. Root detection ignores lib_deps. pio pkg list -e <env> — already parsed for platforms — prints the
library tree with real nesting and real roots. Would using it remove the "not referenced by anyone" heuristic and
fix #3 at the same time?

8. All [env:*] sections are analyzed, ignoring [platformio] default_envs. Defensible for ORT (report
everything), but pio pkg install then runs for environments the user never builds.

9. pio pkg install mutates the user's global ~/.platformio store; only .pio/libdeps is stashed. Inherent to
the tool and consistent with other package managers — worth stating in the class KDoc, which currently mentions only
the project-local stash.

10. Project identity is PlatformIO::platformio.ini: via getFallbackProjectName. If [platformio] name/description are set, the pio project config --json-output call in getEnvironments already has that data.

Nits

  • PlatformIo.kt:67output.substringAfterLast("version") returns the whole string when "version" is absent.
  • PlatformResolver.kt:75,80platform.json is read twice, once via
    readResolvedPackage(platformDir, PLATFORM_MANIFEST_FILE) and again via platformManifestFile.readText().
  • Model.kt:139maintainer is ignored, though the spec documents it alongside author.
  • PlatformIoDependencyHandler.ktPlatformIoPackage.dependencies being a var assigned post-construction is
    load-bearing for the cycle guard; a comment at the declaration (not just at the assignment) would help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants