🤖 Update Benchmark Results - #5449
Conversation
There was a problem hiding this comment.
Benchmark Update Review
This is a straightforward automated benchmark data refresh for TUnit v1.29.0 (up from v1.28.7), with MSTest also bumped to v4.2.1.
Performance Summary
Runtime wins for TUnit v1.29.0:
- AsyncTests: 537.0ms → 528.7ms (~1.5% faster), AOT: 122.7ms → 120.9ms
- DataDrivenTests: 477.77ms → 473.79ms (~0.8% faster), AOT: 24.35ms → 22.25ms (~8.6% faster — notable AOT improvement)
- SetupTeardownTests: 582.0ms → 577.9ms (~0.7% faster)
- ScaleTests AOT: 36.53ms → 36.02ms (~1.4% faster)
Slight regressions (within noise):
- BuildTime: 1.789s → 1.842s (+3%); all frameworks moved up similarly (NUnit: +0.05s, xUnit3: +0.027s), suggesting CI environment variance rather than a real regression
- MassiveParallelTests: 655.8ms → 665.1ms (+1.4%); again, NUnit and xUnit3 also moved up, pointing to environmental noise
- ScaleTests: 445.59ms → 455.16ms (+2.1%); MSTest actually improved from 553ms to 526ms in the same run, so this is likely CI noise
Observation on DataDrivenTests: MSTest appears to have regressed significantly in this benchmark (475.82ms → 590.47ms, ~24% slower). That's a notable jump — though this is their version bump from 4.1.0 → 4.2.1, not a TUnit concern.
Structural Quality
The automated pipeline produces consistent JSON and Markdown output. All files (JSON data sources, rendered Markdown, historical trends) are updated atomically in a single PR, which is the right approach.
No issues found. The changes are correct, internally consistent, and the performance story for TUnit remains strong.
Up to standards ✅🟢 Issues
|
Automated Benchmark Update
This PR updates the benchmark documentation with the latest results from the Speed Comparison workflow.
Benchmarks Produced
Individual benchmark artifacts are available for download:
benchmark-DataDrivenTestsbenchmark-AsyncTestsbenchmark-ScaleTestsbenchmark-MatrixTestsbenchmark-MassiveParallelTestsbenchmark-SetupTeardownTestsbenchmark-BuildTimebenchmark-summary(aggregated overview)Runtime Benchmarks
Build Benchmarks
Changes
docs/static/benchmarks/latest.jsondocs/static/benchmarks/historical.jsondocs/docs/benchmarks/index.mddocs/static/benchmarks/summary.jsonWorkflow Run
🤖 This PR was automatically created and will be merged automatically once CI checks pass.