Skip to content

🤖 Update Benchmark Results - #5449

Merged
thomhurst merged 1 commit into
mainfrom
automated-benchmarks-update
Apr 8, 2026
Merged

thomhurst merged 1 commit into
mainfrom
automated-benchmarks-update

Conversation

@thomhurst

Copy link
Copy Markdown
Owner

Automated Benchmark Update

This PR updates the benchmark documentation with the latest results from the Speed Comparison workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

  • benchmark-DataDrivenTests
  • benchmark-AsyncTests
  • benchmark-ScaleTests
  • benchmark-MatrixTests
  • benchmark-MassiveParallelTests
  • benchmark-SetupTeardownTests
  • benchmark-BuildTime
  • benchmark-summary (aggregated overview)

Runtime Benchmarks

  • AsyncTests
  • DataDrivenTests
  • MassiveParallelTests
  • MatrixTests
  • ScaleTests
  • SetupTeardownTests

Build Benchmarks

  • BuildTime
  • 2026-04-08
  • Ubuntu Latest • .NET SDK 10.0.201

Changes

  • Updated benchmark data in docs/static/benchmarks/latest.json
  • Updated historical trends in docs/static/benchmarks/historical.json
  • Regenerated benchmark documentation in docs/docs/benchmarks/index.md
  • Updated benchmark summary in docs/static/benchmarks/summary.json

Workflow Run

  • Run ID: 24111336347
  • Triggered: schedule
  • Date:

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

@thomhurst thomhurst added automated benchmarks documentation Improvements or additions to documentation ignore-for-release labels Apr 8, 2026
@thomhurst
thomhurst merged commit 83840f3 into main Apr 8, 2026
11 of 12 checks passed
@thomhurst
thomhurst deleted the automated-benchmarks-update branch April 8, 2026 00:42

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Update Review

This is a straightforward automated benchmark data refresh for TUnit v1.29.0 (up from v1.28.7), with MSTest also bumped to v4.2.1.

Performance Summary

Runtime wins for TUnit v1.29.0:

  • AsyncTests: 537.0ms → 528.7ms (~1.5% faster), AOT: 122.7ms → 120.9ms
  • DataDrivenTests: 477.77ms → 473.79ms (~0.8% faster), AOT: 24.35ms → 22.25ms (~8.6% faster — notable AOT improvement)
  • SetupTeardownTests: 582.0ms → 577.9ms (~0.7% faster)
  • ScaleTests AOT: 36.53ms → 36.02ms (~1.4% faster)

Slight regressions (within noise):

  • BuildTime: 1.789s → 1.842s (+3%); all frameworks moved up similarly (NUnit: +0.05s, xUnit3: +0.027s), suggesting CI environment variance rather than a real regression
  • MassiveParallelTests: 655.8ms → 665.1ms (+1.4%); again, NUnit and xUnit3 also moved up, pointing to environmental noise
  • ScaleTests: 445.59ms → 455.16ms (+2.1%); MSTest actually improved from 553ms to 526ms in the same run, so this is likely CI noise

Observation on DataDrivenTests: MSTest appears to have regressed significantly in this benchmark (475.82ms → 590.47ms, ~24% slower). That's a notable jump — though this is their version bump from 4.1.0 → 4.2.1, not a TUnit concern.

Structural Quality

The automated pipeline produces consistent JSON and Markdown output. All files (JSON data sources, rendered Markdown, historical trends) are updated atomically in a single PR, which is the right approach.

No issues found. The changes are correct, internally consistent, and the performance story for TUnit remains strong.

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

TIP This summary will be updated as you push new changes. Give us feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated benchmarks documentation Improvements or additions to documentation ignore-for-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant