This is a simple repository to setup and run NumPy asv benchmarks. The results
are stored with dvc on
DagsHub.
Since the data is stored with dvc we have some additional steps.
micromamba create -f environment.yml
micromamba activate numpy-bench
# Follow the dvc repo setup instructionsPublished results: haozeke.github.io/asv-numpy
The gh-pages branch is a normal asv publish tree with the
asv-tachyon static SPA installed on top (same index.json /
graphs/** / regressions.json data contract). No server-side app; works
on GitHub Pages and any other static host.
pip install -U asv-tachyon
# after asv publish into html_dir (or after dvc checkout of published outputs)
asv-tachyon install path/to/html_dir
# commit/push the html_dir to the gh-pages branchOr from a checkout of gh-pages:
git checkout gh-pages
asv-tachyon install .
git add -A && git commit -m "ENH: refresh asv-tachyon UI" && git push origin gh-pagesWe welcome contributed results as PRs.
Please ensure at a minimum that the machine being used is tuned for benchmarks.
export OPENBLAS_NUM_THREADS=1
export MKL_NUM_THREADS=1
export OMP_NUM_THREADS=1
sudo python -mpyperf system tuneIf running within docker keep the seccomp issues (among others) in mind. It
would be best to use an unloaded bare metal machine.
Additionally, try to have at-least a few isolated CPUs via the isolcpus kernel
parameter.
We welcome contributed results as PRs. For generating subsets of interest:
# Get commits for tags
# delete tag_commits.txt before re-runs
for gtag in $(git tag --list --sort taggerdate | grep "^v"); do
git log $gtag --oneline -n1 --decorate=no | awk '{print $1;}' >> tag_commits_only.txt
done
# Use the last 20
tail --lines=20 tag_commits_only.txt > 20_vers.txt
asv run -m $(hostnamectl hostname) -j HASHFILE:20_vers.txtNote that each tag and the corresponding commit is already shipped in subsets
so this can be simply:
cat subsets/tag_commits.txt | awk '{print $2;}' | tail --lines=20 > 20_vers.txt
asv run -m $(hostnamectl hostname) -j HASHFILE:20_vers.txtWhen there are new benchmarks, skip the existing ones by running:
asv run HASHFILE:20_vers.txt -kWe use dvc and DagsHub for storing the machine data.
MIT.