A lightweight, general-purpose framework for evaluating GPU kernel and benchmark.
-
Updated
Aug 21, 2026 - Python
#
inference-benchmark
Here are 16 public repositories matching this topic...
Deep Learning Inference benchmark. Supports OpenVINO™ toolkit, TensorFlow, TensorFlow Lite, ONNX Runtime, OpenCV DNN, MXNet, PyTorch, Apache TVM, ncnn, PaddlePaddle, etc.
caffe deep-learning mxnet tensorflow inference pytorch benchmark-framework paddlepaddle ncnn tflite openvino dli tensorflow2 dgl iree onnx-runtime inference-benchmark spektral apache-tvm executorch
-
Updated
Aug 10, 2026 - HTML
First public benchmark of llama.cpp speculative decoding on Qwen3.6-35B-A3B with a single RTX 3090 (post PR #19493 merge, 2026-04-19). 19 configurations covering ngram-cache, ngram-mod, and classic draft with vocab-matched Qwen3.5-0.8B. Finding: no variant achieves net speedup on Ampere + A3B MoE. Raw JSON, plots, full reproducibility.
benchmark cuda moe ampere mixture-of-experts inference-benchmark llama-cpp ggml local-llm llm-inference qwen speculative-decoding qwen3 rtx-3090
-
Updated
May 16, 2026 - Python
WER and inference-cost benchmark for Indian-English ASR, and an audit of the evaluation itself: 9 systems x 3 corpora, cluster-bootstrap CIs with Holm correction, a reference-artifact taxonomy confirmed by human re-transcription, and a speaker-disjoint fine-tuning study across 6 seeds x 3 sizes.
benchmark reproducible-research evaluation speech-recognition whisper asr model-evaluation nptel wer fine-tuning parakeet huggingface inference-benchmark accented-speech indian-english qwen3 academic-speech aesrc2020
-
Updated
Aug 4, 2026 - Python
Benchmark for GB10 - Nvidia DGX Spark
quantization model-evaluation huggingface inference-benchmark llama-cpp vllm gptq local-llm local-ai tool-calling agent-benchmark llm-benchmark nvfp4 nvidia-gb10 dgx-spar dgx-spark-benchmark nokast
-
Updated
Aug 16, 2026 - Python
Benchmark your GPU against any GGUF model and contribute to the public leaderboard. Measures throughput, TTFT, ITL, and VRAM limits across quantizations and context sizes.
quantization vram gpu-benchmark huggingface home-lab inference-benchmark llm llama-cpp local-llm poor-pauls-benchmark
-
Updated
Jun 7, 2026 - Python
Vram budgeting and tail-latency simulation for three models sharing one triton gpu
-
Updated
Aug 21, 2026 - Python
InfernoBench: open-source AI model installers, local inference benchmarks, and reproducible browser labs. Wan2.1 and Sulphur-2 included.
linux self-hosted nvidia-cuda open-source-models video-generation text-to-video ai-models inference-benchmark model-benchmarking audio-generation diffusers generative-ai ai-video-generator ai-video comfyui local-ai wan-ai sulphur-2 wan2-1 model-installer
-
Updated
Jul 25, 2026 - Python
Local benchmarking tool to explore Vision Transformer scaling and WSI-level inference constraints under real hardware settings.
-
Updated
Feb 18, 2026 - Python
Local-first Edge AI inference run registry and comparability checker for multi-target benchmark evidence.
python raspberry-pi benchmarking sqlite odroid typer jetson tensorrt onnx edge-ai experiment-tracking onnxruntime edge-inference inference-benchmark ai-inference
-
Updated
Jun 23, 2026 - Python
Real-time EEG neurological classification benchmark with latency-accuracy tradeoffs, OpenNeuro data, ONNX inference, and parallel FFT speedups.
-
Updated
Aug 1, 2026 - Python
-
Updated
Nov 6, 2025 - Python
Benchmark speculative decoding performance for Qwen3.6-35B-A3B on an RTX 3090 GPU using llama.cpp to evaluate model throughput and structural regressions.
benchmark cuda moe ampere mixture-of-experts inference-benchmark llama-cpp ggml local-llm llm-inference qwen speculative-decoding qwen3 rtx-3090
-
Updated
Aug 22, 2026
Dynamic batching inference server with FastAPI, dynamic padding, and benchmark visualizations
-
Updated
Jul 17, 2026 - Python
Measuring the KV-transfer tax of disaggregated prefill/decode LLM serving (vLLM + NIXL, 4x A10G): on PCIe-only hardware, disagg loses to plain data-parallel replication — measured, committed, reproducible.
kv-cache inference-benchmark llm-serving vllm llm-inference nixl disaggregated-serving prefill-decode goodput
-
Updated
Jul 13, 2026 - Python
Reproducible benchmarking framework for signal-processing algorithms and ML inference supporting real and synthetic ECG inputs.
benchmarking machine-learning pytorch ecg tensorrt onnx openvino biomedical-signal-processing onnxruntime inference-benchmark ml-benchmark
-
Updated
Apr 7, 2026 - Python
Improve this page
Add a description, image, and links to the inference-benchmark topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the inference-benchmark topic, visit your repo's landing page and select "manage topics."