Build software better, together

FareedKhan-dev / kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

Updated Sep 10, 2026
C

OpenMOSS / MOSS-TTS-Nano

Star

A 100M-parameter multilingual TTS model for real-time CPU inference, voice cloning, and 48 kHz stereo generation

multilingual text-to-speech realtime tts speech-synthesis english chinese streaming-audio onnx voice-cloning multi-modality cpu-inference voice-clone audio-tokenizer

Updated Sep 6, 2026
Python

kennethleungty / Llama-2-Open-Source-LLM-CPU-Inference

Star

Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

Updated Nov 6, 2023
Python

Vui Nano — a small, context-aware text-to-speech model trained on real conversations. 219M active params (305M total), Apache 2.0, voice cloning, streaming, runs on CPU (dependency-free C build). Ships with a full real-time voice assistant: WebRTC, ASR, local LLM, OpenAI Realtime API compatible.

Updated Sep 12, 2026
Python

lucienhuangfu / eLLM

Star

eLLM: Run Long-Horizon Inference Faster on CPUs Than on GPUs

cpu-inference long-horizon qwen llm-infernece rust-llm kimi-k3

Updated Sep 10, 2026
Rust

brontoguana / krasis

Star

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

transformer inference-engine inference-optimization mixture-of-experts cpu-inference large-language-models gpu-inference llm-inference high-performance-inference hybrid-inference gguf-model-support llama-cpp-alternative

Updated Sep 3, 2026
C++

facex-engine / facex

Star

Full face stack that runs entirely in the browser. Detection, 576-point 3D mesh, recognition, anti-spoof, smile — all WebAssembly, zero server. Apache 2.0.

computer-vision deep-learning c99 simd biometrics face-recognition low-latency avx2 production-ready face-verification face-embedding avx-512 zero-dependencies embedded-ai edge-ai onnx-runtime cpu-inference

Updated May 14, 2026
C

Dicklesworthstone / franken_ocr

Sponsor

Star

Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.

rust ocr neon simd moe quantization avx512 int8 cpu-inference int4 vision-language-model deepseek-ocr

Updated Sep 4, 2026
Rust

FoxNoseTech / diarize

Star

Speaker diarization for Python — "who spoke when?" CPU-only, no API keys, Apache 2.0. ~10.8% DER on VoxConverse, 8x faster than real-time.

python audio-analysis speech-to-text speaker-recognition speech-processing speaker-diarization spectral-clustering voice-activity-detection onnx speaker-embedding diarization apache-2 rttm cpu-inference meeting-transcription who-spoke-when

Updated May 6, 2026
Python

CoderLSF / fast-llama

Star

Runs LLaMA with Extremely HIGH speed

llama inference-engine cpu-inference llama2

Updated Nov 21, 2023
C++

gabriele-mastrapasqua / qwen3-tts

Star

Pure C inference engine for Qwen3-TTS text-to-speech. No Python, no PyTorch — just C and BLAS. Supports 0.6B and 1.7B models, 9 voices, 10 languages.

multilingual c text-to-speech metal cuda emotion inference simd tts speech-synthesis pure-c inference-engine voicecloning cpu-inference voiceclone qwen qwen-tts qwen3-tts

Updated Sep 12, 2026
C

rbitr / llm.f90

Star

LLM inference in Fortran

ai chatbot transformer llama language-model mamba state-space-model cpu-inference llm llamacpp llama2 phi-2

Updated May 30, 2024
Fortran

saurabhv749 / supertonic3-voice-clone

Star

Train voice styles for Supertone/supertonic-3 model.

text-to-speech speech-synthesis speaker-embedding voice-cloning onnxruntime fast-tts tts-model cpu-inference supertone hindi-tts supertonic-3 voice-builder

Updated Jul 8, 2026
Python

FareedKhan-dev / glm-5.2-in-c

Star

GLM-5.2, a 744 billion parameter mixture of experts model, in a pure C inference engine: quantized to int4, experts streamed from disk, deployed and benchmarked. Generates in 16 GB of RAM.

c machine-learning deep-learning cuda moe glm quantization systems-programming inference-engine consumer-hardware mixture-of-experts edge-ai cpu-inference int4 large-language-models llm llm-inference speculative-decoding

Updated Jul 15, 2026
C

Scottcjn / pse-vcipher-collapse

Star

Non-bijunctive attention collapse for LLM inference — POWER8 hardware AES (vcipher) + AltiVec vec_perm. Hebbian path selection, cross-head diffusion, O(1) KV prefiltering.

machine-learning deep-learning aes inference transformer attention-mechanism powerpc hardware-acceleration altivec hebbian-learning power8 cpu-inference llm llama-cpp non-bijunctive

Updated Aug 6, 2026
C

Scottcjn / rpi-inference

Star

RPI (Resonant Permutation Inference) — Zero-multiply text generation. 18K tok/s. 868 KB models. Standalone or as speculative draft engine for LLMs. Runs on N64, POWER8, x86, ARM.