A Datacenter Scale Distributed Inference Serving Framework
kubernetes rust routing-engine omni diffusion vllm llm-inference tensorrt-llm sglang disaggregated-serving
-
Updated
Jul 5, 2026 - Rust
#
sglang
Here are 165 public repositories matching this topic...
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
-
Updated
Jul 5, 2026 - C++
OpenClaw-RL: Train any agent simply by talking
async gui-application coding slime tinker memory-systems skill-learning rlhf sglang grpo on-policy-distillation openclaw-skills open-claw
-
Updated
May 23, 2026 - Python
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
cuda inference openai llama maas rocm ascend llm llm-serving vllm genai llm-inference qwen deepseek sglang distributed-inference high-performance-inference mindie
-
Updated
Jul 3, 2026 - Python
A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.
-
Updated
Jul 4, 2026 - Python
MOSS-TTSD is a spoken dialogue generation model designed for expressive multi-speaker synthesis. It features long-context modeling, flexible speaker control, and multilingual support, while enabling zero-shot voice cloning from short audio references.
-
Updated
Mar 23, 2026 - Python
Open Source Continuous Inference Benchmark Research Platform — Kimi K2.7-Code, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™ TPUv6e/v7/Trainium2/3 | 开源持续推理基准研究平台 — Kimi K2.7-Code、MiniMax M3、DeepSeekv4、GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72，即将推出™ TPUv6e/v7/Trainium2/3
benchmark ai amd cuda pytorch nvidia glm minimax rocm kimi llm vllm deepseek sglang gb300 gb200 mi355x
-
Updated
Jul 5, 2026 - Python
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
-
Updated
Jul 4, 2026 - Python
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
serverless inference-engine llm llm-serving vllm llm-inference ollama llm-framework sglang kvcache gpu-sharing kvcached gpu-mutiplexing kvcache-optimization elastic-kvcache online-offline-coserve
-
Updated
Jul 2, 2026 - Python
MOVA: Towards Scalable and Synchronized Video–Audio Generation
-
Updated
Jun 18, 2026 - Python
UniRL is a Framework for Unified Multimodal Model Reinforcement Learning
-
Updated
Jul 5, 2026 - Python
基于SparkTTS、OrpheusTTS等模型，提供高质量中文语音合成与声音克隆服务。
-
Updated
May 18, 2025 - Python
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
k8s llama oracle-cloud model-serving model-as-a-service multi-node-kubernetes llm vllm llm-inference qwen deepseek sglang kimi-k2 pd-disaggregation
-
Updated
Jul 4, 2026 - Go
An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale
reinforcement-learning multi-agent vlm distributed-training post-training multimodal megatron-lm llm ray-serve rlhf qwen sglang grpo agentic-rl
-
Updated
Jul 5, 2026 - Python
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
chat mcp routing gemini openai claude llm anthropic vllm sglang anthropic-api inference-gateway tokenspeed responses-api tensorrtllm trtllm lightseek
-
Updated
Jul 4, 2026 - Rust
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
kubernetes inference huggingface llm modelscope llamacpp vllm text-generation-inference ollama sglang inference-platform
-
Updated
Jan 26, 2026 - Go
Improve this page
Add a description, image, and links to the sglang topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the sglang topic, visit your repo's landing page and select "manage topics."