vLLM Blog the company blog of vLLM: latest posts and feed
vLLM · Company blog · 14 a month · last post Oct 23, 2026 · Active
Posts
67 on file since Jun 2, 2026Reliability and performanceOct 23 1dyesterdayGLM-5.3 Optimizations, Part 2: PCP, DCP, and Hybrid HiSparse on 16 H200svLLM serves GLM-5.3 on 16 H200s with PCP and DCP prefill and Hybrid HiSparse decode, reaching 1M-token contexts and 1.7–3.2× the per-user speed of data-parallel P/D on the AgentX agentic benchmark.InfrastructureOct 23 1dyesterdayvLLM Delivers Day-0 Support for MiniMax-M3.1-FlashDay-0 vLLM support for MiniMax-M3.1-Flash on NVIDIA Blackwell, NVIDIA Rubin, and AMD MI350-series GPUs, with NVFP4 MSA kernels, NVFP4 MegaMoE, DSpark speculative decoding, AgentX results, and day-0 RL.Reliability and performanceOct 21 3d3 days agoDay-0 Support for Flint on vLLMvLLM supports both Flint checkpoints (40B total, 2.5B active) from day 0, decoding a single request at about 500 tokens/s on one GPU.AI and agentsOct 21 3d3 days agoAnnouncing Humming: Fast, Flexible Kernels for Quantized InferenceIntroducing Humming, a quantized inference kernel library from Ant Group and the vLLM community, with broad GPU and quantization support, dense and MoE performance results, and vLLM integration.InfrastructureOct 20 4d4 days agoIntroducing vLLM TorchTPUvllm-torchtpu is a new vLLM backend for Google Cloud TPUs built on TorchTPU, with Pallas kernels, multi-host parallelism, KV cache offloading, and disaggregated serving.Reliability and performanceOct 19 5d5 days agoAchieving near-zero-downtime scaling in Elastic Expert ParallelismHow asynchronous preparation and NIXL EP enabled near-zero-downtime Elastic EP scaling in vLLM.Reliability and performanceOct 16 8d8 days agoServing Reflection Beam on vLLM from Day 0vLLM supports Reflection's Beam-1.0 (501B total, 23B active) from day 0, with agent-driven optimization that cut decode latency 2.71× and DFlash2 speculative decoding with adaptive verification.Reliability and performanceOct 16 8d8 days agoHyper Optimizing Kimi K3 in vLLM: Moving Fast and ResponsiblyHow agent-assisted CuTe DSL optimizations accelerate Kimi K3 decode on NVIDIA GB300, and how scoped changes, correctness tests, and reproducible benchmarks help ship them responsibly.Reliability and performanceOct 16 8d8 days agoScaling Kimi K3 on AMD Instinct MI355X: Matching NVIDIA HGX B300 Throughput per GPU on AgentXInside the Kimi K3 optimization effort: understanding the bottlenecks and the decisions that made inference fasterProduct and companyOct 15 9d9 days agoAnnouncing Day 0 Support for NVIDIA Nemotron 3.5 Super VL on vLLMRun NVIDIA Nemotron 3.5 Super VL on vLLM for multimodal agent workflows with image, document, and video understanding, OpenAI-compatible serving, and multi-token prediction.Data engineeringOct 15 9d9 days agoDay 0 support for Muse Spark Open on vLLMDay-0 vLLM support for Muse Spark Open: 816B MoE, 1M context, multimodal inputs, DSpark speculative decoding, NVIDIA and AMD serving, initial TPU support, and LoRA RL.Reliability and performanceOct 14 10d10 days agoFinish RL Training Faster: Bitwise-Consistent, High-Performance, Agent-DrivenWe make vLLM rollout and the training forward give bitwise identical logprobs on DeepSeek-V4-Flash and Nemotron-3.5-Lightning-30B-A3B (NVFP4), keep 91% of rollout throughput on DeepSeek-V4-Flash, and open-source the agent skill.AI and agentsOct 14 10d10 days agoPush-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLMA push-based NIXL connector for vLLM disaggregated serving: the prefill instance writes KV directly into decode memory over RDMA, taking the router relay, decode-side allocation, and transfer setup off the TTFT critical path.Reliability and performanceOct 14 10d10 days agoAdaptive Verification Without a Confidence HeadAn online-fitted acceptance estimator that lets every draft-model speculator use Adaptive Verification, by predicting the target's acceptance probability from the drafter's own logits and refitting continuously during serving.AI and agentsOct 12 12d12 days agoImprove TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLMBi-directional KV transfer lets vLLM prefill nodes reuse previously computed KV from decode nodes on later conversation turns, cutting redundant prefill recompute and reducing TTFT by up to 3.3x on long multi-turn prompts.Reliability and performanceOct 9 2w2 weeks agovLLM Support for NVIDIA Vera Rubin NVL72: 7.8x Throughput over GB200 NVL72vLLM now runs on NVIDIA Vera Rubin NVL72 with day-0 model support, Rubin-tuned FlashInfer kernels, and locality-aware MoE, delivering 7.8x the per-GPU throughput of GB200 NVL72 on AgentX.Reliability and performanceOct 7 2w2 weeks agoDeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.Machine learningSep 29 3w3 weeks agoTaking vLLM Apart: A Practical Guide to Disaggregated ServingWhat disaggregated serving actually buys you, how to run it end to end in vLLM today with prefill/decode plus the new GPU-less frontend and the things we're still working on.Reliability and performanceSep 24 4w4 weeks agoWatermarking in vLLMHow vLLM implements distribution-preserving Gumbel-max text watermarking with efficient GPU kernels, statistical detection, speculative decoding, and repeated-context safeguards.Product and companySep 22 4w4 weeks agoAnnouncing vllm-metal: Concurrent Serving on Apple Siliconvllm-metal brings vLLM's paged, continuously batched serving stack to Apple Silicon, with flatter TTFT under concurrent agent load, batched MTP, and automatic M5 prefill acceleration.Reliability and performanceSep 21 4w4 weeks agoPD Serving of Qwen3.8-2.4THow vLLM reaches 5K throughput and 180 interactivity on Qwen3.8-2.4T with GB300 NVL72 PD serving and how to reproduce results yourself.Reliability and performanceSep 18 5w5 weeks agoScaling Multi-GPU Video Captioning with PyNvVideoCodec and vLLMHow to leverage NVIDIA Hardware Video Decoders to Achieve Multi-GPU Scaling in Video Captioning and Description tasks.Reliability and performanceSep 15 5w5 weeks agovLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B300Novita AI has open-sourced Chord, a high-performance W4A16 MoE CUDA kernel for Kimi K2.x serving shapes, with a Humming-compatible indexed path and grouped SM90 operators.Reliability and performanceSep 15 5w5 weeks agoHow we trained the fastest DSpark for Kimi-K3 using GB300 NVL72How Speculators and Mooncake enabled multi-node DSpark training for Kimi K3.InfrastructureSep 14 5w5 weeks agovime × RL-Kernel × AMD: Bitwise Train–Rollout Consistency on ROCmvime and RL-Kernel align selected-token logprobs bit for bit across Megatron training and vLLM rollout on AMD Instinct MI300X, with zero mismatches across 200 GRPO steps. About this blog
vLLM's own blog on vllm.ai, found by the daily check.
Posts per week, last 26 weeks
14 a month over the last six months (or since the feed's oldest post)
History
Listed on Blogs Oct 7, 2026. No changes to its feed, status or cadence since.
Blogsby Ibis