Machine learningvLLM BlogSep 29 3w3 weeks agoTaking vLLM Apart: A Practical Guide to Disaggregated ServingWhat disaggregated serving actually buys you, how to run it end to end in vLLM today with prefill/decode plus the new GPU-less frontend and the things we're still working on.vLLMRead
Reliability and performancevLLM BlogSep 24 4w4 weeks agoWatermarking in vLLMHow vLLM implements distribution-preserving Gumbel-max text watermarking with efficient GPU kernels, statistical detection, speculative decoding, and repeated-context safeguards.vLLMRead
Product and companyvLLM BlogSep 22 4w4 weeks agoAnnouncing vllm-metal: Concurrent Serving on Apple Siliconvllm-metal brings vLLM's paged, continuously batched serving stack to Apple Silicon, with flatter TTFT under concurrent agent load, batched MTP, and automatic M5 prefill acceleration.vLLMRead
Reliability and performancevLLM BlogSep 21 4w4 weeks agoPD Serving of Qwen3.8-2.4THow vLLM reaches 5K throughput and 180 interactivity on Qwen3.8-2.4T with GB300 NVL72 PD serving and how to reproduce results yourself.vLLMRead
Reliability and performancevLLM BlogSep 18 5w5 weeks agoScaling Multi-GPU Video Captioning with PyNvVideoCodec and vLLMHow to leverage NVIDIA Hardware Video Decoders to Achieve Multi-GPU Scaling in Video Captioning and Description tasks.vLLMRead
Blogsby Ibis