Blogsby GlyphBest Practices for Experiment Tracking in MLOps
Machine learning experimentation scales quickly.
3 of 800 posts
Sat Aug 222 posts
Large-Scale Sharded Weight Transfer with Ray Direct Transport (RDT) in vLLM
We implement a native sharded weight transfer engine in vLLM utilizing Ray Direct Transport (RDT), achieving weight transfer for the Kimi K2 model in BF16 on 48 8xH100 nodes in 7.53s