Blogsby GlyphAurora
1.25x over a well-trained static speculator. Aurora is an open-source RL framework that turns speculative decoding from a one-time offline setup into a self-improving system that learns from every request it serves.
12 of 479 posts
Tue Mar 311 post
Thu Mar 261 post
Plan, divide, and conquer: How weak models excel at long context tasks
As context windows grow, LLM performance degrades in unexpected ways. We show how a "Divide & Conquer" framework, breaking long documents into parallel chunks with a planner, workers, and manager, lets smaller models like Llama-3-70B and...
Wed Mar 181 post
Together AI expands fine-tuning service with tool calling, reasoning, and vision support
Together AI expands fine-tuning with native support for tool call, reasoning, and vision-language models, plus 100B+ model training, up to 6× higher throughput, and job cost and ETA estimates.
Tue Mar 171 post
Mamba-3
Meet Mamba-3: the SSM built for inference. Faster than Transformers at decode, stronger than Mamba-2, and open-source from day one.
Mon Mar 161 post
Together AI at NVIDIA GTC 2026: Explore our latest innovations across research and products
Together AI arrives at NVIDIA GTC 2026 with new launches in inference, agents, voice AI, and open models, plus technical sessions from its research and engineering leaders.
Thu Mar 121 post
Build real-time voice agents on Together AI
Build real-time voice agents on Together AI with co-located STT, LLM, and TTS infrastructure, native Deepgram and Cartesia support, and end-to-end latency under 500ms.
Wed Mar 111 post
Together AI Brings NVIDIA Nemotron 3 to Developers on Day 0
NVIDIA Nemotron 3 Super is now available on Together AI Dedicated Inference, delivering efficient multi-agent reasoning, a 1M-token context window, and production-grade deployment on managed infrastructure.
Tue Mar 101 post
New in Together GPU Clusters: Autoscaling, observability, and self-healing
Together GPU Clusters now include built-in autoscaling, RBAC, full-stack observability, and self-healing node repair, giving teams production-ready GPU infrastructure that scales efficiently, stays resilient, and supports shared enterprise...
Thu Mar 52 posts
FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
As GPU throughput outpaces memory bandwidth, kernels must evolve. We introduce FlashAttention-4, featuring new pipelining for maximum overlap, 2-CTA MMA modes to reduce shared memory traffic, and a hardware-software hybrid approach to...
Key research and product announcements at the AI Native Conf
At AI Native Conf, Together AI announced breakthroughs across kernels, RL, and inference optimization, including FlashAttention-4, ThunderAgent, and together.compile. Research that ships to production. That's the AI Native Cloud.
Wed Mar 41 post
Cache-aware prefill–decode disaggregation (CPD) for up to 40% faster long-context LLM serving
Serving long prompts doesn't have to mean slow responses. Learn how Together AI's CPD architecture separates warm and cold inference workloads to deliver 40% higher throughput and dramatically lower time-to-first-token for long-context LLM...
Mon Mar 21 post
Introducing Together AI’s new look
We've refreshed our visual identity, designed with Pentagram to express how Together AI connects open-source innovation, systems research, and builders to unlock new possibilities.