Blog

vLLM, SGLang, TensorRT-LLM, and MAX Serve are all built on top of increasingly sophisticated KVCache management. This blog explores the evolution and role of the KVCache in these inference engines.

In late August, AMD and TensorWave reached out to collaborate on a presentation for AMD’s Media Tech Day—they asked if we could demo MAX on AMD Instinct™ MI355 on September 16th. There was just one problem: no one at Modular had access to an MI355.

Agentic Building Blocks: Creating AI Agents with MAX Serve and OpenAI Function Calling

Learn how quickly MAX and Open WebUI get you up-and-running with RAG, web search, and Llama 3.1 on GPU

Mojo 24.6 introduces key improvements in argument conventions, memory management, and reference tracking, enhancing code clarity and safety with features like 'mut' for mutable arguments, 'origins' for references, and new collection types.

Imagine unlocking a world of open innovation while ensuring secure, reliable, and enterprise-ready Gen AI deployments—MAX 24.6 enables enterprise AI teams to seamlessly run a vast range of cutting-edge AI models from Hugging Face on NVIDIA GPUs.

Build a Chat Application with Llama 3 and MAX Serve

Measuring state of the art GPU performance compared to vLLM on Modular's MAX 24.6

A deep dive into the complexities of optimizing code for SIMD instruction sets across multiple platforms.