Engineering Articles (X)

Measuring state of the art GPU performance compared to vLLM on Modular's MAX 24.6

We are pleased to announce the release of a new tool aimed at simplifying the management of stacked pull requests (PRs) on GitHub - stack-pr. This tool is still in its early development days, but we are excited to share it with the community and welcome your contributions.

Developer tooling is a big priority for Mojo and MAX, we want to vastly improve the debugging experience compared to the traditional Python, C++, and CUDA stack. Machine learning often requires inspecting the state of a program after a long running process, requiring more control than what "print debugging" gives you. Over time this tooling will extend to GPUs, allowing you to step through CPU code into GPU calls with the same developer experience.

Last week, Chris Lattner sat down for an interview on the Developer Voices podcast with Kris Jenkins. It was a wide-ranging episode that explored a variety of topics, including the motivations behind creating Mojo, what it offers to both Python and non-Python programmers alike, how it is built for performance, and which performance features actually matter. This post recaps a number of highlights from the podcast, edited for clarity and brevity. You can find the full 90 minute interview on YouTube.

Modern software and delivery mechanisms are more vulnerable to security attacks than ever before. It is estimated that in 2024, 96% of total codebases contained open-source code with an average of 500+ components per app.

Mojo as a language offers three main benefits, namely the 3 P’s: Performance, Programmability and Portability. It enables users to write fast code, do so easier than many alternative languages, and allows code to be run across different CPU platforms, with GPU support on the roadmap.

Mojo is built on the latest compiler technology in MLIR, an evolution of LLVM which Rust lowers to, and so it can be faster. It depends largely on the skill of the programmer and how far they're willing to go with optimizations. Mojo's goal as a language, is to meet Python developers where they are, and allow them to learn some new tricks to optimize their code to the performance limits of any hardware.

Open any introductory programming book and you’ll find several pages dedicated to structured programming concepts, i.e. making use of loops, conditions and functions extensively for better clarity and maintainability for your code. It helps you express your ideas and solutions neatly and elegantly in code. However, these benefits come at a cost: performance overhead. 

Did you catch Fabian's presentation at SINTEF? If you missed out, he'll be doing it again in Oslo, Norway on the 15th of February!By popular request, we're publishing the slides here, and will soon have video to go with it. Make sure to subscribe to our YouTube channel to get the alert!

Jeff, Abdul, and Chris presented a talk about Mojo and how it relates to MLIR and LLVM, at the LLVM Developer Conference 2023.