Modular has partnered with San Francisco Compute to create Large Scale Inference, the best priced OpenAI-compatible inference in the world. On most open source models, we're 85%+ cheaper than other options. San Francisco Compute built LSI in close partnership with a tier-1 AI lab to help them print trillions of tokens of synthetic data, saving them tens of millions of dollars compared to the leading competitor.

Unlike other providers, inference prices are market-based. The token price tracks the underlying market-based compute cost on SFC & current load. In other words, we give you the best available price.

Our MAX inference engine consistently outperforms other providers across key benchmarks like DocVQA, MathVista, and ChartQA on accuracy. We save you money and get you more accurate results while we do it.

LSI natively supports very large scale batch inference, with far higher rate limits & throughput than other providers. Unlike other services, we don't force you to upload petabytes of data to us. Our batch inference reads & writes to an S3-compatible object store, so your sensitive data isn't stored indefinitely on our servers. LSI natively handles multimodal use cases without forcing you to publicly share links to your content.

LSI is designed for large scale, mostly enterprise, use cases. That lets us be more hands on than traditional, self-serve providers.

• Want a deployment behind your private network?
  

• Need to hit specific latency, throughput, or uptime requirements?
  

• Is there a model that's performing better in your evals, but we're not serving it?
  

Between Modular's world-class engineering & SFC's dramatic price optimization, we'll work with you to get the best possible price & performance.

We currently support the following models below the cost of every current provider on average. Exact prices, latency, and throughput depend on the use case & current market conditions. For a technical demo, get a quote.