The Best LLM Inference Providers in 2026: Groq vs Together vs Fireworks vs Baseten and More

The best LLM inference provider in 2026 depends on the workload you're serving. For the lowest latency on open models, use Groq or Cerebras; for the broadest open-model catalogue with fine-tuning, Together AI or Fireworks AI; for your own models on dedicated GPUs, Baseten or Modal; and for image, video and audio generation, fal or Replicate, which Cloudflare has owned since December 2025. Most of the LLM hosts speak an OpenAI-compatible API, so you can keep two in play behind a router and switch on speed, price or availability.

Last reviewed: September 2026.

Prices change often; each tool page lists current pricing, and the LLM API pricing guide compares per-token rates across major models.

The summary table

Pick by workload

Lowest latency: Groq or Cerebras

Both built their own inference silicon and publish per-model speeds; the catch is catalogue size. Choose Cerebras when raw output speed on a large open model is the requirement and your model is one of its two. Choose Groq for a wider production menu, including Whisper and its compound systems, plus a free plan to prototype on.

Broad catalogue and fine-tuning: Together AI or Fireworks AI

These are the general-purpose open-weight clouds: large catalogues, per-token serverless, dedicated capacity when traffic steadies, and fine-tunes served on the same platform. Choose Together AI for more ways to buy capacity and fine-tunes you can download. Choose Fireworks AI if you'll run reinforcement fine-tuning or want Anthropic-compatible clients.

Dedicated or custom deployments: Baseten or Modal

Choose Baseten for a managed, autoscaling production endpoint for your own or open models, with per-token Model APIs for popular ones. Choose Modal when the model is one part of a larger Python job and you're happy to deploy the inference server yourself.

Generative media: fal or Replicate

Choose fal for production image, video, audio and 3D features billed per output. Choose Replicate, part of Cloudflare since December 2025, for the breadth of its catalogue, and watch idle billing on private models.

Routing across providers

OpenRouter isn't a host: it sends your request to a model provider and, per its FAQ, passes provider pricing through with no inference markup, charging a fee when you buy credits instead. Hugging Face Inference Providers works similarly at https://router.huggingface.co/v1, with :fastest, :cheapest and :preferred suffixes; its partners include every provider on this page except Modal, plus DeepInfra. For a self-hosted gateway, read LiteLLM vs OpenRouter and LLM gateways compared. DeepInfra and SambaNova, which have no page here, also turn up in evaluations.

The providers, one at a time

Groq: LPU speed, now with an NVIDIA partnership

Groq serves open models on its LPU chips through GroqCloud. As of September 2026 its production models are openai/gpt-oss-120b (500 tokens per second on its models page), openai/gpt-oss-20b (1,000), Whisper Large v3 and v3 Turbo, plus llama-3.1-8b-instant and llama-3.3-70b-versatile, which are now labeled Enterprise with contact-sales pricing. Qwen, MiniMax and text-to-speech models are in preview as of September 2026, and no Mistral model is listed.

In December 2025 Groq signed a non-exclusive licensing agreement with NVIDIA for its inference technology; founder Jonathan Ross, president Sunny Madra and other team members joined NVIDIA. Groq said it "will continue to operate as an independent company" and that "GroqCloud will continue to operate without interruption." It was a license, not an acquisition. Since then Groq has named a new CEO (Adam Winter, as of June 2026), raised new funding in June and August 2026, become an NVIDIA Cloud Partner (August 2026), and said it will be among the first to deploy NVIDIA Groq 3 LPX, with no date given. How LPU and NVIDIA capacity will mix hasn't been stated.

The API is "mostly compatible" with OpenAI's client libraries at https://api.groq.com/openai/v1, minus logprobs, logit_bias, top_logprobs, messages[].name and n above 1. The free plan is rate-limited per model; the Developer plan adds higher limits plus Batch and Flex processing. Groq doesn't fine-tune: enterprise customers can serve externally trained LoRA adapters on Llama 3.1 8B only.

Verdict: a low-friction start for fast inference, especially GPT-OSS and Whisper, with a free plan to prove the latency gain. If you relied on Groq's Llama models, check your plan: they're enterprise-only now.

Cerebras: wafer-scale speed, small menu

Cerebras describes its WSE-3T processor as "Four trillion transistors. 250 petaflops. One silicon wafer." Each chip holds model weights in 44 GB of on-chip SRAM, which the company credits for its speed. Cerebras Inference lists two production models as of September 2026: gpt-oss-120b at about 3,000 tokens per second and qwen-3.8-27b at about 1,500. Public endpoints serve "the original, unpruned versions", with weight-only quantization used only in storage.

The company went public on Nasdaq as CBRS in May 2026. In January 2026 it announced that OpenAI will deploy 750 megawatts of Cerebras systems in stages starting in 2026, and since August 2026 it has powered Ultrafast, an OpenAI API tier that runs GPT-5.6 Sol at up to 750 output tokens per second; OpenAI sells that tier, which launched in limited preview in August 2026. Its Q2 2026 results named customers including OpenAI, AWS, Cognition and Figma.

The API is OpenAI-compatible "for many common workflows" at https://api.cerebras.ai/v1, but n must be 1, images must be base64, and gpt-oss-120b rejects tools combined with response_format. A free trial with credits and tight limits leads to a self-serve Developer tier with 10x the free rate limits, and Enterprise adds custom weights, fine-tuning and dedicated queue priority. It's also sold through AWS Marketplace, OpenRouter, Hugging Face and Vercel.

Verdict: the pick when output speed on a large open model is the product requirement, as in latency-bound multi-step agents. If your model isn't on its list, it's not an option outside Enterprise custom weights.

Together AI: the open-model cloud with the most ways to buy

Together AI serves open-weight chat, vision, image, audio, video, transcription, embedding, rerank and moderation models on NVIDIA GPUs up to GB200 NVL72. It sells capacity four ways: serverless per-token with a Batch API, Provisioned Throughput Units (fixed reserved capacity, priced per unit), Dedicated Inference on single-tenant GPUs, and GPU clusters.

The API is compatible with OpenAI's REST API and SDKs across chat, completions, vision, image generation, text-to-speech and embeddings at https://api.together.ai/v1. Fine-tuning covers LoRA (the default) and full fine-tuning with SFT and DPO, and you can deploy the result on a dedicated endpoint or download it. There's no free trial: the platform is prepaid and requires a minimum credit purchase before your first call.

Verdict: the best all-rounder if you want to prototype on serverless, fine-tune, and graduate to reserved or dedicated capacity without changing vendors. Downloadable fine-tunes keep you portable.

Fireworks AI: built for customized open models

Fireworks AI lists 100+ supported models across text, vision, audio, image and embeddings. Serverless bills per token across Standard, Priority and Fast tiers, batch at 50% of serverless, and dedicated deployments per GPU second with no charge for start-up time. From September 1, 2026, US-only serverless models carry a 50% premium and on-demand GPU rates went up; region-restricted deployments cost 1.5x.

Fine-tuning is the center of gravity: SFT and DPO priced per training token, reinforcement fine-tuning per GPU hour, and fine-tuned models served at base-model prices. Fireworks says over 95% of the 40 trillion-plus tokens it serves daily come from models specialized on customers' data. The API is an OpenAI drop-in and also supports Anthropic's /v1/messages endpoint, streaming included, at https://api.fireworks.ai/inference for the Anthropic SDK.

Verdict: the pick when you'll fine-tune or reinforcement-train an open model and serve it at scale, and one of two hosts here that accept both OpenAI- and Anthropic-style clients. Read the September 2026 pricing changes before you pin a region.

Baseten: your models, run as a production service

Baseten sells Dedicated Inference for models you bring, Model APIs for popular open models, and Training. The Model APIs are OpenAI- and Anthropic-compatible, per token, at https://inference.baseten.co/v1. Dedicated deployments bill per minute of GPU time, T4 through B200; you package models with Truss, its open-source (MIT) framework, and deployments autoscale down to zero. New accounts get credits to experiment, and Baseten is SOC 2 Type II certified and HIPAA compliant.

Verdict: the managed middle ground between a token API and your own Kubernetes: bring a fine-tuned or custom model and get an autoscaling, compliant endpoint, with Model APIs for everything you don't need to host.

Modal: programmable GPUs for everything around the model

Modal is serverless compute billed per second, from T4 to B300 GPUs plus CPU and memory. You declare the container image and GPU in Python, and Modal runs functions, web endpoints, batch jobs and Sandboxes for untrusted code. There's no hosted model catalogue: you deploy an inference server such as vLLM yourself, and the scaffold-vllm-config command generates a serving config with an OpenAI-compatible endpoint. Plans are Starter (with monthly free credits), Team and Enterprise.

Verdict: the choice when the model sits inside a larger Python workload, such as batch embedding, evaluation, RL loops or agent sandboxes, and you want per-second GPUs without Kubernetes. For plain tokens from a popular open model, a per-token host is less work.

fal: generative media at production speed

fal, a "generative media platform for developers", offers 1,000+ image, video, audio and 3D models behind one API, with SDKs for Python, JavaScript, Swift, Kotlin/Java and Dart. Model APIs bill per output (per image, per second of video); GPU compute bills per hour and serverless per second. Its homepage says the fal Inference Engine is "up to 10x faster", and it supports private or fine-tuned deployments, LoRA training such as FLUX LoRA, and newer Agent, Sandbox and Workflows products.

Verdict: the production default for media features: curated models, per-output billing that maps to product usage, and mobile SDKs. It isn't an LLM host, so pair it with one of the providers above.

Replicate: the long tail, now part of Cloudflare

Replicate runs community and official image, video, audio and language models behind one API; Cloudflare cites 50,000+ models and fine-tunes. You package your own with Cog (Apache-2.0), and billing is per second of hardware, from CPU to H100.

Cloudflare announced the acquisition in November 2025 and closed it on December 1, 2025. Replicate said it would carry on as a distinct brand and that "The API isn't changing." By April 2026 the team had joined Cloudflare's AI Platform team, and Cloudflare is moving Replicate's models onto its own infrastructure and into AI Gateway, with Workers AI deployment promised "soon". Watch private models: most run on dedicated hardware and bill for setup and idle time, except "fast booting fine-tunes".

Verdict: the place to try an unusual open model in minutes. For steady private workloads, compare the idle-time bill against Baseten or Modal, and follow the Cloudflare integration if you already use Workers AI.

OpenAI-compatible endpoints at a glance

Compatible rarely means identical: test the parameters your code actually sends, especially tool calling with structured output and image inputs, before you swap a base URL.

What changed in 2026

• Groq: the NVIDIA licensing deal (December 2025), a new CEO by June 2026, new funding in June and August 2026, NVIDIA Cloud Partner status, and Llama models moved to Enterprise.
• Replicate: Cloudflare's acquisition closed on December 1, 2025, and the team merged into Cloudflare's AI Platform team by April 2026.
• Cerebras: the OpenAI partnership (January 2026), the Nasdaq IPO as CBRS (May 2026), and OpenAI's Ultrafast tier (August 2026).
• Together AI: a Series C (July 2026), with Provisioned Throughput Units now on the pricing page.
• Fireworks AI: a Series D (July 2026), then a US-only serverless premium and higher on-demand GPU rates from September 1, 2026.
• Baseten: a Series E (January 2026) and a Series F, with Blaxel joining.
• Modal: a Series C (May 2026). fal: a Series D (December 2025).

How to evaluate providers on your own workload

1. Fix the model and prompts. Use the same model, prompts and max tokens everywhere, and ask each provider what precision it serves.
2. Measure under your concurrency: time to first token, output tokens per second and p95 latency with your real parallelism. Headline speeds don't show tail latency under your load.
3. Hit the rate limits on purpose. Groq's free plan allows 30 requests per minute on openai/gpt-oss-120b and Cerebras's trial allows 5, so test the tier you'll actually buy.
4. Send your real request shapes. Tool calls, JSON output, images and logprobs expose compatibility gaps that a hello-world prompt hides.
5. Cost per completed task, not per token. Include cached-token discounts, batch pricing, retries, idle time on dedicated hardware and regional premiums, with rates from each tool page.
6. Wire a fallback before launch. The provider fallback wrapper skill adds timeouts, bounded retries and an alternate provider, and the model router designer skill sends easy requests to cheaper models behind an eval gate.

How to choose

• Latency is the product: Cerebras if your model is on its list, otherwise Groq.
• You need many open models and will fine-tune: Together AI for flexible capacity, Fireworks AI for reinforcement fine-tuning and Anthropic-compatible clients.
• You have your own model: Baseten for a managed endpoint, Modal for code-first control.
• You generate images, video or audio: fal for production, Replicate (Cloudflare-owned since December 2025) for breadth.
• You want several of them: a router such as OpenRouter or a self-hosted gateway, compared in LiteLLM vs OpenRouter.

If you're weighing any of these against your own GPUs, read self-host vs API first, and hand the serving work to the LLM inference engineer agent when you make the move.