Together AI
A cloud for running, fine-tuning, and deploying open-source models (Llama, DeepSeek, Qwen) via an OpenAI-compatible API plus dedicated GPU endpoints.
Together AI is a hosted cloud for open-source models: serverless inference across a broad catalog (Llama, DeepSeek, Qwen, and more) behind an OpenAI-compatible API, plus fine-tuning and dedicated GPU endpoints and clusters. It's the managed way to run open-weight models without operating your own inference stack.
Together AI is a hosted cloud built around open-source models. Instead of running your own inference servers, you call a large catalog of open-weight models — Llama, DeepSeek, Qwen, and many others — through an OpenAI-compatible API, so existing SDKs work by changing the base URL and key. It targets developers and ML teams who want production access to open models without operating the underlying GPU stack.
Beyond serverless inference, the platform spans the full lifecycle of open weights: fine-tuning (supervised fine-tuning, preference optimization, LoRA and full), dedicated endpoints that keep a single-tenant model warm, and GPU clusters for training and large-scale workloads. For bursty pipelines, an asynchronous batch API trades latency for a meaningful discount over synchronous calls.
Highlights
- Broad open-model catalog — serverless inference across a large set of open-weight chat, code, embedding, image, and reranking models.
- OpenAI-compatible API — drop-in for the OpenAI SDK; swap providers by changing the base URL and key.
- Fine-tuning without infra — adapt open models (SFT, DPO, LoRA, or full) through the API, no training cluster to manage.
- Dedicated endpoints and GPU clusters — reserve single-tenant capacity per GPU-hour, up to on-demand and reserved clusters on current NVIDIA hardware.
In an AI-assisted workflow
curl https://api.together.xyz/v1/chat/completions \
-H "Authorization: Bearer $TOGETHER_API_KEY" \
-d '{"model":"<open-weight-model>","messages":[{"role":"user","content":"hi"}]}'Because the API is OpenAI-compatible, it slots into the same gateways and frameworks as other providers. For when to self-host open weights versus calling a hosted API like this, see Self-host vs. API; for deciding which open model to use, see Choosing the Right Model.
TIP
Together AI fits when your stack is open-weight: prototype on serverless inference, fine-tune the model that wins, then move steady traffic to a dedicated endpoint for predictable per-hour cost — all without leaving the platform.
Good to know
The platform is proprietary SaaS; the official Python SDK is open source under Apache-2.0. Pricing is freemium — free credits to start, then per-token serverless rates, per-GPU-hour dedicated and cluster pricing, and a batch discount for asynchronous jobs. Exact rates and the model catalog change over time, so confirm current details on the official pricing page before committing. Compare with Fine-tune vs. RAG vs. Prompt when deciding whether fine-tuning is even the right lever.
Frequently asked questions
- What is Together AI?
- Together AI is a hosted cloud platform for open-source models. It offers serverless inference over a large catalog of open-weight models — Llama, DeepSeek, Qwen, and many others — behind an OpenAI-compatible API, plus fine-tuning (LoRA and full) and dedicated GPU endpoints and clusters for teams that need reserved capacity.
- Is Together AI free?
- It's freemium: you start with free credits, then pay as you go. Serverless inference is billed per million input and output tokens, with a batch discount for asynchronous jobs; dedicated endpoints and GPU clusters are billed per GPU-hour. The platform is proprietary, but the official Python SDK is open source (Apache-2.0).
- How does Together AI compare to OpenRouter or Groq?
- All expose open and hosted models through an OpenAI-compatible API, but the center of gravity differs. OpenRouter is a router across many providers; Groq optimizes for very low-latency inference on its own hardware. Together AI is a full open-model cloud — inference plus fine-tuning, dedicated endpoints, and raw GPU clusters — so the same team can serve, customize, and scale open weights in one place.
Related
- Deploying LLMs to Production: A Reliability & Cost ChecklistTake an LLM feature from prototype to production: API vs self-host, provider fallback, retries, caching, observability, eval gates, and safe rollout.
- Self-Host vs API: When Does Running Your Own LLM Actually Pay Off?The real economics of self-hosting an LLM vs. calling a hosted API — GPU utilization, privacy, latency, and the hidden ops costs that decide the crossover.
- Best Tools for Running LLMs Locally in 2026The local LLM stack, ranked by job: Ollama for serving tools, LM Studio and Jan for desktop exploration, llama.cpp for control, vLLM when it's real serving.
- Fine-Tune vs RAG vs Prompt vs Distill: The 2026 Decision TreeWhen to reach for prompt engineering, RAG, fine-tuning, or distillation — what each actually changes, where each fails, and how to combine them.
- Choosing the Right Model: Haiku vs Sonnet vs OpusHow to pick the right Claude model tier — Haiku, Sonnet, or Opus — for any Claude Code agent or task, with a clear decision rubric and per-agent examples.