Choosing a Model
The right model depends on your task, budget, and latency requirements. Always start from the Model Library or GET /v1/models — only live models are callable, and IDs change as the catalog grows.
Quick decision tree
Do you need Arabic as the primary language? → Filter the Model Library for live chat models. Multilingual models (Qwen-class, etc.) are a common starting point.
Do you need a context window larger than 8K tokens? → Sort the Model Library by context window and pick a live model that fits your document.
Is speed and cost your top priority? → Sort by input price ascending and test the cheapest live chat model first.
Do you need the best quality for complex tasks? → Try a larger live model (70B class or similar). Benchmark on your own prompts before committing.
By use case
| Use case | Where to look | Why |
|---|---|---|
| Customer support chatbot | Cheapest live chat model | Fast loops; upgrade if quality falls short |
| Arabic customer support | Live multilingual chat models | Strong Arabic without a separate integration |
| Code generation | Larger live chat models | Better instruction-following on code |
| Document summarization | Small/mid chat models under 8K context | Fast; fits most short docs |
| Long document analysis | Highest context window in catalog | Fit the full doc in one request |
| Creative writing | Larger chat models | More nuanced tone |
| Data extraction / JSON | Models with tools or response_format in supported_parameters (GET /v1/models) | Structured output |
| Translation | Multilingual chat models | Cross-language quality |
| High-throughput batch | Lowest input price per 1M tokens | Minimise cost at scale |
| Embeddings / RAG | Live embed models | Vectors for retrieval — separate endpoint from chat |
| Research / reasoning | Largest live chat model you can afford | Test on your hardest prompts |
Cost optimization tips
- Start cheap. Test with the lowest-priced live chat model; upgrade only when quality isn't enough.
- Develop on a small model. Iterate fast, then switch for production if needed.
- Set
max_tokens. Don't let the model generate more than you need. - Keep a stable prompt prefix. The API is stateless: every request re-sends, and is billed for, all of its messages, system prompt included. Put what never changes (system message, tool definitions, reference text) first and what changes last, so models with prompt caching can reuse the repeated prefix. Cached tokens are billed at the model's cache-read rate where its pricing lists one, otherwise at the normal input rate, and some models cache only what you mark — see Prompt Caching.
- Batch when possible. Every request counts against your RPM, so pack work into fewer requests where the API allows it (for example, up to 2,048 inputs in one embeddings call) — see Rate Limits.
Switching models
The request format is the same across chat models — change one string. Parameter support can differ by model: some return 400 unsupported_parameter for seed, logprobs, logit_bias, n > 1 or response_format: json_object.
response = client.chat.completions.create(
model="gpt-oss-20b", # any live id from GET /v1/models
messages=same_messages,
temperature=same_temperature,
)See Models Overview for the live comparison table.