Where to Run Qwen 3 in Production: Inference Providers Compared

Compare popular Qwen inference providers based on recommended use case, relative price, speed, and infrastructure control. Feel free to take this table as a baseline to help you determine which provider is best suited for your workload

There is no universal winner. Start with the workload:

• Select a managed serverless API if you’re dealing with unpredictable traffic and want to minimize operational headaches.
• Go with a provider focused on speed if you absolutely need low latency for interactive chat, voice calls, and coding.
• For situations where model portability and provider fallback are significant considerations, you can choose a router.
• Choose dedicated inference when sustained traffic requires predictable capacity.
• Choose self-hosting when you need infrastructure control, custom weights, private networking, or specialized inference optimizations.

The cheapest advertised price per token does not always yield the lowest production cost. Evaluate providers with identical prompts, the same model version, output length, and concurrency levels.

We’ve found the following metrics to be the most critical:

• Time-to-first-token (TTFT) is the duration from when a request is submitted to when the first token of output is generated. This metric greatly impacts the perceived responsiveness of a chatbot or coding assistant.
• Output speed measures how many tokens the system generates per second after the first token. Fast output is important for long answers, reasoning traces, code generation, and multi-step agents.
• End-to-end latency encompasses the full request lifecycle, including queueing, prompt processing, generation, and network transfer times.
• Tail latency attempts to capture slow edge-case requests. This is usually measured as p95 or p99 latency. P99 latency is the value below which 99 percent of requests will complete. Average latency can often hide terrible production behavior if 1 out of every 100 requests takes significantly longer than the others.
• Throughput measures the overall tokens or requests processed over a period of time. It’s more important than per-request speeds for batch workloads and high-concurrency APIs.

Also test support for structured outputs, tool calling, prompt caching, long-context, regional latency differences, rate limits, and failure behavior. Some providers may advertise OpenAI compatibility but only support a subset of the OpenAI parameter set.

OpenAI-compatible APIs accept requests that use endpoints and JSON request/response structures similar to the OpenAI API. Switching between providers can sometimes be as simple as changing the API key, model identifier, and base URL. API compatibility can reduce migration work, but it does not guarantee identical behavior.

DeepInfra and Novita are often good starting points for cost-sensitive Qwen inference; however, comparisons should be between equivalent model versions and service tiers.

For instance, DeepInfra listed Qwen3.5-27B for ~$0.26/million input tokens and ~$2.60/million output tokens on the regular tier at the time of review. Its Qwen3.5-397B-A17B listing was priced at ~$0.45 for input and ~$3.00 for output. Flexible tiers can cost less but can provide a lower scheduling priority. See the current DeepInfra Qwen3.5-27B listing and Qwen3.5-397B-A17B listing.

Smaller models can be dramatically cheaper. This can matter because flagship models are not required for many classification, retrieval, summarization, and routing tasks. Well-evaluated 4B, 9B, or 27B models can outperform much larger models economically, even if the per-active-parameter price is lower for the larger model. Calculate cost using the expected input-to-output ratio:

Suppose an agent request consumes 10,000 input tokens and produces 2,000 output tokens. At $0.26 per million input tokens and $2.60 per million output tokens:

This gets us to about a $7,800 inference bill at 1 million requests, excluding retries, embeddings, storage, routing fees, or other service uses.

Avoid making price comparisons based on some arbitrary 50:50 token split between input and output tokens. RAG and document analysis systems typically generate many more input tokens than output tokens. Reasoning and code-generation style apps may generate many more output tokens. Because output tokens are typically more expensive, the ratio of tokens used by your workload can affect which provider is cheaper.

DeepInfra, Together AI, Fireworks AI, Groq, Novita, OpenRouter, and DigitalOcean offer APIs that support OpenAI-compatible API endpoints to varying degrees. This means developers can leverage the OpenAI Python SDK with a different base_url.

Here is a reusable example:

import os
import time
from openai import OpenAI

client = OpenAI(
   api_key=os.environ["INFERENCE_API_KEY"],
   base_url=os.environ["INFERENCE_BASE_URL"],
)

start = time.perf_counter()
first_token_time = None
output = []

stream = client.chat.completions.create(
   model=os.environ["QWEN_MODEL_ID"],
   messages=[
       {
           "role": "system",
           "content": (
               "You are a production reliability assistant. "
               "Return concise, technically accurate recommendations."
           ),
       },
       {
           "role": "user",
           "content": (
               "A Qwen inference endpoint has rising p99 latency while "
               "average latency remains stable. List likely causes."
           ),
       },
   ],
   temperature=0.2,
   max_tokens=400,
   stream=True,
)

for event in stream:
   if event.choices and event.choices[0].delta.content:
       if first_token_time is None:
           first_token_time = time.perf_counter()

       token_text = event.choices[0].delta.content
       output.append(token_text)
       print(token_text, end="", flush=True)

end = time.perf_counter()

print("\n")
print(f"TTFT: {first_token_time - start:.3f} seconds")
print(f"End-to-end latency: {end - start:.3f} seconds")

# Configure it through environment variables rather than placing credentials in source code:
export INFERENCE_API_KEY="your-api-key"
export INFERENCE_BASE_URL="https://provider.example.com/v1"
export QWEN_MODEL_ID="provider-specific-qwen-model-id"

python app.py

Obtain the provider-specific model ID and base URL from the current provider’s documentation. Qwen3.5-27B may not always be spelled the same way.

This abstraction makes initial migration easier. However, for portable production code, you’ll need to do some additional work. Models can vary by tool-call schema, reasoning controls, JSON enforcement, token usage tracking, context window size, safety filtering, and error codes. Consider building your own model gateway and provider-neutral evaluation suite if you expect to switch providers.

OpenRouter is best considered as an aggregator and routing layer. It provides a single API for developers to access models that can be served by various underlying providers. This is helpful when building applications that require automatic fallback, centralized billing, rapid model comparison, or access to multiple commercial and open models.

For instance, OpenRouter showed Qwen3.5 Plus with a one-million-token context window and separately priced inputs/outputs. You can view current models in its Qwen model catalog, including context sizes, prices, and available routes.

The following example sends a Qwen request through OpenRouter. It prioritizes lower-cost routes, permits fallback when another provider is available, and excludes endpoints that may collect request data:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["OPENROUTER_API_KEY"],
    base_url="https://openrouter.ai/api/v1",
)

response = client.chat.completions.create(
    model="qwen/qwen3.5-plus-20260420",
    messages=[
        {
            "role": "user",
            "content": "Explain continuous batching in three sentences."
        }
    ],
    max_tokens=200,
    extra_body={
        "provider": {
            "sort": "price",
            "allow_fallbacks": True,
            "data_collection": "deny"
        }
    }
)

print(response.choices[0].message.content)

When interacting with an aggregator, there is less visibility and control over the infrastructure actually serving a request. Data handling requirements must be examined both at the routing layer and the underlying provider. Pinning to a provider can improve predictability, while dynamic routing can offer higher availability or price.

Agentic systems repeatedly make requests to a model, sending it plans, tool results, retrieved documents, and conversation history. When selecting providers for these types of systems, low TTFT, reliable tool calling, structured output support, large contexts, prompt caching, and stable rate limits should all be weighted heavily.

Agentic development teams that want to own their production infrastructure and tailor models may want to consider Together AI and Fireworks. DeepInfra may be compelling if agents can generate high token volumes, and cost is your main constraint. Groq is attractive if the agent makes sequential steps and every model call introduces noticeable latency. OpenRouter can provide fallback when an agent must remain available during a provider incident.

Qwen3.5 models are especially promising for tool-oriented and multimodal agents. However, a model’s actual capability with tools should be tested with your application’s specific tools. A benchmark score cannot reliably show whether a model produces valid arguments for your internal functions.

Build an evaluation set of conversations that include failed tool calls, ambiguous instructions, long conversation histories, prompt injection, and faulty tool outputs. Benchmark both the completion rate and cost per completed task rather than cost per token.

Serverless inference is typically ideal for prototypes, workloads with variable traffic patterns, and teams that don’t want to operate/manage GPUs. Pricing is based on tokens used, with the vendor managing batching, scaling, and model serving.

Potential downsides include cold starts, shared-capacity queueing, rate limits, limited customization, and less predictable tail latency. The magnitude of these effects varies by provider and service tier.

Dedicated inference allocates capacity for a single organization. Use when you have sustained traffic, strict latency requirements, or need predictable throughput from your application. It may also allow more control over networking, scaling, observability, and model configuration.

The break-even point depends on utilization:

A dedicated GPU can be economical at high utilization but expensive when idle. Serverless shifts utilization risk to the provider.

DigitalOcean provides serverless and dedicated inference. They provide dedicated inference per GPU-hour and list AMD MI300X as well as NVIDIA H100, H200, and B300 offerings. You can see DigitalOcean’s current inference pricing page for configurations.

Use self-hosting if you require custom weights/adapters/quantization, private networking/data residency, or fixed model versions / direct control of the inference engine.

RunPod and GPU Droplets are infrastructure solutions, not hosted model endpoints. Your team chooses the GPU, deploys an inference server (e.g., vLLM, SGLang), configures autoscaling, and manages monitoring and upgrades.

A basic vLLM deployment can expose Qwen through an OpenAI-compatible endpoint:

# 1) Install vLLM (pin a version to avoid surprises)
pip install "vllm>=0.8.4"

# 2) Run vLLM with Qwen3-8B over an OpenAI-compatible HTTP API
export LOCAL_API_KEY="your-local-key"

vllm serve Qwen/Qwen3-8B \
 --host 0.0.0.0 \
 --port 8000 \
 --api-key "$LOCAL_API_KEY" \
 --max-model-len 32768 \
 --gpu-memory-utilization 0.90 \
 --trust-remote-code

# The application can then call it with the same OpenAI client:
# ---------
# Python: call Qwen3-8B via OpenAI client
# ---------

import os
from openai import OpenAI

# Read the same API key used by vLLM; fall back to a default for demos
api_key = os.getenv("LOCAL_API_KEY", "your-local-key")

client = OpenAI(
   api_key=api_key,
   base_url="http://localhost:8000/v1",
)

response = client.chat.completions.create(
   model="Qwen/Qwen3-8B",
   messages=[
       {"role": "user", "content": "Explain continuous batching simply."}
   ],
   temperature=0.2,
   max_tokens=300,
)

print(response.choices[0].message.content)

This example is intended for learning purposes and not meant to be a ready-for-production deployment. For a production service, you would want to add TLS, authentication, health checks, metrics, request limits, container security, multiple redundant replicas, autoscaling, deployment rollback, GPU-capacity planning, etc. to your service as well.

DigitalOcean’s GPU Droplets support this self-managed approach. Lower-cost GPUs can be used to run smaller quantized Qwen models. For the large MoE variants, you may require multiple high-memory GPU accelerators. DigitalOcean recently launched 1-Click Models that let you quickly deploy supported open models with an OpenAI-compatible endpoint. Check out DigitalOcean’s guide on hosting models on GPUs.

Don’t assume a provider is compliant with your regulated workloads because they display a SOC 2 badge. It will depend on the service, contract terms, region, data flow, and configuration.

Ask each provider:

• Is the inference service included in its SOC 2 report?
• Will they sign a business associate agreement for HIPAA workloads?
• Are prompts or responses retained?
• Will customer data be used for training?
• Where are requests and data processed and logged?
• Are private networking and customer-managed keys available?
• Can subprocessors or routed inference providers access the data?
• What audit logs and deletion controls are provided?

If you use an aggregator, be sure to ask additional questions since another company will be running the underlying inference. Self-hosting will give you more control over architecture, but your organization will own security, patching, and audits.

The best place to run Qwen 3 or Qwen 3.5 depends on the workload, not the provider’s overall reputation.

DeepInfra and Novita stand out for cost-sensitive inference. Groq offers compelling performance for interactive applications with low latency requirements, but verifies their current model availability and deprecation timeline. Together AI and Fireworks offer more full-fledged production environments if your team needs more advanced serving optimization, customization, or dedicated infrastructure. OpenRouter makes it easy to route to multiple providers and fallback, at the expense of adding an additional routing layer. DigitalOcean offers a nice continuum from serverless inference at a pay-per-token rate to fully dedicated inference and even self-managed GPU Droplets. RunPod works well for teams that prefer to manage their own infrastructure and run their own serving stack.

The safest production strategy is to avoid permanent dependence on untested claims. Choose a specific model version, prepare a realistic evaluation dataset, benchmark a few providers under practical concurrency levels, and estimate your cost using the actual input-output token ratio. Keep your application behind an OpenAI-compatible internal interface. However, be sure to test any provider-specific differences in tool usage, structured output, context limits, and error handling.

Inference catalogs will continue to change as Qwen introduces newer generations. A reproducible evaluation process is therefore more valuable than any static provider ranking.

• Qwen models on Hugging Face
• Qwen3.5-27B API reference and pricing
• Inference overview: serverless, provisioned throughput, and dedicated
• Available serverless models
• Querying text models through serverless inference and dedicated deployments
• Supported models
• Provider routing and provider selection