Why Spiky Inference Traffic Breaks the Dedicated GPU Math

A dedicated GPU running spiky LLM inference traffic clears a specific, calculable throughput floor, 1,910 billable tokens per second sustained, before it beats per-token billing; the derivation and its utilization-percentage equivalent follow later in this article. This article applies that floor to llama3.3-70b-instruct on DigitalOcean, works through three traffic shapes with explicit hourly distributions, and covers the three capacity patterns, scheduled capacity, a reserved floor with serverless overflow, and pure serverless, that spiky workloads actually choose between.

The standard advice is that high-volume workloads graduate from serverless to dedicated capacity. That advice is correct for steady traffic and incomplete for spiky traffic, because it treats volume as the deciding variable when the deciding variable is whether you know, in advance, which hours the GPU will be busy. A dedicated GPU is worth renting only for hours you can keep above the sustained floor, and you can only rent exactly those hours if you know when they are. Genuinely unpredictable bursts cannot be scheduled around; predictable ones can, and that difference decides whether the dedicated GPU math breaks or holds, not the size of the peak-to-trough ratio or the monthly token volume alone.

The clearest way to see this is to hold the traffic identical and change only what you know about it. Take one GPU serving an 8-hour daily peak at full load and nothing the rest of the day. Held around the clock, that GPU is busy 33.3% of the month (8 of 24 hours), below the 46.9% crossover derived later in this article, and it costs $3,263.10 a month against a serverless-equivalent bill of $2,318.37 for the same tokens, a loss of $944.73. Rented only for the 8 hours it is actually needed, the same hardware serving the same workload costs $1,087.70 for those hours, a saving of $1,230.67, 53.1% cheaper than the serverless-equivalent cost. Same hardware, same workload, different bill. The only variable that changed is whether you paid for the 16 hours you did not need, and the only thing that lets you skip paying for them is knowing in advance which 8 hours to buy.

DigitalOcean’s own guidance already points at this distinction. From the Dedicated Inference documentation:

Choose serverless inference over dedicated inference when you need to get started quickly without managing any components behind an inference endpoint, don’t have a custom model to host or optimize, or have unpredictable or spiky inference traffic.

This article operationalizes that guidance rather than arguing against it: it gives you the exact throughput floor, the archetypes where the guidance holds and where it gets more nuanced, and the capacity pattern to reach for once you know your traffic’s shape. Scope: this article covers llama3.3-70b-instruct FP8 on a DigitalOcean H200, billed by total tokens (input plus output). Self-hosting decisions independent of traffic shape, quantization tradeoffs, and batch-size tuning are covered in the companion cost framework and are out of scope here.

• Effective August 1, 2026, a DigitalOcean H200 GPU Droplet and an H200 Dedicated Inference endpoint cost the same $4.47/hr, so they now share one break-even point: 1,910 sustained billable tokens per second, 46.9% of the measured 4,071.6 tok/s total-token ceiling, to beat DO Serverless Inference at $0.65/1M tokens. Before that date, the GPU Droplet ran at $3.44/hr with a separate, lower 36.1% threshold; the choice between the two products is now operational, not financial.
• What breaks the dedicated GPU math is predictability, or the lack of it: you cannot say in advance which hours the GPU will be busy. One GPU serving an identical 8-hour daily peak loses money held around the clock (33.3% average utilization, below the 46.9% crossover) and wins decisively rented only for those 8 hours (100% utilization of billed time). Predictability, not the size of the spike, decides the outcome.
• Holding the peak-to-trough ratio fixed at 10:1 and changing only the monthly volume still changes the verdict, but the post-rate-change margins move against dedicated: at Archetype 1’s scale, serverless now wins outright by 27.5%, not a near-coin-flip. At roughly 3.5x the volume, only the single-GPU reserved floor still beats pure serverless; the 2-GPU floor and pure dedicated sized to peak both now lose to serverless. Ratio alone never determined the winner; the volume at which peak first exceeds one GPU’s ceiling did, and at the new rate that ceiling has to be cleared by a wider margin to pay off.
• A consumer traffic shape with unpredictable 2-hour bursts cannot use scheduled capacity at all, even though its burst is short enough that a scheduled window at that load would otherwise be worth building. Scheduling requires knowing the window in advance; when the burst can land at any hour, the only way a dedicated GPU covers it is to run continuously, which returns the shape to the failed 14.8%-utilization floor test.
• Scheduled capacity, a GPU rented only for a predictable business-hours window with the rest of the day routed to serverless, saves 37.3% against an all-serverless baseline for a B2B traffic shape sitting near the crossover. The saving narrows to 34.5% once realistic per-day provisioning overhead (30 minutes of billed creation and teardown time) is priced in, and DigitalOcean bills GPU Droplets from creation, not from readiness.

Every comparison in this article reduces to one question: does a given hour of GPU time deliver more than 1,910 billable tokens per second, on average? Below that line, DigitalOcean Serverless Inference is cheaper for those tokens. Above it, the dedicated GPU is cheaper, and the margin grows with every additional point of utilization.

Methodology: Why Total Billable Tokens

DigitalOcean prices llama3.3-70b-instruct on Serverless Inference at a flat $0.65 per 1 million tokens, input and output, symmetric. Because both sides of the bill are priced identically, total billable tokens (input plus output) is a valid common unit for comparing serverless spend against dedicated GPU throughput, and every number in this article, on both sides of every comparison, counts the same thing. This is the one methodology decision the article makes, and it is stated here once.

That basis matters because the measured throughput anchor comes from a benchmark run at a 1,024-input/1,024-output token ratio, not from a ratio necessarily representative of your traffic. Total-token throughput at that ratio was 4,071.6 tok/s (2,036 tok/s output plus roughly the same in input, measured on a single H200 running llama3.3-70b-instruct FP8 with vLLM 0.24.0). If you run that benchmark yourself, take the Total token throughput (tok/s) row rather than the Output token throughput (tok/s) row when applying the floor formula in this article, since the floor is stated per billable token. A different input:output ratio changes prefill-versus-decode balance and does not simply halve or double that total figure. If your production ratio departs substantially from 1:1, benchmark your own configuration with the harness in the cost framework’s methodology section before trusting the 1,910 tok/s floor at face value.

The Floor Identity

This derivation is independent of throughput measurement, it depends only on the GPU’s hourly rate and the serverless per-token rate:

floor_tokens_per_second = gpu_hourly_rate / (serverless_rate_per_token * 3600)

floor = 4.47 / ((0.65 / 1_000_000) * 3600)
print(f"{floor:,.0f} billable tokens per second")

Output
1,910 billable tokens per second

A DigitalOcean H200 GPU Droplet must sustain 1,910 billable tokens per second, averaged over the billing period, before it costs less per token than DO Serverless. Below that, every additional second of idle capacity is billed at the full $4.47/hr rate and delivers zero tokens against it.

Converting the Floor to a Utilization Percentage

The floor becomes a percentage once you divide it by the GPU’s measured total-token ceiling:

total_tps = 4071.6  # measured total (input + output) throughput, single H200
gpu_hourly = 4.47    # H200 GPU Droplet, effective August 1, 2026; also the current Dedicated Inference rate
serverless_rate = 0.65 / 1_000_000

floor = gpu_hourly / (serverless_rate * 3600)

print(f"H200 crossover (GPU Droplet and Dedicated Inference): {floor / total_tps:.1%}")

Output
H200 crossover (GPU Droplet and Dedicated Inference): 46.9%

Before August 1, 2026, the Dedicated Inference endpoint ($4.47/hr, managed serving stack included) needed a higher sustained floor than the raw GPU Droplet ($3.44/hr), because its $1.03/hr management premium had to be earned back in the same per-token comparison. Effective August 1, 2026, the GPU Droplet rate rose to match Dedicated Inference at $4.47/hr (DigitalOcean, “Upcoming GPU Pricing Updates,” published July 21, 2026), so both products now share the same 46.9% crossover and the same $3,263.10 monthly cost for one GPU. The choice between them is now operational rather than financial: pick the GPU Droplet to manage the serving stack yourself, or Dedicated Inference to have DigitalOcean manage it, since neither carries a cost premium over the other anymore. Every archetype and capacity pattern later in this article checks its utilization against 46.9% unless stated otherwise.

Reference Constants

Every number in this article traces back to this table. Rounding convention: 4,071.6 tok/s and 730 hours per month throughout, so that tables agree with each other to the cent. Pricing reflects the H200 GPU Droplet rate effective August 1, 2026, at which point it converged with the existing Dedicated Inference rate.

It is tempting to reach for the peak-to-trough ratio as the single number that should drive the serverless-versus-dedicated decision: a 3:1 shape looks safe for dedicated, a 20:1 shape looks obviously wrong for it. That instinct does not survive contact with volume. Hold the ratio fixed at 10:1 and change only the monthly scale, and the correct architecture changes with it.

The B2B business-hours shape in the next section runs at 85% utilization for 8 hours and 8.5% for 16 hours, exactly a 10:1 peak-to-trough ratio, at a scale where one GPU covers the entire peak. At that scale, at the post-August-1 GPU rate, pure dedicated ($3,263.10) costs 27.5% more than pure serverless ($2,364.74), a clean serverless win rather than a near-tossup. Now take a workload at the same 10:1 ratio (12,215 billable tok/s peak for 8 hours, 1,221 tok/s trough for 16 hours) scaled up until the peak exceeds one GPU’s 4,071.6 tok/s ceiling. At that scale, reserving one GPU and routing overflow to serverless costs $7,899.84 a month against $8,346.13 for pure serverless, a 5.3% saving, and against $9,789.30 for pure dedicated sized to peak, a 19.3% saving. Same ratio, larger volume, and the winning architecture is still not the same as at Archetype 1’s scale, but the margin for the pattern that wins is thinner than it was before the rate change, and neither the 2-GPU floor nor pure dedicated sized to peak beats serverless anymore. The ratio told you nothing about which side would win; the volume, specifically whether it pushed peak demand past a single GPU’s ceiling, did.

Nor does spiky traffic as such make dedicated capacity uneconomic. A predictable spike can be scheduled around: reserve the GPU only for the hours it is needed, and it captures roughly half the bill compared to running it around the clock (worked through in the capacity patterns section below). What actually breaks the math is not knowing which hours to reserve. An unpredictable spike, one that can land at any hour with no advance signal, cannot be scheduled around at all, because scheduling requires knowing the window before it opens. The only way a dedicated GPU covers a genuinely unpredictable burst is to run continuously and wait for it, which forces the GPU back onto the 24-hour average utilization test it was trying to avoid.

That is the actual thesis of this article: a dedicated GPU is worth renting only for hours you can keep above 1,910 billable tokens per second, and you can only rent exactly those hours, and only those hours, if you know in advance when they are. Traffic that spikes on a schedule you can name is a scheduling problem with a scheduling solution. Traffic that spikes without warning is not, regardless of how large or small the spike is relative to the trough.

Utilization below is the fraction of one H200’s total-token capacity (4,071.6 billable tok/s), time-weighted across 24 hours. Each archetype has an explicit hourly shape, not just an average, so you can reproduce the number yourself. Traffic shapes are illustrative constructions, chosen to bracket the crossover from both sides rather than drawn from any measured dataset. They are worked examples showing how the floor test behaves across a range of shapes, not precedent for your own workload. Profile your own hourly distribution before applying any verdict here.

Archetype 1: B2B Business Hours

avg_util = (85 * 8 + 8.5 * 16) / 24
print(f"{avg_util:.1f}%")

Output
34.0%

This shape’s peak is 3,461 billable tok/s (85% of the 4,072 tok/s ceiling) for 8 hours a day, comfortably under one GPU’s ceiling, so a single H200 covers every hour without overflow. This shape’s 34.0% average utilization sits 12.9 percentage points below the post-August-1 46.9% crossover, and the 27.5% gap between serverless and dedicated is a clean serverless win, not noise: at the current GPU rate, a business-hours shape like this one no longer sits near the threshold. The real answer for a business-hours pattern like this one is still scheduled capacity, worked in the next section, which beats both pure options by 37.3%.

Archetype 2: Consumer Viral Spikes

avg_util = (2 * 0.90 + 22 * 0.08) / 24
print(f"{avg_util:.4f}")

Output
0.1483

At 14.8% average utilization, this shape is 68.4% cheaper on serverless than on dedicated, the widest gap of the three archetypes. That gap alone might suggest scheduling around the 2-hour burst the way Archetype 1 schedules around its 8-hour window. It cannot be done here, and the reason why is the article’s core point: scheduled capacity cannot help here. Renting a GPU only for the busy hours requires knowing which hours those are. When the burst can land at any time, the only way a dedicated GPU covers it is to run continuously, which returns you to the floor test this shape fails.

A 2-hour burst is short enough that a scheduled window at that load would ordinarily be worth building, the same way Archetype 1’s 8-hour window is. The difference is entirely that Archetype 1’s window has a known start and end time and this one does not.

Archetype 3: Steady API Backend (Control)

avg_util = 75.0
monthly_tokens = avg_util / 100 * 10_700_164_800
serverless_cost = monthly_tokens * 0.65 / 1_000_000
print(f"{monthly_tokens/1e6:,.0f}M tokens -> ${serverless_cost:,.2f} serverless vs $3,263.10 dedicated")

Output
8,025M tokens -> $5,216.33 serverless vs $3,263.10 dedicated

This is the honesty anchor. At flat 75% utilization, well above the 46.9% crossover, dedicated wins by 37.4%, though the margin has narrowed from before the rate change: a pricier GPU-hour eats into dedicated’s advantage even at high utilization. This archetype supports one conclusion: dedicated infrastructure is the right tool for this shape. It does not support a broader indictment of dedicated infrastructure; nothing in this article argues against dedicated GPUs for steady, high-throughput workloads, it argues that traffic which cannot promise a sustained floor should not be sized against one.

Once you know a workload’s shape, there are three architectures to choose between, not two.

Scheduled Capacity, Worked on Archetype 1

Archetype 1’s business-hours window is predictable, so it is the shape scheduled capacity is built for: rent the GPU only for the 8-hour peak window and route the remaining 16 hours to serverless.

Scheduled capacity saves 37.3% against the cheaper of the two pure options (all-serverless). That figure assumes zero billed provisioning time, which is not realistic: DigitalOcean bills a GPU Droplet from the moment it is created, not from the moment it finishes booting and is ready to serve. Publishing the provisioning sensitivity keeps the saving figure honest.

gpu_hourly = 4.47
serverless_rate = 0.65 / 1_000_000
total_tps = 4071.6
days_month = 730 / 24

gpu_cost = gpu_hourly * 8 * days_month
trough_tokens = 0.085 * total_tps * 3600 * 16 * days_month
trough_cost = trough_tokens * serverless_rate
scheduled_base = gpu_cost + trough_cost

for extra_min in (0, 10, 20, 30):
    extra_cost = gpu_hourly * (extra_min / 60) * days_month
    total = scheduled_base + extra_cost
    saving = (2364.74 - total) / 2364.74 * 100
    print(f"{extra_min:>2} min provisioning: ${total:,.2f}/mo, {saving:.1f}% saving vs. all-serverless")

Output
 0 min provisioning: $1,481.82/mo, 37.3% saving vs. all-serverless
10 min provisioning: $1,504.48/mo, 36.4% saving vs. all-serverless
20 min provisioning: $1,527.14/mo, 35.4% saving vs. all-serverless
30 min provisioning: $1,549.80/mo, 34.5% saving vs. all-serverless

The table does not price everything a scheduled-capacity architecture costs in practice: daily create-and-destroy automation to build and maintain, a cold endpoint at the start of every window, teardown verification against orphaned resources that keep billing after a script fails partway, and no guarantee that GPU capacity is available in your region the moment a window opens. At a 34.5% to 37.3% margin, that operational overhead is worth carrying.

Reserved Floor with Serverless Overflow

This pattern only becomes distinct from pure dedicated once peak demand exceeds a single GPU’s ceiling. Archetype 1’s peak is 3,461 billable tok/s against a 4,072 tok/s ceiling, so one GPU covers every hour, no overflow ever occurs, and the pattern collapses into plain pure dedicated at $3,263.10. The pattern only becomes a distinct option at a scale where peak actually exceeds the ceiling.

Larger workload: peak 12,215 billable tok/s for 8 hours, trough 1,221 billable tok/s for 16 hours, 12.84B billable tokens per month, the same 10:1 peak-to-trough ratio as Archetype 1 at roughly 3.5 times the volume.

One reserved GPU wins, and the reason why is the decision rule to take from this pattern. A reserved GPU always serves up to its full ceiling whenever demand exists, it is never throttled below its ceiling to manufacture overflow, so the question is never how the workload averages out overall. The question is whether that specific GPU’s own average utilization clears the 46.9% crossover:

• The first reserved GPU runs at its 4,071.6 tok/s ceiling through all 8 peak hours and still carries the entire 1,221 tok/s trough overnight, because the trough never exceeds what one GPU can serve. Its own average utilization across the month is 53.3%, above the 46.9% crossover, so it earns its rate.
• A second reserved GPU would only ever run during the 8-hour peak, because the first GPU already absorbs the full trough by itself. That is 33.3% duty at best, below the crossover, so a second GPU loses money even though the workload as a whole is large enough to look like an obvious dedicated-capacity candidate.

This is a sharper illustration of the marginal-GPU rule than it was before the rate change. At the prior $3.44/hr rate, pure dedicated sized to peak ($7,533.60) and even the 2-GPU floor ($7,340.77) both beat pure serverless ($8,346.13); the difference between the options was a matter of degree. At the current $4.47/hr rate, only the 1-GPU floor still wins: pure dedicated now costs $9,789.30 and the 2-GPU floor $8,844.57, both above pure serverless. The marginal-utilization figures themselves do not change, GPU #1 still runs at 53.3% and GPU #2 still runs at 33.3% duty, since those depend on traffic shape rather than price, but the cushion against the crossover has shrunk: GPU #1’s utilization now clears the crossover by 6.4 points, down from 17.2 points before the rate change. Adding a GPU past the one that clears the floor is now more likely to lose to serverless than it was before.

Do not read the trough level itself as the requirement. The trough alone does not need to sustain 1,910 tok/s; a reserved GPU fills first and takes the busiest hours available to it, so what matters is the marginal GPU’s own average utilization across the full month, not whether the trough by itself clears the floor.

Pure Serverless

Pure serverless wins whenever neither of the other two conditions holds, meaning the window cannot be scheduled and no single GPU’s marginal utilization would clear the crossover. Archetype 2 is the clean example: its burst is short and unpredictable, so it cannot be scheduled, and it never generates enough sustained load for even one reserved GPU to clear 46.9%, so a reserved floor does not help either. For this shape, pure serverless is the correct answer, not merely a fallback.

Cold Starts and Burst Latency on Serverless

Cost is not the only axis. Serverless trades burst latency for elasticity, and that tradeoff is dedicated capacity’s legitimate counterargument. A reserved GPU that is already warm serves the first request of a spike at the same latency as the thousandth. A serverless endpoint may not.

Two distinct effects hide behind the phrase “cold start,” and they need separate measurement because they have different causes and different mitigations:

• Idle cold start. Time to first token on the first request after a quiet period, when weights may need staging and capacity may need scheduling. This is what most people mean by cold start, and it is measured with a single request after a deliberate idle window.
• Queue wait under concurrency. When a burst arrives faster than the serving pool admits it, requests wait to be scheduled into a batch rather than being served on arrival. Continuous batching makes this efficient in aggregate, but an individual request’s time to first token then reflects its position in the queue rather than the model’s load time. The signature is unmistakable: several requests receive their first token at the same wall-clock instant, so measured time to first token falls as send time rises.

Conflating the two produces misleading numbers. A test that fires 32 concurrent requests immediately after an idle window measures both at once and attributes the total to cold start.

Measure them separately against your own account:

• Steady-state time to first token at low concurrency, as the baseline.
• Time to first token on a single request after an idle window of 15 to 30 minutes, which isolates idle cold start.
• Time to first token across a concurrency ramp from a warm state, which isolates queue behavior. Record the send timestamp of every request, not just its latency, because the wall-clock arrival pattern is what separates queueing from cold start.
• Recovery time back to the steady-state median once the burst clears.

Report percentiles rather than means, and state the ramp shape alongside any burst figure, because a number without a ramp is not reproducible. Run the client in the same region as the endpoint. A remote client adds its round trip to every measurement and inflates the baseline more than the burst, which compresses the very ratio you are trying to observe.

This article does not publish a single burst latency figure, because that number is specific to account tier, region, model, and time of day. The protocol above is what makes your own measurement defensible.

This article’s floor formula and its measured throughput anchor come from the cost framework piece, which derives the general effective-cost-per-token formula for dedicated GPU inference. This article applies that formula specifically to traffic predictability rather than to utilization in the abstract.

That framework piece states its crossover as 72.2% where this article now states 46.9%. As of this writing, that gap has two independent causes, not one, and repricing alone would not close it. The first cause is the token basis: the framework derives cost per output token, using the benchmark’s 2,036 tok/s output throughput, while this article derives cost per billable token, using the same benchmark’s 4,071.6 tok/s total throughput, because DigitalOcean Serverless bills input and output alike for llama3.3-70b-instruct. At the benchmark’s 1:1 input-to-output ratio, total throughput is twice output throughput, so a threshold expressed per billable token sits at half the utilization of the same threshold expressed per output token, on its own accounting for a factor of two. The second cause is the pricing date: this article uses the H200 GPU Droplet rate effective August 1, 2026, $4.47/hr, following DigitalOcean’s rate change (DigitalOcean, “Upcoming GPU Pricing Updates,” published July 21, 2026); the framework piece’s published 72.2% figure predates that change and, as of this writing, has not been repriced, so it still reflects the prior $3.44/hr rate. If the framework piece repriced to $4.47/hr without also correcting its token basis, its crossover would rise to roughly 93.8% (the same output-token-basis math behind the Dedicated Inference figure earlier in this article), which would widen the gap between the two pieces rather than close it. Treat the framework piece’s 72.2% figure as stale on both counts until it is updated, and recompute your own crossover from your own current rate and your own token basis rather than reconciling the two published numbers against each other.

A related DigitalOcean tutorial, Serverless vs. Dedicated vs. Self-Hosted LLM Inference Cost (published July 10, 2026), measures Qwen3-32B on MI300X and reports a 22% to 48% duty-cycle break-even. That is not the same threshold as this article’s 46.9% crossover, now shared by both the GPU Droplet and Dedicated Inference, and the two are not directly comparable: different model, different GPU, different hourly rate, and that piece measures duty cycle in the abstract where this piece measures whether traffic predictability lets you hit a duty cycle at all. DigitalOcean’s August 1, 2026 pricing update covers AMD as well as NVIDIA GPU Droplets, so the MI300X rate that piece was priced against also changes on that date; confirm whether its 22-48% figure has been repriced before treating it as current, for the same reason this article flags its own comparison to the framework piece above.

For the mechanics of routing overflow traffic from a reserved GPU to serverless, as used in the reserved-floor pattern above, see How to Use the Inference Router.

This article treats input and output tokens as a single billable unit because DigitalOcean prices them identically for llama3.3-70b-instruct. That symmetry is not universal. For models that charge a premium on output, the cost basis splits and the input:output ratio of your workload starts to matter independently of its traffic shape, which is covered in The Hidden Cost of Output Token Pricing for Llama 3.3 70B.

Is Serverless or Dedicated Cheaper for Bursty LLM Traffic?

It depends on whether the bursts are predictable, not on how large they are. A dedicated GPU needs to sustain 1,910 billable tokens per second, 46.9% of one H200’s ceiling, averaged over the hours it is billed, to beat DO Serverless Inference at $0.65 per 1M tokens for llama3.3-70b-instruct. If you can schedule a GPU for exactly the hours your traffic clears that floor, dedicated wins for those hours. If the busy hours cannot be predicted in advance, running the GPU continuously to catch them usually fails the floor test, and serverless is cheaper.

What Sustained Throughput Justifies a Dedicated GPU?

For a DigitalOcean H200 GPU Droplet running llama3.3-70b-instruct FP8, 1,910 billable tokens per second sustained, averaged across the billing period, is the break-even point against DO Serverless Inference. That is 46.9% of the measured 4,071.6 tok/s total-token ceiling, at the GPU Droplet rate of $4.47/hr effective August 1, 2026. The H200 Dedicated Inference endpoint shares that same $4.47/hr rate and therefore the same 46.9% threshold; before the rate change, the GPU Droplet ran at $3.44/hr with a lower 36.1% threshold, and Dedicated Inference carried a separate, higher floor for its managed-serving premium.

Can a Dedicated GPU Scale to Zero?

No. DigitalOcean bills GPU Droplets and Dedicated Inference endpoints from creation, and that billing continues whether the resource is actively serving traffic, idle, or powered off. Powering off a GPU does not stop the charge.

Does a Hybrid Setup Always Save Money?

No. A reserved floor with serverless overflow only saves money when the marginal reserved GPU’s own average utilization, across the full month, clears the 46.9% crossover. A workload large enough to need multiple GPUs at peak does not automatically justify all of them: in this article’s larger-workload example, the first GPU clears 53.3% utilization and earns its rate at $7,899.84 against $8,346.13 for pure serverless, while a second GPU at 33.3% duty does not, and adding it pushes the total to $8,844.57, above pure serverless. At the current rate, even pure dedicated sized to peak ($9,789.30) loses to serverless, so the rule cuts harder than it used to: apply it per GPU, not to the workload’s average, and do not assume that because one GPU clears the floor, adding more capacity is free money.

How Do I Calculate the Crossover for My Own Traffic?

Run the floor identity with your own GPU’s hourly rate and your provider’s per-token rate: floor_tokens_per_second = gpu_hourly_rate / (serverless_rate_per_token * 3600). Divide that result by your GPU’s measured saturated total-token throughput to get a percentage. Measure your own saturated throughput with the benchmark harness in the cost framework’s methodology section rather than reusing the 4,071.6 tok/s figure in this article, which is specific to llama3.3-70b-instruct FP8 on a single H200 at a 1,024:1,024 input:output ratio.

What Are DigitalOcean Serverless Inference Rate Limits?

Two different pages describe two different limits, and both matter. The Inference APIs reference states a flat 5,000 requests per hour and 250 requests per minute per OAuth token. The Inference Limits page documents tiered requests-per-minute and tokens-per-minute quotas that scale with your account tier, from 120 RPM and 500K to 750K TPM at Tier 1 up to 4,500 RPM and 3.5M to 70M TPM at Tier 5. Check your account’s tier on the Resource Limits page in the Control Panel before sizing a burst against either figure.

The most reliable source is your own account rather than either page. Every Serverless Inference response carries x-ratelimit-limit-requests, x-ratelimit-limit-tokens-per-minute, and x-ratelimit-limit-tokens-per-day headers, each with a matching remaining and reset value. Read your actual quota from those headers before sizing a burst, and expect them to be the ground truth if they disagree with the documentation.

This article covered one governing number for llama3.3-70b-instruct on a DigitalOcean H200: a GPU Droplet needs 1,910 sustained billable tokens per second, 46.9% of its measured ceiling, to beat DO Serverless Inference at $0.65 per 1M tokens. It applied that floor across three traffic archetypes, business-hours, viral-spike, and steady-state, and three capacity patterns, scheduled capacity, a reserved floor with serverless overflow, and pure serverless, using the 8-hour daily peak example to show the same GPU serving the same workload losing money held around the clock and winning decisively rented only for the hours it is needed. Across every case, what decided the outcome was never the size of the spike; it was whether you knew, in advance, which hours the GPU would be busy.

Two things follow for sizing your own traffic. Check volume against a single GPU’s ceiling before trusting a peak-to-trough ratio alone: holding a 10:1 ratio fixed and scaling volume up still moves the correct architecture, but at the current GPU rate the larger workload’s two multi-GPU options, pure dedicated sized to peak and the 2-GPU reserved floor, both now lose to pure serverless; only the single-GPU reserved floor still wins, and by a thinner margin than before the rate change. And sort by predictability before size: a predictable business-hours spike and an unpredictable viral spike, similar in size, land on opposite sides of the decision, one scheduled around for a 37.3% saving and the other unable to be scheduled around at all, because scheduling requires knowing the window before it opens.

Start with predictability, then volume. If you can name the hours your load will clear the floor, scheduled capacity or a reserved floor with overflow will beat both pure options. If you cannot, DigitalOcean’s own guidance already tells you where to start: serverless is built for traffic you cannot predict. Profile your sustained token floor against your own measured throughput, using the formula in this article, before committing to reserved GPU capacity for a workload that spikes without warning.

You can also refer to the following tutorials from our Inference in Production series to get started:

1. Why Your LLM Bill Is 3× What You Expected
2. How to Choose the Right LLM Model for Inference Use Case
3. Prompt Caching in Practice: From 7% to 74% Hit Rate
4. Multi-Provider LLM Routing Is Not a Problem, It’s Your Architecture

• Token Economics Across Traffic Profiles on Dedicated GPUs — Derives the effective-cost-per-token formula and benchmark harness this article’s 1,910 tok/s floor builds on.
• DigitalOcean Inference Mode Comparison for Your Each Use Case — Compares serverless, dedicated, batch, and Inference Router modes so you can match a hosting choice to your traffic shape before running the floor math.
• Serverless vs. Dedicated vs. Self-Hosted LLM Inference: When Self-Hosting Actually Gets Cheaper — Measures duty-cycle break-even on a different model and GPU, useful as a cross-check against this article’s 46.9% crossover.
• Dedicated vs. Serverless Inference as You Scale — Covers when predictable baselines justify migrating from serverless to dedicated capacity as volume grows.
• Multi-Model API Cost Governance with the Inference Router — Shows how to route overflow traffic across model tiers, the serverless side of the reserved-floor pattern worked in this article.
• Metrics that Matter with Serverless Inference — Explains which latency, throughput, and cost metrics to measure before sizing a burst against serverless rate limits or SLAs.
• Why Serverless Inference Consistency Varies on the Same Model — Benchmarks idle cold start and queue-wait behavior that the burst-latency section above points you to measure yourself.
• The Hidden Cost of Output Token Pricing for Llama 3.3 70B — Covers when input:output pricing asymmetry changes the floor math beyond the total-token billing basis used here.
• Continuous Batching vs. Static Batching in LLM Inference — Explains how continuous batching affects saturated throughput, the ceiling the 1,910 tok/s floor is expressed as a percentage of.
• Upcoming GPU Pricing Updates — Documents the August 1, 2026 H200 rate change to $4.47/hr that repriced every crossover figure in this article.