DeepSeek Inference Engine Cost : How DeepInfer Cuts the Price of Serving AI
When a company says one cluster serves 155 trillion tokens a day and earns US$38 per GPU-hour even at its cheapest pricing window, that is not a model story. It is a cost story. DeepSeek published those figures when it open-sourced DeepInfer, the engine it uses to run V4.1 Flash, and they explain a lot about why its API prices are low. This post looks at DeepInfer through one lens only: what it says about the cost of serving AI, and what that means whether you are a buyer, a builder or just a curious user.

I read DeepSeek's repository and its long deployment Q&A to separate what is released from what is only described. Watch the one-minute version below:
Why the engine decides the price
A language model is a file of numbers. Turning it into a service that answers millions of people means loading it across many chips, batching requests, remembering what it has already read and keeping every chip busy. The software that does this is the inference engine, and its quality shows up directly in the price per token you pay.
DeepSeek's own framing is that its system was built to serve more requests with less compute and energy, so that the service can be offered at a lower price. That is a claim about intent, not a measured price comparison, and I would treat it that way. What is useful is that DeepSeek shows the mechanisms behind it, and each one is a cost lever.
The cost levers DeepSeek describes
Keep the chips full. DeepSeek runs the part of the model called experts across 64 to 128 GPUs for decoding. Spreading experts that widely lets each GPU process bigger batches, which raises throughput per chip. It also keeps spare copies of busy experts so no single GPU becomes the bottleneck.
Do not read the same text twice. The engine is designed around a very large cache of already processed context. DeepSeek says it keeps that cache for more than 72 hours, uses about 100 PiB of SSD storage for it, and sets a lifetime of 3 days for agent traffic and 7 days for chatbot traffic. Every cache hit is prefill work that never has to be paid for again.
Cheaper numbers. The engine uses 4-bit formats for the main KV cache, the indexer and the expert weights. Smaller numbers move less data per token, and moving data is a large part of the cost of running a big model.
Fewer wasted checks. Its speculative decoding method, DSpark, verifies fewer drafted tokens per step. In DeepSeek's table the gain at its production load of roughly 60 sequences per GPU is about 4 to 7% more throughput, rising to 11% at heavier load. That is a modest gain, but at this scale a few percent is a lot of hardware.
Less recomputation. A technique called SWA bounded replay rebuilds only the last 128 tokens of context instead of replaying a long history. DeepSeek says this nearly halves total prefill computation on the decoder side.
The headline numbers, with the right caveat
| Figure | What DeepSeek reports |
|---|---|
| Tokens served | 155 trillion per day on one cluster |
| Revenue | US$38 per GPU-hour, at valley (off-peak) prices |
| Load balancing | Roughly 80% of a perfectly balanced offline benchmark |
| Cache retention | More than 72 hours |
These are DeepSeek's own numbers. They have not been checked by anyone outside the company, and revenue per GPU-hour is not the same as profit. Still, they show the scale at which this design is meant to work.
What the open-source release actually gives a buyer
The release is under the Apache 2.0 license, supports only DeepSeek V4.1 Flash, and runs on NVIDIA SM100 GPUs or Huawei Ascend 950 chips. It gives you the optimized kernels, the model and sampler runtime, expert parallelism with expert load balancing, DSpark, SWA bounded replay, asynchronous scheduling, structured outputs and deterministic sampling.
It does not give you the KV cache system, the load balancers for prefill and decode, or the full prefill/decode disaggregation. DeepSeek says the cache and balancer code is tied to its internal infrastructure. Those are exactly the pieces behind the cache savings described above, so a team that downloads the engine today gets part of the cost advantage, not all of it.
If you are buying AI by the token
For most buyers nothing changes today. You still pay your provider's price, and DeepInfer does not lower it. What it does is give hosting providers a documented way to serve V4.1 Flash, and DeepSeek says its recipe component keeps third-party API behavior aligned with the official one. Over time, more providers able to run the model well is the likely route to more competition on price. That is my reading, not a promise from DeepSeek.
If you compare providers, ask the practical questions: which engine do they run, do they offer cached-input pricing, and how do they handle long prompts. Cache-aware pricing is where this design shows up on an invoice.
If you are thinking of self-hosting
Be realistic about the hardware. DeepSeek says a one-prefill plus one-decode setup can run, but the smallest efficient unit is 64 GPUs for prefill plus 64 for decode. For caching it recommends storage servers with 20 SSDs of 30.73 TB each and at least 40 GB/s of read speed. This is data-center territory. If you only need V4.1 Flash answers, calling an API will almost always be cheaper than building this yourself.
My take
DeepInfer is more valuable as a map than as a download. It shows where the money goes when you serve a frontier-scale model, and it shows a team willing to publish the details. Buyers should read it as a reason to expect pressure on serving costs, not as a coupon. I would wait for independent benchmarks before treating any throughput claim as settled.
For the full breakdown with the five-step infographic, the included-versus-missing table and the FAQ, read my complete DeepInfer guide on SoftReviewed.

Frequently asked questions
Does DeepInfer make DeepSeek's API cheaper for me today? No. Your price is set by DeepSeek or your provider. The engine is the reason DeepSeek says it can offer low prices.
Is DeepInfer free to use? The code is Apache 2.0 licensed. The model weights carry their own license on Hugging Face.
Can a small team get the same savings? Not fully. The cache system and load balancers are not included, and the efficient deployment size is large.
Which hardware is supported? NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
Comments
Post a Comment