Posts

Showing posts with the label Inference

DeepSeek Inference Engine Cost : How DeepInfer Cuts the Price of Serving AI

Image
When a company says one cluster serves 155 trillion tokens a day and earns US$38 per GPU-hour even at its cheapest pricing window, that is not a model story. It is a cost story. DeepSeek published those figures when it open-sourced DeepInfer, the engine it uses to run V4.1 Flash, and they explain a lot about why its API prices are low. This post looks at DeepInfer through one lens only: what it says about the cost of serving AI, and what that means whether you are a buyer, a builder or just a curious user. I read DeepSeek's repository and its long deployment Q&A to separate what is released from what is only described. Watch the one-minute version below: Why the engine decides the price A language model is a file of numbers. Turning it into a service that answers millions of people means loading it across many chips, batching requests, remembering what it has already read and keeping every chip busy. The software that does this is the inference engine, and its quality sho...