In short. A DL145 Gen11 with an NVIDIA accelerator is a node for inference at the edge, not for training models. The distinction matters: training needs tens of kilowatts and a cluster, inference needs one card and a couple of hundred watts. The typical scenario in Uzbekistan is on-site video analytics — plate recognition, people counting, PPE compliance — where pushing video to the centre costs more than processing it locally. The model is not in the ITsmart catalogue; we size such nodes on request.
Server for video surveillance: sizing and prices →
Jump to: inference versus training · which card · how much video memory · power and cooling · when it pays off · questions and answers
Inference versus training — different machines
Confusing the two is expensive. Training a model means weeks of a cluster of cards at 300–700 W each, fast interconnects and a dedicated cooling loop. Inference applies an already trained model to a data stream: one card, tens or a couple of hundred watts, an ordinary server.
Ninety per cent of business tasks are inference. The model is taken off the shelf or fine-tuned once in the cloud, and then runs on site for years. Edge nodes with a single accelerator exist for exactly that.
Which card
- 70–75 W cards with passive cooling and half length are the mainstream edge choice. They draw power from the slot, need no extra cables and fit a short chassis.
- 150–300 W cards handle more streams per node but require a full-size chassis, the high-performance fan kit and larger power supplies. In a shallow cabinet that is usually impossible.
- 400 W and above is data centre territory, not an edge site.
Before choosing, work out not how many cards fit but how many camera streams must be processed and at what frame rate. People counting runs fine at 2–5 frames per second, and one card handles dozens of streams; plate recognition at speed needs far more.
How much video memory
Video memory is the first limit hit in practice. The model must fit entirely in memory together with the processing buffers; if it does not, swapping to system memory begins and throughput drops several times over.
As a guide: small detection and classification models fit in a few gigabytes, recognition and segmentation models need noticeably more, and language models on site need tens of gigabytes and usually will not run on an edge card. The card is chosen against the specific model, not from general reasoning.
Power and cooling
- Budget the whole cabinet: server, accelerator, switch, UPS. A 75 W card adds roughly the same again at peak; a 300 W card changes the class of the entire installation.
- Passive cards are cooled by the server's own fans. That means the fan kit must be specified for the card rather than for an empty slot — a separate configuration line.
- Size the UPS on peak draw with the accelerator under load, not on the rated output of the power supplies.
When it pays off
- the link to the centre is thin or expensive and the video stream is constant;
- the decision has to be made in a fraction of a second — a barrier, a turnstile, an alarm;
- the data cannot leave the site perimeter under the customer's requirements.
If none of these hold, it is cheaper to send frames to the centre, keep one powerful node for the whole organisation and leave an ordinary server without an accelerator on site — a DL320 Gen11 at 41 000 000 UZS, or a DL380 Gen11 at 49 000 000 UZS if slot headroom is needed.
We will size the node against your model and stream count — card, power, cooling and storage. Get in touch.
Questions and answers
How does inference differ from training?
Training creates the model and needs a cluster of accelerators, weeks of time and tens of kilowatts. Inference applies a finished model to a data stream and fits within one card and a couple of hundred watts. Edge sites do inference.
Which GPU is needed for on-site video analytics?
Edge deployments usually take a passive half-length card at 70–75 W: it draws power from the slot, needs no extra cables and fits a short chassis. More powerful cards require a full-size server, reinforced cooling and a different UPS.
How much video memory is needed?
The model must fit entirely in video memory along with its buffers. Small detection models fit in a few gigabytes; recognition and segmentation need noticeably more. Size it against the model you actually intend to run.
Can models be trained on such a server?
Technically yes, practically pointless: training on a single edge card takes weeks where a cluster takes hours. Train in the cloud or in a data centre and deploy the finished model to the site.
How many cameras can one node handle?
It depends on the model and the required frame rate. At 2–5 frames per second for people counting and event detection one card handles dozens of streams; for plate recognition at speed, only a few. The sizing is done per task.
