Architecture
Letting the server stop doing the thing it is bad at
The obvious move is to run everything on the server, because the server is the thing with all the cores. That turned out to be exactly backwards for one specific workload.
The assumption I started with
- The storage server has by far the highest core count of anything in the house, so it looked like the natural place to run video encoding.
- Encoding is embarrassingly parallel, which reinforced the idea that more cores would win.
Why it lost anyway
- The server is built on a chip generation that predates AVX2, and modern encoders lean on those instruction sets heavily.
- Per core it loses badly to anything recent, and no amount of core count closed the gap.
- A laptop-class chip from a recent generation was finishing jobs faster than a dual-socket server with far more threads.
The reframe
- The question stopped being "how do I make the server faster at this" and became "why is the server doing this at all".
- The server is genuinely good at holding disks and moving bytes. That is what it was bought for.
- Encoding is a compute problem, and the compute is sitting idle elsewhere in the house most of the day.
What I built
- A scheduler farms encode jobs out to whichever machine is currently idle.
- A Mac mini runs continuously and takes the steady load; a desktop workstation contributes opportunistically when it is not being used for anything else.
- Failed jobs get quarantined rather than silently dropped, so a bad file surfaces as something to look at instead of disappearing.
- The server keeps the storage and the library, and hands off the part it is bad at.
The lesson
- Match work to the hardware that is actually good at it, rather than to the hardware that looks most impressive on paper.
- Core count is not a proxy for throughput when the instruction set is the bottleneck.
- Spare capacity you already own is cheaper than upgrading the machine you assumed should do the job.