Home
Working notes on production machine-learning systems — diffusion inference, GPU infrastructure, and the distance between a published method and what actually ships.
Posts
-
Owning the Inference Path: Handwritten fp8 Kernels, 2.5x Cheaper GPU Inference
An image-generation product lives or dies by its GPU bill. This is how per-image compute on Narcis fell 2.5x — no new model, no extra hardware: strip the library stack, use torch.compile as a teacher, and replace it with seven handwritten fp8 Triton kernels. 1.75s to 0.687s per step on a 6.15B diffusion transformer, serving production on NVIDIA L4 today. -
Narcis.ai: Production ML Systems Engineering
Production systems built through deep understanding and minimal complexity — infrastructure foundations, a two-stage diffusion pipeline, and what it takes to run face-identity generation live.
subscribe via RSS