Infrastructure at scale.
Building and operating production inference platforms: vLLM on EKS, bare-metal Kubernetes clusters, and load testing to characterise where things break under pressure.
Lead SRE. I run GPU inference at scale, build observability into it. Background in fintech and regulated industries.
Solutions →kagent at Scale
102 pods, 6 agent types, one finding: Go runtime converges at 6.6 MB idle regardless of tools or system prompt size. Python ADK idles at 219 MB. At 100 agents per type, that is 660 MB vs 21.9 GB.
Read → Oct 2026GPU Inference Observability: Real Numbers Under Load
DCGM + Prometheus + Grafana on top of the Karpenter/vLLM EKS stack. A load test generates real data: GPU at 99%, KV cache at 4.7%, TTFT rock-solid at 485ms. The numbers explain why GPU utilization is not the metric that tells you when inference is failing.
Read → Oct 2026GPU Spot Inference on EKS: Nine Bugs Before It Worked
AWS approved 4 vCPUs for G-family spot instances. Nine bugs later, Mistral-7B was serving at under 500ms TTFT on a T4. Architecture decisions and a full debugging log.
Read →