AI infrastructure.
GPU inference. Kubernetes. Bare metal.
Building and operating production AI platforms: vLLM inference on EKS, bare-metal Kubernetes clusters, and the observability to know when things break.
SRE focused on AI infrastructure. I run GPU inference at scale, build observability into it. Background in fintech and regulated industries.
Solutions →kagent at Scale
102 pods, 6 agent types, one finding: Go runtime converges at 6.6 MB idle regardless of tools or system prompt size. Python ADK idles at 219 MB. At 100 agents per type, that is 660 MB vs 21.9 GB.
Read → Oct 2026GPU Inference Observability: Real Numbers Under Load
DCGM + Prometheus + Grafana on top of the Karpenter/vLLM EKS stack. A load test generates real data: GPU at 99%, KV cache at 4.7%, TTFT rock-solid at 485ms. The numbers explain why GPU utilization is not the metric that tells you when inference is failing.
Read → Oct 2026GPU Spot Inference on EKS: Nine Bugs Before It Worked
AWS approved 4 vCPUs for G-family spot instances. Nine bugs later, Mistral-7B was serving at under 500ms TTFT on a T4. Architecture decisions and a full debugging log.
Read →