AI Platform & Infrastructure
The shared serving, model-routing, and ML infrastructure that powers AI across the company.
Articles in AI Platform & Infrastructure
Putting Machine Learning in the Checkout Path Without Making Checkout Depend on It
How we run multiple production ML predictors inside a latency-sensitive purchase funnel — and why the funnel never blocks on inference.
LegalZoom Engineering · · 8 min read
One API in Front of Every LLM: Building a Cost-Aware Model Gateway
Cost-per-token is a sticker price that tells you almost nothing. After an autonomous agent quietly burned through its monthly model budget and went dark for two days, we built a gateway that optimizes for what actually matters: realized cost per completed task.
LegalZoom Engineering · · 8 min read
From Model Sprawl to a Shared ML Platform
How we consolidated dozens of one-off model deployments into a single FastAPI-based, Kubernetes-served paved path that carries everything from classic prediction services to LLM agents and shared MCP tool servers — without turning shared infrastructure into a bottleneck.
LegalZoom Engineering · · 7 min read