PythonGoKubernetes operatorsFastAPILLM serving
AI/ML model deployment platform
Software Engineer, Booz Allen Hamilton · 2024 – 2026
The platform data scientists used to get AI/ML models, including LLMs, into production. I led the design of its control plane and of the feature that made prompt iteration cheap.
- Led the design of a control plane that treats the model lifecycle as custom Kubernetes resources: install registers metadata, warm pulls and caches weights, deploy stands up a serving endpoint. The same model then runs under many configurations without re-pulling weights.
- Led a feature that lets data scientists deploy an LLM once and iterate on prompts, inputs, and configurations against it, turning a full redeploy into a config change.
- Extended the control plane to carry guardrail configuration through to running models, so guardrails on inputs and responses could be set from the CLI and UI without a redeploy.
- Rebuilt the Python CLI that packages analytics for deployment, with validation that catches malformed packages before they ship.
- Wrote and reviewed the design records behind these features, reviewed the team's junior engineers, and set the pull request conventions we worked by.
An internal platform, so there is nothing I can link you to.
