Evals & measurement
Golden sets, claim-level scorers, injected-error tests and paired-bootstrap intervals, with the failures written down.
Applied AI Product Leader
About
I’m an applied AI product leader, currently on NVIDIA’s GenAI platform. Before that I built products at Lyft, Intuit, Meta and Visa.
Outside work, I build and measure AI systems: I benchmark clinical scribes and coding agents, serve large open models on workstation GPUs, and run a small studio of AI agents that makes Everdust (opens in a new tab) and the physics work in the Lab.
What I’m good at
Agentic building
A fleet of AI coding agents across five machines, run on written specs and measured gates.
Product judgment
Platforms and products shipped at NVIDIA, Lyft, Intuit, Meta and Visa.
Career
Full résumé- 2022 — presentNVIDIAApplied AI Product Leader
- 2019 — 2022LyftSenior Product Manager
- 2017 — 2019IntuitProduct Manager
- 2015 — 2017MetaProduct Management
- 2014 — 2015VisaAssociate Product Manager
- EarlierAmgenProject Manager
Earlier product case studies are in the archive.
Contact
- sipratt@gmail.com
- sethpratt (opens in a new tab)