About the role
What will you do at Protege?
We are looking for a Forward Deployed Machine Learning Engineer with 3+ years of experience to be the first MLE dedicated to Protege's Benchmarks and Evaluations vertical. You'll work directly with the GM and researchers to build the technical foundation for AI model evaluation — a critical capability at one of the most exciting data infrastructure companies in AI, backed by a16z, CRV, and others. What will you be doing?
Partner with the GM and early customers to define, design, and build benchmarks and evals across different domains and modalities Build and own the backend infrastructure: data pipelines, execution environments, storage, and orchestration Stand up sandboxed environments for agentic evals requiring tools, code execution, or multi-step tasks Own the engineering portion of customer engagements end to end Identify repeatable eval patterns and infrastructure gaps to drive product opportunities Key Requirements 4+ years of engineering experience with hands-on ML model evaluation work Prior ownership of backend and infrastructure (data pipelines, execution environments) Experience building benchmarks, evals, or human data pipelines for LLMs (strongly preferred) High ambiguity tolerance and a strong bias to action in fast-moving environments Strong written communication skills for customer-facing and cross-functional work
Which skills does this role require?
Make your next move
Build a shortlist and prepare
Identify the requirements you can demonstrate, then choose examples from your work to discuss with the hiring team.
- Build a focused shortlist before you applyCompare role requirements with your experience and give each application a clear reason.
- Machine learning jobsCompare current openings and review what to look for in this role.
- Practice explaining your experience in an interviewRehearse your answers before meeting the hiring team.
Other roles to compare
Review the responsibilities and requirements before adding an opening to your shortlist.
Senior AI Engineer
Blue Orange Digital · Washington, District of Columbia, United States
GTM AI Engineer -Deal Desk
Motive Agency · United States
Sr. Applied AI Engineer
phData · United States
Senior Applied AI Engineer
QuEra Computing Inc. · Boston, Massachusetts, United States
AI Engineer
Booz Allen Hamilton · Reston, Virginia, United States
Staff AI Engineer - Personalization, Brand, Communications Tech
American Express · New York, New York, United States
Role information can change. Prepin can help you prepare, but does not submit an application for this role.