Machine Learning Engineer, ML/GenAI Evaluation

Would you like to contribute to Machine Learning and Generative AI technologies?

Are you passionate about measuring what matters and ensuring AI systems work

reliably for everyone? Do you believe that rigorous evaluation — including

holding models accountable to fairness standards — is what separates great ML

from good ML? We truly believe it is! We are defining what exceptional looks

like for machine learning across Wallet, Payments, and Commerce. As a Machine

Learning Engineer specializing in Evaluation, you will establish the evaluation

criteria, metrics frameworks, and quality standards that determine when models

are ready to reach hundreds of millions of users. Your judgment shapes model

quality and earns the confidence to ship. You'll work at the intersection of

rigorous ML science and high-impact product decisions, collaborating closely

with ML Engineering, Product, Privacy, and Legal teams. This unique opportunity

puts you at the center of model quality — designing adversarial test strategies,

surfacing failure modes before they reach users, and owning the sign-off process

that ensures Apple's financial features meet the highest bar for accuracy,

robustness, and reliability.

DESCRIPTION

The ideal candidate is a rigorous, curious ML practitioner who believes that how

you measure a model is just as important as how you train it. You think

critically about what metrics actually capture, know how models break in the

real world, and hold quality standards others find uncomfortably high —

including on dimensions like fairness. You will own the full evaluation

lifecycle for ML models across Wallet features — designing test frameworks,

adversarial corpora, and benchmarks that reflect the diversity of Apple's global

user base, then making the final quality call before any model ships. Your

findings directly shape model development priorities and product decisions at

scale.

MINIMUM QUALIFICATIONS

M.S. in Machine Learning, Computer Science, Statistics, Applied Mathematics, or

a related technical field strongly preferred. Bachelor's degree with 7+ years

hands-on experience in ML evaluation, model quality, or applied research will be

considered 5+ years of hands-on ML experience, with deep expertise in model

evaluation, offline metrics design, and behavioral testing Strong track record

designing evaluation frameworks for production ML systems — not just

accuracy/F1, but precision-recall tradeoffs, calibration, fairness, and

task-specific quality dimensions Creative mindset with the ability to translate

standard ML evaluation metrics (F1, AUC, etc.) into utility and user trust

measures Experience testing for distribution shift, out-of-distribution

generalization, and temporal drift in real-world deployed models Proven ability

to construct adversarial test suites, aggressor scenarios, and edge-case corpora

that surface model failure modes before they reach users Experience with

structured and semi-structured document understanding, OCR pipelines, or

financial data extraction is a strong plus Strong programming skills in Python;

fluency with evaluation tooling, data pipelines, and experiment tracking (e.g.,

MLflow, W&B, or equivalent) Excellent communication skills — ability to

translate metric results into product-quality narratives for engineering and

executive audiences Experience owning model quality sign-off in a

cross-functional launch process

PREFERRED QUALIFICATIONS

PhD in Computer Science, Data Science, Statistics, AI/ML, or a related field.

Experience with Bayesian or causal graph-based approaches to data generation.

Experience with causal approaches to fairness evaluation — counterfactual

fairness, causal Shapley values, or structural causal model–based bias auditing.

Experience evaluating models under privacy constraints or on-device inference

settings is a plus. Familiarity with confidence calibration techniques and

uncertainty quantification a plus Background in financial services, fintech, or

consumer payment products