AI Experience Researcher, Product Evaluation, Vision Products Group

We are seeking a highly motivated and analytical AI Experience Researcher to

join our team. This role blends cognitive and human sciences, data sciences,

systems design, and product evaluation to ensure AI-powered products deliver

exceptional and intuitive customer experiences. You will work alongside a small

but impactful team, collaborating with ML and data scientists, software

engineers, designers, project managers, and other cross-functional teams at

Apple to define success criteria for AI experiences, and create rigorous

evaluations that measure these criteria in iterative product development cycles.

If you're passionate about applying scientific rigor to real-world problems,

thrive on innovation, and want your work to impact hundreds of millions of

users, this role offers an exceptional opportunity to make a lasting

contribution to products people use every day.

DESCRIPTION

The central challenge of this role is figuring out what "good" means for an AI

experience, and then designing rigorous evaluations that measure those qualities

reliably and at scale. This requires both deep theoretical grounding in human

experience and a solid analytical mindset to operationalize that understanding

into scalable evaluation frameworks. Leaning on research in human sciences, you

will decompose complex AI interactions into their constituent parts, reason

about how those parts interact, and build evaluation frameworks that hold up

under the scrutiny of non-deterministic nature of AI experiences and the

pressures of iterative product development. You will derive experimental

designs, create golden data sets, write tests, and turn them into prompts for

LLM judges or instructions for human raters. You will run automated evaluations,

analyze results, and present findings to diverse stakeholders. Candidates who

bring both quantitative rigor and a qualitative sensibility — to recognize

patterns in model behaviors and outputs, and to develop an interpretive

understanding of what the data is and isn't capturing from a human perspective —

will thrive in this role.What matters most is the ability to hold both

orientations at once — to think carefully about what makes an experience work,

and to measure complex human dimensions with precision. We are also looking for

someone who is excited to co-create what this discipline looks like going

forward — bringing intellectual curiosity and a point of view about where

human-centered AI evaluation should be headed.

MINIMUM QUALIFICATIONS

Advanced degree in Cognitive Psychology, Human-Computer Interaction (HCI), User

Experience (UX) Research, Learning Sciences, Learning Analytics, Psychometrics,

Applied Behavioral Science, or a related field with a focus on human cognition,

behavior, and empirical evaluation A strong data-driven mindset with experience

designing and conducting rigorous empirical research or evaluation — including

experimental design, data analysis, and interpretation of various qualitative

and quantitative data — particularly in the context of complex human-system

interactions Ability to reason from theoretical grounding about what makes an

experience good in a given context, and to translate that reasoning into

evaluation frameworks and measurement designs Demonstrated ability to

operationalize research literature, qualitative user feedback, and quantitative

behavioral data into actionable evaluation criteria, observable metrics, and

product insights Proficiency in data analysis and interpretation, with a strong

understanding of statistical validity in evaluation contexts Exceptional

collaboration skills with a track record of working effectively in

cross-functional teams that include engineering, ML, design, QA, leadership, and

subject matter experts of diverse domains Strong communication skills, with the

ability to translate complex research findings and evaluation results into

clear, actionable recommendations for both technical and non-technical audiences

PREFERRED QUALIFICATIONS

Familiarity with methods for capturing experiential quality beyond task success

— such as cognitive interviews, think-aloud protocols, interaction analysis, or

discourse and conversation analysis Experience designing and implementing

automated evaluation pipelines, including writing prompts for LLM judges and

constructing human-in-the-loop or multi-turn evaluation setups Experience

working with multimodal or agentic systems, AI/ML models, preferably Large

Language Models Familiarity with automated testing frameworks and tooling

Experience with data generation and annotation workflows, including curating

datasets, scenarios, and tasks that represent realistic usage Portfolio

demonstrating previous evaluation frameworks, research findings, or measurable

contributions to product improvement Background in learning sciences or

instructional design, with experience reasoning about what makes a complex human

experience effective is a plus