Principal Product Manager Technical, AWS Applied AI Solutions - Core Services

As part of the AWS Applied AI Solutions organization, we have a vision to

provide business applications, leveraging Amazon's unique experience and

expertise, that are used by millions of companies worldwide to manage day-to-day

operations. We will accomplish this by accelerating our customers' businesses

through delivery of intuitive and differentiated technology solutions that solve

enduring business challenges. We blend vision with curiosity and Amazon's

real-world experience to build opinionated, turnkey solutions.

We are looking for a Principal Product Manager, Technical to own and drive the

product strategy, revenue, and adoption for AI Agent Evaluations within our Core

Services AI Foundations team. You will be responsible for defining the long-term

vision and building the business around evaluation capabilities that enable

development teams to measure, benchmark, and continuously improve the quality,

safety, and reliability of their AI powered agents. This includes owning the

full product lifecycle from concept through GTM for evaluation tooling, quality

scoring methodologies, regression testing frameworks, and human review

workflows. You will own one or more products in production while also getting in

on the ground floor of new projects that bring evaluation capabilities to

customers worldwide.

As a Principal Product Manager, you will be part of the larger product

leadership community at AWS. This community plays a critical role in broad

business planning, working closely with senior management to develop business

targets and resource requirements, influencing long term technical and business

strategy, helping hire a talented team of PMs, and enabling us to deliver

solutions rapidly. You will be seen as the subject matter expert for AI

evaluation within Amazon AI.

A successful candidate will bring a passion for technology services, strong

business acumen and judgment, proven experience launching products and building

business around them, ability to define products, desire to have an industry

wide impact, and ability to work within a fast moving environment in a large

company to rapidly deliver services that have broad business impact. You will

work backward from the needs of development teams building agentic AI

applications and define how they assess agent behavior across dimensions

including correctness, safety, groundedness, and customer satisfaction. You will

partner closely with engineering and applied science teams to translate complex

evaluation methodologies into products that scale across diverse use cases and

agent architectures.

Key job responsibilities

• Lead product definition for AI agent evaluation capabilities, owning the

customer working backwards strategy, tenets, long term goals, and working

backwards documents including customer and market feedback, competitive

analysis, and business metrics to inform direction

• Define product vision including all aspects of future roadmap, investment,

innovation, and experimentation for automated evaluation pipelines, human review

workflows, and quality benchmarking tools

• Own execution of product planning and development including customer goals and

business requirements for product releases, ensuring implementation is aligned

with product goals, and ownership of product positioning

• Lead product launches owning GTM plans that deliver results ensuring customer

and business goals are met in operational launch plans

• Create and revise GTM strategy and be responsible for pricing new features and

updating pricing for existing features working closely with finance

• Work with field and BD teams to create customer funnel and drive POC to pilot

to production motions

• Be responsible for creating and reporting on the service's revenue and

customer traction through weekly and monthly business reports shared with

leadership

• Partner with applied scientists and engineers to translate research into

production grade product features

• Drive alignment across dependent teams (identity, observability, data

platform) to ensure evaluation signals integrate seamlessly with the broader

Core Services AI Foundations platform

• Lead interaction with the technical team including helping make tradeoffs

based on customer requirements and QA/testing of the product

A day in the life

You split your time between building the business (developing partnership

strategy, earning trust with stakeholders, defining GTM) and building the

product (writing requirements, working closely with engineering, driving

launches). On any given day you might be meeting with a customer's engineering

leadership to understand how they assess agent quality at scale, presenting

product strategy and business results to senior leadership, reviewing a pricing

model with finance, working with field teams on a customer POC, or collaborating

with applied scientists on how to productize new evaluation research. You

operate with high independence across ambiguous problem spaces, make tradeoffs

across competing priorities, and own outcomes from requirements through launch

and sustained business growth.

About the team

ABOUT AWS:

Diverse Experiences

Amazon values diverse experiences. Even if you do not meet all of the preferred

qualifications and skills listed in the job description, we encourage candidates

to apply. If your career is just starting, hasn’t followed a traditional path,

or includes alternative experiences, don’t let it stop you from applying.

Why AWS

Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted

cloud platform. We pioneered cloud computing and never stopped innovating —

that’s why customers from the most successful startups to Global 500 companies

trust our robust suite of products and services to power their businesses.

Work/Life Balance

We value work-life harmony. Achieving success at work should never come at the

expense of sacrifices at home, which is why flexible work hours and arrangements

are part of our culture. When we feel supported in the workplace and at home,

there’s nothing we can’t achieve in the cloud.

Inclusive Team Culture

Here at AWS, it’s in our nature to learn and be curious. Our employee-led

affinity groups foster a culture of inclusion that empower us to be proud of our

differences. Ongoing events and learning experiences, including our

Conversations on Race and Ethnicity and AmazeCon conferences, inspire us to

never stop embracing our uniqueness.

Mentorship and Career Growth

We’re continuously raising our performance bar as we strive to become Earth’s

Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and

other career-advancing resources here to help you develop into a better-rounded

professional. Basic

Qualifications

• - 7+ years of working as a Technical Product
• Manager experience
• - 5+ years of technical (software development, network development, IT, other
• related) experience
• - Experience delivering large-scale SaaS, PaaS or LaaS products where you are
• responsible for the full product lifecycle, from concept through GTM (go to
• market)
• - Experience with machine learning or AI systems, including familiarity with
• model evaluation, testing methodologies, or quality assurance frameworks
• - Demonstrated ability to write clear, detailed technical product documents
• (PR/FAQs, technical specifications, architecture diagrams) and drive engineering
• teams to consensus

Preferred Qualifications

• - MBA or advanced degree in a
• technical discipline
• - Experience with LLM evaluation, AI safety testing, or automated quality
• assurance for generative AI systems
• - Track record of building and launching evaluation or testing platforms at
• scale
• - Experience working with applied scientists to productize research into
• customer facing features
• - Familiarity with AWS services (AgentCore, SageMaker) and agentic AI frameworks
• (LangFuse)
• - Strong track record of shaping business strategy for technical products or
• services for large enterprises or partners
• - Prior experience bringing to life 0 to 1 AI powered offerings
• Amazon is an equal opportunity employer and does not discriminate on the basis
• of protected veteran status, disability, or other legally protected status.
• Our inclusive culture empowers Amazonians to deliver the best results for our
• customers. If you have a disability and need a workplace accommodation or
• adjustment during the application and hiring process, including support for the
• interview or onboarding process, please visit
• https://amazon.jobs/content/en/how-we-hire/accommodations
• [https://amazon.jobs/content/en/how-we-hire/accommodations] for more
• information. If the country/region you’re applying in isn’t listed, please
• contact your Recruiting Partner.
• The base salary range for this position is listed below. Your Amazon package
• will include sign-on payments and restricted stock units (RSUs). Final
• compensation will be determined based on factors including experience,
• qualifications, and location. Amazon also offers comprehensive benefits
• including health insurance (medical, dental, vision, prescription, Basic Life &
• AD&D insurance and option for Supplemental life plans, EAP, Mental Health
• Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy
• Reimbursement coverage), 401(k) matching, paid time off, and parental leave.
• Learn more about our benefits at https://amazon.jobs/en/benefits
• [https://amazon.jobs/en/benefits].
• USA, WA, Seattle - 181,100.00 - 245,000.00 USD annually