Principal Engineer, GKE Platform for AI Inference Workloads

MINIMUM QUALIFICATIONS:

* Bachelor's degree in Computer Science, a related technical field, or

equivalent practical experience.

* 15 years of experience in software engineering, or 15 years of experience

with an advanced degree.

* Experience building distributed systems and driving technical strategy for

platform-level infrastructure.

* Experience with Kubernetes, container runtimes, and AI/ML infrastructure

(e.g., inference serving, LLM, hardware accelerators).

PREFERRED QUALIFICATIONS:

* Master's degree or PhD in Computer Science or related technical field.

* Experience interacting with senior customer stakeholders (CTOs, chief

architects) to represent the technical vision of the organization.

* Demonstrated track record of significant technical contributions to the

Kubernetes open-source project or related CNCF AI/ML projects (e.g., Kueue).

* Demonstrated track record of influencing cross-functional teams (product,

engineering, research) to deliver complex technical outcomes.

* Deep technical understanding of high-performance networking (RDMA, NCCL),

storage/caching architectures for massive model weights, and accelerator

virtualization/sharing mechanisms.

ABOUT THE JOB:

Google Kubernetes Engine (GKE) is the industry standard for container

orchestration and the core of Google Cloud’s modernization strategy. We are now

embarking on a mission to reinvent GKE and Kubernetes as the premier substrate

for the next generation of computing: AI inference at massive scale. We believe

that serving foundation models and large language models represents a paradigm

shift in cloud computing. These workloads demand a fundamental rethink of

orchestration, moving from CPU-bound microservices to accelerator-bound,

memory-bandwidth intensive workloads that require specialized scheduling,

heterogeneous compute pools, and ultra-high-speed networking.

As the Principal Engineer, you will lead the technical and architectural

reinvention of GKE to become the inference engine for the world. This leader

will provide critical LLM Debugger (llm-d) leadership, defining and driving the

long-term strategic technical priorities for integrating high-scale AI Inference

and the llm-d stack as a core competency into the GKE platform, while leading

our contributions to the broader open-source ecosystem.

Google Cloud accelerates every organization’s ability to digitally transform its

business and industry. We deliver enterprise-grade solutions that leverage

Google’s cutting-edge technology, and tools that help developers build more

sustainably. Customers in more than 200 countries and territories turn to Google

Cloud as their trusted partner to enable growth and solve their most critical

business problems.

Individual pay is determined by factors including job-related skills,

experience, and relevant education or training.

US: $307000 - $427000 (USD) + 30% bonus target + equity + benefits

Learn more about benefits at Google

[https://www.google.com/about/careers/applications/benefits/].

RESPONSIBILITIES:

* Lead the architectural direction for llm-d, ensuring a highly optimized,

scalable foundation for distributed LLM and Reinforcement Learning (RL)

serving across the GKE fleet.

* Define GKE's evolution to support massive-scale inference and RL, solving

novel orchestration problems in dynamic resource allocation, multi-host

TPU/GPU scheduling, and high-throughput networking.

* Partner with strategic AI model builders, DeepMind, and Vertex AI to

co-develop an AI-first roadmap, leveraging Google's custom silicon to

optimize throughput and compute density.

* Lead the broader Kubernetes ecosystem and Open Source Software (OSS)

community, driving key upstream initiatives to establish industry standards

for AI, RL, and accelerator orchestration.