Senior Machine Learning Engineer (Inference Platform)

ABOUT WIZARD AI

At Wizard AI, we’re building the top-performing AI Shopping Agent that delivers

the best products from across the web with unmatched accuracy, quality, and

trust. Our ML models power the core of our platform, and we’re looking for a

Senior Machine Learning Engineer to own how they run in production reliably,

efficiently, and at scale.

THE ROLE

As a Senior ML Engineer on our Inference Platform, you’ll own the end-to-end

lifecycle of production ML serving systems from model packaging and deployment

to monitoring, optimization, and scaling. This is not a traditional MLOps role

focused solely on pipelines and tooling. You’ll be responsible for the inference

infrastructure powering a live conversational shopping agent, operating multiple

specialized serving engines under real-world production load.

You’ll own critical decisions around serving architecture, performance,

reliability, and scalability, working closely with ML Engineers, Data teams,

Product, and DevOps to ensure models move seamlessly from experimentation into

high-performance production systems.

WHAT YOU'LL DO

* Own and evolve our multi-engine inference platform, supporting a variety of

model types and serving requirements.

* Build and improve production ML pipelines — taking models from

experimentation to reliable, high-throughput serving.

* Define and implement model versioning, rollout, rollback, and lifecycle

management strategies that ensure reproducibility and operational

reliability.

* Define and enforce serving-layer SLAs, including latency, availability, GPU

utilization, Time-to-First-Token (TTFT), and Inter-Token Latency (ITL).

* Build observability, monitoring, alerting, and operational tooling for

production inference systems.

* Apply software engineering best practices, including testing, CI/CD

integration, and reproducibility across ML workflows.

* Optimize inference performance through efficient resource utilization,

hardware-aware serving strategies, and cost-conscious infrastructure design.

* Ensure ML serving systems are secure, scalable, and operationally resilient.

* Partner with ML, Data, Product, and DevOps teams to turn ideas into

production systems, driving the technical decisions on serving and scale.

WHAT WE'RE LOOKING FOR

* Bachelor's or Master's degree in Computer Science, Data Science, Engineering,

or a related field, or equivalent practical experience.

* 5–8+ years of experience in Software Engineering, ML Engineering, Platform

Engineering, or Infrastructure Engineering, with direct ownership of

production ML serving systems.

* Hands-on experience running an LLM serving engine (vLLM, TGI, TensorRT-LLM,

or SGLang) in production under real load — not just managed or hosted

endpoints.

* Strong Python skills and software engineering fundamentals, combined with

deep systems and infrastructure knowledge.

* Experience with cloud platforms such as AWS, GCP, or Azure, and familiarity

with ML lifecycle tooling, experimentation platforms, and model registries.

* Strong grasp of inference performance — continuous batching, KV-cache and

GPU-memory behavior, quantization, and CPU-versus-GPU bottlenecks — with the

instinct to profile before tuning.

* Experience serving heterogeneous workloads, including LLMs, embedding models,

and extraction models, each with distinct latency, throughput, and scaling

requirements.

* Demonstrated ability to balance latency, throughput, reliability, and

infrastructure cost while operating production-scale ML systems.

* Experience in high-growth startup environments and comfort operating in

fast-moving, evolving technical landscapes.

WHAT SUCCESS LOOKS LIKE

RELIABLE, SCALABLE INFERENCE SYSTEMS

Production serving infrastructure operates with clear SLAs, strong

observability, and minimal downtime. Latency, availability, throughput, and GPU

utilization are actively measured and optimized as platform demands grow.

END-TO-END OWNERSHIP

You own the complete serving lifecycle — from deployment and release management

through monitoring, optimization, and scaling — enabling ML engineers to ship

quickly while maintaining reliability and reproducibility.

TECHNICAL LEADERSHIP AND IMPACT

You shape the future of Wizard's inference platform, driving key architectural

decisions that improve performance, reduce infrastructure costs, and support the

next generation of AI-powered shopping experiences.