About the role

What will you do at EnCharge AI?

EnCharge AI is a leader in advanced AI hardware and software systems for

edge-to-cloud computing. EnCharge’s robust and scalable next-generation

in-memory computing technology provides orders-of-magnitude higher compute

efficiency and density compared to today’s best-in-class solutions. The

high-performance architecture is coupled with seamless software integration and

will enable the immense potential of AI to be accessible in power, energy, and

space constrained applications. EnCharge AI launched in 2022 and is led by

veteran technologists with backgrounds in semiconductor design and AI systems.

About the Role

EnCharge AI is seeking an LLM Inference Deployment Engineer to optimize, deploy,

and scale large language models (LLMs) for high-performance inference on its

energy efficient AI accelerators. You will work at the intersection of AI

frameworks, model optimization, and runtime execution to ensure efficient model

execution and low-latency AI inference.

Responsibilities

* Deploy and optimize LLMs (GPT, LLaMA, Mistral, Falcon, etc.) post-training
from libraries like HuggingFace
* Utilize inference runtimes such as ONNX Runtime, vLLM for efficient
execution.
* Optimize batching, caching, and tensor parallelism to improve LLM scalability
in real-time applications.
* Develop and maintain high-performance inference pipelines using Docker,
Kubernetes, and other inference servers.

Qualifications

* Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, or
related field.
* Experience in LLM inference deployment, model optimization, and runtime
engineering.
* Strong expertise in LLM inference frameworks (PyTorch, ONNX Runtime, vLLM,
TensorRT-LLM, DeepSpeed).
* In-depth knowledge of the Python programming language for model integration
and performance tuning.
* Strong understanding of high-level model representations and experience
implementing framework-level optimizations for Generative AI use cases
* Experience with containerized AI deployments (Docker, Kubernetes, Triton
Inference Server, TensorFlow Serving, TorchServe).
* Strong knowledge of LLM memory optimization strategies for long-context
applications.
* Experience with real-time LLM applications (chatbots, code generation,
retrieval-augmented generation).
EnchargeAI is an equal employment opportunity employer in the United States.
The salary range for this position is $180,000 to $240,000 USD ($175,000 to
$245,000 CAD) per year. Actual compensation offered will be determined based on
job-related knowledge, skills, and experience.

Which skills does this role require?

LLM Inference DeploymentModel OptimizationRuntime EngineeringPythonAI FrameworksDockerKubernetesTensor ParallelismBatchingCachingInference PipelinesReal-Time ApplicationsMemory OptimizationGenerative AIHuggingFaceONNX RuntimeAILLMInferenceDeploymentOptimizationRuntimevLLMTensorRT-LLMDeepSpeedInference ServerChatbotsCode GenerationRetrieval-Augmented GenerationPyTorchTensorFlowLLMs

Make your next move

Build a shortlist and prepare

Identify the requirements you can demonstrate, then choose examples from your work to discuss with the hiring team.

Build a focused shortlist before you applyCompare role requirements with your experience and give each application a clear reason.
Machine learning jobsCompare current openings and review what to look for in this role.
Practice explaining your experience in an interviewRehearse your answers before meeting the hiring team.

Review the responsibilities and requirements before adding an opening to your shortlist.