About the role

What will you do at True Talent Solutions Partners?

About the Role

Join the engineering team as its first dedicated quality-infrastructure hire. This is a hands-on software engineering role focused on building AI-powered verification and reliable release systems, so teams can trust test results, ship safely, and make releases more predictable. The work is centered on automated systems, not manual test execution.

What You'll Do

Build and maintain agent-powered test harnesses, LLM graders, evaluation logic, and regression coverage for AI systems.
Improve the trustworthiness of test signals and close gaps that allow defects to reach customers.
Develop automated release gates that make readiness visible and prevent unsafe releases based on evidence.
Own CI/CD pipelines, container builds, artifact publication, and release automation using GitHub Actions.
Improve the reliability of pre-production, staging, and demo environments.
Participate in business-hours escalations and make issues easier to diagnose, without formal after-hours production on-call.

What We're Looking For

Seven to 15 years of experience in quality engineering, release engineering, or platform engineering.
Recent hands-on quality engineering experience applying AI-native testing or evaluation to LLM or agentic systems.
Practical experience with LLM APIs, tool calling or MCP, and evaluation methods.
Strong familiarity with GitHub Actions and GitHub-based CI/CD, including container builds and pipeline automation.
Experience with Python, TypeScript, Docker, Kubernetes, and Helm, plus familiarity with a major cloud or data platform such as AWS, Azure, Snowflake, or Databricks.
Background in an early-stage, fast-moving startup or experience modernizing legacy QA or DevOps practices, with the ability to own systems independently.
Authorization to work in the United States or Canada without visa sponsorship.

Compensation

& Benefits

Annual salary range: $170,000 to $200,000 USD. Visa sponsorship is not available.

Location

Remote, with the primary location listed as New York, New York, United States. Candidates must be authorized to work in the United States or Canada.

Which skills does this role require?

AI-powered verificationRelease engineeringLLM gradersEvaluation logicCI/CD pipelinesGitHub ActionsPythonTypeScriptDockerKubernetesHelmAgentic systemsAI Quality EngineerRelease EngineerSoftware EngineeringLLMAgentic SystemsTest HarnessesRegression CoverageCI/CDContainerizationQuality AssuranceAutomationPlatform EngineeringTool CallingMCPLLMs

Make your next move

Build a shortlist and prepare

Identify the requirements you can demonstrate, then choose examples from your work to discuss with the hiring team.

Build a focused shortlist before you applyCompare role requirements with your experience and give each application a clear reason.
Practice explaining your experience in an interviewRehearse your answers before meeting the hiring team.

Review the responsibilities and requirements before adding an opening to your shortlist.