Senior/Lead Quality Engineer - Chatbot

Team: IT

Location: Canada

Commitment: Full-time

Workplace Type: remote

Accountabilities:

The Senior/Lead Quality Engineer will be responsible for establishing and evolving quality practices for AI-driven conversational products. This role will combine hands-on testing, technical collaboration, and strategic quality improvements to ensure reliable releases and strong user confidence.

• Develop and continuously improve quality strategies, testing frameworks, and release processes for chatbot products.
• Establish evaluation methodologies to measure chatbot quality, reliability, performance, and regression risks.
• Design testing approaches for dynamic and non-deterministic AI systems that go beyond traditional testing methods.
• Identify, validate, and improve AI safety measures, including protection against prompt injection, unexpected outputs, and model-related risks.
• Collaborate closely with software engineers through code reviews, technical discussions, and debugging sessions to improve quality throughout the development lifecycle.
• Analyze existing testing practices, identify gaps, research new approaches, and recommend practical improvements.
• Design proof-of-concepts for innovative testing tools and methodologies, helping productionize solutions that deliver value.
• Support release readiness by defining quality signals, validation processes, and acceptance criteria.
• Contribute performance testing strategies for AI and LLM-powered workflows.
• Share knowledge, promote quality engineering best practices, and help build a strong quality culture across engineering teams.
• Mentor engineers and influence testing practices within cross-functional product teams.

Requirements:

The ideal candidate has strong software quality engineering experience combined with technical curiosity and an interest in testing AI-driven systems. They should be comfortable operating in uncertain environments and creating practical solutions for complex engineering challenges.

• Strong hands-on experience in software quality engineering, SDET, test automation, or a similar engineering role.
• Experience testing chatbot, conversational AI, RAG, LLM-powered applications, or other systems with dynamic and non-deterministic behavior.
• Proficiency in at least one programming language with the ability to understand production code and collaborate effectively with software engineers.
• Experience designing and implementing automated testing strategies and quality frameworks.
• Strong analytical and investigative mindset with the ability to explore complex problems and turn findings into actionable improvements.
• Ability to evaluate technical trade-offs and prioritize quality initiatives based on risk, user impact, and business objectives.
• Excellent written and verbal communication skills, with the ability to explain technical concepts and testing decisions clearly.
• Strong collaboration skills and experience working with Product, Engineering, and Machine Learning teams.
• Experience working in agile, fast-paced, and evolving environments.

Nice to have:

• Experience with AI security, safety testing, or responsible AI practices.
• Experience testing conversational AI platforms, chatbot systems, or agent-based workflows.
• Previous experience as the first Quality Engineer within a team.
• Experience mentoring engineers or influencing quality practices across organizations.
• Passion for staying current with emerging technologies and sharing new methodologies with teams.

Benefits:

• Fully remote work flexibility from Canada.
• Competitive compensation package aligned with experience, skills, and market standards.
• Stock options and opportunity to participate in the growth of an innovative technology company.
• Company-provided laptop, remote setup support, and coworking budget.
• Flexible working hours and location independence.
• Generous paid time off in addition to local public holidays.
• Enhanced parental leave benefits.
• Health and retirement benefits.
• Annual learning and development budget to support professional growth.
• Opportunities to participate in impactful AI projects shaping the future of work.
• Virtual and in-person team events, including annual work opportunities and company gatherings.