GenAI Quality Engineer

Location: Sofia, bg

Job Description

Tieto Tech Consulting (MentorMate from 2001 to 2026) provides design-led, data-centric, and AI-powered digital engineering & consulting services to enterprises worldwide. With a strong heritage of Nordic quality, local presence, and global-scale operations, we are backed by 8,500+ IT professionals. Our clients trust us for advanced data and analytics skills, top industry-specific expertise, specialized software R&D, and the capabilities of a multinational team. We are part of Tieto, a leading technology company with an annual revenue of approximately EUR 2bn.

Tieto Tech Consulting’s team in Bulgaria consists of 500+ professionals specializing in the development of tailored digital solutions in diverse industries. 

We are looking for a Quality Engineer with experience testing AI-powered products and building automated testing solutions. In this role, you will help ensure the quality, reliability, and performance of modern AI-first applications, including products powered by Large Language Models (LLMs). 

You will work closely with engineering, product, and AI teams to define testing strategies, validate complex user scenarios, and build automation frameworks for rapidly evolving products.

This role is based on a hybrid work model, with 3 days per week in our Sofia office.

Key Responsibilities 

• Validate LLM-based applications and AI-generated outputs
• Evaluate quality, reliability, and behavior of probabilistic AI systems
• Create test cases from evolving requirements
• Identify gaps, inconsistencies, and risks in product requirements
• Collaborate with product managers, engineers, and AI specialists throughout the development lifecycle
• Analyze defects, perform root cause analysis, and drive issue resolution

Required Qualifications

• Experience testing AI-powered applications, GenAI solutions, chatbots, copilots, or similar products
• Understanding of AI/ML concepts and Large Language Models (LLMs)
• Strong analytical and problem-solving skills
• Experience creating test plans, test cases, and quality metrics
• Excellent spoken and written English

A significant advantage would be

• Experience with AI testing tools and frameworks
• Experience evaluating LLM responses for accuracy, relevancy, safety, and consistency
• Familiarity with prompt engineering and AI quality assessment techniques
• Knowledge of Python or another programming language used in automation
• Experience working in Agile development environments
• Exposure to cloud platforms such as Azure, AWS, or GCP

Additional Information

At Tieto, we believe in the power of diversity, equity, and inclusion. We encourage applicants of all backgrounds, genders (m/f/d), and walks of life to join our team, as we believe that this fosters an inspiring workplace and fuels innovation. Our commitment to openness, trust, and diversity is at the heart of our mission to create digital futures that benefit businesses, societies, and humanity.
Diversity, equity and inclusion | Sustainability | Tieto

Stay alert to recruitment scams:
• Tieto only communicates via @tieto.com email addresses.
• We never ask candidates to pay fees or share financial details during recruitment.
• If you receive a suspicious message claiming to be from Tieto, don't respond — verify it via our official careers site.
• For more information visit Important Advisory Regarding Recruitment Fraud | Tieto Blog