Data Pipelines Engineering Lead

Company:

Qualcomm India Private Limited

Job Area:

Engineering Group, Engineering Group > Software Engineering

General Summary:

We are looking for an experienced hands-on data engineering lead with 10+ years of experience to lead the definition, design and deployment of big data pipelines systems. Responsibilities include high-level software architecture and design for the whole system, advancing and advocating for software quality as a means of maintaining long-term productivity, and coordinating development teams to ensure consistency and efficiency.

Relevant technologies and skills include AWS Cloud, Kubernetes, Argo Workflows, Observability/Monitoring tools, Bazel, Python and familiarity with modern C++. You will also be coordinating and collaborating with the cloud infrastructure and with product and research teams.

Our teams are primarily in India, Sweden, Romania, Germany, and the US.

Minimum Qualifications:

• Bachelor's degree in Engineering, Information Systems, Computer Science, or related field and 4+ years of Software Engineering or related work experience.
OR
Master's degree in Engineering, Information Systems, Computer Science, or related field and 3+ years of Software Engineering or related work experience.
OR
PhD in Engineering, Information Systems, Computer Science, or related field and 2+ years of Software Engineering or related work experience.

• 2+ years of work experience with Programming Language such as C, C++, Java, Python, etc.

Responsibilities:

•         Architect, develop, and maintain the Autonomous Driving Data platform, both on-premises and AWS Cloud

•         Lead a Distributed Data Systems team while remaining hands-on

•         Translate Product Management and Department requirements into work packages.

•         Collaborate with data scientists and software engineers to define requirements and ensure seamless integration of data jobs into the platform.

•         Optimize the platform’s performance, scalability and costs, considering factors such as resources utilization, throughput requirements and data movement.

•         Monitor and troubleshoot system performance, identifying and resolving issues to ensure the availability and reliability of the data platform.

•         Implement and maintain CI pipelines for application containers, and deployment using technologies like Argo Workflow.

•         Implement and maintain monitoring stack to ensure the health and performance of the data platform.

•         Provide technical guidance and mentorship to junior team members.

What are we looking for:

• Proven experience as Data Ops Engineering Lead or similar role, with a focus on large-scale Data infrastructure.
• Strong expertise in configuring and optimizing Kubernetes clusters for data workloads.
• Experience with the ELK stack, in particular ElasticSearch and Kibana
• Proficient in using the Kubernetes platform, including technologies like Helm, ArgoCD, Argo Workflow, Prometheus, and Grafana.
• Solid programming skills in languages like Python, C++ and experience Bazel build systems.
• Familiarity with containerization technologies such as Docker and orchestration tools.
• Experience with CI/CD pipelines and automation tools (e.g., Jenkins, GitHub, ArgoCD).
• Experience with AWS services such as EKS, EC2, VPC, IAM, S3, EFS, Batch
• Experience with AWS logging and monitoring tools.
• Strong problem-solving skills and the ability to troubleshoot complex technical issues.
• Excellent communication and collaboration skills to work effectively within a cross-functional team.

We would love to see:

• Experience with Data Platforms for Automated Driving.
• Familiarity with data storage and retrieval systems.
• Understanding of security and compliance requirements in Data infrastructure.
• Proven track record of creating software systems that emphasize software quality
• Experience with both system level architecture and class level design
• In-depth understanding of distributed computing and parallel computing.
• Extensive experience with different system architectural concepts like event driven architecture, workflows-based architecture, domain driven design

Applicants: Qualcomm is an equal opportunity employer. If you are an individual with a disability and need an accommodation during the application/hiring process, rest assured that Qualcomm is committed to providing an accessible process. You may e-mail disability-accomodations@qualcomm.com or call Qualcomm's toll-free number found here. Upon request, Qualcomm will provide reasonable accommodations to support individuals with disabilities to be able participate in the hiring process. Qualcomm is also committed to making our workplace accessible for individuals with disabilities. (Keep in mind that this email address is used to provide reasonable accommodations for individuals with disabilities. We will not respond here to requests for updates on applications or resume inquiries).

Qualcomm expects its employees to abide by all applicable policies and procedures, including but not limited to security and other requirements regarding protection of Company confidential information and other confidential and/or proprietary information, to the extent those requirements are permissible under applicable law.

To all Staffing and Recruiting Agencies: Our Careers Site is only for individuals seeking a job at Qualcomm. Staffing and recruiting agencies and individuals being represented by an agency are not authorized to use this site or to submit profiles, applications or resumes, and any such submissions will be considered unsolicited. Qualcomm does not accept unsolicited resumes or applications from agencies. Please do not forward resumes to our jobs alias, Qualcomm employees or any other company location. Qualcomm is not responsible for any fees related to unsolicited resumes/applications.

If you would like more information about this role, please contact Qualcomm Careers.