Principal System Engineering - DevOps - Spark and Azure

Job Description:

This position requires office presence a minimum of 5 days per week and is only located in the location(s) posted. No relocation is offered.

Join AT&T and reimagine the communications and technologies that connect the world. The Chief Information Office is responsible for advancing information technology performance and delivering solutions with a focus on maximizing ROI, increasing efficiency and enhancing the experience of end users. Guided by experienced leaders, Corporate Systems seamlessly integrate with advanced Technology and Operations to drive our enterprise forward. Our Systems Reliability and Software Delivery teams are unwavering in their commitment to excellence, ensuring every solution is robust and efficient. When you step into a career with AT&T, you won’t just imagine the future-you’ll create it.

We are seeking a highly skilled and experienced professional to join our team as an Apache Spark Expert within an Azure Synapse Environment. The ideal candidate will possess deep expertise in Apache Spark and be proficient in working with Databricks. The candidate must have a strong command of Spark job definitions, PySpark, and structured streaming. In addition, the role requires comprehensive knowledge of partitioning, SQL, and Delta Lake capabilities.

The successful candidate will also have hands-on experience with EventHubs, Kafka, and ADLSv2 storage, as well as MongoDB. Familiarity with graph databases, preferably Gremlin, is essential. The candidate will be responsible for leveraging these technologies to drive data processing and analytical solutions within our organization.

Additionally, the ideal candidate should possess strong data science skills, including expertise in time series analysis, graph algorithms, classification, and clustering/NLP. They should also have a deep knowledge of distributed systems, particularly related to Spark, including partitioning for performance and reliability, checkpointing, architecture and provisioning of drivers and executors, sizing of Spark pools, and tuning Spark configurations.

Experience in Azure architecture and developing DevOps pipelines is also required. The candidate should be proficient with Git for version control and collaboration. Moreover, they should have a solid foundation in general computer science skills, including reliability engineering, and be proficient with Kubernetes, Docker, and Helm, including developing and deploying applications on Kubernetes. A strong knowledge of Linux and Linux CLI, as well as programming skills in Java and/or Go, are also necessary.

Furthermore, the candidate should have Open Telemetry developer knowledge, including experience with protobufs, receivers, collectors, and sampling. Familiarity with MLflow internals is also highly desirable.

The successful candidate will bring a combination of advanced technical skills, a strong background in data science and distributed systems, and a solid understanding of DevOps and general computer science principles. This role offers an opportunity to work on cutting-edge technologies and contribute to the development of scalable data solutions within our organization.

Required Skills:

• 8+ years experience
• Expert level experience in Apache Spark, PySpark, SQL, and Delta Lake capabilities, EventHubs, Kafka, and ADLSv2 storage, as well as MongoDB,
• Familiarity with graph databases, preferably Gremlin
• Azure Synapse Environment, Azure architecture and developing DevOps pipelines
• strong data science skills, including expertise in time series analysis, graph algorithms, classification, and clustering/NLP
• Kubernetes, Docker, and Helm, including developing and deploying applications on Kubernetes

Desired Skills:

• Bachelors Degree in Computer Science, Engineering or Mathematics

Our Principal System Engineering, earns between $158,200.00-$237,400.00 USD Annual, not to mention all the other amazing rewards that working at AT&T offers. Individual starting salary within this range may depend on geography, experience, expertise, and education/training.

Joining our team comes with amazing perks and benefits:  

• Medical/Dental/Vision coverage
• 401(k) plan
• Tuition reimbursement program
• Paid Time Off and Holidays (based on date of hire, at least 23 days of vacation each year and 9 company-designated holidays)
• Paid Parental Leave
• Paid Caregiver Leave
• Additional sick leave beyond what state and local law require may be available but is unprotected
• Adoption Reimbursement
• Disability Benefits (short term and long term)
• Life and Accidental Death Insurance
• Supplemental benefit programs: critical illness/accident hospital indemnity/group legal
• Employee Assistance Programs (EAP)
• Extensive employee wellness programs
• Employee discounts up to 50% off on eligible AT&T mobility plans and accessories, AT&T internet (and fiber where available) and AT&T phone.

#LI-Onsite – Full-time office role-

 AT&T is leading the way to the future – for customers, businesses, and the industry. We're developing new technologies to make it easier for our customers to stay connected to their world. Together, we’ve built a premier integrated communications and entertainment company and an amazing place to work and grow. Team up with industry innovators every time you walk into work, creating the world you always imagined. Ready to #transformdigital with us?

Apply now!

Weekly Hours:

Time Type:

Location:

Salary Range:

It is the policy of AT&T to provide equal employment opportunity (EEO) to all persons regardless of age, color, national origin, citizenship status, physical or mental disability, race, religion, creed, gender, sex, sexual orientation, gender identity and/or expression, genetic information, marital status, status with regard to public assistance, veteran status, or any other characteristic protected by federal, state or local law. In addition, AT&T will provide reasonable accommodations for qualified individuals with disabilities.