Lead Big Data Engineering

Job Description:

The Lead Big Data Engineer is responsible for designing, implementing, and optimizing data pipelines, ensuring the availability, performance, and security of large-scale data processing systems. This role requires a deep understanding of big data technologies, data architecture, and data engineering best practices. The Lead Big Data Engineer collaborates closely with data scientists, analysts, and other stakeholders to support data-driven decision-making processes.

Key Responsibilities:

• Design, develop, and maintain scalable and efficient data pipelines using technologies such as Python, PySpark, and Databricks.

• Integrate data from various sources, ensuring data quality, consistency, and reliability.

• Develop and implement data architecture strategies, including data modeling, schema design, and data storage solutions.

• Optimize data processing workflows for performance, scalability, and cost-efficiency.

• Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver data solutions.

• Identify and resolve data-related issues and provide support for data infrastructure.

• Maintain comprehensive documentation of data pipelines, architecture, and processes.

• Drive the design and implementation of specific data models to enhance business decision-making through insights from a combination of external and internal data assets.

• Define data requirements, gather and mine large-scale structured and unstructured data, and validate data using various tools in a cloud environment.

• Support the standardization, customization, and ad-hoc data analysis.

• Develop mechanisms to ingest, analyze, validate, normalize, and clean data.

• Develop and maintain data engineering best practices.

• Contribute to insights on data analytics and visualization concepts, methods, and techniques.

Required Qualifications:

• 5+ years of experience in data engineering, with a focus on big data and cloud technologies.

Technical Skills:

• Strong programming skills in languages such as Python, PySpark, and Scala.

• Working experience with data warehousing solutions like Snowflake.

• Proficiency in SQL.

• Experience with Databricks computing environment.

• Development experience on Azure cloud environments.

Soft Skills:

• Excellent problem-solving abilities.

• Strong communication and collaboration skills.

• Ability to work independently and in a team environment.

• Attention to detail and a commitment to data quality.

Education:

•    MS degree in Computer Science, Information Technology, or a related field preferred.

As a Lead Big Data Engineer, you can earn between $143,800 - $215,800 per year. Not to mention all the other amazing rewards that working at AT&T offers. Individual starting salary within this range may depend on geography, experience, expertise, and education/training.

Joining our team comes with amazing perks and benefits:

• Medical/Dental/Vision coverage

• 401(k) plan 

• Tuition reimbursement program

• Paid Time Off and Holidays (based on date of hire, at least 23 days of vacation each year and 9 company-designated holidays) 

• Paid Parental Leave

• Paid Caregiver Leave

• Additional sick leave beyond what state and local law require may be available but is unprotected

• Adoption Reimbursement 

• Disability Benefits (short term and long term)

• Life and Accidental Death Insurance 

• Supplemental benefit programs: critical illness/accident hospital indemnity/group legal 

• Employee Assistance Programs (EAP)

• Extensive employee wellness programs 

• Employee discounts up to 50% off on eligible AT&T mobility plans and accessories, AT&T internet (and fiber where available) and AT&T phone

If you’re ready to make an impact on our business and your career, bring your bold ideas to a world of possibility. Apply today!

Weekly Hours:

Time Type:

Location:

Salary Range:

It is the policy of AT&T to provide equal employment opportunity (EEO) to all persons regardless of age, color, national origin, citizenship status, physical or mental disability, race, religion, creed, gender, sex, sexual orientation, gender identity and/or expression, genetic information, marital status, status with regard to public assistance, veteran status, or any other characteristic protected by federal, state or local law. In addition, AT&T will provide reasonable accommodations for qualified individuals with disabilities.