Senior Manager, Data Engineering

Senior Manager, Data Engineering

Key Responsibilities:

1. Data Infrastructure and Architecture:
• Lead the design, development, and maintenance of scalable data infrastructure and pipelines.
• Ensure the data architecture aligns with the company’s goals for analytics, machine learning, and other data-driven initiatives.
• Manage cloud-based platforms such as AWS, Google Cloud, or Azure for data storage, ETL processes, and integration with internal and external data sources.
2. Team Leadership:
• Lead a team of data engineers, providing mentorship, setting objectives, and driving professional development.
• Foster a collaborative environment, ensuring the team works cross-functionally with data scientists, analysts, and business stakeholders to deliver projects effectively.
3. Data Quality and Governance:
• Implement best practices for data governance, ensuring high data quality, consistency, and security across all pipelines.
• Oversee data validation processes, data cleansing efforts, and ensure compliance with relevant data privacy regulations (e.g., GDPR).
4. Collaboration with Business and Technical Teams:
• Partner with product managers, business leaders, and other departments to understand data needs and deliver solutions that align with business goals.
• Work closely with IT and software engineering teams to ensure seamless integration of data pipelines with other business systems.
5. Performance Optimization:
• Continuously monitor and improve the performance of data systems, focusing on latency, scalability, and cost-efficiency.
• Develop and enforce standards for system performance monitoring and alerting to prevent data pipeline failures.
6. Innovation and Data Strategy:
• Lead efforts to explore and integrate new data technologies, tools, and platforms that could enhance the company’s data capabilities.
• Provide strategic guidance on the adoption of big data, AI, machine learning, and automation to improve data handling and analysis.
7. Project Management:
• Oversee the end-to-end delivery of data engineering projects, ensuring timely and successful completion.
• Manage resources, timelines, and budgets for large-scale data initiatives.

Key Skills:

• Technical Expertise:
• Strong knowledge of data engineering technologies such as Python, SQL, Spark, Kafka, Hadoop, and data warehouse solutions (e.g., Snowflake, Redshift).
• Proficiency in ETL (Extract, Transform, Load) tools and cloud-based data solutions (AWS Glue, Google BigQuery, Azure Data Factory, etc.).
• Experience with DevOps practices and infrastructure automation tools (e.g., Terraform, Kubernetes) is often required.
• Leadership and Team Management:
• Proven experience leading teams of engineers, fostering a high-performance culture, and managing distributed or global teams is often preferred.
• Strong people management skills, including recruitment, performance management, and fostering growth and innovation within the team.
• Data Governance and Compliance:
• Deep understanding of data security best practices and regulatory frameworks like GDPR and CCPA.
• Experience implementing data governance frameworks to ensure data integrity and compliance with legal requirements.
• Collaboration and Communication:
• Ability to communicate complex technical concepts to non-technical stakeholders.
• Experience working with cross-functional teams to align data engineering efforts with business needs.
• Strategic Mindset:
• Strong business acumen and understanding of how data engineering contributes to broader business goals.
• Ability to align data engineering efforts with long-term business strategy, helping the company leverage data for competitive advantage.
• Project Management:
• Proven experience managing large-scale data engineering projects, including resource planning, risk management, and performance tracking.