Engenheiro(a) de Dados Sênior – Azure Databricks

Team: IT

Location: Brazil

Commitment: Full-time

Workplace Type: remote

Accountabilities

• Design, develop, maintain, and optimize data pipelines using Azure Databricks, PySpark, Azure Data Factory (ADF), Python, and Azure Data Lake.

• Build and support ETL/ELT processes to integrate enterprise data and meet analytical and business requirements.

• Perform data ingestion, transformation, validation, and reconciliation across multiple source systems.

• Conduct data profiling and quality assessments to identify inconsistencies, resolve issues, and ensure data accuracy and reliability.

• Develop, write, and optimize complex SQL queries to improve data processing performance.

• Perform performance tuning and troubleshoot large-scale data workloads to ensure efficiency, scalability, and reliability.

• Provide Level 2 production support for incidents involving Azure Data Factory pipelines, Databricks jobs, and data processing workflows.

• Monitor and maintain production pipelines, ensuring availability, operational stability, and efficient resource utilization.

• Support deployment and release processes across development, testing, and production environments.

• Manage competing priorities across multiple projects, working independently while meeting deadlines and maintaining high delivery standards.

Requirements

• More than eight years of professional experience in data engineering, including the development and maintenance of ETL processes.

• Strong hands-on experience with Azure Databricks and Azure Data Factory (ADF).

• Extensive knowledge of ETL/ELT development, data transformation, and enterprise data integration.

• Advanced SQL skills, including complex query development, optimization, and performance tuning.

• Practical experience with Python and PySpark for data processing and engineering.

• Experience working with Azure Data Lake and integrating data from multiple enterprise systems.

• Proven ability to perform data profiling, validation, reconciliation, and quality assurance.

• Experience monitoring, maintaining, and troubleshooting data pipelines in production environments.

• Advanced English proficiency, both written and spoken, to collaborate effectively in a global working environment.

• Strong analytical and investigative skills, with a structured, solution-oriented approach to complex technical challenges.

• Ability to work autonomously, organize activities, manage competing demands, and prioritize effectively.

• Excellent verbal and written communication skills, with the ability to collaborate across multidisciplinary teams.

• Commitment to code quality, performance, reliability, and timely delivery.

Nice to have:

• Experience implementing CI/CD pipelines and DevOps practices.

• Familiarity with Git, Azure DevOps, or similar version control and deployment tools.

• Knowledge of cloud monitoring, logging, and observability solutions.

Benefits

• Fully remote work arrangement.

• CLT employment contract in Brazil.

• Opportunity to contribute to large-scale enterprise data engineering initiatives in a global environment.

• Salary, healthcare coverage, paid time off, and additional benefits are not specified in the provided job description.