Engenheiro de Dados GCP - Sênior

Team: IT

Location: Brazil

Commitment: Full-time

Workplace Type: remote

Accountabilities:

As a Senior Data Engineer, you will be responsible for designing, modernizing, and optimizing data pipelines while ensuring data quality, scalability, and operational efficiency.

• Reverse engineer complex PySpark-based data logic and translate it into optimized native SQL solutions within BigQuery.
• Use AI-powered tools to accelerate data pipeline conversion, improve productivity, and automate validation processes.
• Modernize and develop data ingestion workflows by replacing legacy tools with native Google Cloud services such as Datastream, Dataflow, and Cloud Run.
• Build and manage data orchestration workflows using Cloud Composer (Apache Airflow).
• Manage and optimize data stored in Apache Iceberg format through BigLake, ensuring performance, scalability, and cost efficiency.
• Design and execute automated testing frameworks to validate data migration accuracy and business logic consistency.
• Perform data reconciliation activities, including row count validation, partition verification, business key comparisons, checksums, and hash validations.
• Validate key business aggregations through accuracy checks involving totals, distinct counts, minimums, and maximums.
• Troubleshoot complex data performance issues, schema evolution challenges, and large-scale data processing scenarios.
• Collaborate with technical stakeholders to ensure successful delivery of cloud data transformation initiatives.

Requirements:

The ideal candidate should have strong experience in cloud data engineering, advanced SQL development, and modern data architecture, with the ability to handle complex, high-volume data environments.

• Minimum 6 years of professional experience working with Google Cloud Platform, especially BigQuery, Google Cloud Storage (GCS), and Cloud Composer (Airflow).
• Strong expertise in advanced SQL and Python/PySpark development.
• Proven experience designing, developing, and optimizing large-scale data pipelines.
• Hands-on experience with open table formats, especially Apache Iceberg and Delta, including conversion and data management.
• Experience with data transformation orchestration tools such as Dataform or dbt.
• Strong understanding of cloud-native data services and migration strategies.
• Ability to solve complex performance, scalability, and schema evolution challenges in Terabyte-scale environments.
• Analytical mindset with strong problem-solving skills and attention to data quality.
• Ability to work independently in remote environments while collaborating effectively with technical teams.

Benefits:

• Fully remote work model.
• Opportunity to work on challenging cloud data modernization projects.
• Temporary project contract with an expected duration of up to 4 months, with potential continuation.
• Exposure to modern Google Cloud technologies, AI-assisted engineering practices, and large-scale data platforms.
• Opportunity to collaborate with experienced professionals in data, cloud, and artificial intelligence environments.
• Continuous learning and professional development opportunities.
• Inclusive environment open to professionals with diverse backgrounds.