Mid-Senior Data Engineer

Main Responsibilities

• Acquire data from various data sources (APIs, relational and non-relational databases, queues …) by developing scripts, workflows, and ETL pipelines using our stack of both “small” and big data;
• Participate in modeling business processes with data models;
• Maintain existing data models’ integrity and structure in the data warehouse;
• Identify, design, and implement internal process improvements such as automating manual processes, and optimizing data delivery;
• Assess the effectiveness and accuracy of data-gathering techniques;
• Develop and deploy processes and tools to monitor and analyze pipeline performance and data accuracy;
• Discover opportunities for data acquisition, diagnostics, mapping, and correction;
• Employ a variety of development languages and tools to blend data systems together;
• Recommend and validate different ways to improve data reliability, efficiency, and quality;
• Troubleshooting the data pipeline;
• Ad-hoc dataset creation;
• Work with other teams to understand their individual needs and objectives to enable them through data availability.

Core Requirements

• 3+ years of experience performing data acquisition tasks; 
• Deep, hands-on experience in Python;
• Strong knowledge of Apache Spark;
• Knowledge of Git;
• Knowledge of Bash;
• Experience with Airflow is a plus.

Salary Range

• Gross Salary 22000 - 29800 PLN/month.