[GFA] Azure Senior Data Engineer
Location: Kraków, Lesser Poland Voivodeship, pl, Remote
Company Description
Software Mind develops solutions that make an impact for companies around the globe. Tech giants & unicorns, transformative projects, emerging technologies and limitless opportunities – these are a few words that describe an average day for us. Building cross-functional engineering teams that take ownership and crave more means we’re always on the lookout for talented people who bring passion and creativity to every project. Our culture embraces openness, acts with respect, shows grit & guts and combines employment with enjoyment.
Job Description
Project – the aim you’ll have
Our customer provides innovative solutions and insights that enable our clients to manage risk and hire the best talent. Their advanced global technology platform supports fully scalable, configurable screening programs that meet the unique needs of over 33,000 clients worldwide. Headquartered in Atlanta, GA, they have an internationally distributed workforce spanning 19 countries with about 5,500 employees. Our partner perform over 93 million screens annually in over 200 countries and territories.
We are seeking a Senior Data Engineer with proven expertise in Databricks PySpark development, comprehensive data modeling experience (including fact/dimension tables, SCD Type 2 and incremental loads) and event-based architecture skills to join our Data Engineering Team and drive the evolution of our Azure-based Data Analytics Platform.
Position – how you’ll contribute
- Develop reusable, metadata-driven data pipelines using Databricks Lakehouse architecture and PySpark
- Design and implement comprehensive data models
- Build robust ETL/ELT solutions with advanced features: Merge operations, SCD Type 2 implementations, etc.
- Implement incremental data loads with idempotency patterns and overlap joins optimization
- Automate and optimize data platform processes with focus on performance and reliability
- Build integrations with data sources and consumers using event-driven patterns
- Cooperate with infrastructure engineering team to set up cloud resources
- Initiate and implement improvements to data platform architecture
Qualifications
Expectations – the experience you need
- Databricks expertise: proficient in Databricks Lakehouse architecture and PySpark development
- Data modeling mastery: extensive experience in data model definition including fact tables, dimension tables, measures, grain analysis, SCD Type 2, surrogate keys, late-arriving records handling, overlap joins, and incremental loads with idempotency
- Programming: advanced Python and PySpark skills for ETL/ELT development
- Databricks optimization: deep knowledge of optimize, zOrder, Liquid clustering, ACID transactions and performance tuning
- Event-based architecture: proven experience in designing and implementing event-driven data solutions
- Azure data platform: experience working with Azure-based datasets and data pipelines
- SQL proficiency: strong SQL skills for complex data transformations
- Large-scale data processing: experienced in handling large and complex datasets efficiently
- CI/CD: experience in developing automated deployment pipelines
- Networking fundamentals: understanding of basic networking concepts
- Agile methodology: familiar with Scrum and agile development practices
Additional skills – the edge you have
- Understanding of stream processing challenges and Spark Structured Streaming
- Experience with Infrastructure as Code (Terraform, Bicep)
- Experience with containerized applications (Azure Container Apps, Kubernetes)
- Knowledge of Azure cloud native solutions (Azure Data Factory, Azure Function App, Azure Container Instances)
Additional Information
Our offer – professional development, personal growth:
- Flexible employment and remote work
- International projects with leading global clients
- International business trips
- Non-corporate atmosphere
- Language classes
- Internal & external training
- Private healthcare and insurance
- Multisport card
- Well-being initiatives
Position at: Software Mind Poland
This role requires candidates to be based in Poland.
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say