Sr. Data Platform Engineer
Team: Engineering
Location: Hyderabad
Commitment: Hybrid
Workplace Type: onsite
The Data Platform Engineer is responsible for ensuring the reliability, availability, and cost efficiency of the Airbyte-based data ingestion platform running on Kubernetes. This role focuses on operating and monitoring production data sync infrastructure, ensuring data is reliably synced from source databases into downstream data pipelines, and maintaining strong observability and cost controls across AWS resources.
You will be responsible for:
-
Key Responsibilities :
- End-to-End Data Migration: Architect and execute large-scale data migrations from legacy source databases to cloud targets using AWS DMS, Azure migration tools, or Airbyte.
- Kubernetes Orchestration: Own and operate the Airbyte infrastructure and data pipelines running on Kubernetes, ensuring seamless production handling and rapid incident resolution.
- Advanced Python Engineering: Design and build robust Python-based frameworks for automated monitoring, alerting, and high-performance ETL/ELT processing.
- Platform Reliability (SRE): Ensure high availability (99.9%+) of data ingestion workflows by performing root cause analysis (RCA) and implementing self-healing infrastructure.
- Cost & Performance Optimization: Manage and optimize cloud infrastructure costs (AWS/Azure) related to data sync workloads through precise resource tagging and performance tuning.
- Cloud Infrastructure & Security: Configure and maintain AWS resources (S3, IAM, EKS), ensuring strict adherence to the principle of least privilege and data security standards.
Let’s talk about your skills/expertise:
-
Required Skills & Expertise
- Experience: 5–8 years of professional experience in Data or Platform Engineering roles.
- Migration Tools: Expert-level knowledge of AWS DMS, Azure Data Migration, or similar enterprise-grade replication tools.
- Coding Mastery: Strong Python skills for building custom ingestion utilities, monitoring agents, and automation scripts.
- Data Engineering Core: Deep understanding of Data Lakehouse architectures, dimensional modelling, and SCD Type 2 implementation.
- Kubernetes Proficiency: Hands-on experience with Docker/K8s (EKS/AKS), including pod health, rollouts, and service connectivity.
- Cloud Services: Solid experience with AWS (S3, IAM, CloudWatch) or Azure equivalent services.
- Observability: Familiarity with logging and metrics systems (Prometheus, Grafana, or ELK) to maintain platform health.
- Rapid Adaptability: Ability to quickly learn and adapt to new production environments, transitioning from design to deployment at a "production-ready" pace.
- Collaborative Mindset: Act as a bridge between infrastructure and downstream data consumers to resolve complex ingestion bottlenecks.
- Clean Code Advocate: Committed to Git-based workflows, CI/CD best practices, and structured documentation.
Cultural & Operational Fit
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say
