Data Engineer
Department: Engineering
Location: Bangalore Office
Employment Type: FullTime
Key Responsibilities :
1. Design & Develop Scalable Data Pipelines: Architect and build efficient data ingestion pipelines that empower the business with timely and actionable insights to drive strategy and operations.
2. Collaborate with Cross-Functional Teams: Work closely with technical and business teams to gather requirements and translate them into technical specifications and solutions.
3. Optimize Data Architecture: Continuously refine and improve the performance, reliability, and scalability of data pipelines to ensure seamless integration of data across systems.
4. Ensure Data Integrity: Perform rigorous data quality checks and uphold best practices to maintain the accuracy, consistency, and integrity of data.
5. Build ETL Solutions: Develop ETL frameworks to collect, transform, and integrate data from diverse sources such as Kafka, Scylla, PostgreSQL, MongoDB, APIs, and various file formats.
6. Adopt Best Practices: Implement best-in-class engineering practices around reporting and analysis, ensuring data integrity, testing, validation, and comprehensive documentation.
Basic Qualifications :
1. Bachelor’s degree in Computer Science, Information Systems, or a related technical field, or equivalent work experience.
2. 3+ years of hands-on experience with cloud-based data technologies, including message queues, event grids, relational databases, NoSQL databases, data warehouses, and big data technologies (e.g., Spark).
3. Proficiency in Spark (Java, Python, SQL): Expertise in developing and optimizing Spark-based applications for large-scale data processing.
4. Advanced SQL Skills: Ability to create complex queries, manage database structures, and ensure optimal performance.
5. Experience with Docker and Kubernetes: Familiarity with deploying applications using modern containerisation and orchestration tools.
6. DevOps and CI/CD: Solid understanding of modern DevOps practices, including Git, continuous integration, and continuous deployment pipelines.
7. Agile Development Experience: Comfortable working in an Agile environment, particularly using Scrum methodology.
Preferred Qualifications
1. API Frameworks & OOP: Experience with API frameworks and object-oriented programming to integrate services and improve data workflows.
2. Business Acumen: Ability to collaborate closely with leadership to deliver innovative solutions tailored to evolving business needs.
3. Strong Communication Skills: Excellent verbal and written communication abilities, with a knack for explaining complex technical concepts to both technical and non-technical stakeholders.
4. Ability to leverage tools like langchain, llama index ..etc to build llm based applications.
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say