Description
Duration: 12 months Contract
Evaluation: Performance will be reviewed after 8 months to assess the potential for a permanent role.
CULTURE OF DATA:
As BPM continues to build a data-driven culture, the Data Engineer will be supporting the continued success of that effort. For BPM, the culture of data represents the firm’s approach to data management, stewardship, lineage, architecture, collection, storage, and utilization for delivering analytic results. It also includes tools, applications and approaches which lead the firm to have access to timely and accurate data. This role will play an integral part in the delivery, maintenance, building of business relationships and success to the culture of data.
JOB OVERVIEW:
As member of the Business Insights and Analytics team in the Business Transformation Office, this Data Engineer role is continuing the build, maintenance, and governance of data pipelines with best of breed technology in an Azure environment. This position is responsible for developing the pipelines to the data Lakehouse while ensuring confidence with the business and executive stakeholders by developing a reliable, scalable data solution that is high quality, secure, current, and usable to generate insights and support decision making. The Data Engineer stays up to date with the latest technology and methodologies with a pragmatic mindset. Participates in technology road maps and maintains the data pipeline and tool documentation.
DATA ENGINEER RESPONSIBILITIES/REQUIREMENTS:
· Undergraduate degree in the field of data or computer science, information technology, statistics, or mathematics preferred.
· Minimum of 2 years work experience as a Data Engineer working in a Databricks environment with specific expertise in Databricks Delta Lake, notebooks, and clusters.
· Data Vault Modeling required.
Data Pipeline
Experienced in developing, deploying, and supporting high-quality, fault-tolerant data pipelines.
Builds the infrastructure required for optimal extraction, loading, and transformation of data from a wide variety of data sources using best practices.
Supports the architecture for observing, cataloging, and governing data.
Knowledge of big data technologies and tools, such as Hadoop, Spark, and Kafka.
Strong understanding of relational data structures, theories, principles, and practices.
ELT Development
Hands-on experience in ETL functionality by way of coding (Python, DbT, SQL).
Build and optimize ETL.
Monitor and troubleshoot ELT processes to ensure data accuracy and reliability.
Monitors and analyzes performance metrics and recommends improvements.
Proficiency in programming languages such as Python and SQL.
Strong understanding of data modeling, algorithms, and data transformation strategies and techniques for data science consumption.
Data Governance and Security
Ensures security and integrity of data.
Implement data governance and access controls to ensure data security and compliance.
Collaborate with the security team to implement encryption, authentication, and authorization mechanisms.
Monitor and audit data access to maintain data privacy and integrity.
Proven experience with business and technical requirements analysis, elicitation, modeling, verification, and methodology development.
Ability to create systematic and consistent requirements specifications in both technical and user-friendly language.
Collaboration
Collaborates with the team as well as key stakeholders across the organization.
Understands the need to profile the data source to effectively send the data to the Lakehouse.
Utilizes development best practices including technical design reviews, implementing test plans, monitoring/alerting, peer code reviews, and documentation.
Ability to communicate with both technical and business end-user colleagues to deliver meaningful outcomes.
Excellent critical thinking skills and the ability to understand the relationships between data and business intelligence.
Ability to work with the Information Technology Department in partnership.
Physical Demands and work environment
Fully remote position
Sitting, and typing (repetitive motion) are strong components of the role.
Ability to work flexible schedule (am, pm & weekends as needed)
Minimal travel might be required.
Other Jobs from BPM
.NET Developer
Bengaluru, India
Business Development Research Manager
Bengaluru, India
Similar Jobs
Senior Data Scientist
Dublin, Ireland
Staff Data Engineer (8+ years, Python, Spark, Hadoop, BigData)
Remote
Bengaluru, India
Data Engineer (1-2 years, Python, Spark, Hadoop, BigData)
Remote
Bengaluru, India
Sr. Data Scientist - ML Ops
Remote
Bengaluru, India
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got about 70,000 jobs from 5,000 vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 5,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say