Description
Data Engineer
Team: Enterprise Data
Location: Toronto
Commitment: Full Time
Workplace Type: hybrid
Purpose of Job:
The Data Engineer will have the responsibility for the design and maintenance of enterprise data warehouse for various projects, as well as providing on-going support for activities impacting all enterprise databases. The ideal candidate will have hands-on experience with Azure Data Factory, Azure Data Lake, Azure Synapse, Azure Function App, Logic App, REST APIs, mapping dataflow, and Event Hub. As a key member of our team, you will collaborate with various stakeholders to ensure efficient data flow, storage, and integration, while employing best practices in data engineering.
Main Activities:
- 1. Developing of Data Pipelines (50%)
- 2. Requirements gathering and Technical Documentation (10%)
- 3. Data Strategy and Planning (10%)
- 4. Support and Maintenance of Enterprise Data Warehouse and BAU Tasks (30%)
· Design, develop, and maintain scalable Synapse pipelines for data ingestion, processing, and transformation in Azure data platform.
· Develop and optimize T-SQL and Python-based data processing scripts and integrate them with Synapse pipelines for ETL tasks and data transformations.
· Design and develop mapping dataflow solutions for complex data transformations.
· Develop and maintain data storage solutions, ensuring efficient data retrieval and storage on Azure Data Lake and databases.
· Implement and maintain API integrations, including REST APIs, for seamless data flow between systems.
· Design and test the end-to-end process for data ingestion, data transformation, data cleansing, data delivery, data quality and data timeliness.
· Implement and ensure data security and privacy measures, complying with industry standards and regulations.
· Monitor and optimize the performance of Azure data services, Azure Synapse pipelines, Function Apps, and Logic Apps.
· Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
· Participate in Joint Application Development sessions with business units to gather and understand their data and reporting requirements.
· Create and maintain technical documentations for tools, development, and procedures.
· Provide inputs and assist in enablement of Cloud based data solutions.
· Develop and recommend innovative approaches to solve business and technical problems.
· Work and collaborate with the larger team to exchange knowledge, solutions, and practices to build a more consistent, robust approach to development.
· Continuously improve performance and proactively identify and resolve bottlenecks that will reduce time to build and deliver our products.
· Troubleshoot and resolve data-related issues in a timely manner for production and other environments (DEV,UAT,QA)
· Perform business as usual tasks (BAU) that assigned
Skills/Knowledge Requirements:
- Bachelor's degree or equivalent in the field of Computer Science or Engineering
- 3+ years of Proven experience as a Data Engineer, specifically working with Microsoft Azure technologies.
- Proficient in T-SQL, Pyspark, Mapping Dataflow and Python programming language
- Hands-on experience with Azure data services, including Azure Data Factory, Azure Data Lake, Azure Function App, Azure Logic App, Azure SQL Database, Azure Synapse Analytics, mapping dataflow, and Event Hub.
- Understanding of data warehousing concepts and best practices.
- Experience with Data modeling.
- Experience in working with structured, semi-structured and unstructured data.
- Excellent written and verbal communication skills
- Excellent multi-tasking and organizational skills
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say