Key Responsibilities
- Design, build, and maintain large-scale data pipelines to ingest, process, and transform complex financial data
- Develop data ingestion frameworks to handle high-volume and high-variety data feeds
- Implement data quality control measures to detect and handle data anomalies, duplicates, and inconsistencies
- Optimize data pipeline performance for scalability, reliability, and latency
- Collaborate with data scientists to integrate machine learning models into the data pipeline and ensure seamless data flow
- Develop and maintain model serving infrastructure to support real-time predictions and batch scoring
- Implement model monitoring and logging to track performance, data quality, and model drift
- Manage cloud-based infrastructure to support data pipeline and model serving workloads
- Troubleshoot data pipeline issues, data quality problems, and model performance degradation
- Collaborate with cross-functional teams, including data science, product, and engineering teams to ensure alignment and effective communication
Required Skills, Experience, and Qualifications
- 3+ years of experience in a data engineering or similar role, with a focus on building and deploying data pipelines and machine learning models
- 2+ years of experience with Databricks, PySpark, and Apache Spark
- 1+ year of experience with Apache Airflow, Apache Kafka, and Kubernetes
- Strong Python skills, with 3+ years of experience
- Experience with production machine learning model deployment and management
- Knowledge of cloud-based infrastructure and services AWS and Azure
- Experience with data warehousing, data lakes, and data marts
- Familiarity with containerization using Docker and orchestration using Kubernetes
- Understanding of messaging and streaming using Apache Kafka
- Experience with workflow management using Apache Airflow
- Strong problem-solving skills, with the ability to troubleshoot complex data pipeline and machine learning model issues
- Excellent communication and collaboration skills, with the ability to work with data scientists, engineers, and senior stakeholders
- Degree educated in Computer Science or relevant subject
- Excellent spoken and written English
- Experience of driving innovation within teams
- Positive, can-do approach to work, delivering on commitments
- Able to earn respect and build trust through high-quality delivery and by demonstrating value added
- Act with integrity, with the ability to make decisions, taking accountability for the consequences of these decisions
- Creative approach to problem solving, can look at existing situations and problems in novel ways and propose innovative solutions
Other Jobs from ION
Web UI Developer (React), New York
Senior Quality Engineering Manager
Senior Fullstack Developer, Budapest
Senior Full-Stack Developer, New York
Senior Software Engineer - Full-Stack
Similar Jobs
SENIOR DATA ENGINEER
Java Developer (Data Platform)
Engineer II - Java (Remote)
Senior Data Engineer
Staff Software Engineer, Aggregation & Search
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 401 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got about 70,000 jobs from 5,000 vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 5,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say