Data Engineer

About the Role

Ethos is an extremely data driven company. We take every decision backed by data. Making data and insights available to everyone in the company is the mission of data engineering team. You will work with some of the brightest people helping drive infrastructure and insights to disrupt one of the oldest and largest industries in the country.

Our stack: Our backend is built in Node.js and Postgres and is totally hosted on AWS. Our front end is built in React/Redux.

Duties and Responsibilities:

• Develop and maintain robust ETL pipelines to support data needs across the organization.
• Contribute to the development and governance of the data warehouse.
• Build and maintain data marts to support various business functions.
• Optimize queries, and refine data structures to ensure efficient data retrieval and reporting.
• Build and maintain systems to ensure data accuracy, consistency, and availability across systems.
• Set up and maintain automated CI/CD pipelines for model training and deployment.
• Develop and maintain a centralized feature store to ensure consistency between training and serving features.
• Develop end-to-end automation of ML model deployment, ensuring smooth transition from development to production.
• Implement tools and processes to monitor the performance of ML models in production, ensuring that they are performing as expected and maintaining their accuracy.
• Ensure that all data warehouse activities adhere to regulatory standards, data privacy rules, and company policies.
• Conduct regular reviews of data infrastructure to identify areas of inefficiency, underutilization, or redundancy.
• Research and implement best practices for cost optimization across the data stack.
• Work closely with other teams, including but not limited to data scientists, business analysts, and product managers, to understand and meet their data requirements.

Qualifications and Skills:

• 4+ years of experience in Software engineering.
• Proficiency in a programming language, preferably Python.
• Proficiency in SQL & data modeling.
• Strong understanding of SWE methodologies, CI/CD and testing.
• Strong understanding of ETL processes, data warehousing, and data governance principles.
• Experience building and managing data intensive applications.
• Experience with large scale MPP databases.
• Preferred experience with MLOps, including ML model deployment, monitoring, and lifecycle management. Familiarity with tools and platforms like MLflow, Kubeflow, or sagemaker.
• Preferred experience with Snowflake, Airflow & DBT.
• Preferred experience with stream processing frameworks such as Flink.

#LI-Hybrid

#LI-JS4