Data Engineer III

Team: Data Management and Governance

Location: Doral, FL

Commitment: Regular

Workplace Type: onsite

Job Description

Build Scalable Data & ML Infrastructure
 
• Design and implement medallion architecture (Bronze/Silver/Gold) using Databricks or reliable data processing and ML model training 
• Develop automated data pipelines that process structured and unstructured data from multiple sources into analytics-ready formats 
• Create robust ETL/ELT workflows using Apache Spark and modern data engineering practices for both batch and streaming data 
• Build and maintain data quality monitoring and validation systems across the entire data and ML lifecycle Drive ML Platform Excellence 
• Implement MLOps best practices including automated model training, validation, deployment, and monitoring using MLflow and Databricks workflows 
• Design scalable ML inference systems that handle high-volume, low-latency predictions in production environments 
• Create comprehensive monitoring and alerting systems for model performance, data drift, and system health 
• Build self-service ML capabilities that enable data scientists to deploy and monitor their own models efficiently Enable Advanced Analytics & Business Intelligence 
• Design and maintain data models that support both machine learning workloads and business intelligence requirements 
• Create integration points between ML systems and business intelligence platforms (Tableau, PowerBI, Qlik Sense) 
• Implement data governance standards and metadata management systems that ensure data quality and compliance 
• Collaborate with analysts and data scientists to optimize data architecture for both predictive modeling and reporting needs Ensure Data Quality & Governance 
• Implement comprehensive data governance frameworks including data lineage tracking, quality monitoring, and compliance controls 
• Design and maintain data catalogs and metadata management systems that enable efficient data discovery across the organization 
• Establish data quality standards and automated testing frameworks for both analytical and ML workloads 
• Work with stakeholders to define data definitions, business logic, and governance policies Integrate with Enterprise Systems 
• Build integrations with MAVEN Smart Systems (Palantir Foundry) environments to support operational and predictive analytics 
• Connect Databricks-based systems with enterprise data warehouses, streaming platforms, and business applications 
• Implement security and compliance controls that meet enterprise requirements while enabling self-service capabilities 
• Collaborate with platform engineers to integrate ML systems with broader application architecture and infrastructure 

Education and Years of Experience

• Strong data engineering skills in Python, PySpark, and Spark SQL with experience implementing medallion architecture and modern data platform patterns
• Production ML systems experience including model deployment, monitoring, and MLOps practices in cloud environments 
• Data architecture expertise with experience designing scalable data processing systems and implementing data governance frameworks 
• Experience integrating with platforms such as Qlik, Tableau, PowerBI, MAVEN Smart System (Palantir), or similar. 

Required Skills

• 5+ years of technical experience, including 3+ years building production data pipelines and ML infrastructure using distributed computing platforms like Databricks. 
• Strong data engineering skills in Python, PySpark, and Spark SQL with experience implementing medallion architecture and modern data platform patterns 
• Production ML systems experience including model deployment, monitoring, and MLOps practices in cloud environments 
• Data architecture expertise with experience designing scalable data processing systems and implementing data governance frameworks 
• Experience integrating with platforms such as Qlik, Tableau, PowerBI, MAVEN Smart System (Palantir), or simil 

• Happy - Be Infectious. Happiness multiplies and creates a positive and connected environment where motivation and satisfaction have an outsized effect on everything we do.
• Helpful - Be Supportive. Being helpful is the foundation of teamwork, resulting in a supportive atmosphere where collaboration flourishes, and collective success is celebrated.
• Honest - Be Trustworthy. Honesty serves as our compass, ensuring transparent communication and ethical conduct, essential to who we are and the complex domains we support.
• Humble - Be Grounded. Success is not achieved alone, humility ensures a culture of mutual respect, encouraging open communication, and a willingness to learn from one another and take on any task.
• Hungry - Be Eager. Our hunger for excellence drives an insatiable appetite for innovation and continuous improvement, propelling us forward in the face of new and unprecedented challenges.
• Hustle - Be Driven. Hustle is reflected in our relentless work ethic, where we are each committed to going above and beyond to advance the mission and achieve success.