Data Engineer

Location: Hyderabad, India

Department: IT-Software

Experience: 5+ Years

• Pipeline Design & Operation: Design, build, and operate scalable ELT/ETL pipelines that ingest data from IoT/smart-cart telemetry, video events, operational systems, and external partners into our cloud data lake/warehouse.
• Infrastructure Management: Build and maintain robust data infrastructure, including databases (SQL and NoSQL), data warehouses, and data integration solutions.
• Data Modeling: Establish canonical data models and definitions (schemas, event taxonomy, metrics) so teams can trust and reuse the same data across products, BI, and analytics.
• Data Quality Assurance: Own data quality end-to-end by implementing validation rules, automated tests, anomaly detection, and monitoring/alerting to prevent and quickly detect regressions.
• Consistency & Governance: Drive data consistency improvements across systems (naming, identifiers, timestamps, joins, deduplication) and document data contract expectations with producing teams.
• Root Cause Analysis: Troubleshoot pipeline and data issues, perform root-cause analysis, and implement durable fixes that improve reliability and reduce operational load.
• Collaboration & Analytics: Partner with BI Analysts and Product teams to create curated datasets and self-serve analytics foundations (e.g., marts/semantic layer), as well as support internally facing dashboards to communicate system health.

• Lifecycle Management: Own the production lifecycle for the cart classification capability, including data collection/labeling workflows, evaluation, threshold tuning, and safe release/rollback processes.
• Pipeline Implementation: Implement and optimize machine learning pipelines, from feature engineering and model training to deployment and monitoring in production.
• Evaluation & Monitoring: Build and maintain an evaluation harness (offline metrics + repeatable test sets) and ongoing monitoring (accuracy drift, data drift, false positive/negative analysis).
• Cross-Team Collaboration: Collaborate with the FaceFirst ML team to incorporate improvements (model updates, feature changes) while keeping client’s production integration stable.
• Integration: Work with software engineers to ensure the classifier integrates cleanly into the product workflow with robust telemetry, logging, and operational runbooks.

• Core Engineering: Strong experience building and operating production ELT/ETL pipelines and data warehouses.
• Programming: Fluency in SQL and Python (or similar) for data transformation, validation, and automation.
• Cloud Platforms: Experience with cloud data platforms (Azure and/or GCP), including object storage, security/access controls, and cost-aware design.
• Tooling: Hands-on experience with orchestration and transformation tooling (e.g., Airflow/Prefect) and batch processing frameworks (e.g., Spark/Databricks).
• Quality Practices: Practical experience implementing data quality practices (tests, monitoring/alerting, lineage/documentation) and improving data consistency across systems.
• Operations: Collaborate with operational teams to identify, diagnose, and remediate in-field system issues.
• Bachelor’s degree in computer science, Software Engineering, Information Systems, Mathematics, Statistics, or a related technical field.

• Rule Engines: Working knowledge and understanding of Rule Engines and the integration of those engines to process complex data relationships.
• ML & Vision: Experience with computer vision/video analytics concepts or deploying ML models to production.
• Streaming & IoT: Experience with streaming/near real-time data (e.g., Kafka, Pub/Sub), IoT telemetry pipelines, or edge computing.
• MLOps: Familiarity with MLOps practices (model/version tracking, reproducible training, monitoring).