Description
AVP – Data Engineering Lead
Team: Group Digital Futures
Location: Abu Dhabi
Commitment: Experienced hires
Workplace Type: onsite
JOB PURPOSE
The AVP – Data Engineer (AI Architect) is a hands-on practitioner responsible for architecting, building, and scaling the organization’s data engineering and AI platforms. This role combines deep technical execution with architectural ownership to deliver secure, scalable, and production-grade data pipelines and AI solutions, including Generative AI and LLM-powered systems.
You will actively design and implement data and AI architectures, write production-quality code, prototype new capabilities, and work closely with data engineers, data scientists, and product teams to operationalize analytics and AI use cases at enterprise scale.
ROLES AND RESPONSIBILITIES
Architect, build, and optimize scalable data pipelines using Python, Spark, and Databricks for batch, streaming, and real-time workloads.
Design, develop, and maintain ELT/ETL pipelines, data transformations, and feature engineering workflows.
Design, prototype, test, and productionize AI and Generative AI solutions, moving beyond PoCs to enterprise-ready deployments.
Build and deploy cloud-native data and AI solutions on Microsoft Azure, leveraging services such as Azure OpenAI, Azure Databricks, AKS, App Services, Azure Functions, and Azure Storage.
Develop containerized data and AI services using Docker and Kubernetes, following DevOps and CI/CD best practices.
Design and implement secure APIs and microservices to expose data and AI capabilities to downstream applications.
Implement monitoring and observability for data pipelines and AI systems, including latency, data quality, model performance, drift, and cost metrics.
Optimize cloud and AI workloads for performance, scalability, and cost efficiency.
Actively contribute code, perform design and code reviews, and troubleshoot complex production issues across data and AI systems.
Serve as a hands-on technical mentor, leading by example within data engineering and AI teams.
Influence technical standards and long-term architecture while remaining deeply involved in day-to-day delivery.
RELATED YEARS OF EXPERIENCE
7+ years of progressive, hands-on experience in data engineering, AI engineering, or cloud platform architecture
Proven experience architecting and operating enterprise-scale data platforms and AI solutions, from ingestion to consumption, with high availability and security.
Demonstrated success taking AI and Generative AI solutions from experimentation to production, including scaling, monitoring, and continuous improvement.
Strong experience with lakehouse architectures and modern data platform patterns
Hands-on expertise with relational databases (PostgreSQL, MySQL, SQL Server) and NoSQL / distributed data stores (MongoDB, Cassandra, Cosmos DB).
Experience with cloud-native data platforms such as Azure Synapse, Snowflake, or BigQuery, with a strong preference for Azure-based ecosystems.
Practical experience designing real-time and event-driven architectures, using Kafka, Spark Structured Streaming, Event Hubs, or Flink.
FIELD OF EXPERIENCE
Data Engineering, AI Architect
TECHNICAL AND INTERPERSONAL SKILLS
Expert-level Python for data engineering, AI pipelines, and backend services.
Strong practical experience with Databricks and Azure services, including: Azure OpenAI, Azure Kubernetes Service, Azure App Services & Functions, Azure API Management etc.
Familiarity with vector databases and search platforms (Azure AI Search, Pinecone, FAISS etc.).
Experience with data orchestration and workflow management (Airflow, Azure Data Factory, N8N).
Strong understanding of CI/CD, containerization and cloud-native engineering (Docker, Kubernetes)
Ability to move seamlessly between architecture design and hands-on implementation.
Strong communication skills and ability to explain complex systems to non-technical stakeholders.
QUALIFICATION
Bachelor’s degree in Computer Science, Data Engineering, AI, or a related field
Master’s degree and/or cloud certifications (Azure Solutions Architect, Azure Data Engineer, Azure AI Engineer, Databricks) are a plus.
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say