Solution Architect or Data Bricks Technical Lead

Location: Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia

Department: Information Technology

Workplace: on_site

Employment Type: full

Description

We are looking for Solution Architect or Data Bricks Technical Lead

with 10 to 12 years of experience.

Mandatory Skills: Databricks, Azure, Microsoft Fabric

Job Summary:

Required Technical Skill Set: Proficiency in Azure, Databricks and Microsoft Fabric.

Must-Have

• Databricks Expertise

• Strong hands-on experience with: 
• Delta Lake
• PySpark / Spark SQL
• Unity Catalog governance
• Workflows/Jobs/Task orchestration
• Cluster management and optimization
• MLflow experiment tracking & model deployment
• Lakehouse design patterns and advanced data partitioning strategies.

• Microsoft Fabric Expertise

• Deep knowledge of: 
• Lakehouse tables & Medallion architecture
• Fabric Data Factory pipelines
• Notebooks, Spark compute, and orchestration
• Real-Time Hub ingestion
• Power BI semantic modeling & DAX
• Designing data estates fully within Fabric or hybrid with Databricks.

• Azure Cloud Ecosystem

• Azure AD authentication & RBAC
• ADLS Gen2, Azure SQL/MI, Synapse, Event Hub, ADF
• Data governance, data security, encryption, and networking fundamentals (VNets, private endpoints)

• Architecture Skills
• Data modeling (Kimball, Data Vault, Star/Snowflake)
• API-based data integrations (REST, Graph, OData)
• Designing for scalability, performance, and resiliency
• CI/CD pipelines and IaC (ARM/Bicep/Terraform is a plus)
• Soft Skills
• Strong communication and stakeholder management
• Ability to translate business needs into architectural blueprints
• Leadership, mentoring, and solution ownership mindset

Good-to-Have

• Experience with Microsoft Purview for governance and lineage 
• Knowledge of Synapse serverless SQL pools, KQL, Timeseries analytics 
• Expertise in MLOps: Databricks Feature Store, Model Registry, Fabric ML workloads 
• Background in Data Science (Python, ML frameworks) 

• Experience designing solutions for: 

• Customer 360 platforms
• Supply chain analytics
• Digital media analytics
• Experience with cost governance and FinOps 

• Certifications (nice to have): 

• Databricks Certified Data Engineer/Architect
• Microsoft Fabric Analytics Engineer
• Azure Solutions Architect Expert
• DP-203 (Azure Data Engineer)

Expectations from the Role:

• The Data Solutions Architect will be responsible for designing and implementing robust, scalable, and secure data solutions leveraging Microsoft Azure, Microsoft Fabric, Power BI and Databricks. This role requires deep expertise in medallion data architecture principles, ensuring compliance with governance standards, and guiding delivery teams through complex data integration and design challenges.
• Solution Design & Documentation:

• Develop & update comprehensive Solution Design Architecture documents for data platforms and applications.
• Create high-level end-to-end architecture designs aligned with business and technical requirements.

• Architecture & Solution Design

• Lead the design of modern data architecture leveraging: 
• Azure Databricks (Delta Lake, Unity Catalog, Workflows, MLflow)
• Microsoft Fabric (Lakehouse, Data Engineering, Data Factory, Data Science, Real-Time Hub, Power BI)
• Azure Cloud Services (Azure Data Lake Storage Gen2, Azure Synapse, Event Hub, Data Factory, Cosmos DB, Azure SQL/MI)
• Define and implement Lakehouse architectures, delta-based pipelines, unified governance strategies, and multi-zone data models (bronze/silver/gold).
• Architect end-to-end ingestion and transformation frameworks using both Databricks and Fabric pipelines.
• Develop real-time streaming solutions using Auto Loader, Event Hub, Fabric Real-Time Analytics, or Spark Structured Streaming.
• Create enterprise-grade data models, metadata frameworks, semantic layers, and dimensional/denormalized designs to support analytics and AI.

• Governance, Security & Compliance
• Implement Unity Catalog, Fabric governance, data lineage, and access control across the data estate.
• Ensure enterprise-wide data quality, observability, PII protection, encryption-at-rest/in-transit, and compliance with GDPR/ISO frameworks.
• Enforce data governance, security, and compliance standards across all data solutions.
• Collaborate with security and compliance teams to ensure adherence to regulatory requirements.
• Documentation & Processes:
• Develop and maintain detailed documentation for architecture design, implementation processes, and troubleshooting procedures.
• Technical Leadership & Mentorship:
• Provide guidance and mentorship to delivery teams on resolving issues related to data pipelines, migrations, and system integrations.
• Act as a subject matter expert for Azure, Microsoft Fabric, Power BI and Databricks technologies.