UpGrowth HR Sdn. Bhd.

Senior Data Engineer

Kuala Lumpur
Python pandas PySpark SQL Airflow Prefect Luigi AWS GCP Azure Snowflake BigQuery Redshift Databricks Docker Power BI Tableau
Description

JOB RESPONSIBILITIES:

  • Design, build, and maintain ETL/ELT pipelines for automated data ingestion, transformation, and validation.

  • Establish a centralized data storage system, starting with lightweight solutions and scaling toward cloud-based infrastructure.

  • Collaborate with analysts and business teams to translate reporting needs into scalable, reusable data models.

  • Implement data quality checks, logging, and monitoring to ensure reliability and transparency of data flows.

  • Refactor existing Python/Jupyter workflows into structured, production-ready codebases.

  • Evaluate and recommend tools for data warehousing, orchestration, and cloud migration.

  • Define and promote best practices in version control, documentation, and reproducibility.


JOB REQUIREMENTS:

  • Strong proficiency in Python (pandas, PySpark, or similar libraries for data processing).

  • Solid understanding of SQL and experience with relational databases.

  • Hands-on experience designing or maintaining ETL pipelines and workflow orchestration tools (e.g., Airflow, Prefect, Luigi).

  • Good grasp of data modeling, schema design, and data governance principles.

  • Experience handling large datasets and optimizing data performance.

  • Comfortable working in environments with minimal existing infrastructure and building systems from scratch.

  • Strong analytical thinking, problem-solving skills, and ability to work with both technical and non-technical stakeholders.


NICE TO HAVE:

  • Experience with cloud platforms (AWS, GCP, or Azure) and cloud-native data services.

  • Knowledge of data warehousing technologies (Snowflake, BigQuery, Redshift, or Databricks).

  • Familiarity with CI/CD pipelines, Docker, or containerized workflows.

  • Exposure to business intelligence tools (Power BI, Tableau, or similar).

UpGrowth HR Sdn. Bhd.
UpGrowth HR Sdn. Bhd.

0 applies

0 views

There are more than 50,000 engineering jobs:

Subscribe to membership and unlock all jobs

Engineering Jobs

60,000+ jobs from 4,500+ well-funded companies

Updated Daily

New jobs are added every day as companies post them

Refined Search

Use filters like skill, location, etc to narrow results

Become a member

🥳🥳🥳 452 happy customers and counting...

Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.

To try it out

For active job seekers

For those who are passive looking

Cancel anytime

Frequently Asked Questions

  • We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
  • We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
  • We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
  • We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
  • Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
  • Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
  • Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅

What Fellow Engineers Say