Software Engineer, Platform Engineering - Merchandising and Advertisement Department (MAD)

Job Description:

Business Overview

The AI & Data Division (AIDD) creates powerful, customer-focused search, recommendation, data science, advertising, marketing, price and inventory optimization solutions to a variety of businesses in commerce industries. We design, develop, and deploy high performance, fault-tolerant distributed systems used by millions of Rakuten customers every day. We strive to deliver the most innovative solutions that are helpful to people and societies around the world.

Department Overview

The Advertisement & Marketing Optimization Section is a diverse cross function team of talents from around the world. Our solutions optimize Rakuten experience and services for thousands of consumers, merchants, advertisers, and Rakuten’s ecosystem of partners and services, globally. Our specializations range from personalized recommendations to search advertising and targeting and incentives optimization. Leveraging our scale, data, and core machine learning capabilities and algorithms we are driven with our mission to deliver better optimized experiences to the society, online and offline.

Position:

Position Details

We are looking for a talented Software Engineer with a strong interest in Platform, DevOps & MLOps Engineering, and experience with technologies like Python, Ray, Rust, Kubernetes (k8s), JupyterHub, Docker, Terraform, Prometheus, and CephFS. You’ll leverage your skills to develop and maintain our platform running on Kubernetes, while also contributing to data processing pipelines and in-house frameworks.

Key Responsibilities:

• Contribute to Platform: Design, code, test, release, and maintain various components of the platform.

• Collaboration and Delivery: Collaborate with software engineers & data scientists to address their requests and ensure timely delivery of software solutions.

• Continuous Improvement: Proactively propose and implement system and process improvements, such as refactoring, adopting new technologies, and enhancing system architecture.

• Cloud Infrastructure Management: Design, deploy, and manage cloud infrastructure on AWS, Azure, or Google Cloud. Optimize resource usage and control costs in the cloud environment.

• Containerization & Orchestration: Implement and manage containerized applications using Docker and Kubernetes. Automate deployments and manage container orchestration to ensure scalability and availability.

• Infrastructure as Code (IaC): Develop and maintain infrastructure using IaC tools like Terraform to ensure consistency in deployments and version control.

• CI/CD Pipelines: Design and implement continuous integration and continuous deployment pipelines. Automate testing and deployment processes to streamline software delivery.

• Automation & Scripting: Write scripts in Python or Bash to automate repetitive tasks, improving operational efficiency. Create tools to simplify the management and monitoring of applications and infrastructure.

• Monitoring & Logging: Implement monitoring solutions such as Prometheus and Grafana to track system performance and health. Set up logging frameworks like the ELK stack for log analysis and issue diagnosis.

• Documentation & Support: Document processes, configurations, and best practices for future reference and onboarding. Provide support and guidance to development teams on deployment and operational concerns.

Mandatory Qualifications:

• Bachelor's Degree (BS) in Computer Science or a related field, or equivalent education and experience.

• Over three years of proven experience as a Python Developer.

• Familiar with Kubernetes, with hands-on experience in developing and deploying operators(Airflow, Flink, Spark) on it.

• Experience working with database systems such as Couchbase, Redis, or Cassandra.

• Excellent problem-solving and analytical abilities.

• Able to work independently and as part of a team.

• Strong communication and collaboration skills.

Desired Qualifications:

• Knowledge of Linux administration (Red Hat/Ubuntu) is a plus, including encryption (LUKS/x.509), scripting, monitoring, security, logging, networking, and SSH.

• Familiarity with distributed filesystems is a plus, such as CephFS, HDFS, Lustre, or BeeGFS.

• Experience with Infrastructure as Code tools is a plus, including Chef, Ansible, Salt, Puppet, or Terraform.

• Knowledge of cloud platforms like AWS, Azure, or Google Cloud Platform is a plus.

• Familiarity with CI/CD tools is a plus.

• Certifications, such as Certified Kubernetes Administrator or Certified Kubernetes Application Developer, are a plus.

• Knowledge of big data processing frameworks like Hadoop, Spark, Flink, or Druid is a plus.

#ai #engineer #python #devops #mlops #platform #kubernetes #ceph #cloud

Languages:

English (Overall - 4 - Fluent)

Languages: