Job Description:
Business Overview
Rakuten Group is based in Japan and serves about 100 million customers domestically and 1 billion customers globally. We offer over 70 services in various fields such as e-commerce, payment services, financial services, telecommunications, media, and sports.
Department Overview
The Business Support Development Department is responsible for part of the RMS (ASP service) provided to stores by Rakuten Ichiba. Additionally, we develop and operate various applications that support the business staff who run Rakuten Ichiba. Our mission is to provide high-quality services using cutting-edge technology so that stores can efficiently expand their businesses.
The EC Business Data Analytics & Platform Section builds the data infrastructure that supports the massive transaction data of Rakuten Ichiba. We handle enormous data, including 50,000 stores, 300 million products, and an annual transaction volume exceeding 3 trillion yen, acting as a data hub to support business decision-making and data utilization. We also process Japan's largest traffic data during Rakuten Super SALE and other large events, striving to provide stable systems and continuously improve service quality.
Position:
Why We Hire
We are looking for new members to expand our services for both B2B (for stores) and B2C (for consumers) on Rakuten Ichiba. We use Hadoop and Google Cloud Platform for data storage and efficiently store and process data using Python, Spark, and Kafka.
This recruitment is for a data engineer responsible for expanding and optimizing the data pipeline that handles the massive transaction data of Rakuten Ichiba. This position plays a crucial role in providing data that supports important decision-making in Rakuten Ichiba, directly contributing to business growth and service improvement.
We are seeking an experienced lead software engineer. The ideal candidate will have a strong background in big data technology, cloud platforms, and data migration, with the ability to lead a team, manage projects, and provide high-quality solutions that promote operational efficiency and cost reduction.
Position Details
- Building Large-Scale Batch Data Pipelines: Use data processing frameworks and technologies like Spark, Hive, Kafka, Spark-Streaming, Hadoop, and Google Cloud Platform to build large-scale batch data pipelines. Utilize data pipelines to provide important decision-making data for data checks, customer acquisition, and other key business performance indicators in line with Rakuten Ichiba's guidelines.
- Data Pipeline Management: Build and maintain data pipelines for a large-scale e-commerce platform.
- Automation: Automate data ingestion processes to reduce manual data processing.
- Cloud Migration: Migrate on-premises data infrastructure to Google Cloud Platform.
- Application Management: Manage applications on the platform that ingest and process large amounts of data.
- Solving Complex Data Storage and Retrieval Issues: Resolve and clearly explain complex data storage and retrieval issues in big data pipelines.
- Debugging and Maintenance in Production: Debug and maintain applications in production using logging, alerting, and monitoring tools.
- Effective Collaboration: Collaborate effectively with other teams and cross-team members.
- Team Leadership: Lead a team of software engineers to achieve project goals.
Work Environment
Development Environment
Python / Hadoop / Hive / Spark / Kafka / Google Cloud Platform
Mandatory Qualifications:
- Experience in developing and operating Hadoop, Spark, and Kafka applications
- Development experience in Python or Java/Scala
- Over 5 years of experience as a data engineer
- Development and operational experience with Google Cloud Platform or BigQuery
- Advanced knowledge of SQL and experience with relational databases
- Experience in building and optimizing data pipelines and datasets
- Experience in data modeling
- Development experience on Linux
- Strong analytical skills for data processing and transformation
- Project management skills to manage projects and provide high-quality solutions
- Team leadership experience in leading and mentoring software engineering teams
- Native-level proficiency in Japanese (reading, writing, speaking)
Desired Qualifications:
- Development experience using big data
- Experience in database operations and performance tuning
- Experience in infrastructure management with Docker, Kubernetes, etc.
- Deep understanding and experience with DevOps/Microservice architecture
- Operational experience using Jenkins or other CI technologies
- Experience in Scrum/Agile development
Others Information:
Additional information on Location
Osaka or Tokyo
Additional information on English Qualification
TOEIC score 800 or above
#engineer #datascientist #researcher #commerce
Other Jobs from Rakuten
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 401 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got about 70,000 jobs from 5,000 vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 5,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say