Data Engineer - Video Architecture -LiveNet (Live Streaming Networks)

TikTok is the leading destination for short-form mobile video. At TikTok, our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, and its offices include New York, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.

Why Join Us
Creation is the core of TikTok's purpose. Our platform is built to help imaginations thrive. This is doubly true of the teams that make TikTok possible.
Together, we inspire creativity and bring joy - a mission we all believe in and aim towards achieving every day.
To us, every challenge, no matter how difficult, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always.
At TikTok, we create together and grow together. That's how we drive impact - for ourselves, our company, and the communities we serve.
Join us.

About the Department
Video Arch is one of the world's leading video platforms that provides media storage, delivery, transcoding, and streaming services. We are building the next generation video processing platform and the largest live streaming network, which provides excellent experiences for billions of users around the world. Popular video products of TikTok and its affiliates are all empowered by our cutting-edge cloud technologies. Working in this team, you will have the opportunity to tackle challenges of large-scale networks all over the world, while leveraging your expertise in coding, algorithms, complexity analysis, and large-scale system design.

About the team
The Data Warehouse team within Video Architecture provides stable, complete, and high-quality data to DA/DS/RD/PM teams in a cost-effective manner. Our work includes building data pipelines, optimizing data workflows, and tackling other big data challenges using leading big-data infrastructure and platforms. The team's main goal is to help internal teams and stakeholders gain deep insights into their core business metrics, including service costs and quality metrics. Working on this team, you'll collaborate with one of the largest network system teams to build advanced data models and solve sophisticated data challenges.


Responsibilities
- Design and develop optimal data processing architecture and systems for new data and ETL (Extract, Transform, Load) pipelines.
- Architect, implement, and maintain efficient and reliable data pipelines for data movement and transformation across varying scales.
- Drive internal process improvements and automate manual processes to enhance data quality and Service Level Agreement (SLA) management.
- Collaborate with cross-functional teams including Content Delivery Network (CDN), Video Understanding, Video Transcoding, Live Streaming, and Real-Time Communication teams.Minimum Qualifications
- Bachelor's degree in Computer Science or a related technical field involving software/system engineering, or equivalent working experience
- Good programming experience with at least one of the following languages: Spark SQL, Python, NoSQL databases
- Good understanding of data model design and data warehouse architecture theories
- Willingness to embrace agile development for rapid iteration

Preferred Qualifications
- Experience in custom ETL/data pipeline design, implementation, and maintenance
- Experience in troubleshooting large-scale distributed systems
- Good understanding of big-data processing software (Hadoop, Spark, Pig, Hive) and algorithms (MapReduce)
- Good communication skills and willingness to work closely with peer teams for successful deliveries


---
TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

---
In the spirit of reconciliation, TikTok acknowledges the Traditional Custodians of country throughout Australia and their connections to land, sea and community. We pay our respect to Elders past and present and extend that respect to all Aboriginal and Torres Strait Islander peoples today.

---

#LI-Onsite