Senior AI Data Engineer

Location: Palo Alto, California, United States

Department: US R&D

Workplace: on_site

Employment Type: full

Description

We are seeking a forward-thinking AI Data Engineer to bridge the gap between our user data assets and advanced AI capabilities. In this role, you will be the architect of our user data foundation, building a robust data warehouse and a dynamic tagging system. Crucially, you will leverage this data to integrate with third-party Large Language Models (LLMs), enabling intelligent, data-driven interactions and next-generation user experiences.

Key Responsibilities

• User Data Warehouse Construction & Architecture

1. Design, build, and maintain a scalable User Data Warehouse to consolidate data from fragmented sources.
2. Design efficient data models to support high-performance querying and analytics.
3. Implement ETL/ELT pipelines to ensure real-time or near-real-time data availability and quality.

• Data Tagging & Profile System (User 360)

1. Establish a comprehensive User Tagging/Labeling System (User Portrait).
2. Develop algorithms to generate static, behavioral, and predictive tags to accurately segment users.
3. Ensure the tagging system is dynamic and can update in real-time to reflect the latest user interactions.

• LLM Integration & Data Intelligence

1. Lead the integration of Large Language Models with our internal data.
2. Design and implement RAG (Retrieval-Augmented Generation) pipelines to feed structured user profile data and tags into LLMs for personalized outputs.

• Intelligent Interaction Development

1. Develop APIs and middleware that allow downstream applications to interact with data using natural language.
2. Optimize the "Data-to-AI" loop: ensure the LLM understands the context of the user data to provide accurate, hallucination-free responses.
3. Monitor token usage, latency, and response quality of the AI interactions.

Requirements

• Education: Master’s degree in Computer Science, Data Engineering, Artificial Intelligence, or a related field.
• Experience: 3-5+ years of experience in Data Engineering or Backend Development with a focus on data.

• Data Stack:

1. Proficiency in SQL and Python/Java/Scala.
2. Hands-on experience with Data Warehouses (e.g. Snowflake, BigQuery, ClickHouse) and Big Data frameworks (Spark, Flink).
3. Familiar with message middleware (Kafka) and containerization (Docker).
4. User Data Experience: Proven experience in building CDP (Customer Data Platform), DMP, or User Profile/Tagging systems.

• AI/LLM Skills:

1. Experience interacting with LLM APIs (OpenAI, etc.) and inference optimization (vLLM).
2. Familiarity with frameworks like LangChain, LlamaIndex, or Haystack.
3. Understanding of Embedding, vector databases (FAISS, Milvus), and RAG architecture.

• Soft Skills: Strong problem-solving abilities and the ability to translate business needs into technical data requirements.

Preferred Skills (Nice to Haves)

• Experience with Prompt Engineering and optimizing context windows for efficient data feeding.
• Knowledge of Knowledge Graphs (Neo4j, NebulaGraph) and how to combine them with LLMs.
• Experience in model fine-tuning (SFT, RLHF).
• Familiarity with privacy regulations (GDPR/CCPA) regarding user data and AI.
• Experience with mature launched projects serving a large user base on cloud platforms (AWS, etc.).

Benefits

OPPO is proud to be an equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements.

The US base salary range for this full-time position is $100,000-$300,000 + bonus + long term incentives benefits. Our salary ranges are determined by role, level, and location.