Description
*SUMMARY*
Data engineers work closely with Subject Matter Experts (SMEs) to design the ontology (data model), develop data pipelines, and integrate Foundry with external systems containing the data. Data engineers also need to provide guidance and support on how to access and leverage the data foundation to create new workflows or analyze data.
*ESSENTIAL DUTIES AND RESPONSIBILITIES:*
• Integrate new data sources to Foundry using Data Connection
• Implement 2-way integrations between Foundry and external systems
• Develop pipelines transforming tabular or unstructured data
• Implement data transformations in PySpark or Pipeline Builder
to derive new datasets or create ontology objects.
• Set up support structures for pipelines running in production
• Monitor and debug critical issues such as data staleness or data quality
• Improve performance of data pipelines (latency, resource usage)
• Design and implement an ontology based on business requirements and available data
• Provide data engineering context for application development
• Identify opportunities for turning exploratory or analytical
applications into interactive operational workflows to drive business value.
• Maintain applications as usage grows and requirements change
*Qualifications*
*PREFERRED QUALIFICATIONS:*
• Procedural and Logical thinking
• Technical hands-on background
• Between 1 and 3 years of experience, ideally in a customer-facing role
• Experience in Python/PySpark, or experienced in another programming
language and willing to learn Python and PySpark on their own.
• Experience in TypeScript, or experienced in another
programming language and willing to learn TypeScript on their own.
• Data engineering experience preferred over data science
• Programming experience requiring collaborative software development
• Python – complete language proficiency
• SQL – proficiency in querying language (join types, filtering,
aggregation) and data modeling (relationship types, constraints)
• PySpark – basic familiarity (Data Frame operations, PySpark SQL
functions) and differences with other Data Frame implementations
(Pandas)
• Typescript – experience in TypeScript or experienced in another
programming language and willing to learn TypeScript on their own.
• Distributed compute – conceptual knowledge of Hadoop and Spark (driver,
executors, partitions)
• Databases – general familiarity with common relational database models
and proprietary instantiations, such as SAP, Salesforce, etc.
• Git – knowledge of version control/collaboration workflows and
best practices
• Iterative working – familiarity with an agile and iterative working methodology
and rapid user feedback gathering concepts.
• UX design – knowledge of best practices and applications
• Data quality – best practices
• Data literacy – data analysis and statistical basics to ensure correctness in
data aggregation and visualization
*Benefits for all full-time employees include:*
Medical (HMO/PPO Plan Options)
Dental
Vision
Group Term Life Insurance (CTC pays 100% of the premium)
Short-Term Disability and Long-Term Disability (CTC pays 100% of the premium)
Flexible Spending Account
401K
15 paid vacation days (more after 5 years)
9 paid holidays
3 paid sick leave days
Job Type: Full-time
Pay: $99,000.00 - $130,000.00 per year
Benefits:
* 401(k)
* 401(k) matching
* Dental insurance
* Flexible spending account
* Health insurance
* Life insurance
* Paid time off
* Referral program
* Vision insurance
Experience level:
* 3 years
Schedule:
* Day shift
* Monday to Friday
Application Question(s):
* Do you have a legal work authorization to work in the United States permanently? (If hired, proof of legal work authorization will be required)
* Do you now, or will you in the future, require immigration sponsorship for work authorization (e.g., H-1B)?
* Do you have physics background?
Education:
* Bachelor's (Preferred)
Experience:
* Data Engineering: 3 years (Required)
* Foundry: 3 years (Required)
* SQL, Panda, and PySpark: 3 years (Required)
* Palantir-Foundry: 3 years (Required)
Ability to Commute:
* Irvine, CA 92614 (Required)
Work Location: In person
Data engineers work closely with Subject Matter Experts (SMEs) to design the ontology (data model), develop data pipelines, and integrate Foundry with external systems containing the data. Data engineers also need to provide guidance and support on how to access and leverage the data foundation to create new workflows or analyze data.
*ESSENTIAL DUTIES AND RESPONSIBILITIES:*
• Integrate new data sources to Foundry using Data Connection
• Implement 2-way integrations between Foundry and external systems
• Develop pipelines transforming tabular or unstructured data
• Implement data transformations in PySpark or Pipeline Builder
to derive new datasets or create ontology objects.
• Set up support structures for pipelines running in production
• Monitor and debug critical issues such as data staleness or data quality
• Improve performance of data pipelines (latency, resource usage)
• Design and implement an ontology based on business requirements and available data
• Provide data engineering context for application development
• Identify opportunities for turning exploratory or analytical
applications into interactive operational workflows to drive business value.
• Maintain applications as usage grows and requirements change
*Qualifications*
*PREFERRED QUALIFICATIONS:*
• Procedural and Logical thinking
• Technical hands-on background
• Between 1 and 3 years of experience, ideally in a customer-facing role
• Experience in Python/PySpark, or experienced in another programming
language and willing to learn Python and PySpark on their own.
• Experience in TypeScript, or experienced in another
programming language and willing to learn TypeScript on their own.
• Data engineering experience preferred over data science
• Programming experience requiring collaborative software development
• Python – complete language proficiency
• SQL – proficiency in querying language (join types, filtering,
aggregation) and data modeling (relationship types, constraints)
• PySpark – basic familiarity (Data Frame operations, PySpark SQL
functions) and differences with other Data Frame implementations
(Pandas)
• Typescript – experience in TypeScript or experienced in another
programming language and willing to learn TypeScript on their own.
• Distributed compute – conceptual knowledge of Hadoop and Spark (driver,
executors, partitions)
• Databases – general familiarity with common relational database models
and proprietary instantiations, such as SAP, Salesforce, etc.
• Git – knowledge of version control/collaboration workflows and
best practices
• Iterative working – familiarity with an agile and iterative working methodology
and rapid user feedback gathering concepts.
• UX design – knowledge of best practices and applications
• Data quality – best practices
• Data literacy – data analysis and statistical basics to ensure correctness in
data aggregation and visualization
*Benefits for all full-time employees include:*
Medical (HMO/PPO Plan Options)
Dental
Vision
Group Term Life Insurance (CTC pays 100% of the premium)
Short-Term Disability and Long-Term Disability (CTC pays 100% of the premium)
Flexible Spending Account
401K
15 paid vacation days (more after 5 years)
9 paid holidays
3 paid sick leave days
Job Type: Full-time
Pay: $99,000.00 - $130,000.00 per year
Benefits:
* 401(k)
* 401(k) matching
* Dental insurance
* Flexible spending account
* Health insurance
* Life insurance
* Paid time off
* Referral program
* Vision insurance
Experience level:
* 3 years
Schedule:
* Day shift
* Monday to Friday
Application Question(s):
* Do you have a legal work authorization to work in the United States permanently? (If hired, proof of legal work authorization will be required)
* Do you now, or will you in the future, require immigration sponsorship for work authorization (e.g., H-1B)?
* Do you have physics background?
Education:
* Bachelor's (Preferred)
Experience:
* Data Engineering: 3 years (Required)
* Foundry: 3 years (Required)
* SQL, Panda, and PySpark: 3 years (Required)
* Palantir-Foundry: 3 years (Required)
Ability to Commute:
* Irvine, CA 92614 (Required)
Work Location: In person
Similar Jobs
Lead Full Stack Software Developer - Boeing Defense & Space
Poland
Remote Hybrid
Frontend Senior Engineer
Tel Aviv, Israel
Senior Product Security Engineer - SF
San Francisco, CA
Full-Stack Engineer (Backend leaning)
Berlin, Germany
London, UK
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 307 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
Cancel anytime / Money-back guarantee