Picogrid

Senior Data Engineer

El Segundo, CA
API Python Spark Kubernetes AWS TypeScript Microservices Java Docker Terraform Go
Description

Who we are

Picogrid envisions a future where every system, from the smallest sensors to next-generation fighter jets, can collaborate autonomously to create a safer and more prosperous world for all.

Today, we are at the forefront of American defense technology, developing a unified platform that integrates fragmented technologies—such as sensors, cameras, radar, and drones—into sophisticated mission systems.

Our technology is deployed globally, supporting customers including the U.S. Army, U.S. Air Force, CAL FIRE, PG&E, U.S. Fish and Wildlife, and many others.

About this Role

Picogrid is seeking an experienced Senior Data Engineer to design, build, and own our next-generation highly low latency, data ingestion and aggregation system. In this role, you will architect a robust data ingestion pipeline capable of handling diverse data types—from telemetry time-series data to large blob storage—and integrate an automation workflow for downstream processing. You will work in Go, design solutions for cloud, edge-based, and/or hybrid environments, and collaborate with teams using Kubernetes and AWS. The ideal candidate is passionate about data infrastructure, has a strong understanding of data governance and compliance practices, and is excited about building scalable solutions from the ground up.

Key Responsibilities

  • Data Pipeline Architecture & Development:

    • Design and build a highly extensible, scalable data ingestion pipeline to capture and process data from thousands of sensors, IoT devices, and external sources.

    • Ensure the pipeline accommodates different data types (e.g., time-series telemetry, blob storage) and is built to seamlessly onboard new data sources.

    • Architect a secure data platform that meets stringent US government/DoD compliance requirements and is designed for third-party developers to build additional solutions.

  • Workflow & Automation:

    • Develop and implement a workflow automation pipeline that triggers actions based on ingested data, ensuring timely data processing and integration into downstream systems.

  • Third-Party Platform Enablement:

    • Build and maintain a platform that serves as a foundation for internal/external partners and third-party developers, ensuring it’s well-documented, modular, and secure.

    • Provide API endpoints, SDKs, and integration guidelines to facilitate third-party solution development.

  • Software Development:

    • Write and maintain efficient, robust, and clean code using Go, Python, and Typescript.

    • Collaborate on code reviews and implement best practices in software development and testing.

  • Cloud & Hybrid Architecture:

    • Architect solutions that operate effectively in cloud, edge, or hybrid environments using AWS and other relevant platforms.

    • Leverage Kubernetes for container orchestration and management of scalable microservices.

  • Data Governance & Compliance:

    • Integrate data governance practices into the pipeline design, ensuring data integrity, quality, and compliance with relevant regulations.

    • Stay up-to-date with compliance requirements and implement necessary controls in data handling and storage.

  • End-to-End Ownership:

    • Take complete ownership of the data ingestion and aggregation system, from concept and design through to implementation, monitoring, and ongoing improvements.

Required Skills & Qualifications

  • Technical Proficiency:

    • Strong experience in designing and building data pipelines and ETL/ELT processes.

    • Technical expertise in one or more of the following stacks: Go, Node, Java, Python

    • Hands-on experience with AWS services (e.g., S3, Athena, Glue, Lambda, Redshift) and cloud-based infrastructure.

    • Working knowledge of container orchestration (e.g., Kubernetes, Docker Compose) and microservices architecture.

    • Experience in managing and provisioning new data infrastructure with IaC tools such as Terraform.

    • Experience in building scalable solutions in cloud, edge, or hybrid environments.

  • Data Handling:

    • Expertise in ingesting and processing high volumes of diverse data, including real-time sensor and IoT data.

    • Familiarity with managing different storage systems optimized for various data types (e.g., time-series databases, blob storage).

  • Data Governance & Compliance:

    • Awareness of data governance practices, including data quality, lineage, and security.

    • Understanding of compliance regulations (e.g., NIST 800-53r5, Fedramp, CMMC) and how they impact data storage and processing.

  • Soft Skills:

    • Strong problem-solving skills with the ability to troubleshoot complex data pipelines.

    • Excellent communication skills and the ability to work collaboratively across teams.

    • Ability to take initiative and work independently with minimal supervision.

Preferred Qualifications

  • Experience with workflow orchestration tools (e.g., Apache Airflow, AWS Step Functions).

  • Exposure to other programming languages or data engineering tools (e.g., Python, Spark).

  • Prior experience in scaling data systems for early-stage products transitioning to enterprise-grade solutions.

  • Experience working with government or defense-related projects.

Export Control Requirements

To conform to U.S. Government export regulations, applicant must be a (i) U.S. citizen or national, (ii) U.S. lawful, permanent resident (aka green card holder), (iii) Refugee under 8 U.S.C. § 1157, or (iv) Asylee under 8 U.S.C. § 1158, or be eligible to obtain the required authorizations from the U.S. Department of State.

Picogrid
Picogrid
Artificial Intelligence (AI) Government Industrial Military

0 applies

7 views

Other Jobs from Picogrid

Sr Electrical Engineer

Remote El Segundo, CA

There are more than 50,000 engineering jobs:

Subscribe to membership and unlock all jobs

Engineering Jobs

60,000+ jobs from 4,500+ well-funded companies

Updated Daily

New jobs are added every day as companies post them

Refined Search

Use filters like skill, location, etc to narrow results

Become a member

🥳🥳🥳 452 happy customers and counting...

Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.

To try it out

For active job seekers

For those who are passive looking

Cancel anytime

Frequently Asked Questions

  • We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
  • We've got about 70,000 jobs from 5,000 vetted companies. No fake or sleazy jobs here!
  • We aggregate jobs from 5,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
  • We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
  • Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
  • Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
  • Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅

What Fellow Engineers Say

Sid avatar
Sid
Very nice portal for searching jobs in this rough market.
Mar 6, 2025
Michael Duran avatar
Michael Duran
Software Engineer
I've been using this job search site for a while now, and it’s honestly one of the best out there! The clean and easy-to-navigate UI makes the whole job-hunting process so much smoother. Plus, the job postings are always up-to-date, so I never feel like I’m wasting time. The cherry on top is the owner—super kind and always quick to respond. Definitely recommend checking it out if you're on the job hunt!
Aug 21, 2024
Sai avatar
Sai
It’s really great website for finding jobs based on skills it’s really helpful give a go
Aug 21, 2024
Adinadh avatar
Adinadh
What I like most about Echo Jobs is how easy it is to use. The platform helps me quickly find jobs that match my skills and interests, thanks to its great recommendations and filters. Yes, I would definitely recommend Echo Jobs to a friend. It makes job searching simple and efficient, making it a great tool for anyone looking for a new job.
Jul 23, 2024
As a student navigating the job market, I've found LinkedIn increasingly frustrating due to numerous fake postings by consultancies. In contrast, this job posting website has been a game-changer for me. It offers genuine opportunities and a straightforward application process, making it much easier to find and apply for real jobs. Highly recommend it to fellow students seeking reliable job listings!
Jul 16, 2024
Cliff Gor avatar
Echo Jobs has been exceptional in my job hunt where it provides one platform to job hunt and I don't have to open 10 websites just to look for a job. It has also helped me focus much on the job skill and the location filtering out the onsite jobs and remote ones. The only feature that I would request is to display fully remote jobs that are not restricted to a country since the one available shows ie, Remote, US yet. But if it could show remote only, that would be helpful not only to me but to other people applying for full remote and not tied to only US candidates
Apr 22, 2024
I found EchoJobs in 2022, and I love it. It has a lot of remote jobs. It's exclusive to software and technology jobs (helpful for devs like me). What I like the most are its filters and its API. If you're a tech professional seeking remote work, I highly recommend giving it a try to EchoJobs.
Mar 4, 2024
Would definitely recommend it! Excellent product, dedicated founder, Jobs are easier to find. Congrats 🎉 to the entire team!
Mar 3, 2024
Brandon Banks avatar
Brandon Banks
Echo Jobs is really impressive. It provides a great user experience with an ability to quickly search through the many job postings. There is an impressive amount of jobs here and it is quickly updated. The details in the each job posting is helpful when determining if it is worth pursuing. I would highly recommend using Echo Jobs to find the next step in your career.
Mar 2, 2024
Tyler Young avatar
Tyler Young
tylerayoung.com
Best wishes with EchoJobs—it's become my favorite job board overnight!
Dec 16, 2023
Simply put, it's the most up to date tech jobs aggregator I’ve found. I'm like... "I don't have to check 10+ jobs boards daily just to see if there's a new job listing? sign me up!" The filters are also quite helpful! The UI is very clean and straightforward. Love it!
Oct 5, 2023