Lead Data Engineer (Hybrid)

Location: WI Madison

Remote Type: Hybrid

Time Type: Full time

Job Description

American Family Insurance looking for a experienced and driven Lead Data Engineer to guide our Data Platform team. In this important role, you will oversee a team of engineers in advancing our service-oriented data platform. The Data Platform team influences multiple departments, including security and AI, at AmFam. You will create tools and infrastructure that empower other teams and departments within American Family Insurance.

You will report to a Senior Manager, Data Engineering.

In this flex office/home role, you will be expected to work a minimum of 10 days per month from the following office location: Madison, WI 53783.

#LI-Hybrid

Position Compensation Range:

$131,000.00 - $220,000.00

Pay Rate Type:

Salary

Compensation may vary based on the job level and your geographic work location. Relocation support is offered for eligible candidates.

Primary Accountabilities

• Partner with stakeholders to refine undefined use cases into clear requirements, translate those into technical specs, and implement working solutions.
• Design and build platform capabilities that multiple teams consume, including feature stores, observability tooling, evaluation pipelines, and deployment patterns.
• Define and implement reusable patterns and infrastructure: shared LLM serving, evaluation frameworks, CI/CD templates, internal SDKs.
• Assess existing AI applications and establish standards for code quality, testing, deployment, and observability.
• Mentor data scientist and data engineers, conduct meaningful code reviews, and provide architectural guidance across the platform.
• Facilitate communication across project teams, architects, and business partners.
• Document critical workflows, standards, and operational support aspects of the platform.

Specialized Knowledge & Skills Requirements

• Demonstrated experience building AI/ML systems that run in production at scale - not notebooks or prototypes, but real software serving real users.
• Proven ability to take ambiguous, undefined problems and deliver shipped products.
• Strong system design skills with an emphasis on scalability, failure modes, and long-term maintainability.
• Deep software engineering fundamentals: experience with large codebases, design patterns, testability, and enterprise SDLC.
• Hands-on experience with LLMs in production: RAG, agents, evaluation, prompt engineering, or fine-tuning.
• Expert-level Python programming in a software engineering context, not just scripting or pipelines.
• Experience designing and building APIs for internal or external consumption.
• Deep experience with GCP, specifically Vertex AI, Pub/Sub, BigQuery, and NoSQL databases (e.g., Firestore, Bigtable).
• Applied knowledge of Kubernetes for container orchestration and workload management.
• Experience with infrastructure-as-code tools (e.g., Terraform, Docker) and CI/CD pipelines.

Travel Requirements

• Up to 10%.

Additional Information

• Offer to selected candidate will be made contingent on the results of applicable background checks

• Offer to selected candidate is contingent on signing a non-disclosure agreement for proprietary information, trade secrets, and inventions

• Sponsorship will not be considered for this position unless specified in the posting

• Your offer will be contingent on the results of applicable background checks and signing a non-disclosure agreement for proprietary information, trade secrets, and inventions.
• We will not consider candidates who require sponsorship.

We provide benefits that support your physical, emotional, and financial wellbeing. You will have access to comprehensive medical, dental, vision and wellbeing benefits that enable you to take care of your health. We also offer a competitive 401(k) contribution, a pension plan, an annual incentive, 9 paid holidays and a paid time off program (23 days accrued annually for full-time employees). In addition, our student loan repayment program and paid-family leave are available to support our employees and their families. Interns and contingent workers are not eligible for American Family Insurance Group benefits.

We are an equal opportunity employer. It is our policy to comply with all applicable federal, state and local laws pertaining to non-discrimination, non-harassment and equal opportunity. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law.

American Family Insurance is committed to the full inclusion of all qualified individuals. If a reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please email [email protected] to request a reasonable accommodation.

#LI-DB1