echojobs.io
Login

Welo Global

Lead Prompt Engineer

Remote
Python SQL API Machine Learning AI LLM
Search for More Jobs
Description

Project Lion - Lead Prompt Engineer - United States (Remote, Part-Time)

Team: Data Validation

Location: United States

Commitment: Freelance

Workplace Type: remote

We are looking for an experienced Lead Prompt Engineer to guide and manage a team through the full technical migration process, transitioning templates to LLM autoraters. In this role, you will leverage advanced prompt engineering techniques and the client’s internal tools to optimize model performance, ensuring the successful integration and ongoing enhancement of AI systems. As the team lead, you will drive the strategy, mentor junior engineers, and play a key role in shaping the future of our AI-driven solutions.

Responsibilities: 

    • Utilize Automatic Prompt Generation (APG) tools to create baseline prompts for complex parent-child template clusters. 

    • Run and supervise Automated Prompt Optimization (APO) tool, review the outputs, and flag when the APO reaches deadlocks or plateaus.   

    • Manually draft, test, and refine prompts to navigate complex template architectures, overcome anti-patterns, and handle edge cases where tooling is lacking or broken. Solve edge-case scenarios by designing and refining manual prompts. 

    • Monitor shadowbot runs to ensure sufficient disagreements (between human and LLM ratings) are registered, generated, and tracked.   

    • Run prompt versions against established gold data to continuously measure autorater quality against the human crowd baseline, calculating accuracy metrics such as F1 scores, precision, and recall.   

    • Draft technical launch readiness justifications (Launch Certification Documentation) for final. 

Requirement: 

    • Language Skills: Native fluency in English.

    • Location: Must be based in United States. 

    • Education: Master’s, or Doctorate degree in Computer Science, Data Science, Computational Linguistics, Human-Computer Interaction (HCI), Cognitive Science, or a related analytical field.  

    • Prompt Engineering & AI Expertise: At least 7 years' experience as Prompt Engineer. Proven experience tuning Large Language Models (LLMs) for strict, structured outputs, complex classification tasks, and familiarity with chain-of-thought and few-shot learning.  

    • Data Analysis: Strong proficiency in identifying error patterns, analyzing model performance, and using SQL or other data analytics tools.  

    • Technical Agility: Ability to quickly learn and master proprietary tools with minimal supervision.  

    • Communication: Excellent verbal and written communication skills. 

Optional / Preferred Skills:  

    • Familiarity with enterprise-grade LLM interfaces like the Goose API.  

    • Experience in AI model evaluation, data science, computational linguistics, or software engineering.  

    • Hands-on experience with Automated Prompt Optimization (APO) systems or tuning workflows.  

    • Linguistic expertise, including an understanding of semantics and logic.  

Federal Law Compliance   
   
In compliance with federal law, all persons hired will be required to:   
 
- Verify identity and eligibility to work in the United States; and   
- Complete a required employment eligibility verification form. 
 
Please note that in order to verify work authorization as is required by Federal law (I-9 process), all new employees must complete a live video verification with their selected IDs and provide photos of these selected IDs within their first 3 days of employment.
 
To know more details (Click here)
Welo Global
Welo Global

0 applies

0 views

There are more than 50,000 engineering jobs:

Subscribe to membership and unlock all jobs

Engineering Jobs

60,000+ jobs from 4,500+ well-funded companies

Updated Daily

New jobs are added every day as companies post them

Refined Search

Use filters like skill, location, etc to narrow results

Become a member

🥳🥳🥳 452 happy customers and counting...

Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.

To try it out

For active job seekers

For those who are passive looking

Cancel anytime

Frequently Asked Questions

  • We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
  • We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
  • We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
  • We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
  • Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
  • Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
  • Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅

What Fellow Engineers Say