Senior Machine Learning Engineer (Hybrid)

What You'll Do

• Develop and tune novel ML models, metrics and other techniques to classify and explain ML models and LLM prompts/responses
• Setup processes around model building, benchmarking, deployment and MLOps to ensure health and performance of these artifacts in production 
• Develop new types of metrics and evaluation methods for quality and robustness of ML models
• Design enterprise-grade, scalable APIs and frameworks to run analysis and validation of customers’ ML models and workflows
• Work with product and customer facing teams to enhance and expand our product
• Recommend and implement engineering best practices, improve the functionality of existing software, and ensure that the design, application, and maintenance of software meets quality standards
• In this role, you will be able to make a real impact on the safety of large language models and generative AI solutions across different verticals. You will work on the cutting edge of envisioning and building new types of tools and algorithms to monitor, explain and improve such models

What We're Looking For

• BS/MS/PhD in Computer Science, Data Science or equivalent
• 2+ years of experience
• Proficiency with Python
• Experience working with, monitoring and tuning Large Language models in production. Experience with common LLM frameworks (Langchain, Llamaindex, RAG, HuggingFace, BERT models, and eval frameworks like Ragas or Presidio) and MLOps best practices
• Experience debugging and troubleshooting production systems. Experience in designing, consuming and debugging REST APIs and working with common distributed systems infrastructure: Redis, Postgres, RabbitMQ, Celery, Flask
• Excellent organizational, communication, writing and interpersonal skills; also people who are excited by collaboration and teamwork
• You invest in staying up to date with the LLM space, you regularly educate yourself on new capabilities, market trends and needs and are involved in local user groups and/or events relating to LLMs
• Curiosity, ownership, empathy towards customers, willingness to learn new things, and desire to inspire others

Nice to Have

• Experience running and self-hosting LLMs
• Familiarity with any of model explainability, data drift, algorithmic fairness, pattern mining, and model robustness
• Knowledge of infrastructure management (AWS / Google Cloud, Kubernetes, or equivalent on-premises technologies)
• Experience building large-scale distributed systems and working with columnar and vector databases