Senior Search Engineer (Elasticsearch)

About Us

Acryl Data is the company behind DataHub, the leading open-source metadata platform. Originally developed at LinkedIn, DataHub has grown into the largest open-source metadata community with over 12,000 data practitioners and deployments across 3,000+ organizations worldwide.

Through our flagship product DataHub Cloud, we provide enterprise-grade data catalog and observability solutions that enable seamless data discovery, robust data observability, and federated governance across organizations' entire data ecosystem. Our customers range from innovative startups to Fortune 10 companies, all benefiting from our expertise in bringing clarity and control to complex data environments.

Founded by the original creators of LinkedIn DataHub and Airbnb Dataportal, and backed by top-tier venture capital firms, Acryl Data combines deep technical expertise with a proven track record in building industry-leading data platforms.

Role Overview

We're seeking an experienced Search Engineer to optimize and scale our search infrastructure built on Elasticsearch. In this role, you'll be responsible for delivering lightning-fast, highly relevant search experiences across massive datasets. You'll work on complex challenges involving search relevance, performance optimization, and scalability while maintaining sub-100ms response times. This role combines deep technical expertise in search systems with a data-driven approach to relevance tuning.

Key Responsibilities

Search Performance & Scalability

• Architect and optimize Elasticsearch clusters for high-throughput, low-latency operations
• Implement and tune caching strategies to optimize query performance
• Design and implement efficient indexing strategies for large-scale data ingestion
• Monitor and optimize cluster health, performance metrics, and resource utilization
• Implement horizontal scaling solutions while maintaining consistent performance
• Design and implement efficient replication and sharding strategies

Search Quality & Relevance

• Develop and tune ranking algorithms to improve search relevance
• Implement advanced text analysis pipelines using analyzers, tokenizers, and filters
• Design and implement custom scoring functions and boosting strategies
• Create and maintain relevance testing frameworks
• Conduct A/B tests to measure and improve search quality
• Implement semantic search capabilities using vector search and relevant plugins

Technical Implementation

• Design and implement efficient search APIs and query DSL patterns
• Build robust monitoring and alerting for search infrastructure
• Develop tools for search quality analysis and debugging
• Implement advanced features like faceted search, autocomplete, and filtering
• Create documentation and best practices for search implementation

Required Qualifications

• 7+ years of software engineering experience, with at least 4 years focused on search systems
• Deep expertise with Elasticsearch, including:

• • Cluster administration and optimization
  • Query and indexing performance tuning
  • Relevance scoring and ranking
  • Text analysis and processing
  • Monitoring and observability
  • Strong experience with relevance engineering and information retrieval concepts
  • Proficiency in performance analysis and optimization techniques
  • Experience with large-scale data processing and distributed systems
  • Strong coding skills in Java, Python, or similar languages
  • Experience with search metrics (NDCG, MRR, etc.) and A/B testing

Preferred Qualifications

• Experience with other search engines (Solr, Lucene) and vector databases
• Knowledge of machine learning for search ranking
• Experience with semantic search and embedding models
• Background in natural language processing
• Experience with search infrastructure in cloud environments (AWS, GCP)
• Contributions to Elasticsearch or related open-source projects

Essential Knowledge Areas

Deep understanding of

• Information retrieval concepts
• Text analysis and processing
• Query optimization techniques
• Relevance scoring models
• Search infrastructure scaling
• Performance optimization
• Monitoring and observability

If you're passionate about technology, enjoy working with customers, and want to be part of a fast-growing company changing the industry, we want to hear from you!

How we work

Remote first. We're a fully-distributed company, and our interaction culture is deliberately mixed between meeting culture and written. We're writing heavily because it forces clarity of thought; we have plenty of synchronous time to give space for collaborative ideation.

At Acryl Data, representation matters – for us to build the best, most inclusive, accessible product for our community members, our work and team must reflect the lived experiences, unique perspectives, and communities around us. We are proud to be an equal opportunity workplace.

Benefits

• Competitive salary
• Equity
• 99% coverage of medical, dental and vision insurance for US Employees
• Carrot Fertility and Family Planning 
• Remote friendly
• Monthly co-working salary
• One-time home office budget

This position is open to candidates in all time zones of the United States and can be done remotely.