Senior Query Optimizer Software Engineer

Why work at Voltron Data?

• We are Going for Impact: We are a Series A, venture-backed startup assembling a global team to build a new foundation for data analytics with Apache Arrow. This foundation will usher in a wave of innovation in data processing that can take full advantage of the speed and efficiency offered by modern hardware. 
• We are Committed to Bridging Open Source Communities: We are a collection of open source maintainers who have been driving open source ecosystems over the last 15 years, particularly in the C++, Python, and R programming ecosystems.
• We are Building a Diverse, Inclusive Company:  We are creating a representative, equitable, and respectful workplace that prioritizes employee growth. Everyone at Voltron Data is bought into the company’s success; all voices are critical to shaping the organization’s future. 

Upon joining:

• Spending time learning about the Apache Arrow compute primitives, compute intermediate representation, compute engine, and other foundational components.
• Familiarizing yourself with the different partners for compute kernels and the query execution engine on Apache Arrow.
• Learning and embracing the Apache development process.

Within a month:

• Becoming familiar with our SQL parser and query optimizer.
• Benchmarking queries and exploring the effects of different query optimization techniques using our query optimizer.
• Making changes and improvements to the existing query optimization rules and how it creates physical execution plans for the execution engines under development.

Within 6 months:

• Adding new query optimization rules.
• Making improvements to decision making in the cost based optimization based on metadata availability for the tables being queried.
• Integrating non-SQL operations to the optimization framework.
• Working with client interfacing engineers to understand performance bottlenecks in customer queries.

Within 12 months:

• Proposing and implementing improvements to the query parsing and optimization framework.
• Integrating with a stateful inter-query query engine context to optimize the reusability of compute stages across queries that query the same data in similar ways.