Solutions Data Engineer

This is a great opportunity to be part of one of the fastest-growing infrastructure companies in history, an organization that is in the center of the hurricane being created by the revolution in artificial intelligence.

"VAST's data management vision is the future of the market."- Forbes

VAST Data is the data platform company for the AI era. We are building the enterprise software infrastructure to capture, catalog, refine, enrich, and protect massive datasets and make them available for real-time data analysis and AI training and inference. Designed from the ground up to make AI simple to deploy and manage, VAST takes the cost and complexity out of deploying enterprise and AI infrastructure across data center, edge, and cloud.

Our success has been built through intense innovation, a customer-first mentality and a team of fearless VASTronauts who leverage their skills & experiences to make real market impact. This is an opportunity to be a key contributor at a pivotal time in our company’s growth and at a pivotal point in computing history.

We are seeking an experienced Solutions Data Engineer who possess both technical depth and strong interpersonal skills to partner with internal and external teams to develop scalable, flexible, and cutting-edge solutions. Solutions Engineers collaborate with operations and business development to help craft solutions to meet customer business problems.

A Solutions Engineer works to balance various aspects of the project, from safety to design. Additionally, a Solutions Engineer researches advanced technology regarding best practices in the field and seek to find cost-effective solutions.

Job Description:

We're seeking a Big Data Engineer with 1-2 years of SPARK Streaming experience and a solid background in Python and Java, to play a pivotal role in our data processing efforts. The right candidate will have a broad IT expertise, including a strong foundation in data pipeline development and maintenance. This role involves direct collaboration with R&D, customers, and other key stakeholders to deliver tailored data solutions that drive impactful decisions.

Key Responsibilities:

• Develop and maintain high-performance Big Data pipelines using Apache Spark, Python, Apache Kafka, Cloudera, HDFS and Hive.
• Engage directly with R&D and customers to understand their data challenges and deliver solutions that meet their needs.
• Leverage your IT expertise to manage and optimize data storage solutions, employing both SQL and NoSQL technologies.
• Work collaboratively with teams across Sales,Marketing, Product Management,R&D QA, and more, facilitating data-driven decision-making across the organization.

Required Skills & Experience:

• 1-2 years of hands-on experience with SPARK Streaming, with a strong proficiency in Python and Java.
• A total of 6-8 years of IT experience
• Excellent programming skills, particularly in Python and familiarity with other scripting languages.
• In-depth knowledge of SQL, NoSQL, and HDFS for data storage and management.
• Practical experience with the Cloudera platform, especially in a production Hadoop ecosystem.
• Familiarity with TPC-DS benchmarks for data warehousing and analytics performance evaluation.
• Effective communication skills, capable of engaging with diverse stakeholder groups, including direct interactions with customers.