Principal Software Engineer - Data Engineer

About the Role

We are seeking a talented, self-directed Data Engineer to join our growing team. You will play a key role in designing, developing, and deploying innovative software solutions. The ideal candidate relishes working with large volumes of data and thrives in highly complex technical environments. You should have a passion for data science and analytics, as well as a background in handling big data, working with cloud data stores, and automating data management processes.

Proficiency in AWS cloud infrastructure, GenAI services, experience in writing automation tests, and expertise in building and deploying software releases are essential. Additionally, you should have experience in supporting data-driven decision-making and have demonstrated success in creating and using analytics tools, data products, and data sets to drive the understanding, growth, and success of a product.

What You Will Do in This Role

• Design, develop, and maintain high-quality, scalable, distribute, and secure software applications using Java, Scala, Python and other languages.

• Work with AWS cloud services such as EMR, Glue, S3, Athena, Open Search, Lambda, MSK, AirFlow, and DynamoDB to build and deploy applications in the cloud.

• Object oriented design and practical knowledge of applying patterns to application design/implementation.

• Working knowledge of Batch and streaming pipelines for data extraction/ingestion.

• Pipeline design(s) to handle million(s) of records in both streaming/batch fashion.

• Lead initiatives, architect and drive deliverables end to end.

• Continue to take ownership of delivered components and ensure systems are performing as expected in production

• Drive change where needed to improve team quality as well as team deliverables

• Collaborate with cross-functional teams (product managers, data scientists, designers) to understand requirements, translate them into technical solutions, and deliver on project goals.

• Design and implement CI/CD pipelines to automate the build, test, and deployment process.

• Write clean, well-documented, and maintainable code.

• Conduct unit and integration testing to ensure code quality and functionality.

• Participate in code reviews and provide constructive feedback to colleagues.

• Stay up-to-date with the latest advancements in AWS, and AI technologies.

What You’ll Need to be Successful in this Role

• Bachelor's degree in Computer Science

• 8+ years of relevant experience in Data Engineering

• 3+ years of relevant experience in leading large initiatives

• Experience working with AWS cloud services

• Experience working with big data using Hive/SQL,PySpark, Hadoop

• Demonstrated strength in data modeling, ETL development, data warehousing, and datalakes

• Familiarity with Agile methodologies (Scrum) and its principles.

• Knowledge of CI/CD tools and concepts for automating software delivery.

• Basic understanding of Gen AI concepts and techniques (machine learning, natural language processing a plus).

• Excellent problem-solving and analytical skills.

• Strong communication and collaboration skills.

• Ability to work independently and as part of a team.

• Experience in unit testing and integration testing methodologies.

• A passion for learning and staying current with new technologies.

• Good understanding of Data Privacy Laws (Ex. GDPR, CCPA, etc.)

Gen is proud to be an equal-opportunity employer. We celebrate diversity and are committed to creating an inclusive and accessible environment for all employees. All employment decisions are based on merit, experience, and business needs, without regard to race, color, national origin, age, religion, sex, pregnancy (including childbirth or related medical conditions), genetic information, disability (physical or mental), medical condition, marital status, sexual orientation, gender identity or gender expression, military or veteran status, or any other consideration made unlawful by federal, state, or local law. Gen strictly prohibits unlawful discrimination based on such protected characteristics and seeks to recruit the most talented candidates from diverse cultures and backgrounds. 

We also consider employment-qualified individuals with arrest and conviction records. In addition, we will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. Learn more about pay transparency. 

Gen complies with all anti-discrimination laws. 

To conform to U.S. export control regulations, applicant should be eligible for any required authorizations from the U.S. Government.