Backend, Software Engineer, City Data Platform Development

RESPONSIBILITIES:

• Deliver functionality for analysts, data scientists, and other roles to enhance the city's analytic performance and strategy. Develop and maintain data cleansing, profiling, and integration processes, including building data pipelines. Contribute to documentation and code maintainability.
• Automate optimized data pipelines, ETL solutions, and model data stores based on business requirements for real-time analytics and recommendations. Ensure data assets are quality-checked and efficiently stored for easy access.
• Collaborate with cross-functional developers and other divisions to implement the data platform roadmap. Work with data science to create efficient, reusable data assets to speed up machine learning model development.
• Develop best practices and strategies for the data platform to achieve analytics milestones. Deliver solutions using an ML pipeline, research environment, and production-ready models.
• Research and introduce emerging Big Data and Data Lake technologies. Implement data flow infrastructure as code, covering data sourcing, orchestration, and metadata collection. Implement governance tools covering security and protection.

MINIMUM QUALIFICATIONS:

• Bachelor's in Computer Science or related field and 5+ years of work experience for product
• Professional experience in one or more languages, such as C++, C#, Java, Go, Rust or  Python, with a preference towards functional/trait-oriented Python
• More than 3 years of relevant experience with scripting, developing, debugging, and using Big Data  technologies (e.g., Spark, Flink, Kafka, Arrow, Iceberg, DeltaLake, RockSet, Druid, Tableau,  SuperSet, Dagster …), respectively, working with SQL/NoSQL/Graph data store technologies 
• Experience in designing and building dimensional data models, applying data warehouse concepts and methodologies, optimizing data pipelines, and wearing the architect hat in the past
• Proficiency in business level English and good communication skills and experience in cross-functional teams.

NICE TO HAVES:

• Experience with managing complex systems and solving data and systems issues, having a consistent and algorithmic approach to resolving them 
• Has the know-how and the scripting and coding prowess to set up, configure & maintain a machine-learning model development environment 
• Experience architecting, coding, and delivering high-performance microservices serving descriptive or advanced analytics artifacts (such as recommenders) for internal and external users, potentially in the (tens of) millions. 
• Deep understanding of Information Security principles and discipline to ensure complaint handling and management of all data
• Experience working in Agile teams to support digital transformation projects, having a clear  understanding of Agile principles, practices, and Scrum or Kanban methodologies