Software Engineering

Microsoft's vision for Azure Machine Learning (ML) centers around democratizing ML and ensuring its accessibility to all enterprises, developers, and data scientists. We are seeking individuals to join our team entrusted with the responsibility of serving all internal and external ML workloads. With our current efforts, we are already catering to billions of requests daily, encompassing the most cutting-edge scenarios and models throughout the company

As a member of the Inference team, you will contribute to the development of the next generation of model serving. This includes hosting OpenAI models such as ChatGPT, as well as scaling model hosting for Bing and Office, tackling numerous captivating challenges at the intersection of AI and Cloud. We are actively seeking a highly skilled Software Engineer who possesses a profound passion for designing and constructing exceptionally reliable and available platforms, capable of supporting model inferencing on a massive scale

In addition to platform development, you will be tasked with addressing high throughput/low latency scenarios and spearheading performance optimization capabilities. This position provides a unique opportunity to thrive in an environment that fosters innovation, fosters collaborative teamwork, and upholds the pursuit of excellence, all in alignment with Microsoft's mission.

Required and Preferred

• B Tech or M Tech in computer science, engineering, mathematics or a related field, or equivalent industry experience
• 1+ year(s) of software development experience focused C/C++ and/or Python development
• Knowledge and experience in OSS, Docker, Kubernetes, Python, GOLANG programming languages
• Good communication, collaboration skills and a great team player.
• Experience working in a geo-distributed team
• Practical experience hosting and running large scale machine learning models in enterprise grade applications.
• Experience in building enterprise grade applications in C++, Pytho
• Experience in developing and operating low latency, high scale, reliable online service

#AIPLATFORM#

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Responsibilities:

• Engage directly with key partners to understand state-of-the-art LLMs and Diffusion models, run them at scale in performance and cost effective manner
• Leverage latest hardware stack technologies improvements in CUDA, infiniband and fast-moving software stack to deliver best of class inference
• Anticipate, identify, assess, track, and mitigate project risks and issues in a fast-paced start up like environment
• Motivated to build constructive and effective relationships and solve problems collaboratively
• Support production inference for core AI scenarios on one of the largest GPU fleets in the world