Senior Software Engineer

Microsoft's vision for Azure Machine Learning (ML) centers around democratizing ML and ensuring its accessibility to all enterprises, developers, and data scientists. We are seeking individuals to join our team entrusted with the responsibility of serving all internal and external ML workloads. With our current efforts, we are already catering to billions of requests daily, encompassing the most cutting-edge scenarios and models throughout the company.

As a member of the Inference team, you will contribute to the development of the next generation of model serving. This includes hosting OpenAI models such as ChatGPT, as well as scaling model hosting for Bing and Office, tackling numerous captivating challenges at the intersection of AI and Cloud. We are actively seeking a highly skilled Software Engineer who possesses a profound passion for designing and constructing exceptionally reliable and available platforms, capable of supporting model inferencing on a massive scale.

In addition to platform development, you will be tasked with addressing high throughput/low latency scenarios and spearheading performance optimization capabilities. This position provides a unique opportunity to thrive in an environment that fosters innovation, fosters collaborative teamwork, and upholds the pursuit of excellence, all in alignment with Microsoft's mission.

• B Tech or M Tech in computer science, engineering, mathematics or a related field, or equivalent industry experience
• 7+ years of software development experience
• 4+ years of software development experience focused C/C++ and/or Python development
• Experience in building enterprise grade applications in C++, Python Knowledge and experience in OSS, Docker, Kubernetes, Python, GOLANG programming languages
• Preferred Practical experience hosting and running large scale machine learning models in enterprise grade applicationsKnowledge of LLM/Diffusion model architectures e.g. GPT and Stable Diffusion
• Experience in distributed computing and architecture
• Experience in developing low latency systems
• Experience in developing and operating high scale, reliable online services Good communication, collaboration skills and a great team player
• Experience working in a geo-distributed team Understanding of parallel algorithms for communication between GPUs, familiarity with related libraries and frameworks such as DeepSpeed, PyTorch Distributed, Horovod, Megatron, MSCCL, NCCL is a plus

#IDCAIPlatformHiring

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.

Engage directly with key partners to understand state-of-the-art LLMs and Diffusion models, run them at scale in performance and cost effective manner Leverage latest hardware stack technologies improvements in CUDA, infiniband and fast-moving software stack to deliver best of class inference Anticipate, identify, assess, track, and mitigate project risks and issues in a fast-paced start up like environment Motivated to build constructive and effective relationships and solve problems collaboratively Support production inference for core AI scenarios on one of the largest GPU fleets in the world.