Research Scientist, ML H/W-SW Codesign

Description

Reality Labs (RL) is bringing Meta's vision to life through virtual reality (VR), augmented reality (AR), and wearable technology. Meeting the demanding compute and power efficiency needs of these products requires purpose-built silicon. The Reality Labs Silicon team is pushing boundaries with pioneering advances in computer vision, machine learning, mixed reality, graphics, displays, sensors, and novel approaches to human body mapping. Our custom chips will power AR and VR experiences that seamlessly blend the physical and digital worlds throughout everyday life. We're convinced that realizing this vision demands a full-stack approach: spanning transistors, architecture, firmware, and algorithms. Meta is seeking a HW Research Scientist to join our Research and Development teams. The candidate will have experience working on AI models, hardware acceleration, and software systems related topics. The position will involve taking these skills and applying them to solve for some of the most crucial & exciting problems that exist in Meta Reality Labs. The primary objective will be to develop novel solutions that enable compute and power-efficient training and on-device inference of vision and language models for use cases in AR, VR, and edge devices. We are hiring in multiple locations.

Responsibilities

Identify and solve multi-discipline ML acceleration problems involving algorithms, network design, hardware architecture, and AR/VR use cases. Many of these would be first-time solutions in the industry Work across hardware and software, to solve deep co-design problems with other Research scientists working in this area Codesign and invent novel ML accelerator and system architecture solutions, and facilitate the integration of algorithms and software to utilize these enhancements Develop state-of-the-art model compression and scalability techniques using numerics, pruning, distillation, etc Optimize models on hardware accelerators to achieve the best performance given various real-time latency and power constraints Deliver impact directly or by influencing partners through thorough, data-driven analysis Define use cases, and develop a methodology and benchmarks to evaluate different approaches Apply in-depth knowledge of how the ML acceleration interacts with the other systems around it Attend conferences, interpret papers, and stay up to date with the latest research advancements in the field of ML acceleration; patent and/or publish novel outcomes in peer-reviewed conferences and journals

Qualifications

Masters or PhD in Electrical Engineering, Computer Science or equivalent experience 2+ years of specialized experience in one or more of the following machine learning/deep learning domains: Model compression, hardware-aware model optimizations, hardware accelerators architecture, GPU architecture, machine learning compilers, or ML systems, AI infrastructure, high-performance computing, performance optimizations, or machine learning frameworks (e.g., PyTorch), numerics and SW/HW co-design Experience developing AI system infrastructure, AI algorithms, or AI hardware acceleration in C/C++ or Python Must obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employment Experience or knowledge of training/inference of large-scale AI models - CV and/or LLMs Experience or knowledge of architecting ML hardware accelerators and systems Experience working and communicating cross-functionally in a team environment Experience with PyTorch, TensorFlow, or similar machine learning toolsets Experience or knowledge of on-device algorithm development including hardware-aware ML models and/or optimizing ML compilers for efficient deployment on AI accelerators Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as publications at leading workshops, journals, or conferences such as ICLR, NeurIPS, CVPR, ACL, ICML, MLSys, ISCA, MICRO, DAC, ASPLOS, etc Demonstrated research and engineering experience via an internship, work experience, coding competitions, or widely used contributions in open source repositories (e.g., GitHub) Experience solving complex problems and comparing alternative solutions, trade-offs, and diverse points of view to determine a path forward

Compensation: $154,000/year to $217,000/year + bonus + equity + benefits