ACM, the Affiliation for Computing Equipment, in the present day named Torsten Hoefler, a professor at ETH Zurich, the recipient of the ACM Prize in Computing for basic contributions to high-performance computing and the continued AI revolution.
Hoefler, who was a guest on the @HPCpodcast on this web site final November, developed lots of the core capabilities of recent supercomputers and outlined key elements of the algorithms for distributing AI fashions on them, the ACM mentioned.
The ACM Prize in Computing acknowledges early-to-mid-career pc scientists whose analysis contributions have basic affect and broad implications. The award carries a prize of $250,000, from an endowment supplied by Infosys Ltd., a digital providers and consulting firm.
ACM cited the work of Hoefler and his colleagues to scale community design and programming in supercomputers, which has revolutionized the capabilities of those
giant techniques. For instance, AI algorithms can now be processed on tons of of 1000’s of nodes. Hoefler’s advances in interconnection networks, programming, and parallel algorithms broke new floor in facilitating the usage of large-scale massively parallel clusters.
“His quite a few improvements throughout the entire supercomputer stack—together with key contributions corresponding to MPI-3 nonblocking collective operations, foundational parallelism methods for AI fashions, and high-performance networking techniques—have pushed the boundaries of parallel techniques design and translated into dramatic enhancements in supercomputer efficiency and scalability,” the ACM mentioned. “Lots of these improvements are included into the biggest and strongest machines in the present day.”
The ACM cited these examples of Hoefler’s HPC-AI R&D:
Message Passing Interface 3: Hoefler performed a significant position within the evolution of the Message Passing Interface (MPI), a casual business customary for exchanging messages between quite a few particular person nodes all through an HPC community. A messaging customary permits synchronization of the actions of every particular person pc, sharing information between nodes, and path and management of all the parallel community.
The MPI-3 customary was adopted in 2012 and made potential advances in HPC for simulations and AI purposes over the previous a number of years. Hoefler chaired each the “Course of Topologies” and “Collective Operations” working teams for MPI-3. His nonblocking collective operations, corresponding to Allreduce, Allgather, Bcast, and their respective blocking variations are included in varied collective communication libraries—even past MPI-3. These operations energy the core of distributed deep studying in the present day.
3D Parallelism: Hoefler was among the many first to develop and uncover the now well-known notion of “3D parallelism,” which drives infrastructure design for the AI business. Subsequently, he and his collaborators continued to develop methods for environment friendly pipelining, sparse communication, mannequin sparsity and quantization. This work has
enabled a cumulative 10-1000x acceleration of AI workloads in trendy computer systems.
Routing Protocols and Community Topologies: The low-level community routing protocols and community topologies that Hoefler and his colleagues developed for such networks as Myrinet and InfiniBand energy 1000’s of AI and HPC supercomputers. These contributions are central to high-performance AI techniques that prepare large-language fashions, corresponding to ChatGPT.
“The capability of high-performance computer systems has turn into mind-boggling,” mentioned ACM President Yannis Ioannidis. “In simply the final two years, we’ve got ushered within the period of
exascale computer systems, which might carry out a billion billion calculations per second. However high-performance computer systems may do little with out the underlying algorithms and requirements that enable them to course of large influxes of knowledge.
“Whereas Hoefler launched many of those improvements whereas he was nonetheless a pupil, they continue to be the definitive option to program massively parallel techniques in the present day,” Ioannidis continued. “The ensuing capabilities of those techniques have allowed vital advances in AI, the pure sciences, and plenty of different areas.”
Salil Parekh, CEO, Infosys, mentioned, “We’re in the midst of an thrilling period of AI, with nice promise for the longer term. Torsten Hoefler performed an necessary position in advancing high-performance computing, which in flip, fuelled the present AI revolution. The ACM Prize in Computing is meant to acknowledge to ‘early to mid-career professionals,’ whose work has depth and affect. With a lot of his necessary work being completed in his 20s, Hoefler is an instance for younger people who age just isn’t an impediment to achievement in computing. Infosys is proud to be the sponsor of this award because it was based in 2007.”
Hoefler is a Professor of Pc Science at ETH Zurich (the Swiss Federal Institute of Know-how), the place he serves as Director of the Scalable Parallel Computing
Laboratory. He’s additionally the Chief Architect for AI and Machine Studying on the Swiss Nationwide Supercomputing Centre (CSCS). Hoefler obtained a Diplom Informatik (Grasp of Pc Science) from Chemnitz College of Know-how and a PhD in Pc Science from Indiana College.
Hoefler’s honors embrace the Max Planck-Humboldt Medal, an award for excellent mid- profession scientists; the IEEE CS Sidney Fernbach Award, which acknowledges excellent contributions within the utility of high-performance computer systems; and the ACM Gordon Bell Prize, which acknowledges excellent achievement in high-performance computing. He’s a member of the European Academy of Sciences (Academia Europaea), a Fellow of IEEE, and a Fellow of ACM.
Hoefler can be formally introduced with the ACM Prize in Computing at ACM’s annual awards banquet on Saturday, June 14 at The Palace Resort in San Francisco.
