Talk
Talk by Sumit K Mandal
Title: Communication-aware Heterogeneous 2.5 D System for Energy-efficient LLM Execution at Edge
Abstract:
Large Language Models (LLMs) are used to perform various tasks, especially in the domain of natural language processing (NLP). State-of-the-art LLMs consist of a large number of parameters that necessitate a high volume of computations. Currently, GPUs are the preferred choice of hardware platform to execute LLM inference. However, monolithic GPU-based systems executing large LLMs pose significant drawbacks in terms of fabrication cost and energy efficiency. In this work, we propose a heterogeneous 2.5D chiplet-based architecture for accelerating LLM inference. Thorough experimental evaluations with a wide variety of LLMs show that the proposed 2.5D system provides up to 972x improvement in latency and 1600x improvement in energy consumption with respect to state-of-the-art edge devices equipped with GPU.
Speaker Bio:Sumit K. Mandal is currently an assistant professor in Indian Institute of Science, Bangalore. He completed his PhD from University of Wisconsin-Madison. He received best paper award from ACM TODAES in 2020 and ESweek in 2022. He is one of the ACM India eminent speakers. He recently received INAE Young Engineer award. His research interest is energy efficient communication architecture for machine learning applications with emerging technologies.
- 20 - 20 July,2026
- 02:00 pm - 03:00 pm
- EE Committee Room