Paper Title: ChipNeMo: Domain-Adapted LLMs for Chip Design

Paper Abstract: ChipNeMo aims to explore the applications of large language models (LLMs) for industrial chip design. Instead of directly deploying off-the-shelf commercial or open-source LLMs, we adopt the following domain adaptation techniques: custom tokenizers, domain-adaptive continued pretraining, supervised fine-tuning (SFT) with domain-specific instructions, and domain-adapted retrieval models. We evaluate these methods on three selected LLM applications for chip design: an engineering assistant chatbot, EDA script generation, and bug summarization and analysis. Our results show that these domain adaptation techniques enable significant LLM performance improvements over general-purpose base models across the three evaluated applications, enabling up to 5x model size reduction with similar or better performance on a range of design tasks. Our findings also indicate that there's still room for improvement between our current results and ideal outcomes. We believe that further investigation of domain-adapted LLM approaches will help close this gap in the future.

Paper Author: Mark Ren, Director of Design Automation Research, NVIDIA

Author Bio: Haoxing Ren (Mark) serves as the Director of Design Automation Research at NVIDIA, where he focuses on leveraging machine learning and GPU-accelerated tools to enhance chip design quality and productivity. Prior to joining NVIDIA in 2016, he dedicated 15 years to EDA algorithm research and design methodology innovation at IBM Microelectronics and IBM Research. Mark is widely recognized for his contributions to physical design, AI, and GPU acceleration for EDA, achievements that have earned him several prestigious awards, including the IBM Corporate Award and best paper awards at ISPD, DAC, TCAD, and MLCAD. He holds Bachelor's and Master's degrees from Shanghai Jiao Tong University and Rensselaer Polytechnic Institute, respectively, and earned his Ph.D. from the University of Texas at Austin. He is a Fellow of the IEEE.