Neural Network Reduction and Optimization

At the core of the relatively recent “AI revolution” are Deep Neural Networks (DNNs), which have demonstrated remarkable breakthroughs on a wide range of machine learning tasks such as image classification, speech recognition, object detection, natural language understanding, etc. However, state-of-the-art DNN models are usually equipped with several millions of connections, sometimes even in the order of billions, which consequently make them very resource inefficient. For instance, recent advances in natural language processing have created GPT-3, a model with 127 billions parameters which needed 285 thousand CPUs and a cluster of 10000 GPUs to be trained, whose costs amounted to roughly 12 million US dollars. Moreover, such DNN models are prohibitally large for applications where frequent communication and deployment to resource-constrained devices is required, such as autonomous cars, mobile phones, wearables, smart home devices, VR/AR glasses, etc. A huge number of applications on such embedded systems would emerge if the size of DNN models could be reduced easily at no or small performance loss.

Our research focuses on developing techniques that reduce the complexity of DNN models on all fronts: size, speed & energy consumption. We develop theoretical underpinnings that lead to practical algorithms which are able to significantly reduce the complexity of DNN models while minimally affecting the prediction performance of the models.

Our most recent research efforts focused on reducing the size of DNNs. Compressing the size of DNNs has direct positive impact in that communication as well as storage costs can be directly reduced. Moreover, smaller sized models can also have higher execution efficiency. Our research efforts led to several publications in this field to top conferences and journals [1-6]. Also, the developed technology has been submitted to ISO/IEC JTC1/SC29/WG04 (MPEG Video) and has been stadardized, as described in the research topic on International Standardization. In collaboration with our colleagues from the Video Coding group, we developed several technologies, such as inter alia DeepCABAC [1], a universal compression algorithm for deep neural networks, resulting in a high compression of neural networks down to less the 5% of their original size without degradation in inference performance and accuracy measures.