Title : Factuality and Attribution for Large Language Models
Abstract : This talk will address the factuality status of generative language model output, and the current approaches to providing attribution to sources of information: retrieval-augmented generation and training data attribution. I will also discuss the problem of detecting generated texts, and the impact of large language models on the information ecosphere and content economy.
Anna Rogers is an Associate Professor at the IT University of Copenhagen. Her main research area is analysis and evaluation of pre-trained language models, as well as their sociotechnical impacts. Anna currently serves as an editor-in-chief of ACL Rolling Review, the peer review platform for all major NLP conferences organized by the Association for Computational Linguistics
Abstract : Artificial Intelligence is becoming both a blessing and a burden, thanks to the power of artificial neural networks. Thetalkstarts with a quick overview of the challenges encountered by the modern artificial neural network models. Next, it investigates in more detail one of these challenges, i.e., the dense connectivity, which prevents us from having scalable and efficient deep learning at both cloud and edge computing levels. An emerging state-of-the-art solution is presented, i.e., sparse-to-sparse training, which reduces the number of connections quadratically while persistently improving performance and generalization. The discussion begins with the first works on static and dynamic sparse training in the typical single-task (un)supervised learning context. Further, it gradually introduces newer approaches in the more challenging contexts of continual and reinforcement learning. Besides the fundamental theoretical novelty, a glimpse into the future, such as truly sparse implementations and deep learning energy and cost efficiency, is also provided and open for discussion.
Decebal Constantin Mocanu is Associate Professor in Machine Learning within the Department of Computer Science at the University of Luxembourg, where he leads the Blue Neural Networks group. He is also a Guest Faculty Member within the Mathematics and Computer Science department at the Eindhoven University of Technology (TU/e) and an alumni member of TU/e’s Young Academy of Engineering. Previously, he worked as an Assistant Professor in Machine Learning at TU/e (2017-2020) and at the University of Twente (2020-2023). In 2017, Decebal received his PhD degree from TU/e. During his doctoral studies and afterward, Decebal undertook four research visits: at the University of Pennsylvania (2014), Julius Maximilians University of Würzburg (2015), the University of Texas at Austin (2016), and the University of Alberta (2022). In the long term, Decebal is interested in studying the synergy between artificial intelligence, neuroscience, and network science for the benefit of science and society.
