Talk
Integrating Knowledge and Explicit Reasoning in Vision & NLP through Neurosymbolic AI.
In the 2010s, as Deep Learning models took over as state-of-the-art AI systems, a large sub-community recognized the need for integrating domain knowledge, and human know-how in such data-driven systems. I will discuss how the initial neurosymbolc solutions were used to integrate commonsense knowledge in applications such as visual question answering, and visual puzzle solving. The advent of massively pre trained (Large) Language Models (from BERT to ChatGPT) challenged the perception that DL systems do not have domain knowledge. Even more so, Transformers were shown to be able to deductively reason over natural language, generate Temporal Logic proofs -- showing some reasoning ability.
Therefore, we set out to understand the extent of knowledge and reasoning in Language Models. We saw that there is a specific set of skills that these models lack. This move from missing "knowledge" to missing "skillset" prompted us to look towards more holistic architectures, where LLMs behave as central agents with access to multiple specialist tools at their disposal. We end this talk with such an implementation, termed MathSensei -- a Tool-augmented LLM for solving complex mathematics. In the end, we discuss what are hindrances towards deployable AI, and our future steps.
Bio: Dr. Somak Aditya is currently an Assistant Professor at CSE, IIT Kharagpur (joined 2021). He was a postdoctoral researcher at Microsoft Research India for nearly 2 years and a full-time researcher at Adobe Research for 1.5 years. He did his PhD thesis on “Knowledge and Reasoning for Image Understanding'' from Arizona State University in 2018. He has a background in Knowledge Representation and Reasoning, NLP, and Machine Learning; and is experienced in conducting interdisciplinary research in the field of AI. Through four workshops in top conferences such as KR, IJCAI, CIKM, and CVPR, Somak Aditya has actively promoted knowledge integration, and the need for reasoning (and neuro-symbolic systems) in the community. He was also a part of the organizing committee of IndoML 2021 (and 2022), and an organizing committee member at CODS 2025. He has filed 2 patents and written 29 research articles in top AI/NLP/CV conferences. He has served as the Area Chair and SPC in all the top-tier NLP and AI conferences; and was a visiting faculty at UKP Lab, TU Darmstadt. His research is funded by SERB, AI4CPS Hub, Microsoft, Rakuten, Toloka and Krutrim AI.
- 07 - 07 April,2025
- 12:00 pm - 01:00 pm
- SIT 001