Talk
Talk by Dr. Dipanjyoti Paul
Title :-
Interpretable Vision Transformers for Fine-Grained Analysis: From Biodiversity to Medical Imaging
Abstract :-
Recent advances in visual recognition have enabled artificial intelligence to extract meaningful insights across fields like biology, ecology, and medicine. In biodiversity research, a key challenge is identifying species across taxonomic groups by analyzing subtle visual traits, while in medical imaging, interpretability helps reveal clinically relevant features, supporting diagnosis and uncovering hidden patterns. In both cases, it is crucial to understand not just what a model predicts, but why. Fine-grained tasks, such as distinguishing between similar bird or insect species, require attention to subtle cues like color patterns or shapes. In this talk, I will present my recent works on interpretable vision transformers that use standard architectures and end-to-end learning methods to guide models toward such discriminative regions. These approaches rely on class-specific queries to highlight meaningful features—such as the color of a bird's eyes or the shape of its beak. These ideas have also been extended into lightweight, adaptable modules that work with pretrained transformers and require minimal changes. This makes them practical and broadly applicable across biodiversity and medical imaging, where understanding model decisions is essential for scientific and clinical insights.
Recent advances in visual recognition have enabled artificial intelligence to extract meaningful insights across fields like biology, ecology, and medicine. In biodiversity research, a key challenge is identifying species across taxonomic groups by analyzing subtle visual traits, while in medical imaging, interpretability helps reveal clinically relevant features, supporting diagnosis and uncovering hidden patterns. In both cases, it is crucial to understand not just what a model predicts, but why. Fine-grained tasks, such as distinguishing between similar bird or insect species, require attention to subtle cues like color patterns or shapes. In this talk, I will present my recent works on interpretable vision transformers that use standard architectures and end-to-end learning methods to guide models toward such discriminative regions. These approaches rely on class-specific queries to highlight meaningful features—such as the color of a bird's eyes or the shape of its beak. These ideas have also been extended into lightweight, adaptable modules that work with pretrained transformers and require minimal changes. This makes them practical and broadly applicable across biodiversity and medical imaging, where understanding model decisions is essential for scientific and clinical insights.
Speaker Bio :-
Dr. Dipanjyoti Paul recently joined as an Assistant Professor in the Department of Information Technology, IIEST Shibpur, India. Prior to that, he was an Assistant Professor in the Department of Computer Science at the University of Tsukuba, Japan. He is also affiliated as a Visiting Scientist in the Department of Computer Science & Engineering at The Ohio State University, USA. He also served as a postdoctoral fellow at The Ohio State University, USA, in the Department of Computer Science & Engineering at the Imageomics Institute. He obtained his Ph.D. from the Department of Computer Science & Engineering at the Indian Institute of Technology Patna, India. His research interests include computer vision; natural language processing; multi-modality; AI-driven biodiversity research; explainable AI; medical Imaging; streaming data, among others. He has published in many journals and conferences, including Knowledge-Based Systems, Pattern Recognition, Expert Systems with Applications, IEEE Transactions on Computational Social Systems, Pattern Recognition Letters, ACM Transactions on Knowledge Discovery from Data, CVPR, ICLR, and more.
Dr. Dipanjyoti Paul recently joined as an Assistant Professor in the Department of Information Technology, IIEST Shibpur, India. Prior to that, he was an Assistant Professor in the Department of Computer Science at the University of Tsukuba, Japan. He is also affiliated as a Visiting Scientist in the Department of Computer Science & Engineering at The Ohio State University, USA. He also served as a postdoctoral fellow at The Ohio State University, USA, in the Department of Computer Science & Engineering at the Imageomics Institute. He obtained his Ph.D. from the Department of Computer Science & Engineering at the Indian Institute of Technology Patna, India. His research interests include computer vision; natural language processing; multi-modality; AI-driven biodiversity research; explainable AI; medical Imaging; streaming data, among others. He has published in many journals and conferences, including Knowledge-Based Systems, Pattern Recognition, Expert Systems with Applications, IEEE Transactions on Computational Social Systems, Pattern Recognition Letters, ACM Transactions on Knowledge Discovery from Data, CVPR, ICLR, and more.
- 26 - 26 November,2025
- 12:00 pm - 01:00 pm
- TBC