Talk by Kumar Ashutosh
Title: Long-form video understanding with text supervision
Abstract: Language models, including recent large language models like ChatGPT, LLAMA, have shown great potential in learning fro efforts have been made in learning from large-scale videos on the internet. These advances enable many tasks in video understandi methods are constrained to short video clips due to computational complexity and difficulty in long-term reasoning. In contrast, e.g., while cooking an hour long recipe, one remembers each step done till now, each several minutes long, including added ingred of our efforts in enabling long-term video understanding using text supervision available on the internet.
The first part of the talk will cover our novel hierarchical video-language model that captures why something is happening, in This hierarchical approach to high-level task understanding fosters strong video-language representation learning. We show that c well as long-term (several minutes) understanding of video-language models.
The second part of the talk will emphasize how human activities follow a natural sequence and can be expressed in the form of a t between activities, e.g., adding sugar' is typically followed by 'stirring the mixture' and 'jack up the car' is followed by fre context using this graph improves keystep understanding in videos.
Bio: Kumar Ashutosh is a third year PhD student in the University of Texas at Austin and also a visiting researcher at FAIR, Meta M.Tech) from IIT Bombay in 2021. His research interests include video understanding, multimodal learning and representation learm CVPR, NeurIPS, AISTATS including two oral presentations. More details on his webpage: https://ashutoshkr.me
- 31 - 31 January,2024
- 12:00 pm - 01:00 pm
- SIT 001