Unnat Jain

I am an Assistant Professor of Computer Science at the University of California, Irvine.
Toward building general-purpose embodied intelligence, my research focuses on the intersection of computer vision (perception) and robot learning (action).

I have worked across industry, academia, and startups at Meta's Fundamental AI Research (FAIR) Labs, Carnegie Mellon University, and Skild AI, collaborating with Abhinav Gupta, Deepak Pathak, and Xinlei Chen. I received my PhD from UIUC, advised by Alex Schwing and Svetlana Lazebnik, and previously graduated from IIT Kanpur.

Helpful links for applicants: UC Irvine offers what I believe are important conditions for focused research: an active AI/ML intellectual community, research infrastructure to support ambitious projects, guaranteed affordable housing for all years (a rarity in California), and a safe environment with year-round access to nature. The campus has established pathways to real-world impact, and proximity to Southern California's AI and robotics industry creates opportunities for meaningful collaborations and diverse career paths. The deadline for PhD applications is December 15, 2025. For opportunities to work with me, see my UCI profile.

CV | E-Mail | Google Scholar | Github | UCI Profile | Twitter

People

Our research spans vision-language-action models, human-to-robot learning, sim-to-real transfer, pre-training strategies for embodied agents, and multi-agent collaboration. We're building systems that enable robots to learn from diverse data sources and work together on complex real-world tasks.

I am extremely fortunate to work with an amazing set of students and collaborators. I'm deeply grateful for their support. We are continuing to grow our group.

Dwip Dalal
Daniel Feng
Aditya Mittal
Sagar Patil
Shivansh Patel
Yuchen Song

Affiliations

IIT Kanpur
2011-2016
UIUC
2016-2022
CMU & Meta
2022-2024
Skild AI
2024-2025
UC Irvine
2025-

Internships

              
UMass Amherst
Summer 2015
Uber ATG
Summer 2017
Allen Institute for AI
Summer 2018, 2020
FAIR (collab w/ UT Austin)
Summer 2019, FA19, SP20
Google DeepMind
Summer 2021, FA21

Updates

Area Chairing Currently: CVPR 2026, ICLR 2026, NeurIPS 2025
CoRL 2025; CVPR 2023, 2024, 2025; ICCV 2025; ICLR 2025; NeurIPS 2023, 2024
Jan 2026 Invited talk at UCSD, on Controlling VLMs for Robot Control.
Dec 2025 ViPRA got best paper award at NeurIPS 2025 Embodied World Models Workshop. Congrats, Sandeep & team.
Oct 2025 First quarter at UC Irvine as an Assistant Professor in Computer Science.
Nov 2024 For those on the faculty job search, here is a note I wrote: Hidden Curriculum of Faculty Job Search
June 2024 Organizing the community-building workshop 'CV 20/20: A Retrospective Vision' at CVPR 2024.
June 2024 Accepted faculty position at UC Irvine. Spending the next year at Skild AI and will start in 2025.
Jun 2023 Organizing the 'Scholars & Big Models: How Can Academics Adapt?' workshop at CVPR 2023.
Dec 2022 Organizing the RoboAdapt Workshop at CoRL 2023.
+ previous news

Publications

[NEW] CRAFT: A Tendon-Driven Hand with Hybrid Hard-Soft Compliance
Leo Lin*, Shivansh Patel*, Jay Moon*, Svetlana Lazebnik, Unnat Jain
Under Review
paper | project | bibtex
@article{lin2025craft,
  title     = {{CRAFT}: A Tendon-Driven Hand with Hybrid Hard-Soft Compliance},
  author    = {Lin, Leo and Patel, Shivansh and Moon, Jay and Lazebnik, Svetlana and Jain, Unnat},
  journal   = {Preprint},
  year      = {2026},
}
[NEW] ViPRA: Video Prediction for Robot Actions
Sandeep Routray, Hengkai Pan, Unnat Jain*, Shikhar Bahl*, Deepak Pathak*
ICLR 2026
Best Paper Award at NeurIPS 2025 Embodied World Models Workshop
paper | project | code | bibtex
@inproceedings{routray2026vipra,
  title     = {{ViPRA}: Video Prediction for Robot Actions},
  author    = {Routray, Sandeep and Pan, Hengkai and Jain, Unnat and Bahl, Shikhar and Pathak, Deepak},
  booktitle = {ICLR},
  year      = {2026},
}
sym
[NEW] Constructive Distortion: Improving MLLMs with Attention-Aware Image Warping
Dwip Dalal, Gautam Vashishtha, Utkarsh Mishra, Jeonghwan Kim, Madhav Kanda, Hyeonjeong Ha, Svetlana Lazebnik, Heng Ji, Unnat Jain
ICLR 2026
paper | project | code | bibtex
@inproceedings{dalal2026attwarp,
  title     = {Constructive Distortion: Improving {MLLMs} with Attention-Aware Image Warping},
  author    = {Dalal, Dwip and Vashishtha, Gautam and Mishra, Utkarsh and Kim, Jeonghwan and
               Kanda, Madhav and Ha, Hyeonjeong and Lazebnik, Svetlana and Ji, Heng and Jain, Unnat},
  booktitle = {ICLR},
  year      = {2026}
}
sym
[NEW] Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations
Shivansh Patel, Shraddhaa Mohan, Hanlin Mai, Unnat Jain*, Svetlana Lazebnik*, Yunzhu Li*
ICLR 2026
paper | project | code | bibtex
@inproceedings{patel2026rigvid,
  title     = {Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations},
  author    = {Patel, Shivansh and Mohan, Shraddhaa and Mai, Hanlin and Jain, Unnat and
               Lazebnik, Svetlana and Li, Yunzhu},
  booktitle = {ICLR},
  year      = {2026},
}
sym
An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels
Duy-Kien Nguyen, Mahmoud Assran, Unnat Jain, Martin R Oswald, Cees GM Snoek, Xinlei Chen
ICLR 2025
paper | bibtex
@inproceedings{nguyen2025pit,
  title     = {An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels},
  author    = {Nguyen, Duy-Kien and Assran, Mahmoud and Jain, Unnat and Oswald, Martin R and
               Snoek, Cees GM and Chen, Xinlei},
  booktitle = {ICLR},
  year      = {2025}
}
Exploitation-Guided Exploration for Semantic Embodied Navigation
Justin Wasserman, Girish Chowdhary, Abhinav Gupta, Unnat Jain
ICRA 2024
Best Paper at NeurIPS 2023 Robot Learning Workshop
paper | project | code | bibtex
@inproceedings{wasserman2024xgx,
  title     = {Exploitation-Guided Exploration for Semantic Embodied Navigation},
  author    = {Wasserman, Justin and Chowdhary, Girish and Gupta, Abhinav and Jain, Unnat},
  booktitle = {ICRA},
  year      = {2024}
}
Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
Xavi Puig*, Eric Undersander*, Andrew Szot*, Mikael Cote*, Ruslan Partsey*, Jimmy Yang*, Ruta Desai*, Alexander Clegg*, Michal Hlavac, Tiffany Min, Theo Gervet, Vladimír Vondruš, Vincent-Pierre Berges, John Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai**, Roozbeh Mottaghi**
ICLR 2024
paper | project | code | bibtex
Media: media logo media logo media logo
@inproceedings{puig2024habitat3,
  title     = {{Habitat 3.0}: A Co-Habitat for Humans, Avatars and Robots},
  author    = {Puig, Xavi and Undersander, Eric and Szot, Andrew and Cote, Mikael and
               Partsey, Ruslan and Yang, Jimmy and Desai, Ruta and Clegg, Alexander and
               Hlavac, Michal and Min, Tiffany and Gervet, Theo and Vondrus, Vladimir and
               Berges, Vincent-Pierre and Turner, John and Maksymets, Oleksandr and
               Kira, Zsolt and Kalakrishnan, Mrinal and Malik, Jitendra and
               Chaplot, Devendra and Jain, Unnat and Batra, Dhruv and
               Rai, Akshara and Mottaghi, Roozbeh},
  booktitle = {ICLR},
  year      = {2024}
}
sym
An Unbiased Look at Datasets for Visuo-Motor Pre-Training
Sudeep Dasari, Mohan Kumar Srirama, Unnat Jain*, Abhinav Gupta*
CoRL 2023
paper | project | bibtex
@inproceedings{dasari2023data4robotics,
  title     = {An Unbiased Look at Datasets for Visuo-Motor Pre-Training},
  author    = {Dasari, Sudeep and Srirama, Mohan Kumar and Jain, Unnat and Gupta, Abhinav},
  booktitle = {CoRL},
  year      = {2023},
}
sym
Pretrained Language Models as Visual Planners for Human Assistance
Dhruvesh Patel, Hamid Eghbalzadeh, Nitin Kamra, Michael Louis Iuzzolino, Unnat Jain*, Ruta Desai*
ICCV 2023
paper | code | bibtex
@inproceedings{patel2023vlamp,
  title     = {Pretrained Language Models as Visual Planners for Human Assistance},
  author    = {Patel, Dhruvesh and Eghbalzadeh, Hamid and Kamra, Nitin and Iuzzolino, Michael Louis
               and Jain, Unnat and Desai, Ruta},
  booktitle = {ICCV},
  year      = {2023},
}
sym
Adaptive Coordination in Social Embodied Rearrangement
Andrew Szot, Unnat Jain, Dhruv Batra, Zsolt Kira, Ruta Desai, Akshara Rai
ICML 2023
paper | code | bibtex
@inproceedings{szot2023zsc,
  title     = {Adaptive Coordination in Social Embodied Rearrangement},
  author    = {Szot, Andrew and Jain, Unnat and Batra, Dhruv and Kira, Zsolt and Desai, Ruta and Rai, Akshara},
  booktitle = {ICML},
  year      = {2023}
}
sym
Affordances from Human Videos as a Versatile Representation for Robotics
Shikhar Bahl*, Russell Mendonca*, Lili Chen, Unnat Jain, Deepak Pathak
CVPR 2023
paper | project | bibtex
Media: media logo media logo media logo media logo
@inproceedings{bahl2023vrb,
  title     = {Affordances from Human Videos as a Versatile Representation for Robotics},
  author    = {Bahl, Shikhar and Mendonca, Russell and Chen, Lili and Jain, Unnat and Pathak, Deepak},
  booktitle = {CVPR},
  year      = {2023},
}
sym
MOPA: Modular Object Navigation with PointGoal Agents
Sonia Raychaudhuri, Tommaso Campari, Unnat Jain, Manolis Savva, Angel X. Chang
WACV 2024
paper | project | code | bibtex
@inproceedings{raychaudhuri2024mopa,
  title     = {{MOPA}: Modular Object Navigation with {PointGoal} Agents},
  author    = {Raychaudhuri, Sonia and Campari, Tommaso and Jain, Unnat and Savva, Manolis and Chang, Angel X},
  booktitle = {WACV},
  year      = {2024}
}
sym
Last-Mile Embodied Visual Navigation
Justin Wasserman*, Karmesh Yadav, Girish Chowdhary, Abhinav Gupta, Unnat Jain*
CoRL 2022
paper | project | code | bibtex
@inproceedings{wasserman2022sling,
  title     = {Last-Mile Embodied Visual Navigation},
  author    = {Wasserman, Justin and Yadav, Karmesh and Chowdhary, Girish and Gupta, Abhinav and Jain, Unnat},
  booktitle = {CoRL},
  year      = {2022},
}
sym
Retrospectives on the Embodied AI Workshop
Matt Deitke, Dhruv Batra, Yonatan Bisk, ... Unnat Jain ... Luca Weihs, Jiajun Wu
arXiv 2022
paper | bibtex
@article{deitke2022retrospectives,
  title   = {Retrospectives on the Embodied {AI} Workshop},
  author  = {Deitke, Matt and Batra, Dhruv and Bisk, Yonatan and others},
  journal = {arXiv preprint arXiv:2210.06849},
  year    = {2022}
}
sym
Learning State-Aware Visual Representations from Audible Interactions
Himangi Mittal, Pedro Morgado, Unnat Jain, Abhinav Gupta
NeurIPS 2022
paper | code | bibtex
@inproceedings{mittal2022replai,
  title     = {Learning State-Aware Visual Representations from Audible Interactions},
  author    = {Mittal, Himangi and Morgado, Pedro and Jain, Unnat and Gupta, Abhinav},
  booktitle = {NeurIPS},
  year      = {2022}
}
sym
Bridging the Imitation Gap by Adaptive Insubordination
Luca Weihs*, Unnat Jain*, Iou-Jen Liu, Jordi Salvador, Svetlana Lazebnik, Aniruddha Kembhavi, Alexander Schwing
NeurIPS 2021
paper | project | code | bibtex
@inproceedings{weihs2021advisor,
  title     = {Bridging the Imitation Gap by Adaptive Insubordination},
  author    = {Weihs, Luca and Jain, Unnat and Liu, Iou-Jen and Salvador, Jordi and
               Lazebnik, Svetlana and Kembhavi, Aniruddha and Schwing, Alexander},
  booktitle = {NeurIPS},
  year      = {2021},
}
sym
Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments
Sonia Raychaudhuri, Saim Wani, Shivansh Patel, Unnat Jain, Angel X. Chang
EMNLP 2021 (short)
paper | project | code | bibtex
@inproceedings{raychaudhuri2021law,
  title     = {Language-Aligned Waypoint ({LAW}) Supervision for Vision-and-Language Navigation
               in Continuous Environments},
  author    = {Raychaudhuri, Sonia and Wani, Saim and Patel, Shivansh and Jain, Unnat and Chang, Angel X},
  booktitle = {EMNLP},
  year      = {2021}
}
sym
GridToPix: Training Embodied Agents with Minimal Supervision
Unnat Jain, Iou-Jen Liu, Svetlana Lazebnik, Aniruddha Kembhavi, Luca Weihs*, Alexander Schwing*
ICCV 2021
paper | project | bibtex
@inproceedings{jain2021gridtopix,
  title     = {{GridToPix}: Training Embodied Agents with Minimal Supervision},
  author    = {Jain, Unnat and Liu, Iou-Jen and Lazebnik, Svetlana and Kembhavi, Aniruddha
               and Weihs, Luca and Schwing, Alexander},
  booktitle = {ICCV},
  year      = {2021},
}
sym
Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents
Shivansh Patel*, Saim Wani*, Unnat Jain*, Alexander Schwing, Svetlana Lazebnik, Manolis Savva, Angel X. Chang
ICCV 2021
paper | project | code | bibtex
@inproceedings{patel2021comon,
  title     = {Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents},
  author    = {Patel, Shivansh and Wani, Saim and Jain, Unnat and Schwing, Alexander and
               Lazebnik, Svetlana and Savva, Manolis and Chang, Angel X},
  booktitle = {ICCV},
  year      = {2021},
}
sym
Cooperative Exploration for Multi-Agent Deep Reinforcement Learning
Iou-Jen Liu, Unnat Jain, Raymond Yeh, Alexander Schwing
ICML 2021 (long oral)
paper | project | code | bibtex
@inproceedings{liu2021cmae,
  title     = {Cooperative Exploration for Multi-Agent Deep Reinforcement Learning},
  author    = {Liu, Iou-Jen and Jain, Unnat and Yeh, Raymond and Schwing, Alexander},
  booktitle = {ICML},
  year      = {2021}
}
sym
MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation
Saim Wani*, Shivansh Patel*, Unnat Jain*, Angel X. Chang, Manolis Savva
NeurIPS 2020
paper | project | code | challenge | bibtex
@inproceedings{wani2020multion,
  title     = {{MultiON}: Benchmarking Semantic Map Memory using Multi-Object Navigation},
  author    = {Wani, Saim and Patel, Shivansh and Jain, Unnat and Chang, Angel X and Savva, Manolis},
  booktitle = {NeurIPS},
  year      = {2020},
}
sym
AllenAct: A Framework for Embodied AI Research
Luca Weihs*, Jordi Salvador*, Klemen Kotar*, Unnat Jain, Kuo-Hao Zeng, Roozbeh Mottaghi, Aniruddha Kembhavi
arXiv 2020
paper | project | code | bibtex
Media: [NEW] [NEW] [NEW]
@article{weihs2020allenact,
  title   = {{AllenAct}: A Framework for Embodied {AI} Research},
  author  = {Weihs, Luca and Salvador, Jordi and Kotar, Klemen and Jain, Unnat and
             Zeng, Kuo-Hao and Mottaghi, Roozbeh and Kembhavi, Aniruddha},
  journal = {arXiv preprint arXiv:2008.12760},
  year    = {2020},
}
sym
A Cordial Sync: Going Beyond Marginal Policies For Multi-Agent Embodied Tasks
Unnat Jain*, Luca Weihs*, Eric Kolve, Ali Farhadi, Svetlana Lazebnik, Aniruddha Kembhavi, Alexander Schwing
ECCV 2020 (spotlight)
paper | project | code | bibtex
@inproceedings{jain2020cordialsync,
  title     = {A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks},
  author    = {Jain, Unnat and Weihs, Luca and Kolve, Eric and Farhadi, Ali and
               Lazebnik, Svetlana and Kembhavi, Aniruddha and Schwing, Alexander},
  booktitle = {ECCV},
  year      = {2020},
}
sym
SoundSpaces: Audio-Visual Navigation in 3D Environments
Changan Chen*, Unnat Jain*, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, Kristen Grauman
ECCV 2020 (spotlight)
paper | project | code | challenge | bibtex
Media: media logo media logo
media logo media logo media logo media logo
@inproceedings{chen2020soundspaces,
  title     = {{SoundSpaces}: Audio-Visual Navigation in {3D} Environments},
  author    = {Chen, Changan and Jain, Unnat and Schissler, Carl and Gari, Sebastia Vicenc Amengual and
               Al-Halah, Ziad and Ithapu, Vamsi Krishna and Robinson, Philip and Grauman, Kristen},
  booktitle = {ECCV},
  year      = {2020},
}
sym
TAB-VCR: Tags and Attributes based VCR Baselines
Jingxiang Lin, Unnat Jain, Alexander Schwing
NeurIPS 2019
paper | project | code | bibtex
@inproceedings{lin2019tabvcr,
  title     = {{TAB-VCR}: Tags and Attributes based {VCR} Baselines},
  author    = {Lin, Jingxiang and Jain, Unnat and Schwing, Alexander},
  booktitle = {NeurIPS},
  year      = {2019}
}
sym
Two Body Problem: Collaborative Visual Task Completion
Unnat Jain*, Luca Weihs*, Eric Kolve, Mohammad Rastegari, Svetlana Lazebnik, Ali Farhadi, Alexander Schwing, Aniruddha Kembhavi
CVPR 2019 (oral)
paper | project | code | bibtex
Talk @ Amazon: video, ppt, pdf
Talk @ CVPR'19: video, ppt, pdf, poster
@inproceedings{jain2019twobody,
  title     = {Two Body Problem: Collaborative Visual Task Completion},
  author    = {Jain, Unnat and Weihs, Luca and Kolve, Eric and Rastegari, Mohammad and
               Lazebnik, Svetlana and Farhadi, Ali and Schwing, Alexander and Kembhavi, Aniruddha},
  booktitle = {CVPR},
  year      = {2019},
}
sym
Two can play this Game: Visual Dialog with Discriminative Question Generation and Answering
Unnat Jain, Svetlana Lazebnik, Alexander Schwing
CVPR 2018
bibtex
@inproceedings{jain2018visualdialog,
  title     = {Two Can Play This Game: Visual Dialog with Discriminative Question Generation and Answering},
  author    = {Jain, Unnat and Lazebnik, Svetlana and Schwing, Alexander},
  booktitle = {CVPR},
  year      = {2018}
}
sym
Creativity: Generating Diverse Questions using Variational Autoencoders
Unnat Jain*, Ziyu Zhang*, Alexander Schwing
CVPR 2017 (spotlight)
paper | video | bibtex
@inproceedings{jain2017creativity,
  title     = {Creativity: Generating Diverse Questions using Variational Autoencoders},
  author    = {Jain, Unnat and Zhang, Ziyu and Schwing, Alexander},
  booktitle = {CVPR},
  year      = {2017},
}
sym
Compact Environment-Invariant Codes for Robust Visual Place Recognition
Unnat Jain, Vinay Namboodiri, Gaurav Pandey
Conference on Computer and Robot Vision (CRV) 2017
bibtex
@inproceedings{jain2017crv,
  title     = {Compact Environment-Invariant Codes for Robust Visual Place Recognition},
  author    = {Jain, Unnat and Namboodiri, Vinay and Pandey, Gaurav},
  booktitle = {CRV},
  year      = {2017}
}

Template credits: Deepak, Jon, Saurabh, and Abhishek