Yunbo WANG

Associate Professor
Artificial Intelligence Institute, School of EIEE
Shanghai Jiao Tong University

Email: yunbow@sjtu.edu.cn
Mail: 5-514, Software Building, 800 Dongchuan RD, Shanghai 200240
Google Scholar


Our group's research lies in the intersection of machine learning and computer vision. We want to understand how the real world works through machine learning approaches (e.g., intuitive physical inference and model-based reinforcement learning).

I received my Ph.D. and M.E. degrees from Tsinghua University, advised by Philip S. Yu, Mingsheng Long, and Yangdong Deng, and my B.S. degree from Xi'an Jiaotong University. I also spent time in Josh Tenenbaum's lab at MIT working with Jiajun Wu in probabilistic planning and model-based reinforcement learning.

For prospective graduate students: I appreciate your interest in joining our group! We have 1 open position for Ph.D. students and 1 open position for Master students in Class 2026 in Department of Computer Science.

Preprints

  1. DimOL: Dimensional Awareness as A New 'Dimension' in Operator Learning
    Yichen Song, Yunbo Wang*, Xiaokang Yang
    [PDF] [PyTorch Code]

  2. Discovering Message Passing Hierarchies for Mesh-Based Physics Simulation
    Huayu Deng, Xiangming Zhu, Yunbo Wang*, Xiaokang Yang
    [PDF]

  3. Automated Data Augmentation for Few-Shot Time Series Forecasting: A Reinforcement Learning Approach Guided by a Model Zoo
    Haochen Yuan, Yutong Wang, Yihong Chen, Yunbo Wang*
    [PDF]

  4. GS-Phong: Meta-Learned 3D Gaussians for Relightable Novel View Synthesis
    Yumeng He, Yunbo Wang*, Xiaokang Yang
    [PDF] [PyTorch Code]

  5. Predictive Experience Replay for Continual Visual Control and Forecasting
    Wendong Zhang, Geng Chen, Xiangming Zhu, Siyu Gao, Yunbo Wang*, Xiaokang Yang
    [PDF] [PyTorch Code]

Publications

  1. Dynamic Scene Understanding through Object-Centric Voxelization and Neural Rendering
    Yanpeng Zhao, Yiwei Hao, Siyu Gao, Yunbo Wang*, Xiaokang Yang
    TPAMI 2025 [PDF] [Website] [PyTorch Code]

  2. Open-World Reinforcement Learning over Long Short-Term Imagination
    Jiajian Li*, Qi Wang*, Yunbo Wang*, Xin Jin, Yang Li, Wenjun Zeng, Xiaokang Yang
    ICLR 2025 Oral (Top 1.8%) [PDF] [Website] [PyTorch Code]

  3. Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learning
    Qi Wang*, Junming Yang*, Yunbo Wang*, Xin Jin, Wenjun Zeng, Xiaokang Yang
    NeurIPS 2024 [PDF] [Website] [PyTorch Code]

  4. Model-Based Reinforcement Learning with Multi-Task Offline Pretraining
    Minting Pan*, Yitao Zheng*, Yunbo Wang*, Xiaokang Yang
    ECML 2024 [PDF] [PyTorch Code]

  5. Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video
    Xiangming Zhu*, Huayu Deng*, Haochen Yuan*, Yunbo Wang*, Xiaokang Yang
    ICLR 2024 [PDF] [Website] [PyTorch Code]

  6. DynaVol: Unsupervised Learning for Dynamic Scenes through Object-Centric Voxelization
    Yanpeng Zhao*, Siyu Gao*, Yunbo Wang*, Xiaokang Yang
    ICLR 2024 [PDF] [Website] [PyTorch Code]

  7. Model-Based Reinforcement Learning with Isolated Imaginations
    Minting Pan, Xiangming Zhu, Yitao Zheng, Yunbo Wang*, Xiaokang Yang
    TPAMI 2023 [PDF] [PyTorch Code]

  8. ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning
    Zhiyu Yao*, Yunbo Wang*, Haixu Wu, Jianmin Wang, Mingsheng Long*
    TPAMI 2023 [PDF] [PyTorch Code]

  9. Improving Masked Autoencoders by Learning Where to Mask
    Haijian Chen*, Wendong Zhang*, Yunbo Wang*, Xiaokang Yang*
    PRCV 2023 (Best Student Paper Award Nomination) [PDF] [PyTorch Code]

  10. Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid
    Wendong Zhang, Yunbo Wang*, Bingbing Ni, Xiaokang Yang
    Pattern Recognition 2023 [PDF] [PyTorch Code]

  11. StockFormer: Learning Hybrid Trading Machines with Predictive Coding
    Siyu Gao, Yunbo Wang*, Xiaokang Yang
    IJCAI 2023 [PDF] [PyTorch Code]

  12. Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models
    Minting Pan*, Xiangming Zhu*, Yunbo Wang*, Xiaokang Yang
    NeurIPS 2022 Spotlight [PDF] [Website] [PyTorch Code]

  13. Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation
    Shanyan Guan, Jingwei Xu, Michelle Z. He, Yunbo Wang*, Bingbing Ni, Xiaokang Yang
    TPAMI 2022 [PDF] [Website] [PyTorch Code]

  14. NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields
    Shanyan Guan, Huayu Deng, Yunbo Wang*, Xiaokang Yang
    ICML 2022 Spotlight [PDF] [Slides] [Website] [PyTorch Code]

  15. Continual Predictive Learning from Videos
    Geng Chen*, Wendong Zhang*, Han Lu, Siyu Gao, Yunbo Wang*, Mingsheng Long, Xiaokang Yang
    CVPR 2022 Oral [PDF] [Website] [PyTorch Code]

  16. PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
    Yunbo Wang*, Haixu Wu*, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long*
    TPAMI 2022 (ESI - Top 1% highly cited) [PDF] [PyTorch Code]

  17. Learning Transferable Features for Point Cloud Detection via 3D Contrastive Co-training
    Yihan Zeng*, Chunwei Wang*, Yunbo Wang, Hang Xu, Chaoqiang Ye, Zhen Yang, Chao Ma
    NeurIPS 2021 [PDF] [Supplement]

  18. VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
    Zhiyu Yao*, Yunbo Wang*, Jianmin Wang, Philip S. Yu, Mingsheng Long*
    TPAMI 2021 [PDF] [PyTorch Code]

  19. Context-Aware Image Inpainting with Learned Semantic Priors
    Wendong Zhang, Junwei Zhu, Ying Tai, Yunbo Wang*, Wenqing Chu, Bingbing Ni, Chengjie Wang, Xiaokang Yang
    IJCAI 2021 [PDF] [PyTorch Code]

  20. Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction
    Shanyan Guan*, Jingwei Xu*, Yunbo Wang*, Bingbing Ni, Xiaokang Yang
    CVPR 2021 [PDF] [Website] [PyTorch Code]

  21. MetaSets: Meta-Learning on Point Sets for Generalizable Representations
    Chao Huang*, Zhangjie Cao*, Yunbo Wang*, Jianmin Wang, Mingsheng Long*
    CVPR 2021 [PDF] [PyTorch Code]

  22. Deep Predictive Learning: Problems and Methods
    PhD Dissertation (in Chinese) [PDF]
    Outstanding Doctoral Dissertation Award, China Computer Federation (CCF), 2020

  23. Unsupervised Transfer Learning for Spatiotemporal Predictive Networks
    Zhiyu Yao*, Yunbo Wang*, Mingsheng Long*, Jianmin Wang
    ICML 2020 [PDF] [PyTorch Code]

  24. DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs
    Yunbo Wang*, Bo Liu*, Jiajun Wu, Yuke Zhu, Simon S. Du, Li Fei-Fei, Joshua B. Tenenbaum
    IJCAI 2020 [PDF] [Website] [PyTorch Code]

  25. Probabilistic Video Prediction from Noisy Data with a Posterior Confidence
    Yunbo Wang, Jiajun Wu, Mingsheng Long, Joshua B. Tenenbaum
    CVPR 2020 [PDF] [Website]

  26. Progressive Adversarial Networks for Fine-Grained Domain Adaptation
    Sinan Wang, Xinyang Chen, Yunbo Wang, Mingsheng Long*, Jianmin Wang
    CVPR 2020 [PDF]

  27. A Multi-Player Minimax Game for Diversifying Multi-Discriminators of Generative Adversarial Networks
    Ying Jin, Yunbo Wang, Mingsheng Long*, Jianmin Wang, Philip S. Yu, Jiaguang Sun
    ICME 2020 [PDF]

  28. Multi-Task Learning of Generalizable Representations for Video Action Recognition
    Zhiyu Yao*, Yunbo Wang*, Mingsheng Long*, Jianmin Wang, Philip S. Yu, Jiaguang Sun
    ICME 2020 [PDF]

  29. Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics
    Yunbo Wang*, Jianjin Zhang*, Hongyu Zhu, Mingsheng Long*, Jianmin Wang, Philip S. Yu
    CVPR 2019 [PDF] [TensorFlow Code]

  30. Eidetic 3D LSTM: A Model for Video Prediction and Beyond
    Yunbo Wang, Lu Jiang, Ming-Hsuan Yang, Li-Jia Li, Mingsheng Long, Li Fei-Fei
    ICLR 2019 [PDF] [Poster] [TensorFlow Code]

  31. PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
    Yunbo Wang, Zhifeng Gao, Mingsheng Long*, Jianmin Wang, Philip S. Yu
    ICML 2018 [PDF] [TensorFlow Code]

  32. PredCNN: Predictive Learning with Cascade Convolutions
    Ziru Xu*, Yunbo Wang*, Mingsheng Long*, Jianmin Wang
    IJCAI 2018 [PDF]

  33. PredRNN: Recurrent Neural Networks for Predictive Learning Using Spatiotemporal LSTMs
    Yunbo Wang, Mingsheng Long*, Jianmin Wang, Zhifeng Gao, Philip S. Yu
    NeurIPS 2017 [PDF] [PyTorch Code]

  34. Spatiotemporal Pyramid Network for Video Action Recognition
    Yunbo Wang, Mingsheng Long*, Jianmin Wang, Philip S. Yu
    CVPR 2017 [PDF]

  35. A Feasibility Study of Ray Tracing on Mobile GPUs
    Yunbo Wang, Chunfeng Liu, Yangdong Deng*
    SIGGRAPH Asia 2014 [PDF]

Teaching

  • JCCX0021: Fundamentals of Artificial Intelligence (Reinforcement Learning), 2022-2025
  • CS0502: Introduction to Computer Science, 2022-2025

Research Group

  • Graduate students: Minting Pan, Huayu Deng (Co-supervised), Xiangming Zhu, Qi Wang (Co-supervised), Haochen Yuan, Yanpeng Zhao, Yumeng He, Yitao Zheng, Yichen Song (Co-supervised), Yuetian Wang (Co-supervised)
  • Alumni: Jingwei Xu (2021), Wendong Zhang (2023), Geng Chen (2023), Shanyan Guan (2024), Siyu Gao (2024), Haijian Chen (2024)

Research Grants

  • PI, Visual Intuitive Learning of Fluid Dynamics, NSFC Original Exploratory Program, 2023-2025
  • PI, Predictive Learning from Spatiotemporal Data, NSFC Young Scientists Fund, 2022-2024