Shuangyu Xie

I am a Postdoctoral researcher in the AUTOLab at UC Berkeley, working with Prof. Ken Goldberg. Prior to joining UC Berkeley, I obtained my Ph.D. and M.S. degree from Texas A&M University, advised by Prof. Dezhen Song.

I design algorithmic frameworks that help robots move from theory to the real world—where uncertainty, complexity, and noise are the norm. My research aims to make robots more capable and resilient in unstructured environments, spanning applications in manipulation, outdoor navigation, and mobile manipulation. Recently, I’ve been exploring how task and motion planning, Vision-Language Models (VLMs), and 3D/4D reconstruction can come together to enable robots to perceive, reason, and act with greater autonomy—whether it’s grasping objects, navigating dynamic spaces, or modeling plants in natural settings.

news

Sep 20, 2025	🎉 Our Robo2VLM paper has been accepted to 2025 NeurIPS Dataset and Benchmark track as a Spotlight! 🚀✨ 📊 The dataset Robo2VLM-1 was downloaded 2,537 times last month at Hugging Face 🤗 !
Jan 30, 2024	🎉 Two papers accepted by ICRA 2025 🚀 Energy Efficient Planning for Repetitive Heterogeneous Tasks in Precision Agriculture and Heterogeneous Sensor Fusion and Active Perception for Transparent Object Reconstruction with a PDM² Sensor and a Camera Thank you for the contributions from all collaborators! 🙏
Jul 24, 2023	🔥We released MapBench, the first dataset specifically designed for human-readable, pixel-based map-based outdoor navigation, curated from complex path finding scenarios. Explore our paper for more details.

selected publications

* indicates co-first author

Energy Efficient Planning for Repetitive Heterogeneous Tasks in Precision Agriculture

Shuangyu Xie, Ken Goldberg, and Dezhen Song

In IEEE International Conference on Robotics and Automation (ICRA), 2025
Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot Manipulation Datasets

Kaiyuan Chen*, Shuangyu Xie*, Zehan Ma, and Ken Goldberg

In Thirty-ninth Annual Conference on Neural Information Processing Systems (NeurIPS), 2025

Spotlight
Botanybot : Digital Twin Monitoring of Occluded and Underleaf Plant Structures with Gaussian Splats

Simeon Adebola, Chung Min Kim, Justin Kerr, Shuangyu Xie, Prithvi Akella, Jose Luis Susa Rincon, Eugen Solowjow, and Ken Goldberg

In IROS, 2025
Coupled Active Perception and Manipulation Planning for a Mobile Manipulator in Precision Agriculture Applications

Shuangyu Xie, Chengsong Hu, Di Wang, Joe Johnson, Muthukumar Bagavathiannan, and Dezhen Song

International Conference on Robotics and Automation (ICRA), 2024
System and Algorithm for Robotic Weed Flaming Using a Quadruped Robot and a 6-Axis Manipulator

Di Wang, Chengsong Hu, Shuangyu Xie, Joe Johnson, Hojun Ji, Yingtao Jiang, Muthukumar Bagavathiannan, and Dezhen Song

International Conference on Intelligent Robots and Systems (IROS), 2024
Dynamic Intrinsic Parameter Rectification Network for Cameras with Optical Image Stabilization in Desktop Augmented Reality Applications

Shuangyu Xie, Di Wang, Shu-Hao Yeh, Wei Yan, and Dezhen Song

IEEE International Conference on Automation Science and Engineering (CASE), 2024
A Pretouch Perception Algorithm for Object Material and Structure Mapping to Assist Grasp and Manipulation Using a DMDSM Sensor

Fengzhi Guo, Shuangyu Xie, Di Wang, Cheng Fang, Jun Zou, and Dezhen Song

International Conference on Intelligent Robots and Systems (IROS), 2023
Algorithm and System Development for Robotic Micro-Volume Herbicide Spray Towards Precision Weed Management

Chengsong Hu, Shuangyu Xie, Dezhen Song, J. Alex Thomasson, Robert G. Hardin IV, and Muthukumar Bagavathiannan

IEEE Robotics and Automation Letters, 2022
Toward Robotic Weed Control: Detection of Nutsedge Weed in Bermudagrass Turf Using Inaccurate and Insufficient Training Data

Shuangyu Xie, Chengsong Hu, Muthukumar Bagavathiannan, and Dezhen Song

IEEE Robotics and Automation Letters, 2021