Workshop on Learning for Task and Motion Planning

This workshop will investigate the role of learning within Task and Motion Planning (TAMP). TAMP has shown remarkable capabilities in scaling to long action sequences, many objects and a variety of tasks. However, TAMP often relies on simplified models, assumes perfect world knowledge, and requires long computation time, which limits the real world applicability. To address these limitations, there has been significant recent interest in integrating learning methods into TAMP [ 1Structured deep generative models for sampling on constraint manifolds in sequential manipulation.
Joaquim Ortiz-Haro, Jung-Su Ha, Danny Driess, Marc Toussaint.
CoRL 2022., 2TAPS: Task-Agnostic Policy Sequencing.
Christopher Agia, Toki Migimatsu, Jiajun Wu, Jeannette Bohg.
arXiv 2022., 3Guided skill learning and abstraction for long-horizon manipulation.
Shuo Cheng, Danfei Xu.
CoRL Workshop on Learning, Perception, and Abstraction for Long-Horizon Planning 2022., 4Learning symbolic operators for task and motion planning.
Tom Silver, Rohan Chitnis, Joshua Tenenbaum, Leslie Pack Kaelbling, Tomás Lozano-Pérez.
IROS 2021., 5Sequence-based plan feasibility prediction for efficient task and motion planning.
Zhutian Yang, Caelan Reed Garrett, Leslie Pack Kaelbling, Tomás Lozano-Pérez, and Dieter Fox.
RSS 2023. ]. Despite this progress, there remain many open questions that must be addressed before learning-based TAMP can be applied in real-world settings and in full generality.

The goal of this workshop is to bring together a diverse set of researchers from TAMP, learning for TAMP, computer vision, classical AI planning, and NLP to discuss not only the current state of learning for TAMP, but also the broader state of learning for planning in robotics. Subtopics include: benchmarks, language, perception, skill learning, manipulation, closed-loop TAMP, and policy learning. All of these discussions will take place against the backdrop of recent progress in foundation models.

Discussion Topics

• Benchmarks: What benchmark environments will help the community better measure the progress of TAMP and integrate learning with TAMP?
• Large Language Models for Task Planning: LLM has shown promise in generalizable task planning. What are the potentials and challenges in integrating LLM with TAMP?
• Perception: Perception in TAMP and the latest developments in this area, including 3D computer vision, open-world recognition, and sensor fusion.
• Error Recovery / Closed-Loop TAMP: Techniques and principles that allow TAMP to be more robust and recover from errors and interventions.
• Skill Learning / Manipulation: Skill learning and manipulation for TAMP, including but not limited to deep reinforcement learning and imitation learning. How do we make the learned skills generalizable and composable?
• Model Learning: Techniques and principles for learning discrete or continuous transition models from interaction data.

Schedule

The workshop happened on July 10 in hybrid mode. The in-person location is in Daegu, Republic of Korea.

The recording of the workshop can be accessed on YouTube.

Please use this Latex paper template and submit via Open Review. Review will be single-blind so there’s no need to anonymize your document.