Lightweight Language-driven Grasp Detection using Conditional Consistency Model

Nghia Nguyen, Minh Nhat Vu, Baoru Huang, An Vuong, Ngan Le, Thieu Vo, Anh Nguyen

Publikation: Beitrag in Buch oder TagungsbandVortrag mit Beitrag in TagungsbandBegutachtung

Abstract

Language-driven grasp detection is a fundamental yet challenging task in robotics with various industrial applications. This work presents a new approach for language-driven grasp detection that leverages lightweight diffusion models to achieve fast inference time. By integrating diffusion processes with grasping prompts in natural language, our method can effectively encode visual and textual information, enabling more accurate and versatile grasp positioning that aligns well with the text query. To overcome the long inference time problem in diffusion models, we leverage the image and text features as the condition in the consistency model to reduce the number of denoising timesteps during inference. The intensive experimental results show that our method outperforms other recent grasp detection methods and lightweight diffusion models by a clear margin. We further validate our method in real-world robotic experiments to demonstrate its fast inference time capability.
OriginalspracheDeutsch
TitelProceedings of the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Seiten13719-13725
DOIs
PublikationsstatusVeröffentlicht - 25 Dez. 2024
Veranstaltung2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) - Abu Dhabi, Vereinigte Arabische Emirate
Dauer: 14 Okt. 202418 Okt. 2024

Konferenz

Konferenz2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Land/GebietVereinigte Arabische Emirate
StadtAbu Dhabi
Zeitraum14/10/2418/10/24

Research Field

  • Complex Dynamical Systems

Diese Publikation zitieren