Language-driven Grasp Detection

An Vuong, Minh Nhat Vu, Baoru Huang, Nghia Nguyen, Hieu Le, Thieu Vo, Anh Nguyen

Publikation: Beitrag in Buch oder TagungsbandVortrag mit Beitrag in TagungsbandBegutachtung

Abstract

Grasp detection is a persistent and intricate challenge with various industrial applications. Recently, many meth-ods and datasets have been proposed to tackle the grasp detection problem. However, most of them do not consider using natural language as a condition to detect the grasp poses. In this paper, we introduce Grasp-Anything++, a new language-driven grasp detection dataset featuring 1M samples, over 3M objects, and upwards of 10M grasping in-structions. We utilize foundation models to create a large-scale scene corpus with corresponding images and grasp prompts. We approach the language-driven grasp detection task as a conditional generation problem. Drawing on the success of diffusion models in generative tasks and given that language plays a vital role in this task, we propose a new language-driven grasp detection method based on dif-fusion models. Our key contribution is the contrastive training objective, which explicitly contributes to the denoising process to detect the grasp pose given the language instructions. We illustrate that our approach is theoretically sup-portive. The intensive experiments show that our method outperforms state-of-the-art approaches and allows real-world robotic grasping. Finally, we demonstrate our large-scale dataset enables zero-short grasp detection and is a challenging benchmark for future work.
OriginalspracheEnglisch
TitelProceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Seiten17902-17912
DOIs
PublikationsstatusVeröffentlicht - 25 Dez. 2024
Veranstaltung2024 Conference on Computer Vision and Pattern Recognition (CVPR) - Seattle, USA/Vereinigte Staaten
Dauer: 16 Juni 202422 Juni 2024

Konferenz

Konferenz2024 Conference on Computer Vision and Pattern Recognition (CVPR)
Land/GebietUSA/Vereinigte Staaten
StadtSeattle
Zeitraum16/06/2422/06/24

Research Field

  • Complex Dynamical Systems

Fingerprint

Untersuchen Sie die Forschungsthemen von „Language-driven Grasp Detection“. Zusammen bilden sie einen einzigartigen Fingerprint.

Diese Publikation zitieren