The CropAndWeed Dataset: a Multi-Modal Learning Approach for Efficient Crop and Weed Manipulation

Research output: Chapter in Book or Conference ProceedingsConference Proceedings with Oral Presentationpeer-review


Precision Agriculture and especially the application of automated weed intervention represents an increasingly essential research area, as sustainability and efficiency considerations are becoming more and more relevant. While the potentials of Convolutional Neural Networks for detection, classification and segmentation tasks have successfully been demonstrated in other application areas, this relatively new field currently lacks the required quantity and quality of training data for such a highly data-driven approach. Therefore, we propose a novel large-scale image dataset specializing in the fine-grained identification of 74 relevant crop and weed species with a strong emphasis on data variability. We provide annotations of labeled bounding boxes, semantic masks and stem positions for about 112k instances in more than 8k high-resolution images of both real-world agricultural sites and specifically cultivated outdoor plots of rare weed types. Additionally, each sample is enriched with an extensive set of meta-annotations regarding environmental conditions and recording parameters. We furthermore conduct benchmark experiments for multiple learning tasks on different variants of the dataset to demonstrate its versatility and provide examples of useful mapping schemes for tailoring the annotated data to the requirements of specific applications. In the course of the evaluation, we furthermore demonstrate how incorporating multiple species of weeds into the learning process increases the accuracy of crop detection. Overall, the evaluation clearly demonstrates that our dataset represents an essential step towards overcoming the data gap and promoting further research in the area of Precision Agriculture.
Original languageEnglish
Title of host publicationProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Number of pages10
Publication statusPublished - 2023
EventWinter Conference on Applications of Computer Vision - Waikoloa, Hawaii, United States
Duration: 3 Jan 20237 Jan 2023


ConferenceWinter Conference on Applications of Computer Vision
Country/TerritoryUnited States
Internet address

Research Field

  • Assistive and Autonomous Systems

Cite this