Awarded work “Advanced deep learning for harvest robotics” presented at EMVA Business Conference in Seville. Next conference 2024 will take place in Gdansk.
Seville, Spain; May 8th, 2023. The EMVA Young Professional Award 2023 goes to Dr. Pieter Blok for his work “Advanced deep learning for harvest robotics”. The awardee was announced on 05 May during the 21th EMVA Business Conference in Seville/Spain, where he also had the opportunity to present his work as part of the regular conference program. Pieter Blok is a researcher Deep Learning and Computer Vision at Wageningen University and Research in the Netherlands. Pieter holds a Bachelor and Master of Sciences in Agricultural Engineering from Wageningen University. In December 2022, he obtained his PhD degree at Wageningen University with the thesis entitled “Perception models for selective harvesting robots in fruit and vegetable production”. This PhD thesis was awarded the distinction cum laude. From July 1st, 2023, Pieter will continue his scientific career as an assistant professor at the Laboratory of Field Phenomics at the University of Tokyo in Japan. In Tokyo, Pieter will focus on machine learning and image processing technologies for plant phenotyping.
Awarded Work: Advanced deep learning for harvest robotics
Manual harvesting of fruits and vegetables is a labor-intensive task that suffers from the current shortage of labor in agriculture. To prevent this labor shortage from leading to a reduced supply of fruits and vegetables, robotic alternatives are currently being sought. For a robot to successfully harvest a crop, the fruits and vegetables must be detected using computer vision methods. Unfortunately, to date, most computer vision methods are unable to perform generically when deployed in different fields. This is unfavorable for the commercial success of these robots.
The goal of Pieter Blok’s PhD thesis was to research and develop new machine vision methods that can help a harvesting robot to deal with the mentioned variations and uncertainties. The focus of the PhD thesis was on three tasks that must be performed by every harvesting robot: crop detection, crop size estimation, and crop quality determination.
In the context of crop detection, Blok’s thesis focused on improving the generalization performance of convolutional neural networks (CNNs) to deal with variations within the same crop. Usually, there are many variations within a crop, which can cause the trained CNN to not be able to generalize sufficiently. Therefore, Blok’s research focused on applying different types of data augmentation to optimize the training of a CNN. With geometric data augmentation (rotation, cropping, and scaling of the image), it was demonstrated that the CNN could better generalize to multiple crop varieties.
Another research topic in Blok’s thesis focused on improving the size estimation of crops. Size estimation is important for a harvesting robot, as it determines whether a crop should be harvested or left in the field to grow further. A challenge, however, when estimating the size of a crop is that crops can be occluded by leaves, which reduces the visibility of the crop. Blok introduced a new perception method to alleviate this problem. His novel method used amodal perception, which is the ability to predict both the visible and occluded parts of objects in an image. By integrating this into a CNN, the larger amodal shape of occluded crops could be accurately estimated for three-dimensional crop size estimation and robot positioning.
The third research topic in Blok’s PhD thesis focused on crop quality determination. Specifically, it focused on using active learning to automatically select and annotate sporadically occurring plant diseases. The newly developed active learning method used the output of the CNN to select unlabeled images about which the network was most uncertain. These selected images were then interactively labelled in a semi-supervised way and used to retrain the network. This active learning method significantly reduced the annotation effort by 1400 image annotations, respectively 120 annotation hours.
The application of Blok’s deep learning technologies to five harvesting robots has enabled the successful commercialization of these robots, thus achieving a unique technology transfer from science to industry.
About the EMVA Young Professional Award
The EMVA Young Professional Award is an annual award to honor the outstanding and innovative work of a student or a young professional in the field of machine vision or image processing. It is the goal of the European Machine Vision Association EMVA to further support innovation in the machine vision industry, to contribute to the important aspect of dedicated machine vision education and to provide a bridge between research and industry. With the annual Young Professional Award the EMVA intends to specifically encourage students to focus on challenges in the field of machine vision and to apply latest research results and findings in computer vision to the practical needs of the industry. The Award winner is presented during the EMVA Business Conference.
The next EMVA Business Conference will take place in Gdansk
Traditionally, at the end of the conference it was announced in which city the next, then 22nd EMVA Business Conference will take place. The machine vision industry will meet June 13 – 15, 2024 in Gdansk/Poland.
Photo: EMVA Young Professional Award Winner Dr. Pieter Blok (left), EMVA President Dr. Chris Yates; Picture source: EMVA