Month: December 2024

Hussain, M., He, L., Schupp, J., Lyons, D. and Heinemann, P. 2023. Green fruit segmentation and orientation estimation for robotic green fruit thinning of apples. Computers and Electronics in Agriculture, 207, p.107734. https://doi.org/10.1016/j.compag.2023.107734

Abstract
Apple is a highly valued specialty crop in the U.S. Green fruit thinning is an important operation of apple production, which is the removal of excess fruitlets in the early summer. The task ensures that remaining fruits at harvest time grow to have good size and quality while reducing the risk of biennial bearing. Current methods of thinning include hand, chemical, and mechanical. However, hand thinning generally requires a large labor force to implement, chemical thinning is non-selective and dependent on timing and weather during application, and mechanical thinning is also non-selective and destructive. A robotic green fruit thinning system could possibly be implemented that does not exhibit the drawbacks of current methods. A vision system is an essential component for a robotic green fruit thinning system that is responsible for green fruit detection and segmentation, decision-making on which fruit to remove, and environment reconstruction for path planning. This study took the first step towards developing a vision system for robotic green fruit thinning. First, green fruit and stem instance segmentation was applied using Mask R-CNN. Then, green fruit and stem orientation estimation was applied using Principal Component Analysis (PCA). Average precision scores for green fruit and stem segmentation on all mask sizes were 83.4% and 38.9%, respectively, whereas these increased to 91.3% and 67.7% if only considering the fruits and stems with mask sizes greater than 322 pixels. Green fruit orientation estimation with correction made 89.3% and 75.5% of estimates accurate within 30° of actual orientations for ground-truth and segmentation-generated masks, respectively. Performances respectively were 97.4% and 84.0% when only unoccluded masks are considered. Orientation correction resulted in considerable improvements in all cases of green fruit orientation estimation, with the greatest improvement seen on unoccluded ground truth masks where estimates accurate within 30° of ground truth orientations increased by 23.9%. Stem orientation estimation achieved very high accuracies with corresponding scores of 99.8% and 99.7%. The outcomes provided guideline information for developing a robust machine vision system for robotic green fruit thinning.

Read full article here: https://www.sciencedirect.com/science/article/abs/pii/S0168169923001229?via%3Dihub

Tian Qiu, Tao Wang, Tao Han, Kaspar Kuehn, Lailiang Cheng, Cheng Meng, Xiangtao Xu, Kenong Xu, and Jiang Yu
Plant Phenomics, 8 May 2024, Vol 6, Article ID: 0179
DOI: 10.34133/plantphenomics.0179

Abstract
The architecture of apple trees plays a pivotal role in shaping their growth and fruit-bearing potential, forming the foundation for precision apple management. Traditionally, 2D imaging technologies were employed to delineate the architectural traits of apple trees, but their accuracy was hampered by occlusion and perspective ambiguities. This study aimed to surmount these constraints by devising a 3D geometry-based processing pipeline for apple tree structure segmentation and architectural trait characterization, utilizing point clouds collected by a terrestrial laser scanner (TLS). The pipeline consisted of four modules: (a) data preprocessing module, (b) tree instance segmentation module, (c) tree structure segmentation module, and (d) architectural trait extraction module. The developed pipeline was used to analyze 84 trees of two representative apple cultivars, characterizing architectural traits such as tree height, trunk diameter, branch count, branch diameter, and branch angle. Experimental results indicated that the established pipeline attained an R2 of 0.92 and 0.83, and a mean absolute error (MAE) of 6.1 cm and 4.71 mm for tree height and trunk diameter at the tree level, respectively. Additionally, at the branch level, it achieved an R2 of 0.77 and 0.69, and a MAE of 6.86 mm and 7.48° for branch diameter and angle, respectively. The accurate measurement of these architectural traits can enable precision management in high-density apple orchards and bolster phenotyping endeavors in breeding programs. Moreover, bottlenecks of 3D tree characterization in general were comprehensively analyzed to reveal future development.

Read full article here: https://spj.science.org/doi/10.34133/plantphenomics.0179

 Yu Jiang1, Tian Qiu2, Terence Robinson1, Lialiang Cheng3, Kaspar Kuehn3, Kenong Xu1 

1Horticulture Section, School of Integrative Plant Science, Cornell AgriTech, Geneva NY | 2School of Electronic and Computer Engineering, Cornell University, Ithaca NY | 3Horticulture Section, School of Integrative Plant Science, Cornell University, Ithaca NY 

The sustainable growth of the apple industry relies on
managing apple trees with optimal architectural traits,
which significantly influence their growth, fruiting potential,
and environmental interactions. For instance, tree height
affects light exposure to lower branches, impacting fruit yield
and quality, while trunk diameter helps determine the ideal crop
load. Accurate assessment of these traits is crucial for maximizing
orchard productivity and fruit quality.

Traditionally, apple tree traits have been measured manually
using tools like tape measures and calipers. However, these
methods are labor-intensive, subjective, and often inadequate for
capturing the complex architecture essential for fruit production.
Visual inspections might miss subtle differences in branch angles
or lengths that affect fruit distribution and overall yield, and the
intricate structure of trees can make it difficult to take accurate
measurements in the field.

Optical sensing technologies, particularly imaging, are becoming
increasingly popular due to their noninvasive, versatile,
and cost-effective nature (Jiang et al., 2020, Jin et al., 2021, Li et
al., 2014). These technologies provide detailed insights into plant architecture and physiology, driving interest in advanced imaging and machine learning (ML) methods for more precise and efficient trait characterization. Leveraging these technologies can overcome the limitations of traditional methods, leading to a better understanding of tree traits and improved orchard management.

Read full article in Fruit Quarterly, Volume 32, Number 3, Fall 2024: https://nyshs.org/wp-content/uploads/2024/11/NYFQ-BOOK-Fall-2024_v3.pdf

Hussain, M., He, L., Schupp, J., Lyons, D. and Heinemann, P. 2024. Green Fruit‐Stem Pairing and Clustering for Machine Vision System in Robotic Thinning of Apples. Journal of Field Robotics. November 2024 https://doi.org/10.1002/rob.2246

ABSTRACT
Apples are one of the most highly-valued specialty crops in the United States. Recent labor shortages have made crop production difficult for fruit growers, including the task of green fruit thinning. Current methods including hand, chemical, and mechanical thinning impose tradeoffs between selectivity, cost, tree damage, and speed. A robotic green fruit thinning system could potentially selectively thin fruit in a quick, cost-effective, and non-damaging manner. The machine vision system would be a critical component for robotic thinning, and would not only need to perform green fruit detection/segmentation, but also fruit-stem pairing and clustering to facilitate proper decision-making for thinning. A neural network-based fruit and stem pairing algorithm was devised and evaluated; an LSTM-based clustering algorithm was devised and compared to the density-based clustering algorithm, OPTICS. The algorithms were evaluated on an image data set consisting of GoldRush, Fuji, and Golden Delicious cultivars at the green fruit stage, with evaluations on overall performance, cultivar-wise performance, cluster size-specific performance, and feature importance. For fruit and stem pairing, the neural network-based pairing algorithm achieved an AP of 81.4% on all fruits and stems, and that reached 90.6% when only fruits and stems with labeled angles were considered. For green fruit clustering, the LSTM-based clustering achieved a clustering success rate of 68.4%, whereas the OPTICS algorithm obtained 50.9%. The algorithms will be further implemented in a pipeline of a future green fruit thinning vision system, as well as integrate the use of point clouds and other 3D orchard information to improve pairing and clustering performance.

Read full article in Journal of Field Robotics: https://doi.org/10.1002/rob.22465