Year: 2024

Hussain, M., He, L., Schupp, J., Lyons, D. and Heinemann, P. 2023. Green fruit segmentation and orientation estimation for robotic green fruit thinning of apples. Computers and Electronics in Agriculture, 207, p.107734. https://doi.org/10.1016/j.compag.2023.107734

Abstract
Apple is a highly valued specialty crop in the U.S. Green fruit thinning is an important operation of apple production, which is the removal of excess fruitlets in the early summer. The task ensures that remaining fruits at harvest time grow to have good size and quality while reducing the risk of biennial bearing. Current methods of thinning include hand, chemical, and mechanical. However, hand thinning generally requires a large labor force to implement, chemical thinning is non-selective and dependent on timing and weather during application, and mechanical thinning is also non-selective and destructive. A robotic green fruit thinning system could possibly be implemented that does not exhibit the drawbacks of current methods. A vision system is an essential component for a robotic green fruit thinning system that is responsible for green fruit detection and segmentation, decision-making on which fruit to remove, and environment reconstruction for path planning. This study took the first step towards developing a vision system for robotic green fruit thinning. First, green fruit and stem instance segmentation was applied using Mask R-CNN. Then, green fruit and stem orientation estimation was applied using Principal Component Analysis (PCA). Average precision scores for green fruit and stem segmentation on all mask sizes were 83.4% and 38.9%, respectively, whereas these increased to 91.3% and 67.7% if only considering the fruits and stems with mask sizes greater than 322 pixels. Green fruit orientation estimation with correction made 89.3% and 75.5% of estimates accurate within 30° of actual orientations for ground-truth and segmentation-generated masks, respectively. Performances respectively were 97.4% and 84.0% when only unoccluded masks are considered. Orientation correction resulted in considerable improvements in all cases of green fruit orientation estimation, with the greatest improvement seen on unoccluded ground truth masks where estimates accurate within 30° of ground truth orientations increased by 23.9%. Stem orientation estimation achieved very high accuracies with corresponding scores of 99.8% and 99.7%. The outcomes provided guideline information for developing a robust machine vision system for robotic green fruit thinning.

Read full article here: https://www.sciencedirect.com/science/article/abs/pii/S0168169923001229?via%3Dihub

Tian Qiu, Tao Wang, Tao Han, Kaspar Kuehn, Lailiang Cheng, Cheng Meng, Xiangtao Xu, Kenong Xu, and Jiang Yu
Plant Phenomics, 8 May 2024, Vol 6, Article ID: 0179
DOI: 10.34133/plantphenomics.0179

Abstract
The architecture of apple trees plays a pivotal role in shaping their growth and fruit-bearing potential, forming the foundation for precision apple management. Traditionally, 2D imaging technologies were employed to delineate the architectural traits of apple trees, but their accuracy was hampered by occlusion and perspective ambiguities. This study aimed to surmount these constraints by devising a 3D geometry-based processing pipeline for apple tree structure segmentation and architectural trait characterization, utilizing point clouds collected by a terrestrial laser scanner (TLS). The pipeline consisted of four modules: (a) data preprocessing module, (b) tree instance segmentation module, (c) tree structure segmentation module, and (d) architectural trait extraction module. The developed pipeline was used to analyze 84 trees of two representative apple cultivars, characterizing architectural traits such as tree height, trunk diameter, branch count, branch diameter, and branch angle. Experimental results indicated that the established pipeline attained an R2 of 0.92 and 0.83, and a mean absolute error (MAE) of 6.1 cm and 4.71 mm for tree height and trunk diameter at the tree level, respectively. Additionally, at the branch level, it achieved an R2 of 0.77 and 0.69, and a MAE of 6.86 mm and 7.48° for branch diameter and angle, respectively. The accurate measurement of these architectural traits can enable precision management in high-density apple orchards and bolster phenotyping endeavors in breeding programs. Moreover, bottlenecks of 3D tree characterization in general were comprehensively analyzed to reveal future development.

Read full article here: https://spj.science.org/doi/10.34133/plantphenomics.0179

 Yu Jiang1, Tian Qiu2, Terence Robinson1, Lialiang Cheng3, Kaspar Kuehn3, Kenong Xu1 

1Horticulture Section, School of Integrative Plant Science, Cornell AgriTech, Geneva NY | 2School of Electronic and Computer Engineering, Cornell University, Ithaca NY | 3Horticulture Section, School of Integrative Plant Science, Cornell University, Ithaca NY 

The sustainable growth of the apple industry relies on
managing apple trees with optimal architectural traits,
which significantly influence their growth, fruiting potential,
and environmental interactions. For instance, tree height
affects light exposure to lower branches, impacting fruit yield
and quality, while trunk diameter helps determine the ideal crop
load. Accurate assessment of these traits is crucial for maximizing
orchard productivity and fruit quality.

Traditionally, apple tree traits have been measured manually
using tools like tape measures and calipers. However, these
methods are labor-intensive, subjective, and often inadequate for
capturing the complex architecture essential for fruit production.
Visual inspections might miss subtle differences in branch angles
or lengths that affect fruit distribution and overall yield, and the
intricate structure of trees can make it difficult to take accurate
measurements in the field.

Optical sensing technologies, particularly imaging, are becoming
increasingly popular due to their noninvasive, versatile,
and cost-effective nature (Jiang et al., 2020, Jin et al., 2021, Li et
al., 2014). These technologies provide detailed insights into plant architecture and physiology, driving interest in advanced imaging and machine learning (ML) methods for more precise and efficient trait characterization. Leveraging these technologies can overcome the limitations of traditional methods, leading to a better understanding of tree traits and improved orchard management.

Read full article in Fruit Quarterly, Volume 32, Number 3, Fall 2024: https://nyshs.org/wp-content/uploads/2024/11/NYFQ-BOOK-Fall-2024_v3.pdf

Hussain, M., He, L., Schupp, J., Lyons, D. and Heinemann, P. 2024. Green Fruit‐Stem Pairing and Clustering for Machine Vision System in Robotic Thinning of Apples. Journal of Field Robotics. November 2024 https://doi.org/10.1002/rob.2246

ABSTRACT
Apples are one of the most highly-valued specialty crops in the United States. Recent labor shortages have made crop production difficult for fruit growers, including the task of green fruit thinning. Current methods including hand, chemical, and mechanical thinning impose tradeoffs between selectivity, cost, tree damage, and speed. A robotic green fruit thinning system could potentially selectively thin fruit in a quick, cost-effective, and non-damaging manner. The machine vision system would be a critical component for robotic thinning, and would not only need to perform green fruit detection/segmentation, but also fruit-stem pairing and clustering to facilitate proper decision-making for thinning. A neural network-based fruit and stem pairing algorithm was devised and evaluated; an LSTM-based clustering algorithm was devised and compared to the density-based clustering algorithm, OPTICS. The algorithms were evaluated on an image data set consisting of GoldRush, Fuji, and Golden Delicious cultivars at the green fruit stage, with evaluations on overall performance, cultivar-wise performance, cluster size-specific performance, and feature importance. For fruit and stem pairing, the neural network-based pairing algorithm achieved an AP of 81.4% on all fruits and stems, and that reached 90.6% when only fruits and stems with labeled angles were considered. For green fruit clustering, the LSTM-based clustering achieved a clustering success rate of 68.4%, whereas the OPTICS algorithm obtained 50.9%. The algorithms will be further implemented in a pipeline of a future green fruit thinning vision system, as well as integrate the use of point clouds and other 3D orchard information to improve pairing and clustering performance.

Read full article in Journal of Field Robotics: https://doi.org/10.1002/rob.22465

Precision Apple Cropload MANagement (PACMAN) technologies, 2nd Annual WNY Fruit Grower Tour. With Dr. Terence Robinson and Dr. Brian Lawrence, Cornell Agri-Tech. At Orchard Dale Fruit Farm, Waterport, NY. Also with Bobby Brown or Orchard Dale. Featuring Vivid-Machines and Outfield. pacman.extension.org vivid-machines.com outfield.xyz Thanks to the Brown Family at Orchard Dale for hosting us Thanks to the Cornell Cooperative Extension Lake Ontario Fruit Program and Lake Ontario Ag Consulting, LLC for the 2nd Annual WNY Fruit Growers Tour: https://lof.cce.cornell.edu/event.php?id=1915 For more information: pacman.extension.org ©2024 Jon Clements and UMass Amherst Extension

It was late at night and snowing when Jim and I arrived in Rochester, New York, and the air was a balmy 28°F (or -2°C). Fortunately the pineapple themed Airbnb that we had rented was nice and warm!

We had flown out to Rochester at the end of March to meet with Outfield users and research partners in New York, the second largest apple growing state in the union. Rochester sits in the middle of the Lake Ontario fruit growing region, so it was the perfect base for us for our two week visit.

One of the main reasons for visiting New York was to launch our collaboration with Cornell University. Building on successful projects together in 2023, Outfield is partnering with the Digital Agriculture team at the Cornell College of Agriculture and Life Sciences, based in Geneva, NY. We are really excited to work with Professor Terence Robinson and Associate Research Professor Yu Jiang to demo the Outfield system on New York orchards, in particular using Outfield for precision crop load management (PCLM).

Read more here…

 LAURA HILLMANN AND TODD EINHORN, Michigan State University einhornt@msu.edu 

 Fruit set prediction models aim to produce timely estimates of fruitlet abscission after thinner applications to guide precision crop load management. The time to generate a prediction after an application is important to facilitate grower decisions to re-apply thinners while they are still efficacious, avoiding expensive hand thinning operations. The fruitlet growth rate (FGR) model, developed by Dr. Duane Greene, is a powerful tool that can accurately predict the percentage of fruitlets that will set in an orchard. Although a Excel data template and App are available to run the FGR model via computer and smartphone, respectively, adoption has been limited by the measurement-intensive procedure. A new approach, termed the ‘Fruitlet Size Distribution (FSD) Model’, described herein, was developed to produce predictions of apple fruit set comparable to the FGR model but achievable with less time investment. The principle underlying both models is the same: the relative growth rate or size of a fruitlet is compared to the most rapidly growing or largest fruitlet within the sample date to determine if it will abscise. Most predictions can be made within 8 days from thinner applications, though the duration of time depends on climatic, biological and horticultural factors. To optimize the FSD model we suggest beginning the model three days after the average fruitlet diameter of the orchard is 6 mm. Thus, the model partners well with thinning applications between bloom and 6 mm. For example, if, a prediction can be achieved by 8 days, assuming an average growth of ~0.8 mm per day, then fruitlets will be ~ 12 mm if another application is needed; 12 mm fruitlets are very sensitive to many thinning chemistries. 

HOW-TO-GUIDE (2024 update includes new procedure to flag 120 flower cluster between pink and full bloom)

XLSM file for FSD Predict here. Please read HOW-TO-GUIDE above first. Note the XLSM file will need to be downloaded and run locally on your computer. (Click the Download icon in upper right after viewing FS Predict in Dropbox link above.) Macros have to be enabled.

International Fruit Tree Association meeting dives into sensor systems making progress on providing accurate, actionable crop load data. March 15, 2024 Good Fruit Grower

The dream of managing crop load with sensors and smart sprayers moves closer to reality with technological innovation every season. 

But it’s a complex dream, with evolving vision systems, new spray technologies and a different way to make crop load management decisions. 

“This is a really challenging thing,” said Tory Schmidt of the Washington Tree Fruit Research Commission. He led a session during the precision orchard technology workshop that preceded the International Fruit Tree Association’s annual conference in Yakima in February to highlight the companies in the “rapidly evolving landscape” of drone, tractor and smartphone imaging sensors that promise to track crop load at varying stages from bud to bin. 

Read more here…https://www.goodfruit.com/dialing-in-crop-load-data-with-machine-learning-management/

Dr. Gregory Peck and his team at Cornell Univesity of David Zakalik, Michael Brown, and Craig Kahlke recently published several papers on the results of their research on crop load management of cider apples. Some cider apple varieties are notorioulsy biennial bearing which “exacerbates supply chain issues for cidermakers in North America.” Their results suggest that summer applications of plant growth regulators do not promote return bloom or reduce biennial bearing in seven cider apple varieties, however, fruitlet thinning did reduce biennial bearing and improve juice quality. These results should be of interest to all cider apple growers. The full papers can be viewed below…

Fruitlet Thinning Reduces Biennial Bearing in Seven High-tannin Cider Apple Cultivars
David Zakalik, Michael G. Brown, and Gregory M. Peck
https://doi.org/10.21273/HORTSCI17455-23

Fruitlet Thinning Improves Juice Quality in Seven High-tannin Cider Cultivars
David L. Zakalik, Michael G. Brown, and Gregory M. Peck
https://doi.org/10.21273/HORTSCI17096-23

Summer Applications of Plant Growth Regulators, Ethephon And 1-Naphthaleneacetic Acid, Do Not Promote Return Bloom or Reduce Biennial Bearing in Seven High-Tannin Cider Apple Cultivars
David L. Zakalik, Michael G. Brown, Craig J. Kahlke,
and Gregory M. Peck
Journal of the American Pomological Society 77(2): 75-92 2023

Summary:
In January of 2023, a series of 3 virtual nationwide meetings was conducted by the PACMAN Research and Extension team to update growers about the current state of precision crop load management of apples. At the end of February and early March, CCE LOF conducted 3 Ag-tech sessions in person in Rochester, NY, and one statewide virtual session. More than 450 people in total attended the PACMAN virtual meetups and the 4 Ag-tech sessions offered by CCE LOF during the winter months of 2023. CCE LOF carried out one pruning severity study on ‘Honeycrisp’, compared and validated the use of two thinning prediction models (FSDM and the FGRM) on ‘Honeycrisp’ and ‘Gala’ with grower collaborators, and worked with several companies who are developing rovers or drones to count flowers and fruitlets (Pometa, Orchard Robotics, Vivid, and Outfield). In July, CCE LOF conducted a very successful fruit summer tour in Wayne County where several digital technologies were featured to more than 250 tour participants. In the 2023 growing season and in a few more orchards in the Lake Ontario Fruit region and at Cornell AgriTech in Geneva, the research, development, and first adoption of several digital technologies has been taking place to help improve the accuracy and labor efficiency of precision crop load management.

Cornell research and extension team: T.L. Robinson (Cornell AgriTech), L. Gonzalez (Cornell AgriTech), S. Howden (Cornell AgriTech), Kathy Campo (Cornell AgriTech), M. Miranda Sazo (CCE LOF), C. Kahlke (CCE LOF), L. Tee (CCE LOF), and D. Acquilano (CCE LOF).

Full PDF of the report download below…