Decision Automation for Orchards and Vineyards (University of Auckland/Bruce MacDonald led MBIE project).
Our programme of work “MaaraTech” has a goal to reduce the variability of human worker performance in orchards and vineyards, using augmented reality technology to help inexpert workers make expert decisions, and reduce the risk associated with a transient workforce, using robotic automation. We will create new technologies that capture and convert data about plant structure, and the activities of expert humans, into smart decisions and actions in fruit growing e.g which grape vine to prune and how much to remove. We will use Artificial Intelligence to interpret data and then identify and communicate actions to less experienced humans and to automated robots. Horticulture industries suffer from high labour costs and increasing shortages of skilled labour, which reduce productivity and quality of crops. Our new technologies will assist fruit growing operations, and help NZ companies to generate premium horticultural products for the global market. Our research will be co-developed with NZ grower organisations, individual growers, including Māori fruit crop companies, and NZ agricultural machinery manufacturers. We will use apple thinning, grape pruning, and new blueberry variety harvesting as case studies. Our programme will deliver a suite of human-assist and automation technologies demonstrated for specific tasks and adaptable to other fruit growing operations. The use of field data for decision making and integration into operations will transform horticulture into a data-enabled industry – changing the focus from manual labour to transferable expert knowledge and skills.
MaaraTech will contribute to:
- Increasing high-value manufacturing exports (human-assist systems, fully automated systems);
- Increasing high-value fruit crops/wine export receipts (higher yields, improved quality);
- Improvements in domestics high-value fruit production (labour cost reductions)
What is the need?
The orchard/vineyard industries say they need:
• Rapid and accurate decision-making by workers in the field
• Improved human operator consistency
• Skills-transfer to address the shortage of experienced workers
• Automation technology to ease the upward cost pressures
• Greater predictability of the quality and yield higher value fruit.
How will it work?
We will combine four key elements:
• Data: Rich database on the crop, tree and workers over space and time. AI tools to optimise orchard/vineyard decisions and actions
• Sensor Fusion: Suite of machine vision, optical, acoustic and microwave sensors capturing information on the crop and operators
• Human Assist Tools: Data and knowledge gained from AI models to develop tools such as augmeted reality, virtual reality and dexterious tools to guide decision making and guide the actions of worker
• Communities: How the technology affects workers and communities will be investigated. A focus will be placed on Mātauranga Māori, cultural heritage, knowledge and IP in the co-design and adoption of new technologies.
Looking ahead
We will fill a significant gap in global fruit industries by creating a high-quality tech augmented workforce (e.g. pruning, thinning, harvesting).
• New wearable sensors will assess the crop canopy in front of an experienced operator (e.g. branches, buds, fruitlets) and monitor their manual actions (gestures, events, tasks)
• New AI methods to learn actions from sensor data
• We can apply the automation platform to other orchard and vineyard operations
• Being highly transferable and addressing a global need, the technology has significant export potential
• The new knowledge New Zealand gains will increase our science capabilities and put us into the R&D frontier of AI‑based automation for horticulture
• Learnt knowledge will support inexpert staff using interface methods such as AR; reducing operator inconsistency
• We will understand technology adoption in the workforce and how automation affects horticultural communities
• We will advance Human Assist technologies into a control system for automated robots
• We will better understand the constraints around technology adoption and consider how we can support social benefits from the introduction of new AI technologies.
Researchers
