Counting Mangoes with YOLO: From the Original Paper to Modern Ultralytics Models
- Kasturi Murthy
- Jun 10
- 7 min read
1. Introduction
Counting fruits on trees is one of the most deceptively difficult problems in computer vision. Mangoes vary in size, shape, lighting, occlusion, and background clutter. Traditional image‑processing pipelines — thresholding, segmentation, contour detection — break down quickly in real orchards.
Modern deep learning changed this landscape. Among all object‑detection architectures, YOLO (You Only Look Once) [1] stands out for its speed, simplicity, and real‑time performance. In this post, I walk through the evolution of YOLO from the original research paper to the latest Ultralytics [2] implementations and show how I used it to detect and count mangoes on trees and on the ground.
2. From ECG Lead Detection to Fruit Counting: My Annotation Journey
Before working on fruit detection, I had already built a full‑fledged ECG lead‑detection and labeling platform — a clinical‑grade annotation tool designed for biomedical signals. This system handled:
multi‑lead ECG waveform segmentation
lead identification
quality checks
human‑in‑the‑loop correction
structured annotation export
Because the platform was modular and built with a strong annotation engine, I extended it to support image‑based labeling. What began as a medical annotation tool naturally evolved into a general‑purpose annotation system. This made it possible to annotate mango images with the same precision and workflow reliability used in ECG labeling.
At present, the fruit‑annotation module supports a single mango class, but the architecture is already prepared for multi‑class fruit variety and maturity detection, targeting major commercial cultivars like Totapuri, Banganapalli, Alphonso, Mallika, and Neelam.
For a resource-constrained smallholder farmer, moving from a single "mango" box to multi-class phenotyping changes everything. Instead of just noting that a fruit exists, the upcoming module iteration allows labeling specific on-tree ripening stages. By training the model to recognize maturity indicators—such as the swelling of the fruit's "shoulders" relative to the stalk or subtle color gradient shifts—the annotation tool lays the groundwork for optimized, staged harvesting.
3. The Original YOLO Paper (2015): A New Way to Detect Objects
YOLO was introduced by Joseph Redmon et al. in the 2015 paper [1] “You Only Look Once: Unified, Real‑Time Object Detection.” The core idea was radically different from earlier detectors like R‑CNN:
Single neural network predicts bounding boxes and class probabilities in one pass
Grid‑based prediction instead of region proposals
Real‑time speed (45 FPS on a GPU in 2015)
End‑to‑end training instead of multi‑stage pipelines
YOLO reframed object detection as a regression problem, not a classification‑plus‑proposal problem. This made it extremely fast and surprisingly robust. For agricultural use cases like mango detection, this single‑shot architecture is ideal because orchard canopies are dense, overlapping environments with highly erratic natural lighting.
4. YOLO Evolves: v2 → v3 → v4 → v5
After the original paper, YOLO evolved rapidly:
YOLOv2 (2016)
Introduced anchor boxes for better shape precision
Higher resolution training
YOLOv3 (2018)
Multi‑scale detection layers
Stronger small‑object performance, crucial for distant or tiny fruits
YOLOv4 (2020)
CSPDarknet backbone and Mosaic data augmentation
YOLOv5 (2020, Ultralytics)
Rewritten entirely in PyTorch
Hugely popular due to its exceptional out-of-the-box ease of use for engineering practitioners
5. YOLOv8 and YOLOv9: The Modern Ultralytics Era
Ultralytics continued to innovate beyond YOLOv5:
YOLOv8 (2023)
Anchor-free detection head
Better small‑object detection
Native Python API with seamless export choices (ONNX, TensorRT, CoreML)
YOLOv9 (2024)
Programmable Gradient Information (PGI)
Better feature retention during deep backpropagation
For mango detection, YOLOv8 and YOLOv9 offer high accuracy on small, heavily occluded fruits, real‑time inference on edge devices, and clean APIs for raw video processing.
6. Building a Mango‑Counting System with YOLO
I trained YOLOv8m on a custom mango dataset containing:
Mangoes on trees and clusters partially hidden by leaves
Mangoes in different maturity stages
Mangoes on the ground (to track windfall/crop damage)
Training Setup
Model: YOLOv8m
Images: ~700 training, ~200 validation
Epochs: 55
Hardware: NVIDIA RTX 4060
Framework: Ultralytics YOLO
The model outputs raw bounding boxes, localized coordinates, and confidence scores per frame.
7. Beyond Raw Counts: Actionable Yield & Ripening Management
While getting bounding boxes on a screen is an excellent milestone, raw per-frame counts don't translate directly to a farmer's ledger. To make this tool truly impactful for resource-constrained farmers, the raw YOLO output must be processed through a localized agronomic pipeline to solve two critical challenges:
A. Translating Detections into Actual Tonnage (Yield Estimation)
If a farmer walks down an orchard row recording video, a raw YOLO model will double- or triple-count the same mango across consecutive video frames. Furthermore, dense tree canopies hide fruit. We bridge this gap using a lightweight post-processing math stack:
Temporal Tracking: By pairing YOLO with a lightweight multi-object tracker like ByteTRACK, the system assigns a unique, persistent ID to each individual mango, ensuring it is counted exactly once as it passes the lens.
To correct for repeated detections of the same mango across multiple frames, we use a visibility‑normalization factor Vf. This leads to the standard multi‑frame counting formula. This approach is widely used in multi‑frame object counting, ecological capture–recapture models, and drone‑based agricultural analytics.
Volumetric Mass Analysis: Because cultivars like Totapuri exhibit a highly predictable, elongated geometry, the pixel dimensions of the YOLO bounding boxes can be mapped to approximate physical volume. Given the known average density of the fruit, the system can estimate total tonnage weeks before harvest.
B. Optimizing the Market Window (On-Tree Ripening Analysis)
Harvesting an entire orchard indiscriminately leads to massive financial losses. By utilizing the multi-class maturity extensions developed in our annotation journey, the edge model categorizes mangoes on the tree into distinct maturity phases (e.g., Immature-Green, Mature-Green, Breaker).
Selective Staged Harvesting: A farmer can scan rows and direct limited, expensive labor to harvest only the specific branches that have reached the targeted maturity.
Smart Market Routing: Fully mature fruit can be routed immediately to nearby fresh markets for premium pricing, while mature-green fruit can be safely allocated for long-distance transport, ensuring it ripens precisely upon arrival.
8. Scaling the Orchard: An Event-Driven Kafka Architecture
When moving from a single smartphone video to managing data from a fleet of drones, tractor-mounted cameras, or multiple edge devices across hectares of land, a single monolithic script falls short. Running object detection, tracking IDs, calculating mass, and writing to a database simultaneously causes major hardware bottlenecks.
To scale the system reliably under real-world farm constraints, we introduce an Event-Driven Architecture powered by Apache Kafka. By decoupling the ingest, inference, and analytics stages into asynchronous microservices, the system becomes highly fault-tolerant.
The Streaming Pipeline Breakdown using KafKa
1. The Edge Producers (The Sensors)
Field workers carrying smartphones or low-cost cameras mounted to tractors act as Kafka Producers.
Heavy Edge: In fields with poor cellular connectivity, a local edge node (like an INT8 quantized model on a Raspberry Pi 5) runs the YOLO inference locally and streams lightweight JSON payloads (bounding box coordinates, timestamps, confidence scores, and GPS telemetry) to a Kafka topic named mango-detections.
Light Edge: If field hardware must remain cheap and low-power, the devices act as simple video capture units, streaming compressed frames into a raw video-ingest topic for a central farm server to process.
2. The Kafka Broker (The Data Highways)
Kafka organizes incoming streams into persistent, partitionable logs called Topics. This acts as a protective buffer, ensuring no data is dropped even if the network drops connection mid-row.
mango-detections: Receives raw bounding box arrays from the field in real time.
yield-aggregates: Holds finalized, tracked count metrics organized by tree ID or orchard zone.
ripening-alerts: Holds high-priority triggers, such as an orchard sector hitting a 75% "Mature-Green" threshold.
3. Distributed Consumers (The Analytics Engines)
Independent consumer microservices subscribe to the Kafka topics to process data concurrently without interfering with one another:
The Yield & Tracking Consumer: Listens to mango-detections, pipes the coordinates through ByteTRACK to resolve duplicates, applies the occlusion correction multiplier, translates pixels to volumetric mass, and commits the finalized yield telemetry to a time-series database.
The Ripening & Phenotyping Consumer: Concurrently reads the same mango-detections stream, aggregates maturity stages, and flags optimal harvest windows.
The Telemetry Alert Consumer: A lightweight microservice monitoring the stream for anomalies—such as a sudden surge of mangoes detected on the ground—triggering an immediate SMS alert to the farm manager regarding wind damage or over-ripening.
Why Kafka Solves Real Farm Constraints:
Network Fault Tolerance: Agricultural fields are notorious for erratic cellular networks. If connection is lost, Kafka's local buffer retains the messages sequentially. The moment the device regains a signal, Kafka manages the backpressure, catching up seamlessly without crashing the pipeline.
Stream Replayability for AI Iteration: Kafka retains data logs for a set period. If we update our YOLO weights next month to an improved version, we can replay the exact historical data stream through the new consumer model. This allows us to re-evaluate and optimize our yield math using identical real-world conditions.
9. Expanding the Horizon: From Mangoes to Universal Crop Phenotyping
The operational blueprint developed here for Totapuri mangoes isn't a single-use solution; it is a fundamental framework for universal vision-based crop phenotyping. Because the system relies on flexible object-detection and time-series event handling, the exact same YOLO-plus-Kafka pipeline can be repurposed across the wider agricultural ecosystem to serve resource-constrained farmers:
Yield & Productivity Across Cultivars: The geometric volume algorithms can be easily recalibrated for other cash crops—ranging from orchards like citrus, apples, and pomegranates to field vegetables like tomatoes and capsicums.
Early Disease & Pest Detection: By broadening the multi-class annotation engine, the model can look past the fruit and monitor leaf canopies for the earliest visual markers of blight, mildew, rust, or localized insect infestations before they devastate an entire field.
Nutritional Status Monitoring: Edge-AI can track subtle chlorosis (yellowing) patterns or leaf curling gradients, giving farmers real-time indicators of nitrogen, phosphorus, or potassium deficiencies without waiting for slow, expensive soil lab turnarounds.
10. Conclusion: Bootstrapped Innovation for the Grassroots
Developing high-impact agricultural tech comes with its own unique set of boundaries. At Intuitus Research and Innovation Services, we are developing this work under a lean, bootstrapped model. Being resource-constrained ourselves gives us a sharp, realistic perspective: we understand that the end-user—the smallholder farmer—cannot rely on expensive enterprise infrastructure or continuous high-bandwidth cloud access.
By prioritizing INT8 model quantization, low-power edge nodes like the Raspberry Pi, and robust offline event stream architectures via Kafka, we prove that precision agriculture doesn't require deep corporate pockets. This framework moves AI out of elite academic labs and transforms it into a practical, democratic, and highly localized tool—built by a lean team, designed precisely to shield vulnerable farmers against crop waste, market exploitation, and financial uncertainty.
Crucially, this approach also brings AI capability building directly to the grassroots level, shifting the center of gravity away from just major cities or urban tech hubs. By deploying, maintaining, and calibrating these intelligent edge nodes right in the fields, we demystify deep learning for rural communities. It creates an environment where local agricultural students, field technicians, and farmers themselves become active stakeholders in data collection, model validation, and edge deployment—fostering high-tech engineering literacy where it matters most.
References
References
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2015). You Only Look Once: Unified, Real-Time Object Detection. ArXiv. https://arxiv.org/abs/1506.02640
Comments