Results from a new stratified evaluation of wheat head detection models reveal that ranking models according to their overall mean performance does not provide the complete picture of their reliability in field use. Nine models trained on the Global Wheat Head Dataset 2021 (GWHD) were retested by country, and the results showed clear performance differences between geographic domains, even when the overall scores were similar.
The analysis, published by the user Saumya Saksena under the name dronefreak on the Hugging Face blog on August 11, 2026, follows up on an earlier open-source release that included nine object detection models: YOLOv8, YOLOv11, YOLOv26, and RF-DETR, with versions ranging from nano to x-large. The models were trained and tuned using GWHD 2021, a dataset created from field images of wheat heads collected across six countries and 18 research institutions to diversify genetic backgrounds, growth stages, and imaging conditions.
Separate Testing for Each Country
The analysis relied on a custom manifest for each image, linking test-set images to country and growth stage. The author explained that this linkage was not part of the original GWHD release, but was created for this analysis from domain metadata. The same evaluation engine used for the overall results was also run again: model.val() from Ultralytics for the YOLO family, and MeanAveragePrecision from the supervision library for RF-DETR models.
Of the 1,382 images in the test set, the country of one image could not be identified because of a filename-duplication issue in the source data; the image was therefore excluded rather than assigned speculatively. The comparisons were based on 1,381 images distributed across the United States with 605 images, Australia with 281, Mexico with 205, China with 200, Japan with 60, and Sudan with 30 images.
China Leads All Models in Performance
China achieved the highest mAP@50 result for all nine models without exception, with scores there ranging from 79.78% for RF-DETR Nano to 92.36% for YOLOv11x. The analysis attributes this to the Chinese domain being one of the largest and most visually consistent domains in the dataset, comprising two institutions and 200 images. Within-domain homogeneity may make evaluation there easier, regardless of the information the model learned from the other countries.
In the previous overall evaluation, YOLOv11x led with 74.25% in mAP@50 and 34.92% in mAP@50:95, with a precision of 83.37%. YOLOv26s provided the best efficiency-performance tradeoff, achieving 70.49% in mAP@50 at 22.8 GFLOPs and 10 million parameters—less than four points behind the top model and using approximately 8.6 times fewer operations than YOLOv11x, which reached 196.0 GFLOPs.
Areas of Weakness Differ Between Model Families
The comparison showed that four of the six YOLO versions—YOLOv26s, YOLOv8m, YOLOv8s, and YOLOv8n—were weakest on the American image set. The United States represents 43.8% of the test images, meaning that the overall score of these models is heavily influenced by their performance in the region they handle worst, rather than necessarily by performance in a typical country.
YOLOv11x was an exception, recording its weakest performance in Australia, while YOLOv26m performed worst in Japan. On the other hand, Australia was the area of weakness for all three RF-DETR versions. The differences between the best and worst countries ranged from 22.79 points for YOLOv26m to 44.38 points for RF-DETR Nano, according to the values reported in the analysis.
Raw Accuracy Does Not Equal Robustness Across Domains
YOLOv26m recorded the narrowest difference between countries, at 22.8 points, from 88.99% in China to 66.21% in Japan, even though it was not the model with the highest overall score. YOLOv11x was close behind with a difference of 23.1 points, indicating better consistency across domains than the other models.
By contrast, RF-DETR Nano achieved the widest difference, at 44.4 points, moving from 79.8% in China to 35.4% in Australia. This difference is larger than the range of variation between the models themselves in China, where the results ranged from 79.8% to 92.4%. For this particular model, the source of deployment images may have a greater effect on the result than choosing another model from the group.
The example of YOLOv26s and YOLOv8m illustrates the importance of this measurement: their overall results were similar—70.49% and 69.55% in mAP@50, a difference of less than one point—but the difference between their best and worst countries varied by more than three points, reaching 25.3 points for the former and 28.6 for the latter. According to the analysis, a single row of overall evaluation results cannot show this difference in reliability.
Comparison Limitations and the Next Step
The author cautions against interpreting the results for Sudan and Japan with the same confidence used for the United States or Australia, because their groups are small and a handful of easy or difficult images may affect mAP more substantially. The current evaluation is also conducted at the country level rather than the genetic-background level; the manifest used includes country, institution, and growth stage, but does not include genetic-background labels.
The next step includes an evaluation split by growth stage, after standardizing a difference in the spelling of one label between “Post-flowering” and “Post-Flowering,” which is likely a formatting difference related to letter case. Model cards now include a section on performance by country, along with the files country_breakdown.json and domain_metadata.json in the dataset repository. The author emphasized that the release and analysis are an independent, unofficial evaluation based on the work of the original dataset authors.