Annotation-free tomato disease diagnosis using a hybrid vision–LLM framework

Authors: MANJUNATHA, HARSHITHA - University Of Texas At Tyler; SUNDARAVADIVEL, PRABHA - University Of Texas At Tyler; TAMIL, LAKSHMAN - University Of Texas At Tyler; Ling, Kai Shu

Submitted to: Meeting Abstract
Publication Type: Abstract Only
Publication Acceptance Date: 3/5/2026
Publication Date: 4/30/2026
Citation: Manjunatha, H., Sundaravadivel, P., Tamil, L., Ling, K. 2026. Annotation-free tomato disease diagnosis using a hybrid vision–LLM framework. Meeting Abstract. https://units.cals.ncsu.edu/2026-ai-ag-conference/.

Interpretive Summary: N/A

Technical Abstract: Plant diseases pose a persistent threat to agricultural productivity, particularly for high-value crops such as tomato (Solanum lycopersicum), where delayed or inaccurate diagnosis can lead to significant yield and economic losses. While deep learning–based disease detection approaches have demonstrated strong performance, their reliance on large, expertly annotated datasets limits scalability and practical deployment in real-world agricultural environments. The purpose of this work is to develop a scalable, annotation-free diagnostic framework that combines unsupervised visual analysis with language-based reasoning to enable robust tomato disease identification, severity assessment, and decision support without dependence on labeled training data. The proposed framework integrates unsupervised computer vision techniques with a multi-stage large language model (LLM) reasoning pipeline. Initially, plant regions are isolated from complex field backgrounds using an unsupervised segmentation strategy that combines focus-based masking with color-space clustering. This approach exploits sharpness cues and plant-specific color characteristics to extract the target foliage without requiring pre-trained models or annotated masks. Once segmented, disease symptoms are quantified through chromatic and textural anomaly analysis. Deviations in color and surface texture are measured relative to healthy plant regions and fused into a composite disease heatmap that spatially localizes and quantifies pathological areas. These visual and quantitative representations are then provided to a multimodal vision–language model, which performs disease inference and severity estimation. A secondary text-based LLM interprets the diagnostic outputs to generate actionable treatment recommendations, including immediate mitigation strategies and longer-term disease management guidance. Evaluation using real-world tomato imagery demonstrates that the proposed framework accurately identifies clear disease cases, including tomato yellow leaf curl virus and tomato brown rugose fruit virus, without reliance on labeled training datasets. The system effectively localizes symptomatic regions and estimates disease severity while maintaining robustness across varying visual conditions. In ambiguous scenarios where symptoms overlap or are weakly expressed, the framework exhibits calibrated uncertainty, avoiding overconfident predictions and appropriately flagging cases for closer inspection. These results highlight the effectiveness of combining unsupervised visual cues with language-based reasoning for practical disease diagnosis. By eliminating the need for expert annotation and integrating interpretable LLM-driven reasoning, this work addresses key barriers to scalable plant disease diagnostics. The hybrid framework enables explainable, data-efficient, and deployable disease monitoring suitable for mobile, edge, and field-level applications in precision agriculture. Its ability to provide both visual localization and natural language decision support bridges the gap between raw sensor data and actionable agronomic insights. The proposed approach offers a practical pathway for extending intelligent plant health monitoring to new crops, environments, and disease types, supporting broader adoption of AI-driven decision support systems in agriculture.