Improving blueberry firmness classification with spectral and textural features of microstructures using hyperspectral microscope imaging and deep learning

Authors: Park, Bosoon; Shin, Tae-Sung; CHO, JEONG-SEOK - Korea Food Research Institute; LIM, JEONG-HO - Korea Food Research Institute; PARK, KI-JAE - Korea Food Research Institute

Submitted to: Postharvest Biology and Technology
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 10/10/2022
Publication Date: 10/20/2022
Citation: Park, B., Shin, T., Cho, J., Lim, J., Park, K. 2022. Improving blueberry firmness classification with spectral and textural features of microstructures using hyperspectral microscope imaging and deep learning. Postharvest Biology and Technology. https://doi.org/10.1016/j.postharvbio.2022.112154.
DOI: https://doi.org/10.1016/j.postharvbio.2022.112154

Interpretive Summary: Postharvest blueberry softening is an important issue in shelf-life quality control. Although blueberries consumption with health benefits has kept increasing, retailers must reject a large volume of blueberries with firmness lower than their standards due to being highly perishable. Firmness measurement is essential to evaluate blueberry quality but has been onerous because of the lack of standard procedure. A traditional firmness measurement is sensory evaluation, in which berry is rolled with the examiner's fingers to evaluate its firmness on a given scale. For objective evaluation, several instrumental procedures are available; however, it is challenging to obtain consistent measurement results with current methods. Blueberry softening has been explained with microstructural changes using light microscopy and cold-stage scanning electron microscopy, implying a strong correlation between blueberry firmness and spectral-spatial features of the microstructures. Recently, the spectral-spatial characteristics of blueberry parenchyma cells were examined using hyperspectral microscope imaging (HMI). In this study, we investigated the textural features of blueberry parenchyma cells to classify blueberry firmness with the hyperspectral image at the cell level. The goal of this study was to enhance HMI methods to understand blueberry softening with texture features. Specifically, we analyzed hyperspectral images of blueberry parenchyma cells to uncover the textural relationship between blueberry softening and microstructural changes and developed classification models with deep learning methods using spectral and textural features of blueberry.

Technical Abstract: Postharvest blueberry softening has hindered shelf-life quality control of blueberries. This study associated the softening with microstructural changes of blueberry parenchyma cells based on spectral and textural features with hyperspectral microscope imaging and deep learning methods. More specifically, we examined textural features from grey-level co-occurrence matrix (GLCM) computed with each hypercube of the blueberry cell images. We extracted GLCMs and their eight features with nine different distances and four orientations from three input image types, including single-band images of 530 nm, 680 nm, and pairing double-band images. With the GLCM features of mean, variance, homogeneity, contrast, dissimilarity, entropy, energy, and correlation, parenchyma cell textures were characterized visually and statistically over different firmness. To exploit the characterized textural features, Fusion-Nets combining 1D-CNN for spectra and ResNet50 for GLCMs with different distances and image types were evaluated. According to the results of textural characterization, contrast, entropy, variance, dissimilarity, homogeneity, and energy features were significantly different (p < 0.1) over two firmness categories, soft (1.96N-3.92N) and firm (3.92N-9.81N) in shear force, for average GLCMs with a distance greater than 32 pixels regardless input image types. A Fusion-Net with average spectra of cell walls and textural features of GLCMs with 256-pixel distance from 530 nm band images distinguished the firmness categories with 90% accuracy and 78.7% Matthew's correlation coefficient, implying textural changes in microstructures highly correlated with blueberry firmness.