Hyperspectral imaging with applications in the agro-food industry.

The quality of fruit and vegetables is very important to producers, retailers and consumers. The consumer decides, based on the observed quality of the product, whether or not to buy the product. Next to the flavour and the texture of the fruit, also the appearance is a critical quality parameter. Nowadays, inspection of this appearance is achieved through visual inspection, although it is subjective, not very precise and prone to human errors. Therefore, an automation of this quality inspection would be beneficial. A promising technique for this automation is hyperspectral imaging, enabling the measurement of spectral information in all the pixels in an image.

Although many researchers have demonstrated the added value of hyperspectral imaging for quality inspection of agrofood products, this technology was still rarely used in the agrofood industry at the start of this PhD research. One of the reasons was the acquisition time of spectral hypercubes which was typically high. However, this acquisition speed is considerably improved by companies like IMEC, that recently developed hyperspectral cameras with a high acquisition speed. The main reason was the large added v needed to build calibration models and the limited added value to justify the price. Therefore, in this PhD research, the focus was on improving the flexibility of hyperspectral imaging and, as the colour and colour distribution of fruit or vegetables is an important parameter to determine its quality and ripeness, on the development of a method to determine the colour of tomatoes in every pixel. In this way, a fast and contact-free method could be obtained. These concepts were elaborated for vine tomatoes, which were used as a model case containing different challenges, as they have a curved and very glossy surface and vine tomatoes consist of different components, which can vary in quality considerably.

First, a semi-supervised algorithm was developed to segment hyperspectral images. It consists of three steps. First, several unsupervised algorithms are tested to split a spectral hypercube in a predefined number of comparable classes. By an operator, based on a visual inspection of the segmentation, the best split is determined. Based on this selection, from each class, a number of pixels is selected using a selection algorithm, which are used as input to develop a supervised segmentation model. This supervised model is used to segment newly measured spectral hypercubes of the same cultivar. Spectra of badly segmented spectral hypercubes are then used to augment the training set and to obtain a more robust model. After training with 10 new spectral hypercubes, the best result was achieved using a Partial Least Squares – Discriminant Analysis (PLS-DA), using a first derivative preprocessing. Applied on 5 additional spectral hypercubes, an overall accuracy on pixel level of 96.95 % for vine tomatoes and 98.52 % for table grapes was achieved. After the initial segmentation of spectral hypercubes in its present classes, the quality of fruit and vegetables can be determined.

Another important quality parameter of vine tomatoes is the ripeness. Ripeness of tomatoes is linked to the colour, as during ripening the concentration of chlorophyll decreases while the concentration of lycopene increases, which results in a colour change from green to red. To measure the average colour and the variability present in each tomato, the colour of each pixel in the segmented tomatoes was determined. Therefore, two different methods are compared. The first method is by using the calculations developed by CIE to determine the L*a*b*-values. This method was suited for flat, matte samples, but in the case of vine tomatoes, which are curved and very glossy, the results were not accurate. Therefore, a databased method is presented. This method was suited to determine the hue-angle (R² = 0.95), the a*-value (R² = 0.93) and the L*-value (R² = 0.86) accurately. However, the disadvantage of this method is that it can only be used to determine the colour of comparable tomatoes.

Next, it was investigated if the colour of a batch of tomatoes during storage could be predicted based on a measurement shortly after harvest. First, a model was built to describe the variability of a batch of tomatoes during storage. Therefore, mixed effects modelling was used. To describe the colour evolution accurately, 2 random effects were needed. As an analytical solution in this case is very difficult, a data-driven approach was developed. By using the algorithm to determine the hue of tomatoes, the hue in every pixel of the tomato in the image could be accurately determined. Next, a distribution was fitted over all the results of each tomato. The mean and skewness of this distribution was determined and used, together with the time after harvest and the time at which the colour should be known, as input in a multiple linear regression. This made it possible to predict the hue of tomatoes at a certain moment in the near future, until 10 days after harvest, with an R² of more than 0.80 and a RMSE of less than 9°.

High quality products need to be free of defects. As a large variability in surface defects can occur, training a method to detect all possible defects is very difficult. Therefore, an algorithm was developed to detect defects on tomatoes by training a model to know good quality tomatoes. Defects are detected as outlying from this good quality. Resulting from an analysis of the spectra of the different tomatoes, the wavelength range between 700 nm and 985 nm was selected and an area-normalization was used as preprocessing. Then, the preprocessed spectra of each individual tomato was used as input of a Principal Component Analysis (PCA), so defective areas could be detected based on a combination of Hotelling’s T²-values and Q-residuals, evaluated by comparing a local difference against a threshold value. After optimisation of this threshold, the algorithm is able to detect puncture damage, but the detection of bruises and cutting damage is more difficult. It could be observed that specular reflections had a large influence on the detection result. By using a cross-polarized hyperspectral setup, it was possible to reduce this effect, reducing the numbers of false-positive detections.

The algorithms described above are an important step towards a more easy, intuitive application of hyperspectral imaging. To improve the results achieved by the defect detection algorithm, the initial segmentation should be optimised. It is important that the number of misclassifications resulting from the semi-supervised segmentation algorithm is as low as possible, as these are an important source of misclassifications during the defect detection. Next, methods which are able to account for the curvature of the products and the presence of specular reflections should be developed, as they lead to low accuracies achieved when measuring the colour using the equations described by CIE. A possible solution that should be investigated, is the application of the cross-polarised illumination to reduce the effect of specular reflections on the colour calculations. To ensure that the developed algorithms are robust, they should also be tested on different products, like grapes or cranberries.