English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Introduction
The quality of food products is very important for the human health. The large population and the increased requirements of food products makes it difficult to arrive the desired quality. For example, sorting tons of fruits and vegetables manually is a slow, costly, and an inaccurate process. Hence food quality evaluation plays an important role in providing defect free food products to the consumers. Quality which defines the internal and external characteristics of the materials. In food quality the external characteristics depends on morphological (includes shape and size), colour, and texture, respectively. In food processing industries, the food products are continuously over the sieves such that hundreds of food products are scanned in fraction of second. For instance, CCD cameras are used to monitor the movement of the food products and finally the defected materials are thrown away from the sieves.
For several years, the food industry has adopted automated vision-based inspection systems in an attempt to reduce operation costs and increase product quality control (Mai, Chetima & Pierre, 2012). In fact, nondestructive detections, like photoelectric detection, the electromagnetic characteristics analysis, Near Infrared Spectroscopy, X-ray analysis, computer vision and so forth, have been used increasingly in the food and agricultural industry for inspection and evaluation purposes as they provide suitably rapid, economic, consistent and objective assessment (Jing-Jin, Guiping, Xiaojuan & Viray, 2009; Narendra & Hareesh, 2010). The potential of computer vision in the food industry has long been recognized and the food industry is now ranked among the top 10 industries using this technology (Tadhg & Da-Wen, 2004). Vision-based inspection systems reduce human interaction with the inspected goods, classify generally faster than human beings, and tend to be more consistent in their product classification (Mai, Chetima & Pierre, 2012; Novaković, Strbac & Bulatović, 2011). Many vision systems have been developed for different food products inspection, such as apples, tomatoes, potatoes, vegetables, eggs, corn, rice, and many other products (Jing-Jin, Guiping, Xiaojuan & Viray, 2009; Tadhg & Da-Wen, 2004). More recently, Velappan, Prakash & Sada (2012), developed an Apple grading system, using vision box hardware with the advantages of high precision and high automatization (White, Svellingen & Strachan, 2006). Therefore, Yeh, Hamey, Westcott & Sung (1995), used Kohonen’s self-organizing map for identification of baking curves in baked goods.
Given these concerns, morphological, color and texture features are the primary information sources for foods and agricultural commodity (i.e. object) inspection, classification, and sorting or grading (Du & Sun, 2004). Computer vision systems have been successfully used to recognize or to classify quality parameters like color and size in several agricultural and food commodities including dry beans (Mahesh, Ganesh & Dongqing, 2013), pistachios (Hanbury, 2002), coffee (Deddy, Usman, Kudang & Dewa, 2010), soya beans seeds (Namias et al., 2012), peanuts (Hong, Jing, Qiaoxia & Peng, 2011), and brazil-nuts (Castelo et al., 2013; Cheng-Jin & Da-Wen, 2008).
In this research, an intelligent system to classify the food products based on morphological, color and texture characteristics using computer vision is developed. The system is applied for six different food products namely food grains, edible nuts, bakery products, vegetables, leafy vegetables and fruits. Although, there are many similarities among systems for all products, a special design and training is required for each product.
Material and methods
Sorting system
The vision based sorting system consists of different sub- systems. Figure1, shows the different components of the sorting system.
Fast single camera or multiple cameras are used and provides more accurate and reliable estimates of the image capture for food products. A single camera with mirrors can be used to check the different sides of the product, while multiple cameras fixed in different directions get more clear images (Velappan, Prakash & Sada, 2012).Usually, isolated box with lighting is used to overcome lighting variation problems and get better images. The captured images are sent to the computer to be processed and analyzed in real time. The decision, "pass" or "fail", is sent as an electronic signal to interfacing circuits. These circuits drives into an electronic valve to open or close the path of the products. By closing the path, the product is pushed to "bad product" store. Finally, the high quality products only will continue to the "pass" store. Sometimes, products are classified into more than two classes. The different classes represents different degrees of quality. Figure 2, shows the different modules of computer vision for food products sorting.
The vision system consists of many modules, and it is required to finish all processing in real time. The image acquisition module captures an image and store the image in computer memory. The size and format of the image affects the speed and accuracy of the sorting system. High resolution images contains many details of the product, but requires large time for processing and classification. Low-resolution image are processed very fast, but the accuracy of the system can be reduced. The suitable resolution should be chosen to give acceptable speed with best accuracy (Yang, 2010).
The first step in processing and sorting the image, is to detect the object or determine the location and borders of the product. This operation is considered as an image segmentation process while the image is segmented into two classes: object and background. After the detection of the object, the area of the object is analyzed again to detect any damages in the product. This process is dependent on the nature of the product and the required classification. Another image segmentation is required to extract these regions (cracks- holes - different colors) from the product area. Features are extracted from product regions. The final step is a trained classifier, which gives the decision. The next sections presents the data set, feature extraction and classification.
Data set
The FoodCast Research Image Database (FRID), was an attempt at standardizing a food related objects (bakery products such as biscuits, fruits, edible nuts, vegetables, leafy vegetables and food grains) dataset. In the dataset, all images size (530x530 pixels) are standardized and stored as .jpg file format. In this study, considered total food related images are 180 and categorized into fruits (30 images), biscuits (30 images), edible nuts (30 images), vegetables (30 images), leafy vegetables (30 images) and food grains (30 images).
Feature extraction and classification
The feature extraction is very important phase in this research. We have used the segmented images of different category from the FRID dataset. Then, we inputted to developed feature extraction method, to extract the features as Morphological, Color and Textural. The Morphological characteristics are size and shape of a product. The size and shape characteristics of a categorized food product are listed out in Table 1.
We have used the CIEL*a*b* colour space, to extract the colour characteristics of a categorized food product to measure luminance and chrominance. The measured colour feature as follows:
Mean (μ): The overall brightness of each color component of an image is measured using the mean.
Standard Deviation (σ): The Standard Deviation is the average distance from the mean of the overall perceived brightness and contrast of each color component in an image (Cheng-Jin & Da-Wen, 2008).
Range (r): This gives us the range of maximum and minimum perceived brightness of each color component in an image (Cheng-Jin & Da-Wen, 2008).
Luminance (L): Luminance describes the “achromatic” component of an image. In general, Luminance represents the brightness of an image (Cheng-Jin & Da-Wen, 2008).
Chrominance (C): Chrominance is the color information of an image, separately from the accompanying luminance. Chrominance is usually represented as two color-difference components (Cheng-Jin & Da-Wen, 2008).
Color Distance Metric (ΔE): It is a metric of difference between colors.
The following shows steps for conversion of RGB to CIE L*a*b*.
PHASE I: This involves the conversion from RGB to
As a first step it must normalize RGB to rgb values (values between 0 and 1) using equations (1) 2 (3).
si-Segmented image
(equation1)
(equation 2)
(equation 3)
Subsequently, this values were converted the 𝑟𝑔𝑏 values to 𝑋𝑌𝑍 𝑅𝐺𝐵 values using the matrix M for a D65-2° illuminant observer shown in equation 4.
(equation 4)
Where:
(equation 5)
Thus developing Eqn. 4 and using matrix Eqn. 5, are obtained 𝑋, 𝑌 and 𝑍 values by equation 6
(equation 6)
PHASE II
This involves the conversion from the XYZ to CIE LAB
Subsequently we obtain the values x, 𝑦 and 𝑧 using equations 7, 8 and 9
(equation 7)
(equation 8)
(equation 9)
Where: 𝑋𝑛, 𝑌𝑛 and 𝑍𝑛 are tri-stimulus values of the white specific object using in this case illuminant D65 (day light) and the observer with the values shown in equation 10
(equation 10)
After we are calculated the values 𝑣𝑎𝑟𝑋, 𝑣𝑎𝑟𝑌 and 𝑣𝑎𝑟𝑍 using equations 11, 12 and 13
(equation 11)
(equation 12)
(equation 13)
Calculate the values of L*, a* and b* using equations 14, 15 and 16
(equation 14)
(equation 15)
(equation 16)
The Mean, Standard deviation and Range of each 𝐿 ∗ , 𝑎 ∗ 𝑎𝑛𝑑 𝑏 ∗ component are determined.
(i) The Mean, Standard deviation and Range of 𝐿 ∗ component are determined using following equations 17, 18 and 19.
(equation 17)
(equation 18)
(equation 19)
(ii) The Mean, Standard deviation and Range of 𝑎 ∗ component are determined using following equations 20, 21 and 22.
(equation 20)
(equation 21)
(equation 22)
(iii) The Mean, Standard deviation and Range of 𝑏 ∗ component are determined using following equations 23, 24 and 25.
(equation 23)
(equation 24)
(equation 25)
From 𝑎 ∗ and 𝑏 ∗ component, determined Chrominance using equation 26.
(equation 26)
The color distance metric determined from 𝐿 ∗ , 𝑎 ∗ and 𝑏 ∗ components using equation 27.
Color distance metric:
(equation 27)
We have used the Haralick textural features, to extract the texture characteristics of a categorized food product from Grey Level Co-occurrence Matrix (P). The measured textural features as follows:
(equation 28)
(equation 29)
(equation 30)
(
equation 31)
(
equation 32)
(
equation 33)
(
equation 34)
(equation 35)
(
equation 36)
(
equation 37)
(
equation 38)
(
equation 39)
Results
In this study, the important and noticeable features such as morphological, color and texture are extracted from the categorized food product image using proposed methods. The Ist method is used to extract the 12 no’s of morphological features. The IInd method is used to extract the colour features. There are CIEL*a*b* colour features of 11 numbers. The IIIrd method is used to extract the texture features using Grey Level Co-Occurrence Matrix. There are 12 texture features. The features are extracted from the bulk of categorized food product image.
The Correlation based Feature Selection (CFS) algorithm is used to reduce the dimensionality of feature set, to obtain a high prediction accuracy of each classification model (Novaković, Strbac & Bulatović, 2011). In this sense, CFS is to evaluate the value of features subset by considering the remarkable predictive ability of each feature and also the amount of redundancy between them. The obtained features subset includes seven morphological features (L, W, A, Eq, CA, S, E), eight colour features (
,) and four texture features (Contrast, Correlation, Energy and Homogeneyti).
In this study, the Meta classifiers lazy classifiers and trees of Weka software ® are considered for classification (Witten & Frank, 2005; Siedliska, Baranowski & Mazurek, 2014). Initially, majority of classifiers are tested on illustrative training and testing data groups. Among all classifiers, eight of them are chosen for comparison with high prediction accuracies. The selected eight classifiers are as follows: Sequential Minimal Optimization (SMO), Naïve Bayes, Logistic, Nearest-neighbor classifier, Simple Logistic, Random forests, Multilayer perceptron and libSVM, respectively.
The graphical interface which is available in Knowledge Flow Interface of Weka, allows the design and execution of configurations for streamed data processing as shown in Figure 3.
Figure 3 The studied classification models knowledge flow interface arrangement for cashew kernels grade with the main components of data flow. Adapted from: Siedliska, Baranowski & Mazurek, (2014)
This interface is used to create the prediction model for six different food products namely food grains, edible nuts, bakery products, vegetables, leafy vegetables and fruits. In fact, we have created dependent variables for all the studied classifiers are 1 for fruits, 2 for edible nuts, 3 for bakery products, 4 for vegetables, 5 for leafy vegetables and 6 for food grains using the Weka Knowledge Flow Interface. This graphical interface allows the design and execution of configurations for streamed data processing (Figure 3). Within this interface, the appropriate data file for each is loaded in native ARFF file format available in Weka. Therefore, with the class assigner, the dependent variable is selected and with the class value picker, a value for the positive class is chosen for each model. The Cross-Validation Fold Maker split the dataset into folds (10 folds are chosen). In a previous work, Witten & Frank (2005), stratified 10-fold cross-validation, which is the standard evaluation technique in situations where only limited data are available and it is regarded as the most rigorous one. The idea of 10-fold cross-validation is that data are partitioned randomly into 10 complementary subsets. Each subset is held out in turn and the learning scheme trained on the remaining nine-tenths. Therefore, an error rate is calculated on the holdout set. The learning procedure is executed a total of 10 times on different training sets. In the earlier chosen Classifiers panel, eight classifiers are included to be executed simultaneously and the results are sent to the Classifier Performance Evaluators, throughout which they were presented (and then stored) as text files in the Text Viewer Panel and as ROC threshold curves (Siedliska, Baranowski & Mazurek, 2014; Yang, 2010).
Discussion
The classification experiments are conducted on the morphological, color and texture features set. The 180 total samples of which 30 of fruits, 30 of biscuits, 30 of edible nuts, vegetables, 30 of leafy vegetables and 30 of food grains (from each categorized food product 15 samples as a training or test set and another 15 samples as a validation set), are chosen randomly. The 10-fold cross validation is used for training and testing. For each fold, the proportion among data are used for training, and data are used for testing was 90-10%. Conversely, the research is to the identification of food products into a category namely food grains, edible nuts, bakery products, vegetables, leafy vegetables and fruits
Fruits
The obtained results of fruits are presented in Table 2. It shows the eight prediction techniques results measured using cross-validation on a given dataset.
For the training or test set, the best obtained prediction accuracy is for Sequential Minimal Optimization (82.27%), Multilayer Perceptron (84.9%), and Simple Logistic (86.07%). A good obtained (more than 79.99%) prediction accuracy is for the Logistic and libSVM models. The validation set resulted in somewhat lower classification accuracy of the classification models, but in the case of three models (i.e. Simple Logistic, Multilayer Perceptron, Sequential Minimal Optimization) it is equal to 85.58%, 83.90% and 82.00%. When comparing the instances of correctly classified, root mean squared error and Kappa statistic, it can be stated that the Simple Logistic, Multilayer Perceptron and Sequential Minimal Optimization models are the best for recognition of fruits.
Food grains
The obtained results of food grains are presented in Table 3. It shows the eight prediction techniques results measured using cross-validation on a given dataset. For the training or test set, the best obtained prediction accuracy is for Sequential Minimal Optimization (83.27%), Multilayer Perceptron (94.9%), and Simple Logistic (88.07%). A good obtained (more than 79.99%) prediction accuracy is for the Logistic and libSVM models. The validation set resulted in somewhat lower classification accuracy of the classification models, but in the case of three models (i.e. Simple Logistic, Multilayer Perceptron, Sequential Minimal Optimization) it is equal to 92.58%, 93.90% and 83.00%.
When comparing the instances of correctly classified, root mean squared error and Kappa statistic, it can be stated that the Simple Logistic, Multilayer Perceptron and Sequential Minimal Optimization models are the best for recognition of food grains.
Edible nuts
The obtained results of Edible nuts are presented in Table 4. It shows the eight prediction techniques results measured using cross-validation on a given dataset.
For the training or test set, the best obtained prediction accuracy is for Sequential Minimal Optimization (89.27%), Multilayer Perceptron (85.9%), and Simple Logistic (88.07%). A good obtained (more than 79.99%) prediction accuracy is for the Logistic and libSVM models. The validation set resulted in somewhat lower classification accuracy of the classification models, but in the case of three models (i.e. Simple Logistic, Multilayer Perceptron, Sequential Minimal Optimization), which is equal to 87.58%, 84.90% and 88.00%, respectively. When comparing the instances of correctly classified, root mean squared error and Kappa statistic, it can be stated that the Simple Logistic, Multilayer Perceptron and Sequential Minimal Optimization models are the best for recognition of edible nuts.
Bakery products
The obtained results of bakery products are presented in Table 5. It shows the eight prediction techniques results measured using cross-validation on a given dataset.
For the training or test set, the best obtained prediction accuracy is for Sequential Minimal Optimization (91.27%), Multilayer Perceptron (87.9%), and Simple Logistic (89.07%). A good obtained (more than 79.99%) prediction accuracy is for the Logistic and libSVM models. The validation set resulted in somewhat lower classification accuracy of the classification models, but in the case of three models (i.e. Simple Logistic, Multilayer Perceptron, Sequential Minimal Optimization) it is equal to 88.58%, 86.90% and 88.00%. When comparing the instances of correctly classified, root mean squared error and Kappa statistic, it can be stated that the Simple Logistic, Multilayer Perceptron and Sequential Minimal Optimization models are the best for recognition of bakery products.
Vegetables
The obtained results of vegetables are presented in Table 6. It shows the eight prediction techniques results measured using cross-validation on a given dataset.
For the training or test set, the best obtained prediction accuracy is for Sequential Minimal Optimization (90.27%), Multilayer Perceptron (89.9%), and Simple Logistic (90.07%). A good obtained (more than 79.99%) prediction accuracy is for the Logistic and libSVM models. The validation set resulted in somewhat lower classification accuracy of the classification models, but in the case of three models (i.e. Simple Logistic, Multilayer Perceptron, Sequential Minimal Optimization) it is equal to 88.58%, 86.90% and 88.00%. When comparing the instances of correctly classified, root mean squared error and Kappa statistic, it can be stated that the Simple Logistic, Multilayer Perceptron and Sequential Minimal Optimization models are the best for recognition of vegetables.
Leafy vegetables
The obtained results of leafy vegetables are presented in Table 7. It shows the eight prediction techniques results measured using cross-validation on a given dataset (Mai, Chetima & Pierre, 2012).
For the training or test set, the best obtained prediction accuracy is for Sequential Minimal Optimization (90.87%), Multilayer Perceptron (89.96%), and Simple Logistic (90.66%). A good obtained (more than 79.99%) prediction accuracy is for the Logistic and libSVM models. The validation set resulted in somewhat lower classification accuracy of the classification models, but in the case of three models (i.e. Simple Logistic, Multilayer Perceptron, Sequential Minimal Optimization) it is equal to 88.53%, 86.92% and 88.90%. When comparing the instances of correctly classified, root mean squared error and Kappa statistic, it can be stated that the Simple Logistic, Multilayer Perceptron and Sequential Minimal Optimization models are the best for recognition of leafy vegetables.
Conclusions
This study evaluated the effects of morphological, color and texture features, which were extracted from food products. Given these concerns, image proved to be the precise method in recognizing categorized one. In fact, the study limited to fruits, leafy vegetables, bakery products, food grains and edible nuts therefore further studies on more individual food products like vegetables such as onion, garlic, etc., are needed. The very high accuracy and prediction performance of the results helped us to develop food product sorting systems.
