Automatic detection of bumblebees using video analysis
Detección automática de abejorros usando análisis de video
Willy Azarcoya-Cabiedes a, Pablo Vera-Alfaro a, Alfonso Torres-Ruiz b & Joaquín Salas-Rodríguez a
a Instituto Politécnico Nacional, México, wazarcoyac1300@alumno.ipn.mx
b Koppert de México S. A. De C. V., México
Received: October 14th, de 2013. Received in revised form: May 21th, 2014. Accepted: June 3th, 2014
]]>Abstract
In this document, we explore and develop techniques to automatically detect bumblebees flying freely inside a greenhouse, where illumination conditions are left unconstrained, and no artifact is used on their bodies. Specifically, we compare a Viola-Jones classifier and a Support Vector Machine (SVM) classifier to detect the presence of bumblebees. Our results show that the latter has a better classification performance.
Keywords: Support Vector Machine classifier; Viola-Jones classifier; bumblebee detection.
Resumen
En este documento exploramos y desarrollamos técnicas para la detección de abejorros que vuelan libremente dentro de un invernadero, en donde las condiciones de iluminación no son controladas y ningún artefacto es colocado en sus cuerpos. En particular, comparamos clasificadores Viola-Jones y Máquinas de Soporte Vectorial (SVM) en su uso para la detección de abejorros. Nuestros datos muestran que el SVM ofrece mejores resultados de clasificación.
Palabras clave: Clasificador tipo Support Vector Machine; Clasificador tipo Viola-Jones; detección de abejorros.
1. Introduction
Pollination is a critical ecosystem service in agriculture [10]. It has been estimated that 75% of human food crops require pollination by insects for adequate production [6,11], bees (Hymenoptera: Apoidea) and bumblebees being the most used in managed pollination programs. To date, bumblebees are used as managed pollinators on more than 40,000 hectares of greenhouses tomato crop in the world [15]. Therefore, monitoring their activity is important from the perspective of ecological research, and when there is the need to know their patterns of activity and how they are affected by greenhouses management practices [5]. Currently, the study of pollination is based fundamentally on direct observations of plant-pollination relationships [8], on offline video monitoring [9,13], and also with the aid of special tags attached to the bumblebees' bodies [3]. Overall, the trend is toward the use of automatic techniques that facilitate biology studies.
]]> The objective of the present document is to report the use of computer vision algorithms to automatically detect the presence of bumblebees, for extended periods of time, and without the need to engineer the environment. To that end, the rest of the document is developed as follow. In Section 2, we survey the related literature. Then in Section 3, we describe the materials and methods used to perform the evaluation. Next, in Section 4, we describe our experimental results. Finally, in Section 5, we conclude the document summarizing our findings and describing potential lines for future research.
2. Related Works
Automatic visual recognition of insects has been used when it has been possible to study static insects, with enough resolution, and in controlled lighting conditions. For instance, Larios et al. [7] represent insects by features based on the curvature of their profiles, analyzed on both local and global scales. On the other hand static and adaptive appearance templates for handling appearance change, and geometry-constrained resampling of particles for handling unreliable features has been used in the past [19]. At their end, Yuefang et al.[4] identify insects by describing their wings with a combination of moment invariants. In addition, the interaction between insects and the environment can facilitate the use of clustering techniques based on color or intensity, as reported by Jinhui et al. [17]. Furthermore, insects can be identified by their traces [12]. Nonetheless, there is a pressing need to increase our understanding for situations where the bumblebees interact freely with their environment.
Detecting and tracking at the beehive entrance has been done in the past making use of computer vision technics in 3D.[20] In our investigation we focus only on the detection of insects either at the entrance of the beehive or at the time of pollination in the flower. Toward that objective, we compare the performance of a Viola-Jones classifier [16] and a Support Vector Machine classifier (SVM) [2]. This work further enhance a previous work [23], where we explored the use of tracking by detection to analyze the arrival of bumblebees to flowers and their motion around beehives.
3. Materials and Methods
A flying cage (6m x 3m x 3m) was covered with an antiaphid net. Inside we placed five tomato (Solanum lycopersicum) and Serrano chili (Capsicum annuun) flowering plants. A MINIPOL™ (Koppert) hive, containing 30 Bombus impatiens (Hymenoptera: Apidae) workers was introduced into the cage two hours before the experiment began. The bumblebees were free to fly inside the cage and a JAI camera model CV-S3200 with an analog interface connected to a National Instrument NI PCI 1411 acquisition board was used to obtain images at a 640 x 480 resolution and a frame rate of 30 fps. Acquisition was done during the day making use of direct sunlight as illumination. The camera was mounted on a metallic support focused at times on a tomato flower as well as to the hive (see Fig. 1). In order to analyze the performance of each classifier, 2,082 images with bumblebees were selected as positive samples and 3,483 without bumblebees as negative samples employing cross-validation. The images included the natural changes of illumination caused by the apparent Sun movement and the occasional Sun occlusion due to clouds. To train the classifiers, we selected 80% of the samples at random using the rest for testing. To construct the Viola-Jones classifier, we used the Open CV library [1], which uses Haar-like features. To construct the SVM classifier, we used the implementation provided by Matlab with a linear kernel, with Histogram of Oriented Gradients (HOG) as features [18]. We constructed Viola-Jones classifiers for 24 x 24 pixel subimages, with 10, 15, 18, 20, and 22 stages. Their corresponding training time was around 4, 8, 12, 18, and 24 hours, respectively. For the HOG features, we used 64 x 64 pixel images, with 8 x 8 pixel cells, and 2 x 2 cell blocks, as seen in Fig. 3. The SVM works by constructing a feature space where the classes to be distinguished are separated using a certain type of kernel. The search for a bumblebee in a particular image takes place using a hierarchical search of a pyramid structure, where each level has twice the resolution [24]. We accepted bumblebee detection when the Viola-Jones classifier gave a positive response and when the margin of the SVM classifier was positive. We use the Receiver Operating Characteristic (ROC) curve [14] to verify the performance of the classifiers. The computer used for these experiments has an Intel i5 microprocessor with four cores, operating at 3.33GHz, with 8 GB of RAM and running on the Windows 7, 64 bit, operating system.
4. Experimental Results
]]> For the Viola-Jones classifier, as the number of stages was varied in the classifier, the performance improved. Fig. 2 illustrates the results as a ROC curve [14]. The reduction of the False Positive Rate (FPR) after the addition of just a few stages in the classifier is remarkable. For instance, the FPR is reduced from ~1.0 to ~0.4 upon changing from a 10-stages classifier to a 15-stages classifier. In fact, the FPR for the 18-stages classifier is 0.04, while the True Positive Rate (TPR) is 0.85. Of course, the TPR decreases accordingly but it does so at a smaller rate. Note that while the TPR is 0.99 with the 10-stages classifier, it is 0.87 and 0.85 with the 15 and 18-stages classifiers, respectively. Similarly, the FPR is below 0.01 for the 20- and 22-stages classifiers, while the TPR is ~0.75. With the SVM classifier, the TPR is 0.98 and the FPR 0.003. These results show a superior performance of the SVM classifier.
5. Conclusion
In this document, we applied Viola-Jones and SVM classifiers to the problem of detecting bumblebees in an unconstrained,green-house-like environment. Furthermore, our results show that the Support Vector Machine classifier, with HOG features, outperforms the Viola-Jones classifier. Managed pollinators like bumblebees are frequently monitored at the hive entrance to determine the foraging activity rate by counting the number of bees coming in or out the hive [21]. This activity rate is an important element of practical pollination studies in greenhouses [22]. A system counting automatically the number of bees flying in and out of the hive or the number of bumblebees arriving to a flower with high accuracy would be very informative. However, in some computer vision systems the illumination changes could affect the outcome of the detection. HOG features are less affected by possible illumination changes, because they are based on the orientation of gradients and the normalization of image blocks. Keeping the natural illumination conditions is important in order not to disturb the behavior of bumblebees. More research should increase the performance of the detectors, highlight other aspects of the insect-pollinators activity, and lead the development of more flexible monitoring tools. For instance, a possible way to increase the performance of either one of the classifiers could involve the use of a detection-and-tracking strategy. Such strategy could be used in combination to fill the gaps whenever a bumblebee is not detected.
6. Figures
]]> Acknowledgements
This research was partially funded through research grant number SIP-IPN/201325. We thank Paul Riley for his comments to improve this document.
References
[1] Gary, B. and Adrian, K., Learning OpenCV: Computer vision with the OpenCV library. 1ra. Ed. California, U. S.: Editor O'Reilly Media, 2008. [ Links ]
[2] Nello, C. and Shawe-Taylor, J., An introduction to support vector machines and other kernel-based learning methods. 1ra. Ed. United Kingdom, Cambridge University Press, 2000. [ Links ]
[3] Chiu, C., En-Cheng, Y., Joe-Air, J. and Ta-Te, L., An imaging system for monitoring the In-and-out activity of honey bees. Computers and Electronics in Agriculture, pp. 100-109, 2012. [ Links ]
]]>[4] Yuefang, G., Hongzhi, S., Xuhong, T. and Yan, C., Identification algorithm of winged insects based on hybrid moment invariants, Proceedings of International Conference on Bioinformatics and Biomedical Engineering, pp. 531-534, 2007. [ Links ]
[5] Cama-Pinto, A., Gil-Montoya, F., Gómez-López, J., García-Cruz, A. and Manzano-Agugliaro, F., Wireless surveillance system for greenhouse crops , DYNA (Colombia), 81 (184), pp. 164-170, 2014. [ Links ]
[6] Klein, A.-M., Vaissière, B.-E., Cane, J.-H., Steffan-Dewenter, I., Cunningham, S., Kremen, C. and Tscharntke, T., Importance of pollinators in changing landscapes for world crops. Proc. R. Soc. B., pp. 303-313, 2007. [ Links ]
[7] Larios, N., Deng, H., Zhang, W., Sarpola, N., Yuen, J., Paasch, R., Moldenke, A., Lytle, D., Correa, S., Mortensen, E., Shapiro, L. and Dietterich, T., Automated insect identification through concatenated histograms of local appearance features: Feature vector generation and region detection for deformable objects, Machine Vision and Applications, 19, pp. 105-123, 2007. [ Links ]
[8] Lingren, B. and Lenker, D.-H., Insect trap and assembly for capturing and monitoring insects, US Patent, 6,516,558, February, 2003. [ Links ]
]]>[9] Lortie, C.-J., Budden, A.-E. and Reid, A.-M., From birds to bees: Applying video observation techniques to invertebrate pollinators, Journal of Pollination Ecology, 6, pp. 125, 2012. [ Links ]
[10] Nabhan, G.-P. and Buchmann, S.-L., Services supplied by pollinators, In Nature's Services, Societal Dependence on Natural Ecosystems, Island Press, pp. 133-150, 1997 [ Links ]
[11] Potts, S.-G., Biesmeijer, J.-C., Kremen, C., Neumann, P., Schweiger, O. and Kunin, W., Global pollinator declines: Trends, impacts and drivers, Trends in Ecology and Evolution Cell Press, 25, pp. 345-353, 2010. [ Links ]
[12] Bok-Suk, S., Eui-Young, C., Kwang-Baek, K., Kyoung-Won, C., Reinhard, K. and Woon, W., Effective feature extraction by trace transform for insect footprint recognition, in 3rd International Conference on Bio-Inspired Computing: Theories and Applications, pp. 97-102, 2008. [ Links ]
[13] Steen, R., Aase, O. and Thorsdatter, A., Portable digital video surveillance system for monitoring flower-visiting bumblebees, Journal of Pollination Ecology, 5, pp. 90-94, 2011. [ Links ]
[14] Swets, J.-A., Dawes, R.-M. and Monahan, J., Better decisions through science, Scientific American, 283, pp. 82-87, 2000. [ Links ]
[15] Velthuis, H.-W. and Van Doorn, A., A century of advances on bumblebee domestication and the economic and environmental aspect of its commercialization for pollination, Apidologie, 37, pp. 421-451, 2006. [ Links ]
[16] Viola, P. and Jones, M., Rapid Object detection using a boosted cascade of simple features, in Proceedings of the Conference on Computer Vision and Pattern Recgnition (CVPR), Kauai, Hawaii, Vol. 1, pp. 511-518, 2001. [ Links ]
[17] Jinhui, Z., Muhua, L. and Mingyin, Y., Study on image recognition of insect pest of sugarcane cotton aphis based on rough set and fuzzy C-means clustering, in: Intelligent Information Technology Application, IITA 2009, Third International Symposium on Vol. 2, pp. 553-555. 2009. [ Links ]
[18] Dalal, N. and Triggs, B., Histograms of oriented gradients for human detection, In Proceedings of the Conference on Computer Vision and Pattern Recognition, San Diego, California, USA, pp. 886-893, 2005. [ Links ]
[19] Maitra, P., Schneider, S. and Shin, M., Robust bee tracking with adaptive appearance template and geometry-constrained resampling, in Applications of Computer Vision (WACV), 2009 Workshop IEEE 2009. [ Links ]
[20] Chiron, G., Gomez-Krämer, P., Ménard, M, and Requier, F., 3D tracking of honeybees enhanced by environmental context, International Conference on Image Analysis and Processing, Springer, pp. 702-711, 2013. [ Links ]
[21] Kevan, G.-P., Cooper, E., Morse, A., Kapongo, P.J., Shipp, L. and Khosla, S., Measuring foraging activity in bumblebee nests: A simple pest-entrance trip recorder, Journal of Applied Entomology, 133, pp. 222-228, 2009. [ Links ]
[22] Torres-Ruiz, A. and Jones, R.W., Bombus impatiens and bombus ephippiatus (Hymenoptera: Apidea) as pollinators of tomato in greenhouses, Journal of Economic Entomology, 105 (6), pp. 1871-1877, 2012. [ Links ]
[23] Miranda, B., Salas, J. and Vera, P., Bumblebees detection and tracking, workshop vis. Observational Anal. Anim. Insect Behav. ICPR. 2012. [ Links ]
[24] Crowley, J.-L., A representation for visual information, Report CMU-RI-TR-82-07, Robotics Institute, Carnegie-Mellon University, 1981. [ Links ]
W. Azarcoya-Cabiedes, received a Bs. on Communications and Electronics in 2006, from the Instituto Politécnico Nacional - IPN, México; an MSc. in Distributed Software Engineering in 2012, from Querétaro State University - UAQ, México; he is currently a PhD student in Computer Vision at IPN, México.
A. Torres-Ruiz received a Bs. in Biology in 1999 from the Queretaro State University - UAQ, México; an MSc. degree in Biological Science in 2002 from the Institute of Ecology at the National University of Mexico - UNAM, México; and his PhD. degree in Management of Natural Resources in 2013, from the UAQ, México. His areas of research interest include: pollination ecology, biological control and mass rearing of insects. He is currently Production and Research and Develop manager at Koppert México.
P. Vera-Alfaro, received a Bs. in Communications and Electronics and an MSc. degree on Advanced Technology in 2007, all of them from Instituto Politécnico Nacional - IPN, México. Currently he is working in IPN, as an associate professor in the area of computer vision.
J. Salas, received a Bs. degree in Computer Science in 1989 from the ITESM, México; an MSc. degree in Electrical Engineering in 1991 from CINVESTAV, México and a PhD. degree in Informatics in 1996, also from ITESM, México. He is a full-time professor at IPN. His areas of research include, computer vision, pattern recognition, and artificial intelligence.
]]>
