Automated detection of photoreceptors in in-vivo retinal images

Detección automatizada de fotorreceptores en imágenes retinianas in-vivo

Piero Rangel-Fonseca ^a,b, Armando Gomez-Vieyra ^c, Daniel Malacara-Hernandez ^a & Mario C. Wilson ^d

^a Centro de Investigaciones en Óptica, León, Guanajuato, México. pierorf@gmail.com, dmalacara@cio.mx
^b Laboratorio de Visión Robótica e Inteligencia Artificial, Universidad de Guanajuato, Salamanca, Guanajuato, México. piero@laviria.org
^c Laboratorio de Sistemas Complejos, Departamento de Ciencias Básicas, Universidad Autónoma Metropolitana, Azcapotzalco, D.F., México. agvte@correo.azc.uam.mx
^d Centro de Investigaciones en Óptica, León, Guanajuato, 37150, México. CONACYT - CICESE, Ensenada, B.C. México. mwilson@cicese.mx

Received: December 8^th, de 2015. Received in revised form: August 8^th, 2016. Accepted: August 26^th, 2016

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Abstract
The inclusion of adaptive optics (AO) into ophthalmic imaging technology has allowed the study of histological elements of retina in-vivo, such as photoreceptors, retinal pigment epithelium (RPE) cells, retinal nerve fiber layer and ganglion cells. The high-resolution images obtained with ophthalmic AO imaging devices are rich with information that is difficult and/or tedious to quantify using manual methods. Thus, robust, automated analysis tools that can provide reproducible quantitative information about the tissue under examination are required. Automated algorithms have been developed to detect the position of individual photoreceptor cells and characterize the RPE mosaic. In this work, an algorithm is presented for the detection of photoreceptors. The algorithm has been tested in synthetic and real images acquired with an Adaptive Optics Scanning Laser Ophthalmoscope (AOSLO) and compared with the one developed by Li and Roorda. It is shown that both algorithms have similar performance on synthetic and cones-only images, but the one here proposed shows more accurate measurements when it is used for cones-rods detection in real images.

Keywords: Photoreceptor; adaptive optics; image processing.

Resumen
La inclusión de la óptica adaptativa (adaptive optics, AO) en la tecnología de imágenes oftálmicas ha permitido el estudio in-vivo de los elementos histológicos de retina, como los fotorreceptores, células del epitelio pigmentario de la retina (retinal pigment ephitelium, RPE), la capa de fibras nerviosas de la retina y células ganglionares. Las imágenes de alta resolución obtenidas con dispositivos oftálmicos con AO son ricos en información, que es difícil y/o tediosa de cuantificar por medio de métodos manuales. Por lo tanto, se requieren herramientas de análisis automatizadas robustas que puedan proporcionar información cuantitativa reproducible del tejido bajo examen. Algoritmos automatizados han sido desarrollados para detectar la posición de células individuales fotorreceptoras y caracterizar el mosaico RPE. En este trabajo, se presenta un algoritmo para la detección de los fotorreceptores. El algoritmo ha sido probado en imágenes sintéticas y reales adquiridas con un oftalmoscopio de barrido láser con óptica adaptativa (Adaptive Optics Scanning Laser Ophthalmoscope, AOSLO) y comparado con el desarrollado por Li y Roorda. Se muestra que ambos algoritmos tienen un rendimiento similar en imágenes sintéticas e imágenes con sólo conos, pero el algoritmo propuesto muestra mediciones más precisas cuando se utiliza para la detección de conos-bastones en imágenes reales.

Palabras clave: Fotorreceptores; óptica adaptativa; procesamiento de imágenes.

1. Introduction

Typically, the retina is shown to have several layers: inner limiting membrane, retinal nerve fiber layer (RNFL), retinal ganglion cell layer, inner plexiform layer, inner nuclear layer, outer plexiform layer, outer nuclear layer, external limiting membrane, photoreceptor layer and retinal pigment epithelium (RPE) layer [1]. The RNFL conducts signals from the eye. The ganglion cell layer contains the cell bodies of ganglion cells and displaced amacrine cells; the ganglion cell dendrites are in the plexiform layer, where they receive bipolar cell and amacrine cell inputs, while their axons are in the RNFL. The photoreceptors are responsible for visual transduction, the transformation of light energy into electrical energy, a process called isomerization [2,3]. Finally, the RPE is vital for the maintenance of photoreceptor function, since it contains nutrients and enzymes [4].

In the retina, there are two types of photoreceptors: cones and rods. Structurally, the photoreceptors are different in shape, as their names indicate, and in size, rods being longer than cones, and cones wider than rods; they also have different functional properties that affect our perception. Cones cover all the retina, although higher density is in the fovea, where there are no rods. There are about 120 million rods, and they are in the periphery of the retina [2,4,5].

It is very important to know where the photoreceptors, cones and rods, are located and how many photoreceptors there are in the retina, which could be used to monitor the evolution of therapies for ocular globe diseases. A method previously used for cell detection and their characterization is shown in [6]; it uses adaptive optics scanning light ophthalmoscopy (AOSLO), which has become an important tool for the study of retinal diseases [6-22]. Our interest is primarily in detecting the cones and rods. Since changes in the photoreceptor mosaic may precede cell death, it is possible that morphometric changes can be measured before larger damages are observed, allowing early therapeutic intervention.

Accuracy in the detection and classification of patterns in biomedical images is central to detecting and monitoring tissue damage, as well as to quantifying its extent. Hence, it is necessary to have robust and reliable methods for classifying and quantifying retinal structures in in-vivo retinal images. This is the reason why, in recent years, there has been an increased attention to this area [6-8,22-29], and it spurred our interest in developing a more efficient algorithm for the detection of retinal cones and rods. The algorithm that we propose quantifies and locates the photoreceptors using the local maximum value, maximum absolute deviation and binary operations.

This paper is organized as follows. In section 2, the methods used for the development of the algorithm, as well as the generation of synthetic and real AO images, are discussed. In section 3, the comparison between the algorithm proposed by Li and Roorda and the one here presented is undertaken. Finally, the main discussion and conclusions are in section 4.

2. Method

2.1. Algorithm

2.1.1. Adaptive threshold using local maximum values

Let I(x,y) be a bidimensional intensity function (a matrix) where x and y are the spatial coordinates of a pixel; let I_max(x,y) be the local maximum value within a window of 3x3 pixels, being:

Thereby, F₁(x,y) is defined as follows:

2.1.2. Adaptive threshold using absolute deviation

Let M(x,y) be the maximum absolute deviation (Eq. 3), which determines the sufficient variance in intensity, with n being the number of elements, and let I_max(x,y) be the local maximum value (Eq. 1). Then, let m be a threshold that determines the sufficient variance in intensity of I(x,y).

The next step in this stage is the generation of a binary image G(x,y) using Eq. 5, through the logic operation AND of the results from Eqs. 2 and 4.

2.1.3. Binary operations

Eq. 6 is a binary function that works within a window of 3 X 3 pixels; it was designed to reduced over-detection of cells, where over-detection of a cell can be represented as elements a1, a2, …, a33 shown in Figs. 2. Element a1 from Fig. X can be represented by a binary function ; where , , , , , , , and ; which means that the central element is eliminated to avoid over-detection. Thus, every element from Fig. 2 can be represented by a binary function where the central element has been eliminated, and it is based on that fact that every element can be represented by a binary function, and the combination of all these functions yields cell location. This equation was reduced by the Quine-McCluskey method; afterward, Postulates and Theorems of Boolean Algebra were used to minimize it. This binary function is applied onto the image I(x,y). Later, the obtained image is rotated 180 degrees, H(x,y) = rotate180(B(x,y)), with the goal of eliminating the multiple detection in the opposite direction. Then, Eq. 6 is applied onto the rotated image H(x,y). Finally, the result is rotated back to the original orientation. This process is performed until algorithm convergence is reached.

2.2. Synthetic images

The synthetic images were generated using a pre-determined number of pseudo-photoreceptors, which in turn were generated through the use of a Gaussian function and an elliptical one [30]. The pseudo-photoreceptors varied at random in terms of size and reflectance, and they were symmetrically (in equidistant square and hexagonal arrays) and randomly distributed. Background illumination was added to each of the generated images, which corresponds to the illumination in the real images acquired with the AOSLO utilized at the University of Rochester [18].

We used AOSLO images obtained for current and previous experiments at the Center for Visual Science of the University of Rochester to test the algorithm. Data shown from human participants are from experiments that were approved by the Research Subjects Review Board at the University of Rochester and adhered to the tenets of the Declaration of Helsinki. Participants gave informed written consent after the nature of the experiments and any possible risks were explained both verbally and in writing.

3. Results

The algorithm developed by Li and Roorda [6] (Algorithm 1) and the algorithm here proposed (Algorithm 2) have been tested on synthetic and real images, Fig. 3 shows the results obtained with both algorithms on synthetic images. In the caption, we show the color code that was used to identify where a pseudo-cell was positioned and the results obtained by each algorithm.

For each function-distribution combination, 1000 synthetic images of 100x100 pixels in size were generated. To those images, we applied algorithms 1 and 2, and the results obtained were later evaluated. The outcome of such evaluation is shown in Tables 1 and 2. Table 1 shows the relative error obtained for each of the combinations, whereas Table 2 shows the number of cells that were correctly detected as corresponding exactly with the place where the center of the pseudo-cells had been positioned.

For all the images, both real and synthetic, a threshold m = 2.0 was used to calculate the maximum absolute deviation M_ij, and it was determined empirically; it is based on different tests that we conducted. If m > 2.0, we have a very prominent peak, but if m < 2.0, there is a tendency to flatness. The manual count presented in Table 3 was carried out on the basis of the authors' appreciation. For algorithm [6], cut_off_frequency = 0.61p and threshold = 0 were used to test the all the images.

Fig. 4 shows the results obtained on real images. In the caption, we describe the color code used to identify the outcome of the algorithm that was utilized in each case. Table 3 shows the number of cells that were manually detected by the authors and those detected by each algorithm.

As shown in Tables 1 and 2 and in Fig. 3, both algorithms exhibited similar performances on synthetic images.

When the algorithm was applied to real images that included cones, rods and vascular tissue, Fig. 4 (a - c), the algorithm developed by Li and Roorda [6] showed detection of cells in the vascular tissue, as well as poor detection of rods. It is evident that there are photoreceptors in these zones, but since the vessels cause the photoreceptor layer to be below or above the blood vessel, it cannot be stated that photoreceptor detection in this zone is adequate.

Both algorithms exhibited similar performances on real images that included only cones, Fig. 4 (d, e). It is worth mentioning that Li and Roorda's algorithm does not show multiple detection for a single photoreceptor, whereas ours does.

As aforementioned, it is very important and useful to know where photoreceptors are located and how many photoreceptors there are in the retina, because that information could be used for the analysis of the distribution and orientation of cones and rods in-vivo as well as to monitor the evolution of therapies for retinal diseases.

The authors thank Alfredo Dubra, PhD, and Ethan Rossi, PhD, for the AOSLO images they kindly provided. We also acknowledge the support of CONACYT through grant 162031 and projects 166326 and 166070. M. W. Acknowledges FONDECYT Nº 3140387.

References

[1] Oyster, C.W., Eye. Sinauer Associates, 1^st ed, 1999. DOI: 10.1046/j.1475-1313.2000.00552.x        [ Links ]

[2] Goldstein, E.B., Sensation and perception. Wadsworthm, 6^th ed, 2002.         [ Links ]

[3] Rodieck, R.W., The first steps in seeing. Sinauer Associates, 1^st ed., 1999.         [ Links ]

[4] Levin, L.A., Nilsson, S.F.E., Hoeve, J.V., Wu, S., Kaufman, P.L. and Alm, A., Adlers physiology of the eye. Elsevier, 11^th ed., 2011. DOI: http://dx.doi.org/10.1016/S0039-6257(03)00069-9        [ Links ]

[5] Curcio, C.A., Sloan, K.R., Kalina R.E. and Hendrickson, A.E., Human photoreceptor topography. The Journal of Comparative Neurology, 292, pp. 497-523, 1990. DOI: 10.1002/cne.902920402        [ Links ]

[6] Li, K.Y. and Roorda, A., Automated identification of cone photoreceptors in adaptive optics retinal images. J. Opt. Soc. Am. A, 24, pp. 1358-1363, 2007. DOI: 10.1364/JOSAA.24.001358.         [ Links ]

[7] Doble, N., Choi, S.S., Codona, J.L., Christou, J., Enoch J.M. and Williams, D.R., In vivo imaging of rod photoreceptor mosaic. Opt. Lett., 36, pp. 31-33, 2011. DOI: 10.1364/OL.36.000031.         [ Links ]

[8] Putnam, N.M., Hammer, D.X., Zhang, Y., Merino, D. and Roorda, A., Modeling the foveal cone mosaic imaged with adaptive optics scanning laser ophthalmoscopy. Opt. Express, 18, pp. 24902-24916, 2010. DOI: 10.1364/OE.18.024902        [ Links ]

[9] Fisher, J.B., Jacobs, D.A., Markowitz, C.E., Galetta, S.L., Volpe, N.J., Nano-Schiavi, M.L., Baier, M.L., Frohman, E.M., Winslow, H., Frohman, T.C., Calabresi, P.A., Maguire, M.G., Cutter, G.R. and Balcer, L.J., Relation of visual function to retinal nerve fiber layer thickness in multiple scleriosis. Ophthalmology, 113, pp. 324-332, 2006. DOI: 10.1016/j.ophtha.2005.10.040.         [ Links ]

[10] Miller, D.T., Willliams, D.R., Morris, G.M. and Liang, J., Images of cone photoreceptors in the living human eye. Vision Research, 36, pp. 1067-1079, 1996. DOI: http://dx.doi.org/10.1016/0042-6989(95)00225-1        [ Links ]

[11] Takayama, K., Ooto, S., Hangai, M., Arakawa, N., Oshima, S., Shibata, N., Hanebuchi, M., Inoue, T. and Yoshimura, N., High-resolution imaging of the retinal nerve fiber layer in normal eyes using adaptive optics scanning laser ophthalmoscopy. PLoS ONE, 7, e33158, 2012. DOI: http://dx.doi.org/10.1371/journal.pone.0033158.         [ Links ]

[12] Hunter, J.J., Masella, B., Dubra, A., Sharma, R., Yin, L., Merigan, W.H., Palczewska, G., Palczewski, K. and Williams, D.R., Images of photoreceptors in living primate eyes using adaptive optics two-photon ophthalmoscopy. Biomed. Opt. Express, 2, pp. 139-148, 2011. DOI: 10.1364/BOE.2.000139        [ Links ]

[13] Liang, J., Williams, D.R. and Miller, D.T., Supernormal vision and high-resolution retinal imaging through adaptive optics. J. Opt. Soc. Am. A, 14, pp. 2884-2892, 1997. DOI: 10.1364/JOSAA.14.002884        [ Links ]

[14] Jonnal, R.S., Rha, J., Zhang, Y., Cense, B., Gao, W. and Miller, D.T., In vivo functional imaging of human cone photoreceptors. Opt. Express, 15, pp. 16141-16160, 2007. DOI: 10.1364/OE.15.016141        [ Links ]

[15] Baraas, R.C., Caroll, J., Gunther, K.L., Chung, M., Williams, D.R., Foster, D.H. and Neitz, M., Adaptive optics retinal imaging reveals s-cone dystrophy in tritan color-vision deficiency. J. Opt. Soc. Am. A, 24, pp. 1438-1447, 2007. doi: 10.1364/JOSAA.24.001438.         [ Links ]

[16] Morgan, J.I.W., Dubra, A., Wolfe, R., Merigan, W.H. and Williams, D.R., In vivo autofluorescence imaging of and macaque retinal pigment epithelial cell mosaic. Investigative Ophthalmology & Visual Science, 50, pp. 1350-1359, 2009. DOI: 10.1167/iovs.08-2618         [ Links ]

[17] Carroll, J., Choi, S.S. and Williams, D.R., In vivo imaging of the photoreceptor mosaic of a rod monochromat. Vision Research, 48, pp. 2564-2568, 2008. DOI: 10.1016/j.visres.2008.04.006        [ Links ]

[18] Rossi, E.A., Chung, M., Dubra, A., Hunter, S.S., Merigan, W.H. and Williams, D.R., Imaging retinal mosaics in the living eye. Eye, 25, pp. 301-308, 2011. DOI: 10.1038/eye.2010.221        [ Links ]

[19] Williams, D.R., Imaging single cells in the living retina. Vision Research, 51, pp. 1379-1396, 2011. DOI: 10.1016/j.visres.2011.05.002.         [ Links ]

[20] Morgan, J.I.W., Hunter, J.J., Masella, B., Wolfe, R., Gray, D.C., Merigan, W.H., Delori, F.C. and Williams, D.R., Light-induced retinal changes observed with high-resolution autofluorescence imaging of the retinal pigment epithelium. Visual Science, 49, pp. 3715-3729, 2008. DOI: 10.1167/iovs.07-1430        [ Links ]

[21] Roorda, A., Zhang, Y. and Duncan, J.L., High-resolution in vivo imaging of the rpe mosaic in eyes with retinal disease. Investigative Ophthalmology & Visual Science, 48, pp. 2297-2303, 2007. DOI: 10.1167/iovs.06-1450        [ Links ]

[22] Rossi, E.A., Rangel-Fonseca, P., Parkins, K., Fischer, W., Latchney, L. R., Folwell, M.A., Williams, D.R., Dubra, A. and Chung, M.M., In vivo imaging of retinal pigment epithelium cells in age related macular degeneration. Biomed. Opt. Express, 4, 2527-2539, 2013. DOI: 10.1364/BOE.4.002527        [ Links ]

[23] Organisciak, D.T. and Vaughan, D.K., Retinal light damage: Mechanisms and protection. Progress in Retinal and Eye Research, 29, pp. 113-134, 2010. DOI: 10.1016/j.preteyeres.2009.11.004        [ Links ]

[24] Garrioch, R., Langlo, C., Dubis, A.M., Cooper, R.F., Dubra, A. and Carroll, J., Repeatability of in vivo parafoveal cone density and spacing measurements. Optometry and Vision Science, 85, pp. 632-643, 2012. DOI: 10.1097/OPX.0b013e3182540562        [ Links ]

[25] Wojtas, D.H., Wu B., Ahnelt, P.K., Bones, P.J. and Millane, R.P. Automated analysis of differential interference contrast microscopy images of the foveal cone mosaic. J. Opt. Soc. Am. A, 25, pp. 1181-1189, 2008. DOI: 10.1364/JOSAA.25.001181        [ Links ]

[26] Sekhar, S., El-Samie, F.E.A., Yu, P., Al-Nuaimy, W. and Nandi, A.K., Automated localization of retinal features. App. Opt., 50, pp. 3064-3075, 2011. DOI: 10.1364/AO.50.003064.         [ Links ]

[27] Takayama, K., Ooto, S., Hangai, M., Ueda-Arakawa, N., Yoshida, S., Akagi, T., Ikeda, H.O., Nonaka, A., Hanebuchi, M., Inoue, T. and Yoshimura, N., High-resolution imaging of retinal nerve fiber bundles in glaucoma using adaptive optics scanning laser ophthalmoscopy. Am. J Ophthalmology, 155(e3), pp. 870-881, 2013. DOI: 10.1016/j.ajo.2012.11.016.         [ Links ]

[28] Rangel-Fonseca, P., Gómez-Vieyra, A., Malacara-Hernández, D., Wilson, M.C., Williams, D.R. and Rossi, E.A., Automated segmentation of retinal pigment epithelium cells in fluorescence adaptive optics images. J. Opt. Soc. Am. A, 30, pp. 2595-2604, 2013. DOI: 10.1364/JOSAA.30.002595.         [ Links ]

[29] Chiu, S.J., Toth, C.A., Rickman, C.B., Izatt, J.A. and Farsiu, S., Automatic segmentation of closed-contour features in ophthalmic images using graph theory and dynamic programming. Biomed. Opt. Express, 3, pp. 1127-1140, 2012. DOI: 10.1364/BOE.3.001127         [ Links ]

[30] Valencia-Estrada, J.C. and Bedoya-Calle, A.H., Trigonometría elíptica para su uso en ingeniería. Jornada de Investigación EIA (Escuela de Ingeniería de Antioquía), 2, pp. 84-92, 2009.         [ Links ]

P. Rangel-Fonseca, is PhD candidate, received his BSc. of Electronics Engineering in 2003, in 2005 he received his MSc. in Electrical Engineering, at Universidad de Guanajuato, Mexico. Currently he is working towards on his PhD. Degree at Centro de Investigaciones en Optica A.C. His main research interest is image processing application in the biomedical area. ORCID: http://orcid.org/0000-0002-6254-0550

A. Gómez-Vieyra, is PhD, received obtained his BSc. of Electronics Engineering in 2004 at Universidad Autonoma Metropolitana, Mexico, in 2005 he received his MSc and in 2010 he received his PhD in Optics at Centro de Investigaciones en Optica A.C. His current research interest includes vision science and optical engineering. ORCID: http://orcid.org/0000-0003-0290-1518

D. Malacara-Hernández, his PhD. in Optics in 1965 from the University of Rochester. He has published more than one hundred and thirty refereed papers in Optical Engineering optics journals. He is a Fellow, both of the Optical Society of America and The International Society for Optical Engineering (SPIE). In 1987 he was elected a Vice-president of the SPIE - International Commission for Optics. He has served the Optical Society of America as a Topical Editor for Applied Optics from 1989 to 1992.He also has served SPIE - The International Society for Optical Engineering, as a member of the Board of Governors from 1990 to 1991.In 1989 he was granted the Rudolf and Hilda Kingslake Chair in Optical Engineering, at the Institute of Optics of the University of Rochester. He also received in 1994 the A. E. Conrady Award for Scientific Achievement by SPIE.-International Society for Optical Engineers and in 1996 the Galileo Galilei Award, by the International Commission for Optics, and the Joseph Fraunhofer Award.-Robert M. Burley, from the Optical Society of America in 2002. He also received in 2012 the Gold Medal Award by SPIE - International Society for Optical Engineers. ORCID: http:// orcid.org/0000-0001-8564-0119