Calibrating a photogrammetric digital frame sensor using a test field

Calibración de una cámara digital matricial empleando un campo de pruebas

Benjamín Arias-Pérez ^a, Óscar Cuadrado-Méndez ^b, Pilar Quintanilla ^c, Javier Gómez-Lahoz ^d & Diego González-Aguilera ^e

^a Cartographic and Land Engineering Department, University of Salamanca, Hornos Caleros, benja@usal.es
^b Cartographic Center, Government of the Principality of Asturias, oscar.cuadradomendez@asturias.org
^c Departamento Técnico, Hispana Suiza de Perfilados, mdpqb71@hotmail.com
^d Cartographic and Land Engineering Department, University of Salamanca Spain, fotod@usal.es ]]> ^e Cartographic and Land Engineering Department, University of Salamanca, daguilera@usal.es

Received: February 5^th, 2013. Received in revised form: January 27^th, 2014. Accepted: May 8^th, 2014.

Abstract
In this paper a twofold calibration approach for a digital frame sensor has been developed which tries to cope with panchromatic and multispectral calibration separately. Although there have been several improvements and developments in calibration of the digital frame sensor, only limited progresses has been made in the context of multispectral image calibration. To this end, a specific photogrammetric flight was executed to try to calibrate the geometric parameters of a large format aerial digital camera. This photogrammetric flight was performed in the "Principado de Asturias" and it has been designed with a Ground Sample Distance of 6 cm, formed by two strips perpendicular between each other, with five images each one and a longitudinal overlap of 60%. Numerous points have been presignalled over the ground, both check points and control points.

Keywords: CCD sensor; large format digital camera; calibration; multispectral image; panchromatic image; aerial photogrammetry.

Resumen
En este artículo se presenta un doble enfoque para la calibración de una cámara digital matricial y que trata la calibración pancromática y multiespectral por separado. Aunque ha habido varias mejoras y novedades en la calibración las cámaras digitales matriciales, sólo se han hecho limitados progresos en el contexto de la calibración de imágenes multiespectrales. Con este fin, fue realizado un vuelo fotogramétrico específico para tratar de hacer la calibración de los parámetros geométricos de una cámara aérea digital de gran formato. Este vuelo fotogramétrico se realizó en el "Principado de Asturias", y ha sido diseñado con un tamaño de píxel en el terreno de 6 cm, formado por dos pasadas perpendiculares entre sí, con cinco imágenes cada una y un recubrimiento longitudinal de 60%. Se han tomado numerosos puntos preseñalizados sobre el terreno, tanto para los puntos de control como para los puntos de chequeo.

Palabras clave: sensor CCD; cámara digital de gran formato; calibración; imagen multiespectral; imagen pancromática; fotogrametría aérea.

In the field of photogrammetry there is a great interest in optimizing the acquisition of data. It has been strengthened in recent years with the exchange of information among the manufacturers of sensors, users and experts in geospatial information. The objective is being achieved with an improvement of the methods as well as the systems used, and the implementation of new production techniques and management and processing of spatial data. The Project of European Spatial Data Research "Digital Camera Calibration & Validation" was divided into two phases: theoretical and empirical. The first was mainly dedicated to the launching of the Project, including the call for experts to form the network. In addition, an extensive report was made, where the different approaches for the calibration of sensors and the calibration methods applied by the manufacturers are documented [1]. In the second phase empirical tests based on the experiences and recommendations of experts on the procedures commonly accepted for calibration were performed. Flights were made with the following cameras: Leica ADS40, DMC from Z/I Imaging and UltraCamD by Vexcel. The data from these flights were distributed among the members of the network who took part in the second phase. The most important results obtained are shown in a report made by Cramer [2, 3]. From these results it should be remarked that the environmental conditions in the taking of frames are different from the laboratory conditions where the manufacturer has done the calibration. So users have to perform the calibrations "in situ" (on site) to validate and refine the calibration parameters provided by the manufacturer. The calibration of the system in-flight is not common, so far, in the traditional aerial Photogrammetry, so there is a general ignorance of the characteristics and advantages of the method.

The camera behaviour is not the same when tested under laboratory conditions as when performing under flying conditions and thus, some additional parameters are typically introduced when the self calibration approach is applied [4].

The results provided by the standard photogrammetric model are usually affected by the departure of the theoretical model from the camera actual geometry as well as by the existence of a certain correlation between the parameters used in it, basically between some of the interior parameters (camera geometry) and some of the exterior parameters (camera position and attitude).

The additional parameters are usually split into three major groups: the first group consists of those parameters that belong to a mathematical or physical model. The second group of parameters does not account for a functional cause but rather uses an empirical expression that has been proven useful from tests. A third group comes from the blending of these two groups.

In any case, the mentioned discrepancies can be determined and assumed with the introduction of additional parameters in the adjustment of the block of images. Specifically, the introduction of additional parameters mainly affects the increase of the vertical accuracy due to the limitation in the height/base ratio of digital cameras.

As an example, diverse works that show that the main point of auto-collimation of this cameras is variable have been published [5], and this produces effects not only on the images obtained with this cameras, but in the whole set of sensors (GPS, Inertial Measurement Unit) involved in the capture of data. In [6], the results of determining the misalignment of the system of inertial measurement are presented by two companies that operate with UltraCamD. For one of them everything worked correctly, but for the other one some unexpected results permit one to detect a systematic trend that is finally due to the principal point of autocollimation of the camera. This reveals the necessity to contrast and to validate the internal parameters of these new photogrammetric aerial cameras. Therefore, the issue of the calibration of digital cameras of large format is in fact a matter of great relevance and high interest. Test flights were performed specifically to contrast the internal parameters of a camera (focal length and position of the principal point) together with additional parameters, especially those related to radial lens distortion and some systematic trends. Likewise, a twofold calibration approach has been developed trying to cope with panchromatic and multispectral calibration separately. Although there have been several improvements and developments in calibration of digital frame sensors, only limited progress has been made in the context of multispectral image calibration. More recently, the results published in [7] show that the geometric calibration of the panchromatic aerial images is well known. However, no attention is paid to the geometric calibration of the multispectral images of these cameras.

The paper has been structured as follows: after this introduction in Section 2, a detailed description about the sensor, the calibration field, the flight requirements and the computation methods are provided. In Section 3 the experimental results are outlined and discussed. A final section is devoted to point out the main conclusions.

2. Materials and methods

2.2. Calibration field and flight requirements
GPS is the name for the Global Positioning System (NAVSTAR) which permits the location of a fixed or moving target on the earth surface within an accuracy of a few centimeters (if the differential GPS is used in any of its varieties) although the expected usual standard accuracy is a few meters. The system has been developed and is operated by the Department of Defense of the USA.

The initial constellation has been completed by several initiatives: GLONASS (Russia), GALILEO (Europe), BEIDOU (China). All these systems share the same purpose: a global positioning. From now on we use the term GNSS for Global Navigation Satellite System.

For an absolute positioning with a single GPS receiver (GNSS), the expected accuracy ranges from a few decimeters to a few meters. To improve this accuracy a second receiver is involved so that they are referenced to each other and not to an absolute framework. This also permits that one of the receivers can work in a dynamic fashion while the other (the base) is kept fixed at one position. When both receivers communicate with each other in real time by radio or modem or wifi, exchanging data received from the system and thus allowing for correcting their relative positions, this technique is known as kinematic relative positioning or Real Time Kinematic (RTK) positioning and leads to an accuracy of some centimeters. It is the way how the control points of this work have been measured.

Having in mind that the smallest Ground Sample Distance (GSD) is 7 cm and assuming an image accuracy of 1/3 of the GSD we get a photogrammetric accuracy of 2.33 cm. Provided that the GNSS technique employed guarantees a precision better than 2 cm we can certify that this data are enough to be used as control points.

The calibration field is located in the Technologic Park of Asturias (Spain), in the council of Llanera, next to the airfield of La Morgal. This area is chosen because, on one hand, it allows the establishment of a set of presignalized control points (evenly distributed over the working area) with good temporary stability and, on the other hand, enables the use of road marks as presignalized points available for both their measurement with GPS techniques as in the images themselves. Besides this, the buildings located in the surroundings have been used to incorporate points at different heights which can be perfectly identified in the images. A total of 52 presignalized control points were measured with GPS techniques (RTK with a baseline of 500 m., with centimetric accuracy) as well as 581 points at road marks obtaining coordinates in the cartographic projection Universal Transverse Mercator-UTM and ellipsoidal heights referred to the Geodetic Reference System, European Terrestrial Reference- ETRS89.

The measurements of the image coordinates both manually and automatically were performed with Match-AT v.5. from Inpho. To give more consistency to the calculation of the internal parameters, 124 tie points located on the roofs of the buildings were manually measured. The flight requirements consist of two strips in the shape of a cross, each with 5 images and with a longitudinal overlap of 60%, covering a surface about 4.6 ha. The first strip was performed in NW-SE direction and included the images: 309, 310, 311, 312 and 313. The second strip was carried out in SW-NE direction with the images: 314, 315, 316, 317 and 318. The GSD used is 6 cm, corresponding with a flight height of 675 meters approximately (Fig. 1).

The calculations for the calibration of the camera are of two types: bundle adjustment and spatial resection [10]. If we use several images with overlap between them, it is preferable to use bundle adjustment, taking advantage of the geometric robustness that provides both automatic and manual measurements of image coordinates of the points in different images. On the other hand, when using a single image, an option for calibration is spatial resection or inverse intersection. In particular, an iterative process is launched in which the redundant parameters are flagged for deletion and eliminated in the next iteration. This automatic selection is made according to various criteria [9].

3. Experimental results and discussion

3.1. Calibration with the panchromatic image
The results are shown in tables with the following data: Control points: number of control points used; c: focal length in millimetres; Sc: standard deviation a posteriori of c in millimetres; x_H, y_H: image coordinates of the main point of autocollimation in millimetres; Sx_H, Sy_H: standard deviations a posteriori for the image coordinates from the principal point of autocollimation in millimetres; s0: standard deviation a priori of the image coordinates in microns; S0: standard deviation a posteriori of the image coordinates in microns; Ratio: quotient between the standard deviation a posteriori and the standard deviation a priori of the photo coordinates.

The calculation of the Bundle Adjustment was separated by using the initial approximations obtained (Table 1) or not using them (these results are pretty much the same to those outlined in Table 1).

The results (c, xH, yH) are very similar whether or not the initial approximations are used, so that in this case they could be omitted. First, the computed focal length, c, barely varies from the nominal value (101.4000 mm). Regarding the main point of autocollimation (xH, yH), it scarcely separates from the origin (0,0) and the displacement could be estimated as 1/4 of the pixel size (1.8 microns). Furthermore, the use of numerous control points does not improve the standard deviations (Sc, SxH, SyH) including the standard deviation a posteriori (S0). Nevertheless, for both cases S0 is lower than the standard deviation a priori (s0). Second, the spatial resection was calculated for all the images except for those placed at the extremes of the flight strips because they had few Control Points and they were not properly distributed along the whole image.

Table 2 shows two calculations for the image 310, depending on the use of only the presignalized control points or on the use of all the points measured (presignalized and roadmarks). The results obtained scarcely vary the initial nominal values. Again, the focal length, c, presents slight variations in relation to its nominal value, whereas the principal point of autocollimation (x_H, y_H) evidences also small variations from the origin (0,0).

As can be observed through an analysis of standard deviations, the results are slightly worse than those obtained by means of Bundle Adjustment. This is coherent since the geometry provided by Bundle Adjustment is more robust. Besides, the use of numerous control points worsens the standard deviations and a similar output is observed in the case of Bundle Adjustment.

Therefore, the following conclusions related with the panchromatic image calibration could be pointed out: Firstly, the use of more control points does not modify the result and worsens the standard deviations. This may be due to the weighting criteria of the control points. Since these points are measured manually, their precisions can be reasonably supposed to be worse than those of the automatic measured points. In any case, the ratio between a priori and a posteriori standard deviations stays under an acceptable threshold. Secondly, it is not required to use initial approximations, so we can afford to work with unknown nominal values and perform the calibration; and lastly, as the standard deviations are slightly better in the case of Bundle Adjustment, the results obtained through space resection are totally valid.

3.2. Calibration with the multispectral image
The UltraCamD camera has four cones to generate multispectral images, corresponding to Red, Green, Blue and NIR. Each cone is associated to a CCD, in such a way that it captures the whole area that is covered by the panchromatic image (through its 9 CCDs) and therefore, they have lower resolution on the terrain. That is why a procedure known as pan-sharpening, widely used in remote sensing, is applied which, based on the fact that the colour is a property of the area, to give the multispectral images the highest resolution that the final panchromatic image offers.

With this flight the calibration of one of the multispectral cones has been made by means of bundle adjustment, the one (cone n° 4) using the presignalized control points since the low resolution that this image offers does not allow the road markings measured on the ground to be correctly distinguished. In this case, the image corresponds to level 0 (without any type of processing), with a focal length of 28 mm and a big radial distortion. So the calibration consists basically in determining radial distortion.

For the calculation we have used the additional parameters of radial distortion, 25 and 26. The results obtained using manual measurements are outlined in Table 3. The results with automatic measurements are identical except for the value of S₀: 2.19 µm.

It should be noted that these computations have been performed using the same software (BINGO) that the manufacturer does.

However, it is not common the use of 7 and 8 parameters together with parameter 26. This could explain the difference obtained between our results and those provided by the manufacturer. In particular, the change of sign in the distortion is due to the different use of the pairs of parameters 7–8 or 25–26. Another important aspect that could explain these differences is the environmental conditions of the image acquisition, since the manufacturer calibration is carried out in laboratory whereas our calibration is performed in a field test.

4. Concluding remarks

In this paper, the results for the calibration of a large format digital camera for aerial photogrammetry UltraCamD model have been presented, with images taken in-flight. This means a change from the usual calibration in the laboratory. Through two methods of calculation, bundle adjustment and spatial resection, the accuracy of calibration parameters for the final image has been verified. The results show a higher accuracy and reliability of the calculations by bundle/beam adjustment in contrast to spatial resection, as was expected. However, the distribution of the image coordinate residuals shows the contribution of the 9 CCDs on the matricial image. One possibility to attenuate the influence of these 9 areas is the application of special additional parameters. Another possibility is the calibration in-flight of the 4 cones for the 9 CCDs of the panchromatic image at level 0, and to introduce the results of this calibration in the processing of the image until reaching level 3. This would be as if the cones were considered as the processing unit and not the whole image itself. To do this, the flight should be planned so that a large overlap between the CCDs themselves (and not between the images) can be guaranteed. This would demand firstly, that the calibration field depending on the image scale should include, a very large number of road marks as candidates to be control points, as well as the presignalized points, so that they are imaged on the same CCD for different images. Secondly, the longitudinal overlap between two adjacent images positions should be of about 80% (flying base of 20%). In this way there would be an adequate overlap between the CCDs (with the standard 60% overlap this is not achieved), and we could perform a calibration by bundle adjustment (since this calculation is much more robust than the option of spatial resection) for the CCDs as a processing unit. Note the impossibility to perform a strip with this 80% overlap for this size of GSD since the camera cannot operate at such a high frequency nor the plane fly so slowly. But this problem can be solved by performing additional strips with exactly the same trajectory as the original ones but with the projection centers shifted along the trajectory half the size of the standard flying base.

References

[1] Cramer, M. Digital camera calibration. Amsterdam: European Spatial Data Research, 2009.         [ Links ]

[2] Cramer, M. The EuroSDR performance test for digital aerial camera systems. Proceedings of the Photogrammetric Week, pp. 89-106, 2007.         [ Links ]

[3] Cramer, M. The ADS40 Vaihingen/Enz geometric performance test. ISPRS-J. Photogramm. Remote Sens. 60 (6), pp. 363-374, 2006.         [ Links ]

[4] Ladstädter, R., Tschemmernegg H., Gruber, M. Calibrating the UltraCam aerial camera systems, an update. Proceedings of International Calibration and Orientation Workshop EuroCOW, 2010.         [ Links ]

[5] Honkavaara, E., Ahokas, E., Hyyppä, J., Jaakkola, J., Kaartinen, H., Kuittinen, R., Markelin, L., Nurminen, K. Geometric test field calibration of digital photogrammetric sensors. ISPRS-J. Photogramm. Remote Sens. 60 (6), pp. 387-399, 2006.         [ Links ]

[6] Arias, B., Nafría, D.A., Blanco, V., Rodríguez, O.O., Blanco, M., Antolín, F.J., Rodríguez, J., Gómez, J. Testing a digital large format camera. Proceedings of International Calibration and Orientation Workshop EuroCOW, 2008.         [ Links ]

[7] Jacobsen, K., Cramer, M., Ladstätter, R., Ressl, C., Spreckels, V. DGPF project: evaluation of digital photogrammetric camera systems - geometric performance. Photogramm. Fernerkund. Geoinf., 2, pp. 85 – 98, 2010.         [ Links ]

[8] Gruber, M., Ladstätter, R., Geometric issues of the digital large format aerial camera UltraCamD. Proceedings of International Calibration and Orientation Workshop EuroCOW, 2006.         [ Links ]

[9] Kruck, E.J. BINGO user's manual. Geoinformatics & Photogrammetric Engineering, 2006.         [ Links ]

[10] Kraus, K., Jansa, J., Kager, H. Photogrammetry. Bonn: Ümmler, 1997.         [ Links ]

[11] Gruber, M., Kröpfl, M. Calibration report. Microsoft Photogrammetry, Graz, Austria, 2007, 66 P.         [ Links ]

B. Arias-Pérez is Professor at Salamanca University since 2010. Previously, he was professor at the University of León between 2004 and 2010, and in the University of Salamanca between 2003 and 2004. He accomplished a BS Degree in Surveying Engineering at the University of Salamanca in 1999. Also he completed a MS Degree in Geodesy and Cartography in 2002 at Salamanca University. He received his PhD from Salamanca University in 2008. He teaches in subjects related with geomatic and his research lines are focused on aerial photogrammetry and radar images.

O. Cuadrado-Méndez is chief of photogrammetric mapping projects in the Cartographic Center of the Principality of Asturias (Spain). He accomplished a BS Degree in Surveying Engineering in 2001. Later he completed a MS Degree in Geodesy and Cartography in 2006 and a Master in Cartographic Geotecnologies in Engineering and Architecture in 2011 at Salamanca University. It is currently developing his doctoral thesis within the doctoral program "Geotechnology Research and Development" at the University of Salamanca. Has an experience of 17 years in the field of Surveying, Mapping and Geographic Information Systems and Remote Sensing. He previously held various positions in the private sector. Currently he is responsible for the implementation of free software in the field of GNSS positioning within the administration of the Principality of Asturias.

P. Quintanilla has obtained her BS Degree in Surveying Engineering in 2009 and her BS Mining Engineering in 1999, both from the University of León. She has worked in construction projects such as irrigation pipelines and road tunnels until 2012. She has completed studies of Building Projects in 2014 at the IES Virgen de La Encina in Ponferrada. Currently she is engaged in building, focusing her work on measurement techniques for the geometric definition of buildings and its 3D modeling.

J. Gómez-Lahoz is Professor at Salamanca University since 1993. He has completed studies at the Complutense University of Madrid in Education Sciences in 1984, at the Polytechnic University of Madrid in Surveying Engineering in 1993 and has obtained his PhD at the University of Salamanca in 1999.

His thesis was the result of his two main academic and professional concerns, the field of computer aided learning / teaching of engineering, and the field of cartographic and photogrammetric engineering. This constitutes one of his researching lines, being the other the application of low cost aerial and terrestrial photogrammetry to world heritage. He has fostered, participated in and disseminated research projects on both lines.

D. González-Aguilera is Professor at Salamanca University since 2002. He accomplished a BS Degree in Surveying Engineering at the University of Salamanca in 1999. Also he completed a MS Degree in Geodesy and Cartography in 2001 at Salamanca University. He received his PhD from Salamanca University in 2005. Based on his thesis's results he obtained two international awards of the International Society of Photogrammetry and Remote Sensing (ISPRS). He has authored more than fifty research articles in international journals and conference proceedings. Currently, his teaching and research lines are focused on close range photogrammetry and laser scanning applied to engineering and architecture.