English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
I. INTRODUCTION
The tertiary road network in Colombia plays an important role in national, regional, and local integration. This approach is rooted in the interconnections of roads between footpaths and access to the national highway by isolated communities. Tertiary roads occupy a percentage of 67% of the total road network in Colombia, representing the largest transport infrastructure in the national territory. The relevance of this road network is also reflected in the policies and efforts carried out by the national government to generate development and connectivity in rural areas affected by the armed conflict. Given that the existing documentation on tertiary roads is scarce, incomplete, and in most cases very outdated, it is necessary to develop tools that allow the roads and points of interest to be easily identified to the people who travel them. One solution for this is through the use of deep learning techniques that make it possible to detect entities in satellite images with several layers in neural networks to carry out the analysis of points of interest in a versatile way. [1], [2], [3], [4].
Recently, studies have been conducted using Deep Learning [5] - [8]. In [5], deep learning techniques are presented to predict the consumer spending of a village from satellite imagery and perform object detection and regression. Furthermore, in [6], deep learning is used for pathway state analysis and anomaly detection. On the other hand, in [7], deep learning techniques based on single class detection are used to identify road safety attributes, that is, to train a model for each attribute. In the case of [8], there is evidence of a monitoring and surveillance system based on the analysis of images captured with low-altitude drones using deep learning techniques.
Comparing the previous contributions, the purpose of this article is the development of a method for the classification and labeling of points of interest such as park infrastructures and sports spaces. In addition to the identification of tertiary roads using supervised deep learning techniques with the topology of FCN network in satellite images from Google Maps and OpenStreetMaps. The collection of complementary and more detailed information is carried out through the implementation of remotely piloted aircraft and the contribution of data by the local community. Consequently, a mobile application in Flutter and a desktop application in Python are developed to facilitate the actions of the consultants. Finally, an augmented reality system developed with Vuforia Engine in Unity is intended for the distribution of content from the road network of the study region. This research project aims to be a support instrument for decision-making in the execution of viable inventories with improvements in the action times of private and state entities to contribute to the connectivity and competitiveness of the region by applying artificial intelligence techniques and augmented reality systems for the visualization of the most relevant road documentation in the region in immersive and interactive scenarios.
The article is organized as follows: The study and planning phase of the proof of concept is presented in Section 2. This section includes data collection, artificial intelligence concepts, mobile and desktop application development, and the design of the augmented reality system for the distribution of virtual content related to the study. In Section 3, the results obtained for the classification and labeling of points of interest and identification of the most distinctive tertiary roads in the region are presented. In addition, the results of the implementation of the mobile application and the visualization of virtual content through the augmented reality system are described. Finally, in Section 4, the final conclusions of the study are presented.
II. METHODOLOGY
For the study and planning phase of the proof of concept, the Logical Framework Approach (LFA) was used to systematically and logically define the objectives of this research, facilitating the coordination and concertation of long-term strategic actions.
The case study comprises the Taminango region in Nariño. In this context, the groups involved have been analyzed and characterized, i.e., the National Planning Department; the Government; the Municipal Council; transportation, energy, communication, and construction companies; and the Taminango community. Thus identifying problems related to the dispersion of official data consulted, lack of road inventories, high financing costs, and lack of academic research in the area. For this reason, a series of alternatives that make up the structure of this article have been proposed.
A. Dataset
Initially, a map of the area of interest was downloaded through SAS.Planet - a free software for acquiring high resolution georeferenced satellite images from multiple free access sources (freeware under the GNU license), for example: Google Earth, Google Maps, OpenStreetMaps, Bing Maps, and GeoHub. In Figure 1, the sample taken from the SAS.Planet program can be observed.
For the registration of points of interest, satellite images have been extracted from Google Maps considering the identification of parks that contain a playing field. By applying image segmentation algorithms in MATLAB with restrictive conditions of color and area, it was possible to optimize the labeling process for the creation of the image bank. Figure 2. illustrates the binarization segmentation process for park labeling in Taminango.
Fig. 2 (A) Decomposition of channels in the satellite image; (B) Identification of the park using segmentation techniques; (C) Final labeling of the point of interest.
During the data acquisition process, there were difficulties in identifying the Taminango road network due to the outdated data and resolution of the images. To correct this situation, satellite images were collected through OpenStreetMaps to later carry out the preprocessing and keep only the information corresponding to the roads. In addition, the routes that were not indicated in the database have been manually marked. In Figure 3. the extraction of images for the recognition of roads can be observed.
In order to facilitate the training of a neural network with the ability to identify the road network of the region, the satellite and pre-processed images of the roads have been divided into equal parts to obtain an image bank, assigning an output to each input desired. This article presents a dataset that contains 441 inputs and 441 outputs in a resolution of 256x256 pixels. Figure 4. illustrates the process of dividing images.
By examining Figure 5. it can be seen how the image bank provides a satisfactory output to the various inputs from the raw satellite images.
B. Artificial Intelligence
For the classification of tertiary roads in satellite images, an FCN-type network architecture with a binary classification model as a problem is proposed, ruling out the use of the transfer learning technique. For the case study, the satellite images will be the training data and the binary images will be the target images, the entire database for training the model had a preprocessing in order to facilitate the training of the neural network. Due to all the tools offered by the TensorFlow library in Python, it was decided to use it for model training. The model receives as input 256x256x3 preprocessed images through the 3 RGB channels of the image, with a loss function binary cross-entropy and Adam as optimizer. Having the model already trained, it is validated with the previously assigned validation data, in this way it is possible to check its performance before being implemented. Figure 6. illustrates an example application of the algorithm.
C. Desktop Application
The design of the graphical interface was carried out in Python with the Tkinter library for the development of GUIs. The interface includes a button that allows loading an image from the user files. Then, there are two markers to select an action: detection of tertiary roads or labeling of points of interest. Figure 7 presents a screenshot of the desktop application development.
D. Mobile Application
The development of a platform called "Atlas-AR" is proposed to host the most relevant contents of the study to reduce the degree of uncertainty and the dispersion of data regarding the number and status of the tertiary road network of Taminango. The design and implementation stages of the application were carried out in the Flutter framework, this is an open-source SDK developed by Google that allows multiplatform mobile applications with a single code base, considerably reducing coding times. In Figure 8 the programming interface where the mobile application was made can be observed.
The identification of areas that are not visible on official maps or statistics required the process of gathering information through a system of remotely piloted aircraft in areas difficult to access. On the other hand, the participation of the local community was required to capture infrastructure images that are not registered in the database and the detailed description of the most relevant roads in the region. This information is stored on an Amazon Web Services server.
E. Augmented Reality
Considering previously processed images, an augmented reality interface was developed using Vuforia Engine and Unity to stimulate user interaction with virtual content generated in Blender and SolidWorks [9], [10]. For the above, multiple layers have been arranged: the first layer illustrates a satellite image, the second layer represents a relief of the region, and the third layer corresponds to the road network. This last layer contains icons and descriptive boxes about the state of the road or labeling of points of interest. The result of this configuration is seen in Figure 9.
III. RESULTS
In Figure 10. the results of the classification and labeling of a park are presented according to the algorithm shown in previous sections. Likewise, it is possible to identify the most distinctive tertiary roads in the region. In Figure 11., the final result of the mobile application is observed; it is composed of 2 stages that bring together both the visualization of the content in the database and the contribution of images through mobile phones by the local community.
Fig. 11 (A) Presentation tab of the mobile application; (B) Tab for the contribution of infrastructure images by the community; (C) Gratitude tab for users.
Finally, the visualization of the virtual content was carried out through hyperlinks in the physical world [11]. The mobile application generates QR (Quick Response) codes that link to the different models and documentation stored in the database, as shown in Figure 12.
IV. CONCLUSIONS
The development of a method for the classification of tertiary roads and the labeling of points of interest was achieved with deep learning techniques, remotely piloted aircraft, and community participation.
It is essential to define the limit of images for the creation of the dataset. This is because a greater number of images required greater processing and storage capabilities. Therefore, the strategy proposed in this document may have implications due to the angle of incidence of the sun (shadows or overexposure), atmospheric conditions, or location of the satellite.
The augmented reality system facilitated the processes of exploration, understanding, and retention of the most relevant information about the road network and points of interest in Taminango.
Currently, various studies are being carried out to scale this proof of concept to other regions using the databases developed. In addition, the implementation of drones for photogrammetry is suggested for the study and for obtaining measurements with precision of dimensions and position of the study places.
