English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
1. Introduction
In the context of government activity, more and more organizations are joining the Open Government initiative. Some authors [1] define this concept as a policy that groups several public concepts to improve aspects of transparency, participation, and collaboration of citizens. The pertinent aspects of access to public information, as well as citizen participation and transparency in public management, both actively and passively, are booming at the international level.
This new paradigm proposes a vision of public management with a special focus on three fundamental principles: a) access to public information (public and open data), b) transparency and c) citizen participation and collaboration. To carry out this model, it is necessary to set specific objectives in the management and definition of the cultural organization of citizens, as well as in the treatment of technological processes that facilitate access and connectivity for all the people. In the United States, for example, president Obama promoted in December 2009, the creation of a system of transparency, public participation, and collaboration with an Open Government directive (Memorandum on Transparency and Open Government) [1]. The axes of this opening strategy are four: I) the online publication of government information in open and reusable formats (open government data); II) improvement in the quality of public information; III) the creation and institutionalization of a culture of Open Government, and IV) the creation of a common regulatory framework to support it [2].
In September 2011, based on the new paradigm shift in matters of an increasingly open State, the Open Government Partnership (OGP) [3] was constituted as an alliance that is formed by people from the government, the civil society, companies, non-profit organizations and more; this network brings together the different initiatives of various governments around the world. Its mission is to introduce governments to the implementation of best practices to carry out an Open Government, both in their reform commitments, as well as management procedures, skills and technologies to be used. This includes non-governmental organizations, as well as experienced private companies [4]. According to the Research Group on Government, Administration and Public Policies (GIGAPP) [5], an essential part of open government consists in applying the term of transparency, which provides information on what the government is doing, and its action plans and data sources. As a result, more governments are joining in this strategic plan of transparency and participation together with citizens. Therefore, different public organizations are increasingly implementing aspects in their open government data portals, which provide data that can be accessed and used for their own needs. This data availability not only facilitates data-driven decision-making, but also directly influences people's trust and satisfaction with the government, since open government data (OGD) allows citizens to monitor the performance and management of the government [6].
In Latin America, countries such as Mexico, Chile, Uruguay, Brazil, Colombia, Peru, among others, have made significant progress in terms of public information, transparency, and open data, and are evaluated in international open data indexes such as the Global Open Data Index (GODI) [7] or the Open Data Barometer (ODB) [8]. The technology is an important element directly related to open data and allows carrying out a policy based on public and open data in each state. This will make it possible, in addition to the dissemination of information, to collaborate in the transparency of governments and in the participation of any citizen without any type of technological obstacles [2]. For this concept to be successful, some of the authors, such as [9], suggest that governments should ensure the reliability of the data and promote spaces for collaboration where the generation of public value is enabled, guaranteeing the confidentiality of personal data. It is important to note that it is necessary to advance in the standardization of quality, accessibility and easy-to-use publication formats for open data to generate value. For these reasons, it is required to have specific software techniques and functionalities that help in the detection of data-focused issues (integer data, complete data, redundant data, etc.). Detecting these weaknesses in time and correcting them will help us maintain the quality of public data and, in turn, will provide greater confidence in government data sources.
This article describes a first observation and analysis of the open data portals of some Latin American countries, randomly selected. The data sets for the study were taken from public government websites of the selected countries. To carry out the analysis, a proprietary software tool called HEVDA was used, which allows the identification of several deficiencies in quality aspects of open data. Finally, the results obtained from the analysis are presented with the aim of making a comparison of the current status of the quality of the data that these countries provide to their citizens.
2. Quality aspects in public data
Public sector information made available as open data is called “Open Government Data” [10]. One of the fundamental aspects mentioned by the Economic Commission for Latin America and the Caribbean (ECLAC), [11] one of the five regional commissions of the United Nations, is that governments collect a large amount of high-quality data. Normally, this translates into the State becoming a powerful data monopoly capable of structuring and homogenizing the interactions between itself and its citizens. These unilateral interactions are expensive and do not necessarily respond to the needs of citizens, unnecessarily limiting government activities [12]. Therefore, it is necessary to have techniques and strategies for verifying the quality of the data provided to the community, and in this way, to obtain an analysis that offers added value to society. Government data openness involves both technical and political considerations. Having open data available on public government portals does not necessarily imply that there is quality in them; that is why governments need to invest in improving the quality of their data, particularly in formats, metadata and publication times. The quality improvement not only allows a positive advance in the use, but in the internal processes of the government, which in turn allows optimizing the decision-making [13].
2.1 About the problem
Maintaining the quality and openness of public data that is offered to people helps governments and different actors in civil society to make better decisions. It also makes it possible to monitor processes, build indicators and make transparent how public money is spent [14]. One of the points to consider in open data portals is that the availability of the data does not necessarily match the quality of the data. Data quality continues to be a problem in Latin America and the Caribbean; this issue is really a major challenge for public policies. The analysis of many of the public datasets represents a crucial problem, without a doubt, since it is scattered, not standardized and, in many cases, out of date. “The same can be said of the information necessary to monitor transparency in public purchases and the evolution of the current COVID-19 pandemic” [15]. The problem in this context is also about updating the technical infrastructure for open data, since data management practices are weak and inconsistent, and change too frequently. Governments must work on technical and organizational transformation to open data. They need to invest in hardware and software tools. Still, other investments are also necessary, such as: “technical standards, training, organizational transformation and decision making processes to give adequate support to data management practices” [16]. Although there are many guidelines focused on the quality of public open data, there are still some barriers that range from the insufficient quality of the published data to the lack of maintenance of the portals where they are published [17]. Several of these guidelines [18] are oriented to the opening of a dataset, but not to the analysis of the potential problems that open data already implemented and published in the government portals may have.
3. Situation in Latin America
Latin America and the Caribbean is a region highly committed to the open data agenda. Several countries in the region have appropriate regulations on access to public information and have implemented open data portals and are building legislation on the protection of personal data [9]. There are initiatives, such as the Latin American Open Data Initiative (ILDA), which seeks to standardize the collection and publication of these public data, and promote its reuse. One of the ILDA reports shows the analysis of the Regional Open Data Barometer (ODB) for Latin America and the Caribbean 2020 [13] which indicates that despite the variability in public open data policies, data user communities are becoming more active and diverse. There are more ambitious projects based on open data seeking to generate social impact. Therefore, technical communities are also involved in issues related to the quality of public data, for example, with public health data in the context of COVID-19. This report covered 24 countries, 16 from Latin America (Argentina, Bolivia, Brazil, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Guatemala, Honduras, Mexico, Nicaragua, Panama, Paraguay, Peru and Uruguay) and 8 from the Caribbean (Bahamas, Belize, Guyana, Haiti, Jamaica, Dominican Republic, Saint Lucia and Trinidad and Tobago). This measurement also classifies the governments of these countries according to a) Preparation for open data initiatives; b) The implementation of programs; c) The impact that open data is having. Among the main findings of the report in the region, the following stand out: a) Latin America and the Caribbean, is today a region that publishes more data with respect to the last evaluation made in 2016; b) The average score for the 24 countries of Latin America and the Caribbean was 40.38 points, out of a total of 100 possible points, this shows a great opportunity [19]. According to studies [20], the countries of the region should make greater efforts to improve the participation of stakeholders in the quality and integrity of the data; that is, more high-value data sets should be shared, and their accessibility improved, that is very important because the evolution of open data is in full swing, there is still a long way to go. Public bodies should invest in improving their data portals and statistical systems to increase the availability and accessibility of their datasets. According to a report [15] by CAF (Development Bank of Latin America) [21], international cooperation and development banks have been instrumental in advancing this agenda in the region, by strengthening the capacities of the public sector in many cases. In this regard, CAF has carried out actions to support open data policies, for instance, in Mexico City for the use of data for predictive models in the face of the COVID-19 pandemic; in Ecuador for the implementation of the Inter-American Anticorruption Open Data Program; in Colombia worked on a model of data intelligence; and in Buenos Aires, where it is working towards a new stage in its policy and strategy for the reuse of open data. Similarly, at the regional level, the use of open data has been supported to promote govtech [22] ventures, the development of data infrastructures for artificial intelligence and recommend open data publication standards related to the management of the COVID-19 crisis [15].
In the following section, the open data portals of six Latin American countries are described and analyzed, together with the guides, standards and good practices that governments recommend in order to maintain the quality of the data provided to citizens and organizations.
Argentina
Argentina is one of the countries that encourages citizen participation, and offers events and open talks to citizens, private and state companies. One of the last meetings was the one held by the Autonomous City of Buenos Aires, regarding the Fourth Open Government Action Plan 2019-2022 [23], which consists of 18 commitments seeking to develop transformative policies to solve specific problems on the public agenda. In addition to the commitments assumed by the country, the areas responsible for carrying out this work were assigned. The participants in this event were: the National Open Government Directorate, on the part of the Chief of the Cabinet of Ministers [24], General Audit of the Nation [25], with the special participation of the person in charge of the Citizen Participation Area, who added new approaches in the country's auditing area, in order to support and promote transparency in the data of the different public agencies, and the scope of the Open Government initiative towards them, showing the commitments focused on the mechanisms for linking with civil society in the audit cycle [26].
On the other hand, one of the actors in raising awareness of this new paradigm is the Access to Public Information Agency [27]. This Directorate promotes some variables for the issuing of public information, such as: Affidavits, Purchases, budgets, among others. The main objective is to provide a communication channel with the public to provide advice on requests for public information and collaborate in directing the request and refining the search (technological platform, legal framework, etc.). The National Magistracy Council [28] is another actor involved in this issue through the Open Council and Citizen Participation Unit [29], which assures that the outbreak of the Pandemic has become a new opportunity to advance in various designs in public policies related to the concept of Open Government, so this generates a change in behavior in the various public bodies. Additionally, the Honorable Chamber of Deputies of the Argentine Nation [30] is another central nucleus that currently promotes the importance of public data and transparency in the country. Likewise, the Fundación Huésped de Argentina [31] participates on behalf of Civil Society, and especially in matters of what is public knowledge on issues related to corona virus; the Foundation is an Argentine organization with a regional scope that has been working since 1989 in areas of public health. This is part of the national Open Government table, as each country must have a link between Government and Society. These affirm that not all organizations have the same interests or resources to participate in Open Government issues, so it is necessary for organizations with fewer resources to be able to articulate with those with more resources to be empowered in this context.
The official government portal of open data in Argentina [32] presents approximately 1,040 datasets. Guidelines on various topics are presented, such as recommendations to publish data in open formats [33], how to structure a table, name files and columns, use basic standards, and work with spreadsheets. On the other hand, good practice guides can be accessed for the use and publication of metadata [34], and data opening guides in organisms of the National Public Administration as well as in Subnational Governments. On the other hand, documents and some resources are presented to promote the openness and consumption of public data [35]. Regarding the quality of the data sets, good practices are presented to improve the quality of the datasets; for example, it is indicated that they must be in enabled open formats, for example: CSV, XML, JSON, among others. For software developers, data transformation and cleaning routines that follow good practices can be implemented through a tool called Data Cleaner [36], which is in the testing stage. This allows the use of script templates to clean or replace characters, indicate line breaks, remove columns and rows, normalize date types and separate specific fields from a dataset. While it is an interesting tool, it does not provide detailed dataset quality analysis. One of the aspects added by the Argentine Government is the standardization in some fields determined as interoperable [37]. This only applies to data of the type of natural and legal persons that follow the standard, as well as geographical data (provinces, departments, municipalities, towns, streets or coordinates) in which an API of the Geographical Data Normalization service can be used [38].
Colombia
This country is highly committed to open data and the quality of its public data offered on its main open data portal [39]. It presents a knowledge base that includes Manuals, Guides, Videos, Tutorials, Presentations, Studies, Conferences and other support materials to learn about open data [40]. It currently has 4,848 datasets on different topics about the Colombian Government to research, develop applications, and create visualizations and stories. In aspects of data quality, Colombia maintains a special section on its website, which presents the evolution of the data, for example: trends, updates of the data sets, as well as representative graphs and disaggregated by date ranges. The government of Colombia has a database that contains the result of the diagnosis made by the quality tool of the national open data portal datos.gov.co [41], where the ratings are presented in terms of the quality criteria of the guide of open data, in addition to having a viewer [42]. In other words, as indicated on their portal, “citizens can graphically observe the result of the quality analysis carried out on the data sets published on the datos.gov.co portal. This measurement was carried out based on the criteria defined in the Guide to Standards of Quality and Interoperability of Open Data of Colombia applied in a consolidated way over the total data sets; it can also be viewed by categories and individually” [43]. The results of the measurement of the quality criteria of all the datasets are divided into categories. For each of these categories, the calculation methodology used is shown by means of a data quality guide [44], in which standards that guarantee quality and allow data interoperability are detailed. The document explains the process of evaluating the quality of the information in the context of the Transparency in Open Government project.
Another document [45] offered on the website of this country provides guidance and good practices for developing strategies for opening and using data. As highlighted in detail, an inventory of resources is presented on the site, where users can track the performance of data releases, maintenance of metadata, and a summary of data from the website [46]. Finally, regarding quality, Colombia recently presented LEILA - Data quality library [47]. The main objective of this library is to offer a tool that facilitates the verification of database content and provides quality metrics so that users can decide if their databases need to be modified to be used in projects. The library was written in the Python programming language and can analyze structured databases that become pandas dataframe objects (Pandas provides tools that allow reading and writing data in different formats). It contains three main modules, the Data Quality module to analyze any database, the Gov Data module to connect with the metadata of the Colombia Open Data Portal and use its databases, and finally, the Report module, which allows generating a quality report using the previous modules [48]. As part of the implemented technology, the Socrata Open Data API [49] allows users to programmatically access a wealth of open data resources from governments, non-profits, and NGOs around the world.
Mexico
Regarding the Government of Mexico, in its open data portal [50], there are 9,415 datasets focused on different categories (with different formats), which are from the states, municipalities and autonomous bodies that are part of this data bank, to increase the benefit of opening public information for citizens. This website shows tools [51] that present a series of interactive resources, APIs, maps and web and mobile applications that facilitate the use, visualization and generation of new knowledge with open data, as well as access to an official blog of the country in aspects of Open Government [52]. The Mexican government indicates that it is important to publish information that is socially useful, of quality, friendly, with language understandable to citizens and that fosters accountability on the priority programs and projects of the federal government, as indicated in the Guide to the Transparency, Open Government and Open Data Policy from the APF 2021-2024 [53], but in its data portal, there are no topics focused or dedicated to the quality of the datasets. On another of the sites related to public data in Mexico, there is a guide to open data in Mexico [54], 4 basic steps are presented: 1) Plan, 2) Publish, 3) Perfect and 4) Promote, which contain specific actions to comply with the provisions established in the Open Data Decree. Some of the aspects that are considered are recommendations on open formats and could be related to quality, but from a general perspective. Some considerations that are analyzed are interoperability (standards, guidelines, suggestions, among others) and usability aspects of the data. Additionally, a tool called ADELA [55] is presented, which is the platform to publish in datos.gob.mx and monitor compliance with the Open Data Policy in an organization, which would implement good practices in the publication of the datasets. Finally, in the Developers section of the portal, there is content on how to access the metadata of the data sets published in datos.gob.mx through APIs, videos, guides and code examples for the use of APIs and open-source tools [56].
Mexico is a great innovator in open data aspects, but the authors of this work did not find reference guides that are 100% focused on quality or validation tools. If citizens or organizations find errors in the datasets (which are already published in the open data portal), they have the possibility of reporting problems that were detected with the data. At the top of the dataset, when clicking on the “report” function, it opens a new tab to indicate the details of the problem or suggest improvements. An email address will be requested to allow the follow-up of this problem.
Paraguay
In the open data portal of the Government of Paraguay [57], there are 182 datasets. These are identified by topics, tags, formats, and organizations. Unfortunately, not all datasets comply with open formats. Paraguay presents a simple portal in which the organizations that collaborate with public data made available to citizens and organizations are presented. On the other hand, brief stories of open data and a small section of applications and ideas are presented to use with datasets through mobile applications (mail, budget, hiring, housing distribution, etc.). Finally, open public data can be requested through an option that shows a form, which must be completed, indicating in which Paraguayan public sector the information is required.
Although a good practice guide on data quality is not shown in the open data portal, there is an Open Data Implementation Guide [58], which is a technical document to publish data in open formats and the necessary conditions that must comply.
Peru
In the open data portal of Peru [59], there are 2,055 datasets, which are identified by formats, labels and categories.
Although the Peruvian Government portal does not show good practices or aspects of data quality, there are guides, such as the Peruvian government data opening guide [60], the model and strategy of open government data in Peru [61], which indicate the importance of a correct process of opening and reusing data, helping public entities to improve the quality of data, and contributing to the improvement of information management. Additionally, a document explaining the National open government data strategy of Peru 2017-2021 [62], indicates several guidelines, but focuses on the importance of strengthening the infrastructure of the technology used for a good opening of data.
Uruguay
In the case of Uruguay, its open data portal [63] shows 2,320 datasets with identification of the organizations to which they belong, and the possibility of filtering by formats, categories, labels, and type of licenses.
This country has a high commitment to the open data issue, since many activities can be observed on its web portal, with more than 54 organizations participating in the opening of public data. On the other hand, a section dedicated to the exhibition of applications developed with the use of open data is presented [64]. Some of these examples are: COVID-19 vaccination data monitor, mobile applications on various topics: citizen participation, data viewer on domestic violence and gender, energy viewpoint, where to recycle, national postal network and local service to citizens, applications on education, student rights, among other innovative ideas. Most of these applications are published on GitHub.
As indicated in its open government portal, the National Open Data Catalog of Uruguay is a tool that allows access to open data from public organizations, academia, civil society organizations and private companies. Anyone can freely use published data to tell stories, develop research, show visualizations, implement civic applications and entrepreneurship [63]. Good practice approaches are provided, regarding quality aspects of the published data, [65]. However, it is clarified that there is no strict control, and that data quality is the responsibility of each of the data portal publishers.
The information provided is complete, and inquiries about the opening of data are supported. In addition, the obligations of the owner of the portal, users, issues related to third-party portal links, protection of personal data and the procedure for the denunciation of available content are explained [66]. In the document [67], a special section is made on the quality of the data to be published in the data portal, considering some aspects such as: a) quality of metadata; b) data quality: adopting open standards, format resources that allow interoperability between software, avoiding unnecessary data, including descriptions, among others.
Looking in more detail, there are practical documents that are indicative to provide a better publication and opening of the data; this is the case of the Active Transparency Open Data publication instructions [68], or the guide for the publication of Active Transparency datasets (containing 26 templates in xlsx format templates, which are orientated for publication [69]). They can be downloaded the document with the detail of templates and standards of Open Data for Active Transparency [70].
4. Proposal. Analyzing the quality of open government data related to COVID-19 in Latin America
As is public knowledge, the world is going through a severe pandemic context. Evidence shows that open data in governments are strategic allies for decision-making. The data helps to fight collectively, and especially in the current health context that people go through, they can help in the administration of the resources destined to face COVID-19, and, in addition, design technological tools that are useful in crisis and that can serve as a support to the actions in the short term [20]. The COVID-19 crisis has demonstrated the need for quality data to make good decisions and an active civil society to monitor how it is used. It also demands greater coordination between all public institutions involved in the management and generation of data, so that the regulation of data privacy enables its use in an ethical manner and under the highest standards of confidentiality [71]. One of the most outstanding works on open data on the pandemic, is the one carried out in Brazil, through the organization Open Knowledge Foundation Brazil [72], which put together a transparency index of Covid-19 data to evaluate the status and quality of information reported by each state in the country.
According to Serale [71], the collaboration to generate quality data is important; managing the “new normality” requires improvements in the quality of the data and its publication in reusable formats”. This requires that the agencies in charge of opening data work actively with all the sectors (transport, health, finance, social development, etc.). Another of the analyzed works [73] indicates the importance of systematizing the available tools and the data use cases to disseminate good practices. It also explains that the Global Partnership for Sustainable Development Data compiles resources [74] to share information and improve data quality. On the other hand, it explains that the Open Data Watch has also launched a site [75], in which the available information is organized according to the data value chain. Finally, Serale [73] explains that both organizations have collaborated with statistics (information repository) [76] through tools and examples to respond to the emergency.
In this context, this article describes the efforts to validate the quality of open data sets from Latin American countries, focusing on the pandemic. To this aim, 15 datasets from the portals of each of the countries presented in the previous section are taken as a sample of health aspects of COVID-19, to validate the quality of critical and non-critical metrics through a validation tool, called HEVDA (Herramienta de Validación de Datos Abiertos, or in English Open Data Quality Validation Tool) [77]. HEVDA provides new functionalities for measuring relevant quality metrics. The main objective is to verify the quality of the public data provided by governments to citizens and observe the main shortcomings of these data sets.
4.1 HEVDA
The HEVDA tool allows validating different proposed metrics for an open dataset in CSV (Comma Separated Variables) format.
In summary, it facilitates an automated analysis, providing detailed information on the erroneous aspects detected in terms of data quality. One point to keep in mind is that this tool does not alter or modify the analyzed source.
The use of this software tool is very friendly for the user, since it allows selecting a file (dataset) and with a single click, it is possible to start with the data analysis corresponding to its content.
Figure 1 shows the initial screen of the tool, with the set of metrics classified as critical and non-critical.
Below is a brief detail of the basic functions of the HEVDA tool focused on data quality metrics.
1) Metric 1 - Decimal numbers:
This metric focuses on identifying the number of errors detected in aspects of decimal numbers.
One of the points that it considers is the thousand separators or the decimal character.
One possible message that the tool might display is a detection alert for cases where the decimal separator is incorrect (for example, comma).
For greater detail, the tool presents the possibility of viewing the cases affected by this quality aspect, identifying the erroneous data, the column in which the data detected as erroneous, and its corresponding record number in the dataset is shown.
2) Metric 2 - Duplicated records:
The analysis of this metric allows finding the number of records affected in redundancy aspects, that is, in repeated content.
This functionality indicates the number and percentage of equal records in data content. For example, the content of record number 4 is equal to the content of record number 29.
3) Metric 3 - Incomplete and complete Data
This metric focuses on the functionality of detecting complete records in the dataset, as well as incomplete records. In addition to the number of records, it identifies the percentage that each case represents in the analyzed file.
For this analysis, there is a classification of the content of the columns:
Null Column
Empty Column
Column Not available: In this case, the possible values of NULL are contemplated among other special characters that refer to the same idea.
4) Metric 4 - Invalid Characters
This metric indicates the number of columns that have invalid characters that prevent a correct reading of the dataset.
5) Metric 5 - Redundancy for the domain of a column
This metric identifies the groups of records that have the same data in the same column but are expressed in different ways.
For example, if we analyze a dataset, for the column named “City”, 6 records are displayed that have the value “CABA” in the “City” column. In addition, 8 records have the value “C.A.B.A.” in the same column, and finally, it is detected that there are 3 records that have the value “Autonomous City of Buenos Aires” in such column . However, the values CABA, C.A.B.A. and “Autonomous City of Buenos Aires”, refer to the same data. Thus, this metric would allow the identification of these cases so that they can be related in a single value.
6) Metric 6 - Redundancy between fields of the same row
This metric measures the values that are repeated between different fields, considering that they are in the same row of the dataset. For the measurement of redundancy between the fields of the same row, to do this, the number of affected records that fulfill repeated values over the total number of records in the dataset is displayed.
The functionality of this metric analyzes the repeated values between the fields of a file, as long as they are in the same record. For example: For record number 28, field 1 and field 7 contain the same data, "Argentina".
The metric indicates the number of records with cases of this type.
On the other hand, a spreadsheet is shown as a summary with the combinations of columns with redundant data detected in their fields, the number of occurrences, and the percentage it represents related to the total cases found.
7) Metric 7 - ID detection
The analysis of this metric is based on detecting the number of fields that have the word “ID” in their content; this is representative in a dataset, since it could have a meaning of relationship or linkage with other data.
This is important to understand if there are numeric fields that link with other description data; for example, the “category_id” field could be associated with a description field called “vaccines”. This serves for a better connection of the data and its future analysis in interoperability between software.
8) Metric 8 - Trivial fields
This metric focuses on analyzing the contents of the fields that are in the same column of a dataset; that is, it analyzes the data that is repeated in the same column.
For example, the data "Buenos Aires" in the "Province" column is repeated in all records. This metric helps to identify and remove redundant data from the datasets, for example, if all the values indicate: “ARGENTINA” in the same column, that column can be eliminated, and that data added in the name of the general dataset. In the following sections, the validation of a set of datasets from different Latin American countries is shown.
5. Materials, methods and results
This work was carried out with 90 datasets from the 6 countries analyzed in the previous sections.
The 6 countries were randomly selected. The datasets were verified with the HEVDA tool for obtaining details about their quality.
5.1 Data Collection
The sample of this study consists of 15 open public data sets containing information related to the COVID-19, that were extracted from the government portals of each of the countries analyzed in the previous section. The files were imported in CSV format to be able to be verified with the HEVDA tool. The situation found in each country is explained below.
In Argentina and Colombia, datasets related to COVID-19 were downloaded without problems.
In the case of Uruguay, although there are numerous datasets in aspects of public data of COVID-19, its data portal shows several visualization options, but little open data. Thus, datasets available from other related categories (i.e., health and public government) were also considered to complete the sample. In the case of Peru, although there are several datasets focused on COVID-19 (there is a special category), several of these present problems when accessed or downloaded. Some files are too large to be downloaded and messages are displayed, such as, “File was too large or unavailable for preview”, on the other hand, some datasets are identified with a specific open format, but when they are downloaded, they are with a xls or xlsx extension, for this reason, for the collection of the sample, the same strategy as Uruguay was used. In the case of Mexico, there is little variety of datasets for COVID-19, but it was possible to extract the open public data on the contracts and hiring carried out to face the pandemic. For the last case of Paraguay, in its open data portal, there are very few datasets, of the countries analyzed, it is the one that presents the least amount in this regard, then the same strategy as Uruguay and Peru was used.
5.2 Results
A total sample of 90 datasets (15 datasets for each of the six selected Latin American countries analyzed in previous sections) was verified with HEVDA. Before carrying out a data quality verification, the HEVDA tool allows detecting the presence of some obstacles or “blockings” that could prevent the verification. The tool allows analyzing three types of locks in the verified files: A) If a file does not have the format corresponding to CSV; B) If a file does not have titles in its first row; C) If the file does not have the same number of columns in each row; D) The file has double character “(quote).
Figure 2 shows the comparative analysis of the country databases with respect to the types of blockings found. It is observed that the country with the largest number of blocking files is Paraguay with 93.33%, followed by Mexico with 73.33%. On the other hand, the countries with the lowest percentage are Argentina with 6.67% and Uruguay with 13.33%. This analysis reveals a high percentage of COVID-19 datasets that do not meet the basic objective of interoperability so that they can be reused.
For the first type of blocking A, the Figure 3 shows that the country with the highest number of blockings is Paraguay with 66.67%, in most of these files the “;” separator is used instead of the comma separator (CSV format), then Peru follows with 40% of the cases detected, for Mexico, there are no cases. For the type of blocking B, Mexico has the highest percentage being 66.67%, while in Paraguay, the value is 13.33%, in the rest of the countries, no cases were observed.
For the case “The file has double character’’ (quote) “, a country with cases of this type is detected, being Peru, with 6.67%. For the case “The file has different numbers of columns in some of its records”, Paraguay had a greater number of cases with 13.33%, and Mexico had a lower number with 6.67%; for the rest of the countries, no cases are observed.
Figure 4 shows the global comparison of quality metrics for each of the countries analyzed from a view of the impact of metric compliance. For example, it is observed that the countries that are in better positions of openness and quality in the open public data are Argentina and Uruguay. In acceptable terms of quality, that is, between ranges of 50% and 70% is Colombia. In aspects of low quality of data, it is observed Peru with ranges between 20% and 53%. Continuing in ranges of lower quality, Mexico has an average of 27%, and finally, Paraguay with an average of 7%.
Figure 5 shows a detailed comparison of each of the metrics and how they are evaluated in each country based on the cases analyzed. In the case of metric 1 (decimal numbers), the countries that best comply with this measurement are Argentina with 73% and Colombia with 60%, and in the last place is Paraguay with 7%. It should be clarified that the low value in this metric for Paraguay is due to the fact that most of its datasets could not be verified since they presented blocking errors (as described above); this same issue occurs in each metric with respect to that country. For metrics 6, Uruguay leads with 87%, and in a lower percentage range, Argentina follows with 60% and Colombia with 53%. For metric 7 (Detection of ID Values), compliance cases are detected with a low percentage, being 53% for Peru and 27% for Mexico. In metric 8 (Trivial Fields), Mexico has a 20% compliance, below the other countries that present compliance between 50% and 80%. This shows that although there are problems and shortcomings in open data, with small changes in the identified aspects, quality can be significantly improved, for example, aspects analyzed in data redundancy or incomplete data.
Figure 6 shows the number of blocking (this was explained in previous sections), and not locking data sets. Paraguay has the highest number of blocking data sets, while Argentina has the highest number of no-blocking data sets. This figure shows the number of blocked datasets that is, that do not meet specific aspects to be analyzed with the HEVDA tool, because they are not completely open.
Unfortunately, detecting several of these cases leads to an incorrect or erroneous interpretation by citizens or organizations that need to access them.
6. Discussion
This research work presents several sources of authors, in which the importance of keeping in mind the quality of the data in aspects of government open data sources is evident. Another of the points analyzed was the content of the various official websites of various Latin American countries, in which aspects of Open Government are increasingly present. On the other hand, works that offer measurement standards and good practices for the evaluation of data sets in reference to quality were analyzed.
Regarding quality measurements, there are measures or indices in different governments, such as the Open Data Barometer (ODB) [8], a global measure of how governments publish and use open data for accountability, innovation and social impact. Leaders Edition analyzes the 30 governments that have adopted the Open Data Charter [78] and those who, as members of the G20 (Group of Twenty is an intergovernmental forum comprising 19 countries and the European Union), have committed to the G20 Anti-Corruption Resources [79]. Other measurements were developed by the Global Open Data Index (GODI) [7], which is the annual global reference point for the publication of open government data, managed by the Open Knowledge Foundation [80]. It works with a crowdsourcing survey, which measures the openness of government data through the GODI methodology [81]. Another project is the Open Government Data [82], which evaluates the efforts of governments through indices to implement open data in the three critical areas: openness, usefulness and reuse of government data. Other measures focus on the quality and impact of open data [83,84], others [85,86] are funded by the CEPAL [11] and the World Wide Web Consortium in Brazil (W3C Brazil) [87]. Several organizations that offer public services do a very good job of dealing with open data quality aspects [88], and also use aspects such as ISO/IEC 25012 [89], which specifies a general data quality model that is defined in a structured format, or the use of standards [90] to establish the criteria that a data set must meet to be considered of quality and interoperable (universal standard, with the Conformal Dimensions of Data Quality [91].
Although there are several studies carried out based on research on Latin American countries [9,12,15,19,92, which guide their research in the opening of data or visualization of contents, but very few works speak of dataset content validation.
The authors consider that it is necessary to periodically carry out an in-depth analysis of the state of the countries guided by the Open Government approach, since this will allow a better awareness among citizens and will allow knowing the current state of the quality of data offered to people around the world. The authors' approach is initially based on Latin American countries, so the authors consider that a study of the content of public websites in these countries is interesting to know their level of commitment and amount of information published and made available to users.
As analyzed in previous sections, there are several shortcomings in aspects of data quality provided in open data, so sharing good practices, learning guides, forums or tools that allow better data analysis and treatment is a benefit for citizens who wish to access public information.
The proposal of the HEVDA tool allows to quickly validate the “health” status of a dataset and analyze some of the proposed metrics in quality aspects; this will help to know if a dataset must be modified or if it is in conditions to serve as a basis and analyze deeper aspects of software interoperability issues to obtain added value.
The contribution of this work not only focuses on a study of the websites of some countries, but also provides a tool that offers a quick analysis of various quality aspects together with a number of critical and non-critical metrics without modifying the original source.
As part of the limitations of this work, it is indicated that the validation tool only allows an analysis of datasets in CSV format.
7. Conclusion
Although more governments are joining this new open government paradigm, there is still a long way to go before achieving our objective. Political support for open data is still insufficient, as well as the resources available to train public servants and strengthen civil society [92]. The study described in this article shows a significant advance in Latin America towards this new paradigm of Open Government, as well as an understanding on the open data importance and its contribution to society.
However, it can be concluded that there are several problems in the opening of the data, formats and quality granted. It is necessary to have advanced tools (more validation features) for verifying the quality of the open public data that is provided to citizens, since it would help to detect possible shortcomings in their distribution and interpretation, as well as to avoid having an erroneous or incorrect analysis of a certain situation or decision making.
In COVID-19 contexts, open data should play a fundamental role in governments, with the aim of informing and evidencing the work carried out by political management to the citizens facing the crisis of public knowledge worldwide. As analyzed in this article, there are several cases of datasets that cannot be analyzed due to blocking problems, hampering interoperability and reuse. Another point that was detected in this work is that some open data portals have little open information; that is, although they have dashboards (monitoring) regarding COVID-19, offered to citizens or organizations, very few datasets are available, as is the case of Paraguay, Uruguay and Peru. Another aspect to consider based on the metrics analyzed with the HEVDA tool is that the metric with the lowest compliance is metric 5 (redundancy for the domain of a column). In this metric, all countries obtained percentages equal to or less than 40%, which leads us to think that it is important to identify repetitive values in datasets, since their data can be restructured to avoid redundancy. Finally, it is important to make the population and public organizations aware that it is necessary to make quality open government data available to provide better knowledge to society and help in decision-making by promoting economic development, transparency, and citizen participation.
