1. Introduction

Europeana is a project oriented towards centralizing the largest amount of digital resources possible. These resources are stored in external repositories so as to be catalogued, centralized and easily accessed by means of metadata indexation. However, by having a centralized catalogue built through collecting metadata from specialized repositories, the results obtained when looking up digital resources vary considerably and also lack accuracy; this leads to a waste of time in terms or search and selection of a particular digital resource.

This study allowed us identify firstly, to what extent Europeana covers certain topics or areas of knowledge, permitting to reuse and develop learning objects of knowledge areas related to the European cultural heritage. In addition, we examined the relationships of part of the metadata elements of these digital resources, to verify the integrity of the search results of the users, and the availability of accessible digital resources. As this process revealed, the metadata of the majority of the Europeana digital resources we browsed, were insufficient. This enabled us to identify a number of deficiencies in search processes of digital resources of a specific field of knowledge. This study was carry out between periods 2012 and 2014.

The purpose of this study is to analyze how well or how poorly Europeana covers certain topics or knowledge areas in the AAT, mainly those topics most used by teachers and student in high schools. In order to achieve this, the coverage study aims to analyze if Europeana, is a digital library that teachers could use for the development of learning objects in a specific knowledge areas, through the reuse of free/open access digital resources. In conclusion, the aim of this research is to analyze the metadata quality of Europeana’s digital resources related to knowledge areas as arts and cultural heritage in order to re-use in learning process.

The motivation behind the present article is to explore if Europeana digital library is a well-structured library in order to provide resources for learning process in different knowledge areas such as heritage culture, architecture and arts. As well lies in the analysis of metadata quality that has been found so far from a set of digital resources that were extracted over Europeana by using data visualization techniques (^{Gaona-García, Sánchez, & Fermoso, 2012}). One of the sections of this article focuses on analyzing the data exchange models that are used by Europeana. Subsequently, the article discusses the process associated to digital resource exploration and coverage analysis through a set of terms used in the Art & Architecture Thesaurus (AAT, 2015). The following sections present the results obtained from metadata quality analysis in terms of completion; these metadata were identified through digital resource extraction. The final section presents the result of this analysis and provides some recommendations in order to use this type of projects in learning process and so on in our educational context.

2. Background

The success of resource location, depends largely on the quality with which the metadata has been designed, making it an essential condition for the results being produced by search engines in the repositories (^{De la Prieta & Gil, 2010}; ^{Muñoz-Arteaga, Calvillo-Moreno, Ochoa-Zezzatti, Santaolaya-Salgado, & Álvarez-Rodríguez, 2010}) and therefore, it is important to improve indexing strategies for learning objects stored in them (Ochoa, Cardinaels, Meire, & Duval, 2005; ^{Stuckenschmidt, Vdovjak, Houben, & Broekstra, 2004}; Wiley, 2002). Moreover, the lack of metadata constitutes a poor classification of the resources. This condition is a key factor that affects search results in a specific knowledge area. As the quality of contents is an indicator that permits us to evaluate digital resources, there are several studies that refer to methods on how to evaluate quality in digital resources (^{Chuanjun, 2004}; ^{Downes, 2007}; ^{Gonçalves, Moreira, Fox, & Watson, 2007}), as well as the quality of the contents existing in collections of digital resources (Chuanjun, 2004; Downes, 2007).

2.3. Europeana Data Model (EDM)

EDM is a data exchange pilot model whose main purpose is to preserve the original metadata as well as serving as a more flexible and expressive model (when compared to ESE). This pilot model allows multiple-register access on a single digital resource. EDM is based on Semantic-web good practices, supporting standards such as Open Archives Initiative Object Reuse and Exchange (OAI-ORE) (^{Lagoze et al., 2007}) for Internet resource exchange and interoperability. The model also uses semantic description languages through Resource Description Framework (RDF) and Simple Knowledge Organization System (SKOS). Additionally, it links resources by means of the Linked Open Data (LOD) project (^{Berners-Lee, 2006}); which is an initiative that has involved 8 providers from 15 different countries and adds up to 2.4 million released metadata using standard Linked Data recipes (Europeana.lab, 2016). Table 1 presents the EDM elements.

Table 1 Europeana Semantic Element present in EDMmodel. 

Source: The ESE Model (Clyphan, 2013).

According to EDM model (^{Doerr, et al., 2010}) actually the present version of EDM integrates the ESE elements. The integration of ESE into EDM is expressed in RDF, this offers the additional advantage of exploiting the Web architecture for linking resources (^{Haslhofer & Isaac, 2011}). For that reason, and taking into consideration the Europeana data model we have just described, in the next sections we will focus our coverage study in the five already mentioned elements of Dublin Core, that have been integrated into EDM: language, type, country, content provider and rights. First we will describe the methodology for coverage analysis and then, the results analysis.

However, as described in the transformation process (from ESE into EDM [^{Haslhofer & Isaac, 2011}]), when taking the same set of metadata described by ESE, only access to a minimum digital resource metadata set is allowed; thus exhibiting poor performance in terms of digital-resource expression, which limits the scope of the model to serve various purposes such as indexation, search and access.

3. Methodology

To carry out this research, we used a collection of digital resources of Europeana digital library as a case study. Europeana was choosen because of the following characteristics: i) it uses a semantic model for data exchange, the Europeana Data Model/EDM (^{Doerr et al., 2010}), ii) it includes a large number of digital resources related to the European cultural heritage (over 53 million digital resources to date), iii) it uses the largest group of content providers on a European level, and finally iv) it allows reusability of open digital resources. Thus, according to the purpose of this study, Europeana is an initiative that offers the opportunity to reuse digital resources, allowing teachers or professionals of the sector of cultural heritage to use them as learning objects for educational purposes. On the basis of these characteristics, we browsed through the Europeana digital resources to determine its level of coverage concerning a branch of knowledge of the Art and Architecture Thesaurus (AAT). This coverage allows us to determine if Europeana is a good option in order to re-use digital resources in learning process.

3.2 Recognition of Digital Resources Through Data Extraction Process

In this second stage, we developed a strategy to link the Europeana digital resources to a specific knowledge area using selected terms of the AAT thesaurus. For this reason, we developed a Web Crawler to browse digital resources discovered in Europeana for each AAT thesaurus term. A Web Crawler analyzes the syntax of a Web page and extracts information according to its structure (metadata) (^{Tadapak, Suebchua & Rungsawang, 2010}). This extraction strategy was selected according to a series of features defined in a preliminary study we carried on relevant tools. These criteria were related to whether the tool: provides a clear definition of its architecture (^{Baeza-Yates & Castillo, 2004}; ^{Tripathy & Patra, 2008}), is highly scalable (^{Boldi, Codenotti, Santini & Vigna, 2004}), favours optimizing processes (^{Edwards, McCurley & Tomlin, 2001}) and is highly efficient for content extraction processes (Castillo, 2005). The tool itself supports its operation by using computational algorithms that allow covering every page by means of search methods, and also by using specialized libraries to define extraction elements.

On the other hand, metadata extracted from each digital resource is selected according to the elements of the Europeana Data Model (EDM). The metadata extracted from digital resources are based on the EDM model -Table 1, section 2.3. In the Figure 1, is shown an example of one the resources obtained as result. This resource presents 11 representative metadata elements describing some characteristics of this digital resource:

• Title: A name given to the resource.

• Creator: An entity primarily responsible for making the resource.

• Contributor: An entity responsible for making contributions to the resource.

• Date: Date of creation of the resource.

• Type: The type of the original analog or born digital object as recorded by the content holder, this element typically includes values such as photograph, painting, sculpture, etc.

• Description: A description of the original analog or born digital object.

• Data provider: The name or identifier of the organization that contributes data to Europeana.

• Provider: Name of the organization that delivers data to Europeana.

• Identifier: An unambiguous reference to the resource within a given context.

• Format: The file format, physical medium or dimensions of the resource.

• Language: A language of the resource.

Figure 1 Metadata properties extracted from a digital resources. 

4. Result of Digital-Resource Exploration

In the third stage, we analyzed the relationship between the extracted digital resources and the thematic areas selected for browsing. For this, with the support of a group of experts in cultural heritage, we reviewed the extracted results and found that most of the extracted resources did not correspond to the knowledge domain defined by the terms of the AAT thesaurus. For this reason, we designed a browsing strategy that used refined searches and keywords related to the domain of knowledge of each term of the AAT thesaurus. This strategy produced more relevant results as far as the knowledge area of each term of the thesaurus is concerned.

From the exploration stage, we found 44,280 digital resources identified in Figure 2(a). In order to conduct a coverage analysis, such digital resources were classified according to format type, country of origin, language, content provider and copyright. The results of this analysis are shown in Figure 2(b), where 23,431 (53%) digital resources were found to lack enough attributes to describe the type of copyright of the digital resource itself; the second biggest category was that of free-access digital resources, which accounted for 40% (17,802) of the resources analyzed; finally, 2,838 (6.5% on average) digital resources required payment for open search.

Figure 2 (a). Total number of digital resources. (Own elaboration) (b). Number of digital resources by Rights. (Own elaboration) 

As a result, we found that a large set of topics associated to the AAT thesaurus knowledge area "Styles and Periods" was not fully covered by Europeana. On the other hand, the variety of results displayed while browsing suggested that Europeana did not support search methods to relate resources to a specific knowledge area. In Figure 3 we can see this distribution.

Figure 3 Percentage of coverage of AAT terms according to digital resources explored in Europeana. 

According to the classification of Figure 5, can be identified a high set of AAT terms that have a low coverage (54.23%) and only a small group of AAT terms with high-level coverage of digital resources (5.93%). Below, we describe these results in more detailed according to language, type of format and copyrights.

4.3. Coverage by Right Information

Figure 4 shows the coverage of digital resources found, according to copyrights. It identifies that a large majority of them (53%), had no copyright description in digital resources, and only (40%) of the total of digital resources found, are resources of free access.

Figure 4 Coverage of digital resources by right information. 

According to the results of coverage by copyright, below, in Figure 5, is presented a detailed analysis of digital resources found, that are free access, non-free access, and do not have a copyright information, classified by type of format.

Figure 5 Description of coverage by right information vs type of format. 

The absence of description for copyright in metadata elements of digital resources, and the variety of results, led to a second study related to metadata quality of the digital resources explored. With this aim, we present a complementary study in the following section, where a detailed analysis to examine metadata quality according to completeness of digital resources explored was performed. Through this analysis we will study if the level of quality of metadata can influence the study of coverage.

According to these results of coverage, is necessary to perform a detailed analysis to examine metadata quality of digital resources explored. To carry out more evidence, the study of metadata quality was made to 555.0000 Europeana digital resources extracted. Next section we analyses this results

5. Results of Quality and Metadata Completion Analysis

The metadata quality are essential to search digital resources. Without them, the results of searches of digital resources are poor and inefficient (^{Cechinel, Sánchez-Alonso & Sicilia, 2009}). The problem of low quality in repositories has been mentioned by other researches. To determine the quality of the metadata, our study will be based on the following areas: completeness and accuracy (^{Bui & Park, 2013}; ^{Manouselis, Vuorikari, & Van Assche, 2010}). We use some analysis criteria strategies of quality assessment dealt with in (Bui & Park, 2013; Manouselis, et al., 2010; ^{Palavitsinis, Manouselis & Sánchez-Alonso, 2014}). In the following sections, we will present the results of this study.

As a result, we found that a large set of topics associated to the AAT thesaurus knowledge area "Styles and Periods" was not fully covered by Europeana. On the other hand, the variety of results displayed while browsing suggested that Europeana did not support search methods to relate resources to a specific knowledge área.

According to the categories defined by the specification ESE: Mandatory, Recommended and Optional elements, has a varied definition of completeness in the metadata. Table 2 describes the case for analyzing completeness of Mandatory elements.

In the case of mandatory elements can be evidenced a low level of 62,71% completeness for metadata elements defined by Europeana, but the metadata elements that have been defined by external providers, have in total a low level of completeness for an average of 56,81%. Table 3 presents results of recommended elements.

Table 2 Completeness of mandatory elements. 

In the case of recommended elements above, identifies a level of completeness with an average half 38.40%. Finally, in Table 4 presents completeness of optional elements.

Table 3 Completeness of recommended elements. 

Table 4 Completness of optional elements. 

In the case of optional elements, the level of completness is rather low with an average 14,66%.

Figure 6 shows the detailed analysis results obtained from the explored metadata, based on the mapping and normalization guide defined by ESE model through a set of mandatory elements, recommended elements, and Europeana’s own elements.

Figure 6 Analysis of metadata used in Europeana digital re sources explored. 

In the first set, namely “Mandatory elements”, there is a (38.13%) attribute absence from the total amount of explored resources. For the second set, namely “recommended elements”, attribute absence reaches (61.60%). Finally, the set of elements belonging to Europeana exhibits attribute absence of (50.52 %) for the metadata. This means that the majority number of metadata elements classified in “Mandatory elements” has a high quality of completeness. Therefore, we can conclude that we have found some important deficiencies in Europeana metadata on the basis of our accuracy analysis: redundancy, absence, ambiguity and inconsistency of its metadata, among other.

6. Conclusions

Europeana’s data exchange model defines a digitalresource search within the same set of general metadata that describe the resource. This yields considerably varied search results (thus not very accurate). Therefore, the model provides a limited set of metadata that do not permit classifying digital resources coming from content providers and aggregators according to a specific knowledge domain.

As a conclusion to this preliminary study of metadata quality associated with the branch of knowledge “styles and periods” of AAT, and in general on any knowledge area, it is important to define metadata to classify the digital resource from a topic or subject area. Offering a limited set of metadata descriptions of digital resources within a digital repository, generate a variety of irrelevant results in search processes. The definition of these metadata and quality themselves, offer a variety of search criteria (thematic area, language, resource type, copyrights, etc.) can be integrated into the development of visual interfaces trough visualization techniques (^{Gaona-Garcia, Martín-Moncunill, Sánchez-Alonso, & Fermoso, 2014}; Gaona-Garcia, Sánchez-Alonso, & Montenegro, 2014).

The results of this analysis showed a lack of completeness of the metadata defined according to the Europeana Data Model/EDM. However, such results are not new, since other studies concerning digital repositories such as ARIADNE (^{Ternier et al., 2009}), the National Science Digital Library (^{Fox, Gonçalves, & Kipp, 2002}) and other collections of digital resources (^{Bui & Park, 2013}), also found deficiencies concerning the definition of metadata. In fact, this deficiency, the lack of precision in the definition of metadata elements, is one of the main factors that directly influence digital resources search. Similarly, the absence of metadata elements for the classification of a digital resource according to topics or knowledge areas was reflected in the poor search results that we obtained through the study of thematic coverage. Those are factors that influence search results, and have also been mentioned by ^{Cechinel et al. (2009}).

Despite the deficiencies in the metadata quality associated to completeness, Europeana presents great opportunities to be a digital library with reusability of digital resources for the development of learning objects. Not only by the results of coverage and the large volume of digital resources available to this library, but by the EDM exchange data model. Strategy, which is emerging as a model that fully cover future issues related to semantic search, and thus facilitate better quality of search process by linking them through Linked Data (^{Bizer, Heath, & Berners-Lee, 2009}; ^{Dietze et al., 2012}; ^{Haslhofer & Isaac, 2011}). However, in order to improve the quality of them, Europeana should make great efforts not only to improve the quality of the metadata of digital resources, but also define strategies by implementing alternative metadata to facilitate the search for digital resources from a subject or area of knowledge. The key issue is the integration of knowledge representation schemes such as the use of ontologies or thesauri, to classify digital resources associated with a branch of knowledge