Development of a digital TV receivers test suite

Desarrollo de una suite de pruebas para receptores de TV digital

Pablo Flores-Guridi ^a, Gustavo Guimerans ^b, Juan Pablo Garella ^a, Javier Baliosian ^a, Eduardo Grampín ^a, Rafael Sotelo ^a & María Simon ^a

^a Facultad de Ingeniería, Universidad de la República, Montevideo, Uruguay, pablof@fing.edu.uy, jpgarella@fing.edu.uy, javierba@fing.edu.uy, grampin@fing.edu.uy, sotelo@fing.edu.uy, maria@fing.edu.uy
^b Centro de Ensayos de Software, Montevideo, Uruguay, gusguime@ces.com.uy

Received October 31^th, 2014. Received in revised form: March 2^nd, 2015. Accepted: July 30^th, 2015

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Abstract
The deployment of digital TV in Uruguay is ongoing, following the ISDB-Tb standard. The channels allocation process was completed, and after the adoption of a testing protocol, the introduction of decoders and receivers began in early 2014. Testing their compliance with the standards is of paramount importance in order to foster migration towards it and to provide a platform to local developers. The government committed the School of Engineering and the Software Testing Centre to the elaboration of the testing protocol. This article reports the key aspects of this work: the analysis of the standards, the definition of a type approval protocol and the experience gained by applying the testing procedures. Its goal is to share the experience and contribute to the improvement of the digital TV adoption process, hopefully aiding to avoid some problems that were detected during this work, both in regulation and in devices compliance to the standards.

Keywords: Digital Television; ISDB-T; TV Receivers Testing.

Resumen
El despliegue de la televisión digital en Uruguay está en curso, siguiendo el estándar ISDB-Tb. Ya se completó el proceso de asignación de canales y, después de aprobado un protocolo de homologación, se ha comenzado a importar receptores o decodificadores desde principios de 2014. La certificación es de capital importancia para acelerar la migración y para brindar una plataforma a los desarrolladores locales. El presente artículo resume los aspectos clave de este trabajo: el análisis de las normas, la elaboración del Protocolo de Homologación y la experiencia obtenida en la aplicación de las pruebas. Su objetivo es compartir estas experiencias para contribuir en los procesos de adopción de la TV digital, con lo que se espera ayudar a que otros países o actores puedan evitar problemas de compatibilidad o cumplimiento de las normas que aparecieron durante este trabajo, tanto en la regulación como en los dispositivos ensayados.

Palabras clave: Televisión Digital; ISDB-T; Pruebas para Receptores de TV.

1. Introduction

Trial broadcasting began in late 2012, while the receivers were just starting to become available. No Uruguayan based companies are producing decoders, and therefore both Set-Top Boxes and TV sets need to be imported. Local manufacturing, in current conditions, could be envisaged only if the foreseen market were wider than the domestic one. The Dirección Nacional de Telecomunicaciones (DINATEL) [4] has ruled that the receivers must be tested and certified in a national laboratory, recruiting the School of Engineering (Facultad de Ingeniería, Universidad de la República - UdelaR) and its partner institution, the Software Testing Centre (Centro de Ensayos de Software - CES) [5], to collaborate in the definition of the test suite. CES is a joint venture between UdelaR and the Chamber of Software Companies (Cámara Uruguaya de Tecnologías de la Información - CUTI). The CES is devoted to software testing and certification, and therefore, its participation in this project guarantees that proper procedures are followed in the testing process.

One may ask what the need is to test and certify TV receivers, being domestic appliances manufactured under a defined standard. There are three reasons: to begin with, scarce experience. Japan, where the ISDB-T standard was first implemented, has the longest history working with it. However, geographical, demographic and cultural conditions are very different from those of our country. Brazil's and Argentina's are yet quite recent deployments. Therefore, to help the local deployment, the Uruguayan government is especially interested in orienting consumers to buy receivers compatible with the standard. Secondly, intuition tell us that, having two different countries (Brazil and Argentina) implementing the standard with different objectives and strategies, some aspects should be carefully tested; this proved to be true, and after in-depth analysis and testing, which are described in this article, it was found that the standard has different and not fully compatible versions, especially regarding interactivity. Thirdly, there is a political intention of fostering the software as well as audiovisual industries, which means that the foreseen deployments are heavily based on the interactive aspects of the standard. Those branches are important in Uruguay, which is the leading per-capita software exporter in South America [6], while audiovisual, among other cultural industries, is in fast development and was responsible for about 0.45% of the GDP in 2011 [7] and is now more than 1%.

The ISDB-Tb standard has two major aspects: interactivity, and signal reception and decoding. Interactivity, implemented by a Brazilian middleware named Ginga [8], is expected to be an important component of public campaigns, advertising and on-line games. Its foreseeable users are the government (education [30], social development, health and culture ministries, among others), broadcasters, and commercial companies.

This article includes the contents of a previous work of the group, which was written immediately after the adoption of the Type Approval Protocol [9] and, therefore, is self-contained. About a year of experience in applying this test suite and following the standards evolution generated a new insight into interactivity aspects and some learned lessons about regulation and devices.

Section 2 is devoted to signal reception and decoding aspects, defining the main features to be tested. Test analysis leads to the conclusion that the theoretical set of exhaustive tests should be pruned. Indeed, the standard allows multiple configurations, not only in source coding, but also in channel coding, modulation and transmission; testing every combination would be non feasible in the real world. However, the decoder architecture implies that some features to be tested reflect the behavior of some functional blocks. When those blocks are completely independent from each other, the corresponding features can be tested with a fixed choice of the remaining parameters. In addition, during the testing process it has been found out that many receivers did not support some of these configurations. The aim of this work was in this sense to achieve a representative subset of tests, which would guarantee a right receiver operation. Finally, the transmission requirements, established in the coverage regulation, were compared with the reception requirements, such as the receiver sensitivity, in order to determine guidelines for the receivers' installation.

Section 3 focuses on interactivity aspects, making a comparison between implementations and the standard. A thoughtful analysis was undertaken, taking into account Ginga standard documents and existing implementations in Brazil and Argentina. Important differences among the implementations were found and, more importantly, none of them are fully compliant with the standards. In some cases there are different understandings, while in many aspects there are clear deviations from the standard. Some bugs were identified that should be corrected.

In Section 4, the current situation of the approval process is referred to, emphasizing the experience gained and lessons learned, and describes some important examples in which corrections were needed.

Finally, in Section 5 the conclusions are summarized and some foreseen further work is pointed out.

This document distinguishes between the abbreviations ISDB-T and ISDB-Tb. The first one refers not only to the original version of the standard, the Japanese version, but also to its international version defined in the International ISDB-T Forum. See, for example, [10]. The second one is used to only refer to the Brazilian version, which unlike the original one, uses MPEG-4 [11,12], instead of MPEG-2 for video and audio compression and uses Ginga as interactivity middleware instead of BML.

2. Signal reception and decoding aspects

2.1. What to test: System description

The ISDB-T transmission system can be configured in many different ways regarding channel coding and modulation in order to prioritize useful bit rate or noise immunity. Such is the case of modulation scheme, transmitting mode, guard interval, convolutional code, etc. Source coding is also important to be defined in every multimedia broadcasting system; ISDB-Tb uses MPEG-4 as audio and video compression standard. Finally, automatic channel tuning, sensitivity, co-channel and adjacent channel interference, and other functionalities have also to be tested. The ISDB-Tb transmission and reception systems are entirely defined in the ABNT NBR 15601 [13] and ABNT NBR 15604 [14] respectively, while the international ISDB-T community has reached binding agreements as described in the ISDB-T Harmonization Document - Part 1: Hardware [15]. These normative references are the basis of Uruguayan Act 143/013 [16], which enforces the receivers tests following the type approval protocol generated in this work.

ISDB-T uses OFDM modulation in a 6MHz channel. This channel is divided into 13 sub-channels named "segments"; each segment has a bandwidth of 6/14 MHz ≈ 428.57 kHz. The remaining sub-channel is used as guard interval at the sides of the OFDM spectrum. It is interesting to note that the segments can be grouped in up to three hierarchical layers named "A", "B", and "C". Each hierarchical layer carries different media contents; since each layer has a different convolutional code rate, time interleaving and modulation scheme, it is possible to adjust the immunity of each media, or group of medias, independently. Two different error correction algorithms are used. The outer code is a Reed Solomon algorithm, RS(204,188), which for every transport stream packet of length 188 bytes, adds 16 bytes of redundancy capable of correcting up to 8 bytes. The inner code is a punctured convolutional code with the following possible rates: 1/2, 2/3, 3/4, 5/6, 7/8. One rate has to be defined for each hierarchical layer, and they can be different for different layers. Thus, each layer can have its own trade off between useful bit rate and error immunity.

Different modulation schemes can be assigned for different layers. The ones defined in ISDB-T are DQPSK and QPSK, where each carrier transmits two bits per symbol; and also 16-QAM and 64-QAM, where each carrier transmits 4 and 6 bits per symbol respectively. ISDB-T differs from other standards, such as DVB-T and ATSC, in its capability of transmitting a low definition signal for handheld receivers in the same bandwidth, using the center segment. This signal can be used for public alarms, such as natural catastrophes warnings. For this purpose, this segment, named "segment 0", is defined as the layer A, with very robust transmission parameters. The other 12 segments are used for the layers B, or B and C, depending on the broadcaster's needs.

Handheld receivers, also named one_seg receivers, tune and demodulate this "segment 0" information. In this case, layer A is usually configured to use QPSK modulation.

Unlike the one_seg receivers, those able to tune and demodulate the full 6MHz channel are named full_seg. Mobile full_seg receivers, including smartphones and tablets, were not included in the first version of the protocol, but its market availability pushed the group to undertake a new version, which is in progress.

The ISDB-T transmission system can be configured to work in three different modes. These modes, keeping the total bit rate constant, increase (mode 3) or decrease (mode 1) the total number or carriers per segment and thus increase (mode 3) or decrease (mode 1) each OFDM symbol length. Particularly, modes 1, 2 and 3 use 108, 216 and 432 carriers per segment, with effective symbol lengths of 252ms, 504ms, and 1004ms respectively.

A time interleaving is added to randomize the transmitted symbols in order to strengthen the transmission against burst errors. The time interleaving parameter can be configured to {0,4,8 or 16}, {0,2,4 or 8} or {0,1,2 or 4}, if the system is configured to work in mode 1, 2 or 3 respectively.

So far, this section shows how complex is the ISDB-T transmission system. In addition, the source coding, consisting in video and audio compression, entails a wide span of choices. It is well known that MPEG- 4 offers many options that contribute with a factor to the cardinality of the theoretical set of tests. If the goal were to test the receivers tuning and demodulation of signals with every possible configuration in the transmitter, such a test battery would imply 960 options, multiplied by the many possible combinations of hierarchical layers. Furthermore, there are 63 possible channels where to test reception (from 07 to 13 in VHF and from 14 to 69 in UHF).

The first and justified conclusion is that the exhaustive test of all possibilities was not feasible within practical times. It was needed to define a strategy able to test a representative subset of configuration parameters that can guarantee the good reception of digital terrestrial television in every approved receiver. It should be noted that the set of tests is not a full Cartesian product of all possible combinations, because independent functional blocks, whose failure is also independent, can be identified in the receiver architecture as can be seen in the Fig. 1. This issue will be explained in more detail in the following subsection.

2.2. How to test: Defined strategy

The defined set of tests was intended only for full_seg receivers, particularly to Set-Top Boxes (STB) and digital television sets; however, it could be easily scalable to other types of receivers such as dongles or tablets.

An STB is a digital converter that, among other features, receives the digital signal and converts it to analogical so that it can be displayed on an analogical TV set. There are some features that any STB must have and digital TV sets do not; such is the case of video and audio output interfaces. Therefore, each test was classified according to its target: STB, digital television or both. Tests were also divided into six groups: hardware, video, audio, reception, functionalities and documentation. Each one includes different requirements in some way independent from the others.

The hardware group clusters all the interfaces and connectors requirements such as antenna input, composite video output and stereo audio output. Every connector is expected to have its own identifying name at the bottom, or in the users manual. Most of these tests were thought to have been done by visual inspection; generally, the technicians in charge only have to certify the presence of the interfaces but not put them to work. It is worthy to note that in [16] the HDMI output and the remote control implementation are not mandatory. Nevertheless, in the protocol both are expected to be present.

The video group was intended to test all video source coding configurations expected to be used by local broadcasters. Although [16] establishes that receivers must be able to decode progressive and interlaced signals, with 25, 30/1.001, 50 and 60/1.001 frames per second, in different resolutions and aspect ratios (AR), only three different configurations were included, all of them using the MPEG-4 standard, part 10: AVC/H.264, High Profile @ Level 4.0:

• 576i (720x576i, AR: 4:3) @ 50 Hz
]]> 576i (720x576i, AR: 16:9) @ 50 Hz
• 1080i (1920x1080i, AR: 16:9) @ 50 Hz

This is because Uruguayan analogical TV standard is PAL-N, which has a resolution of 720×576 pixels presented in an interlaced way, at 50 frames per second. DINATEL organized a meeting with local broadcasters, who claimed they will only use the configurations mentioned above.

The audio group was also intended to test the audio source coding configurations expected to be used by local broadcasters. Although [16] establishes receivers must be able to decode sampling rates of 32, 44.1 and 48 KHz and quantizations of 16 and 20 bits, all of them in mono, stereo and multichannel stereo audio modes; only three different configurations were included, in every case using LATM/LOAS for multiplexing and transmission:

• Profile AAC-LC, Stereo (2/0), Sample Rate 48kHz, Quantization 16 bits.
• Profile HE-AAC, Stereo (2/0), Sample Rate 48kHz, Quantization 16 bits.
• Profile HE-AAC, Multichannel Stereo (3/2 + LFE), Sample Rate 48kHz, Quantization 16 bits.

The strategy used to prune these tests was the same as for the video group.

The sensitivity for every receiver is expected to be at least -77 dBm. This is not an arbitrary value: in Uruguay the coverage area for every digital TV broadcasting station is defined as the region in which the electric field is higher than 51dBmV/m [19], and there is a close relationship between these two values. Indeed, the electrical field strength (E) in dBmV/m and the power in the receiver's terminals in dBm (P) are related by [20]:

where G is the antenna gain in dBi and l is the wavelength of the carrier. Now, if we suppose l ≈ 0.5m, G ≈ 9dBi and a power loss of approximately 4dB in cables and connectors, we have that:

Hence, a sensitivity of at least -77 dBm is implicitly supposed when planning digital TV coverage. Therefore, sensitivity is an important parameter to be tested.

The tested selectivity values are just those specified between digital television channels. That is because in Uruguay analogical TV is limited to the VHF band and digital TV is limited to UHF band. As was mentioned above, testing if the receiver is able to tune and demodulate every possible configuration in the transmitter would lead us to define over a thousand tests. However, it is to be noted that convolutional coding, mapping, time interleaving and the insertion of the cyclic prefix are done in different and independent blocks. So then, for every transmission mode, each possible value must be tested just once. We concluded that the resulting number of configurations to define was 16, including those that broadcasters claimed they will use.

The functionalities group gathers a variety of features such as: the possibility of configuring the receiver to Spanish, the ability to perform an automatic channel tuning in the UHF band (channels 14 to 69), virtual channel support and the ability to recognize and reproduce by default the primary audio stream. In accordance with the government resolution [16], Uruguay requests the sequential channel selection through all logical channels (every service) instead of sequential channel selection trough primary services.

Finally, the documentation group purpose is to require a Spanish-written user manual that includes, at least, the following topics: technical information, installation guide, user manual and service contact support. Although [16] does not require its mandatory inclusion, this decision was based on [21].

The interactivity aspects of the ISDB-Tb standard are implemented on the Ginga middleware, a complex piece of software that interprets applications written in NCL and that can be extended to run code written in other languages. NCL is an XML-based application language that provides support for specifying spatial-temporal synchronization among media objects, media content and presentation alternatives, exhibition on multiple devices, and live producing of interactive non-linear programs. Ginga-J is a Ginga subsystem or extension, especially promoted by the Brazilian broadcasters, that allows Ginga to run Java coded applications. Both Ginga subsystems for digital terrestrial television are standardized in [22].

Since the Ginga framework was initially developed by Telemidia Lab at the Pontifícia Universidade Católica do Rio de Janeiro (PUC) [23], the first Ginga implementation was made by PUC as a reference for the following implementations, and is the base of the ITU-T Recommendation H.761. This piece of software implemented the most relevant or demonstrative parts of the standard to be used as a guide, but was not intended to be deployed inside ISDB-Tb devices. Some industrial vendors, such as Samsung, just implemented the specification given by the standard from scratch, and others such as TQTVD (TOTVS), MOPA and FUCAPI, started from PUC's implementation but all of them included the Ginga-J subsystem too, with the purpose of deploying it in their TV sets (including TV sets from Sony, SEMP-Toshiba, AOC and PANASONIC) and in Nokia and Motorola cellphones.

In the case of Argentina, the government in conjunction with the Laboratorio de Investigación y Formación en Informática Avanzada (LIFIA) [24] decided to distribute low-cost ISDB-Tb Set-Top Boxes with a license-free Ginga implementation. In this circumstances LIFIA worked on top of PUC's Ginga, restricted to the Ginga-NCL part, leaving the Java code out of their implementation. Their development is named Ginga.ar [25].

In this context, the Uruguayan government decided to design a type approval procedure to deal with a market with Brazilian-made TV sets and Argentinian-made STBs and, probably, cellphones of unexpected origin and components. There are TV sets being imported from other origins such as Mexico and China that also use the Brazilian-based Ginga developments.

To simplify this task, DINATEL decided to leave the Ginga-J extension as an optional feature in the approval process.

As a part of this type of approval protocol, the plan was to design a set of tests to check that the Ginga implementation provided by the tested device would comply with the standard. This task was based on the specification given by the standard and on two main sources of tests already developed: one vast and very specific set of tests hosted by the Brazilian-based organization Ginga-NCL Confomance Testing, launched in July 2011 with a set of more than 300 test cases [26]; and another, much smaller but more application-oriented, developed by the Argentinian LIFIA, with more than 60 test cases.

3.1. Problems found

As the Ginga-NCL Confomance Testing was originally meant for IPTV and thus based on the ITU-T H.761 Ginga- NCL middleware standard; therefore, a revision and an adaptation of the test cases has been undertaken.

Smoke tests were performed on a family of Argentinian implementations and also on many Brazilian providers' products. It was found that all implementations diverge from each other and also from the standard's specification. This is reflected in several ways: some methods behave differently than is specified, some methods have different signatures, some features required by the standard were not implemented, or the middleware does implement features that are not specified in the standard.

This somewhat surprising situation led this work to explore two directions: (1) to identify, for every Ginga implementation, the coverage of the standard, and (2) to find interoperable functionalities among every implementation which would enable software developers to produce interesting interactive applications, i.e, the common subset of useful capabilities. As a first approximation, we considered two representative implementations (one Argentinian and the other Brazilian), performing the following activities: studying the existing test suites, evaluating them by the analysis and execution of a subset of the tests on the receivers, and generating corrected versions of the tests. In general, corrections were needed for tests because they were poorly specified, and/or lag behind the latest standard versions. More than 250 tests were executed for more than 15 different configurations, showing an important degree of inconsistency among the tested implementations. For instance, considering 66 LIFIA's compliance tests for one of the implementations, we found that 21 of them, i.e. 32%, fall out of the standard due to a set of un-compliant individual features that have a relevant impact on the application design. This situation is depicted in Fig. 3.

One of the most notorious non-conformance cases, present in the actual Ginga.ar 2.1 and previous versions, is that although ABNT NBR 15606-1:2013, Table A.28, specifies that the "MENU" and "INFO" keys are reserved for the native applications and must not be used by the interactive ones, the Ginga.ar implementation enables this feature. It is possible to define a simple test case implementation to evidence that this is not fully compliant with the Ginga standard and that seriously affects the interoperability among Ginga implementations.

Other issues detected are, for example, the usage of non- standard features that are not specified in the Ginga standard like HTML5 and CSS3 contents that could use features not considered in ABNT NBR 15606-2:2011: Tables 2 and 4. Some new features such as support of elements <section> and <article> are also not considered. A similar case occurs with the "EXIT" key, Ginga.ar assumes it will close the applications while this is not a behavior defined in the standards.

3.2. Defined strategy

The incompatibilities described above result in a very restricted common set to be tested. Therefore, it was decided to leave the interactivity aspects of the standard temporally out of the type approval protocol.

Nevertheless, the knowledge and experience acquired during this work is a relevant asset which would be very useful to assist application developers (governmental and non-governmental organizations, as well as private content providers), since they will face a diverse market where the receivers deploy different interactivity capabilities; some of them may decide to produce applications to reach as many users as possible (typically governmental agencies) and others may prefer to focus their efforts in selected segments of the public (e.g., owners of high-end, expensive devices).

As a result of this experience we reached the conclusion that the ISDB-Tb technology definition, and more specifically, the interactive aspects of the standard, are in rapid change. Deployment of terrestrial digital TV is an important investment of the society, and therefore it is mandatory, in order to guarantee a successful experience, to maintain a technological observatory and test laboratories, with a strong support of an expert team which can follow the evolution. Moreover, the local content developers need support to take benefit from the possibilities offered by the interactive aspects of digital TV. In the long term, we envisage that new convergence standards such as Hybrid Broadcast Broadband TV (HbbTV) will eventually take over. Again, expertise and tools are needed to keep track of the upcoming standards.

4. Current situation: Type approval protocol and its application

4.1. Approved Protocol

The knowledge and experience gained during this process is reflected in the ISDB-Tb Receivers Type Approval Protocol [27], which is currently being used in Uruguay to certify full_seg receivers, either TV sets or STB. Other receivers, especially handheld mobiles or accessories such as USB dongles, are being included in a second part of the protocol.

The Protocol includes 21 tests regarding signal reception and decoding aspects. It also contains a description of the working environment required to perform the tests, normative references, terms and definitions and finally, Annexes specify carefully the configurations required for each test and the transmitter settings in which the receiver is expected to work properly. Each test is classified according to its target, has instructions and expected results. Besides, the presentation order of tests is devised to optimize execution time of the suite.

Those 21 tests were selected taking into account technical and practical aspects. They constitute a reasonable set, to be performed in an affordable time (4 hours) by trained workers, well suited to reveal the main failures to be expected. They cover all aspects of the operation, from the sensitivity to the display formats or the man-machine interface. They were backed by a meeting with the authorized broadcasters, in which they were asked about the options and parameters they planned to employ.

4.2. Experience gained during the type approval process

4.2.1. Phases of work

The type approval protocol was first applied on March 2014. During the first six months a transition period took place, in which the laboratory in charge of the tests reported to the importer or manufacturer the receiver test failures. In such cases, an interactive process was allowed, in which the manufacturer could make changes in the device, mainly on firmware, in order to accomplish the requirements. During this time, about 20% of the tested devices needed at least one correction to certify.

Since August 2014, the laboratory has given no more feedback, and the only tests result is that the receiver is either approved or not. In case the device is not approved, the importer is only informed about which tests failed. So far, approximately 160 different models of receivers have been submitted for certification, 80% were TV sets and 20% STBs. Until now, about 3% have not passed the type approval protocol and thus could not be imported to the country.

Problems found are classified into categories. First, those related to physical problems, which may be related to external components such as connectors or remote control, or internal ones, such as the performance of the front-end tuner. Secondly, the classification is based on problems relating to firmware that were often solved by upgrades. A relevant incident implying a possible omission in an ABNT standard in relation to a technical report from ARIB that is used by manufacturers, is also reported. Some problems were found regarding tuners that are not STBs or TV sets, for which a second version of the protocol is being developed. The concept of families of models should be elucidated, in order to decide when the differences deserve a new type approval test or when the similarities make it unnecessary. Finally, a problem with virtual channels and their specification is reported.

4.2.2. Physical problems

Some gross mistakes were detected, such as TV sets coming with a DVB-T tuner, which is wrong because the approved standard for free air television in the country is ISDB-T. Regarding hardware, in some cases the "F connector" was absent, while the protocol requires its presence at the antenna input. Some were submitted with the so-called "TV aerial plug" connector, also know as "Belling-Lee connector" or "TV game connector" in other regions. This connector is usually used in TV antenna installations in Europe.

Some receivers did not tune when transmission mode 1 or mode 2 were used. Even though mode 3 is expected to be the most used in real operations in Uruguay, the reception in all modes is required. Receivers that did not tune over the entire required band were also found.

Sensitivity problems were detected. Some receivers did not achieve the interference tests by a short margin, about 2 dB less sensitivity than required.

In the man-machine interface domain, TV sets samples presented the remote control written in non classical Latin characters, and some remote controls did not present a "dot" button resulting in difficulties to introduce the virtual channel and service ID, i.e. 4.1. However, this is not mandatory in the protocol.

4.2.3. Firmware

Firmware failures included, among others, the absence of virtual channel support, i.e. the selection of different services by the selection of the physical channel instead of the virtual channel. Malfunction of auto-tuning was also found. Once they had already performed an auto-scan, some STBs needed to be rebooted in order to perform a new auto-scan to discover new channels. Also, some Android-based STBs had pre-stored DVB receiving channels that cannot be deleted.

Appreciable differences between the speed to perform channel scanning, among other functions was remarked upon, but in the protocol the speed is not taken into account.

4.2.4. Possible omission in ABNT 15603

Some TV sets presented a tuning problem during the tests: in certain transmitting modes, after scanning, they presented a blank screen with the message "This video format is not supported". However, the TV sets were able to show the name of the channel, thus evidencing that it was in fact tuned. This behavior appeared every time that the 13 segments were transmitted together in Layer A.

It was found that those TV sets interpreted the signal layers incorrectly only because the transmitted signal had a wrong parameter set in the "Network Information Table" (NIT). According to the Japanese version of the standard, in ARIB TR-B14, the descriptor "TS Information Descriptor" has a field named "Transmission Type Info" that it is used to classify the transmission layers. This field is used to describe the Transmission Parameter Type Value, which can be either Type A, Type B or Type C. This parameter must be specified for every layer and it is defined that Type A is used to define the signal as high protection layer, Type B as medium protection layer and Type C as low protection layer. In the laboratory equipment this field was set to Type C but when there is only one layer defined, i.e. the 13 segments with the same transmission parameters, according to [28] the signal should be defined as Type A.

The laboratory made the corresponding request to the manufacturer of the transmitting equipment. After checking the ABNT NBR 15603 Standard, that describes the NIT for the Brazilian ISDB-T version, the specification for that descriptor could not be found. This working group was also unable to find it. However, after changing this parameter at the transmitting equipment, the TV set was able to properly decode the signal.

4.2.5. Other devices

Some tablets and USB dongles with TV tuners are being presented for approval. The protocol did not contemplate these devices. Its second version is currently under development. It should consider devices both with external antenna connector and with internal or built-in antenna. A subset of the tests used for TV sets and STB has been temporarily performed for tablets and USB dongles with external antenna connector. Devices with internal antennas have not been submitted for approval yet, but the working group is preparing tests for those kinds of receivers. Sensitivity and selectivity tests raise special issues when there is not external antenna connector that implies the creation of a controlled, uniform and adjustable electromagnetic signal.

4.2.6. Definition of families of receivers

The type approval process contemplates the fact that different models of receivers belong to the same manufacturing process, and thus are considered as family members. Once one receiver has been approved in the certification process, the remaining members of the same family do not need to be tested.

In order to consider two or more different models as belonging to the same family, the manufacturer has to properly demonstrate through documentation that the two models will perform equally in the tests. This claim can be based, for instance, in the fact that both receivers use the same tuner, the same or equivalent video processing integrated circuit, the same or equivalent firmware and have also very similar or equal user and technical manuals.

4.2.7. Regulation about virtual channels

In ISDB-Tb, the NIT table fields "original network id" and "network id", that perform as network identifiers and must be different for each broadcaster, are defined in Annex H of ABNT NBR 15603-2 Standard, but only for the Brazilian territory. Thus, when the tests started to be performed there was a legal loophole in the definition of these parameters for Uruguay. So, initially the "original network id" and the "network id" were set identically for every physical channel during every test. This led some receivers to fail. When tuning two or more channels they just stored one of them as if one channel was an update of the other. However, for most receivers this was not a problem.

After identifying this problem, URSEC defined in Resolution 111/2014 [29], emitted the 22nd of May 2014, the values for "original network id", "network id", "service id" and "country code" for each broadcaster in the national territory. After that, these parameters in the tests where changed and the problem disappeared.

4.3. Foreseen evolution of the type approval test suite

The current version of the protocol is 3.1, but the test suite will evolve, since the harmonization document may -and should- have updates arising from the ISDB-T International Forum. The standards for interactivity are especially active development. As previously written, a second version of the protocol is in progress, which will include mobile full_seg receivers, such as tablets or smartphones. So far, interactivity tests are not included as an admission requirement, but the group is working on this aspect, in relation with CUTI, in order to develop a testbed for applications.

5. Conclusion and future work

This paper presents interdisciplinary research that addressed the task of producing a type approval protocol for ISDB-Tb receivers for the Uruguayan market. The work was divided in two areas: on the one hand the electrical, signal reception and decoding aspects of the standard, and on the other, the interactivity functionality.

Regarding signal reception and decoding, the main features to be tested were analyzed and a representative test set was produced (not a comprehensive one), which aims at guaranteeing a right operation of the receiver. This test set covers the functions of signal reception, processing, decoding and display, and fulfills the requirement of being practically feasible and cost-effective, while being able to soundly ensure the compatibility and performance. This test procedure was adopted by the government and is already in production.

The experience gained in this work calls for a critical observation of the standard evolution, which shall be exercised by a technical team with participation of government, industry, broadcasters and academia. Furthermore, interactivity is tightly coupled with Internet and social networking, and cannot be confined solely to the closed broadcast television paradigm. To this end, it is worth considering the evolution of the Hybrid Broadcast Broadband TV (HbbTV), which seems to be a more up-to-date approach.

Acknowledgments

This work was supported by the Dirección Nacional de Telecomunicaciones y Servicios de Comunicación Audiovisual (DINATEL), Ministerio de Industria, Energía y Minería, Poder Ejecutivo, Uruguay. The working group is co-supported by the Agencia Nacional de Investigación e Innovación (ANII).

References

[1] Associação Brasileira de Normas Técnicas, [online], [accessed, April 9^th of 2014]. Available at: http://www.abnt.org.br        [ Links ]

[2] Instituto Nacional de Estadística (INE) y Agencia de Gobierno Electrónico y Sociedad de la Información (AGESIC), Encuesta específica de acceso y uso de TIC (EUTIC 2013), [online], [accessed, October 28^th of 2014]. Available at: http://www.agesic.gub.uy/innovaportal/file/3499/1/presentacion_eutic2013.pdf         [ Links ]

[3] Instituto Nacional de Estadística, Uruguay en Cifras 2013, Uruguay, 2013.         [ Links ]

[4] Dirección Nacional de Telecomunicaciones y Servicios de Comunicación Audiovisual (DINATEL), [online], [accessed on April 09^th of 2014]. Available at: http://www.dinatel.gub.uy.         [ Links ]

[5] Centro de Ensayos de Software, [online], [accessed on April 9^th of 2014]. Available at: http://www.ces.com.uy.         [ Links ]

[6] Cámara de Comercio Uruguayo - Británica, [online], [accessed on April 24^th 2014]. Available at: http://www.camurbri.com.uy/index.php/uruguay.         [ Links ]

[7] Ministerio de Educación y Cultura, [online], [accessed on April 24^th 2014]. Available at: http://www.mec.gub.uy/innovaportal/v/1613/8/mecweb/cuenta_satelite_en_cultura?3colid=690        [ Links ]

[8] Sitio Oficial del Middleware Ginga, [online]. [accessed on October 20^th 2014]. Available at: http://www.ginga.org.br,         [ Links ]

[9] Flores-Guridi, P., Garella, J.P, Guimerans, G., Baliosian, J., Grampín E. and Simon M., Testing ISDB-Tb Digital TV Receivers, XXI International Conference on Electrical Eng., Electronics Eng. and Computer Science (INTERCON 2014), pp 221-226, Arequipa, Peru, August, 2014.         [ Links ]

[10] ISDB-T Official Web Site, [online], [accessed on April 11^th 2014]. Available at: http://www.dibeg.org         [ Links ]

[11] ISO/IEC14496-3:2009 - Information technology - Coding of audio - visual objects - Part 3: Audio.         [ Links ]

[12] ISO/IEC 14496-10:2012 - Information technology - Coding of audio-visual objects - Part 10: Advanced Video Coding.         [ Links ]

[13] Associação Brasileira de Normas Técnicas, ABNT NBR 15601: Digital Terrestrial Television - Transmission System, Brazil, 2007.         [ Links ]

[14] Associação Brasileira de Normas Técnicas, ABNT NBR 15604: Digital Terrestrial Television - Receivers, Brazil, 2007.         [ Links ]

[15] Foro Internacional de ISDB-T 2012, ISDB-T Documento de Armonización - Parte 1: Hardware, [online], [accessed on October 27^th of 2014]. Available at: http://archivo.presidencia.gub.uy/sci/decretos/2013/05/miem_850_anexo1.pdf.         [ Links ]

[16] Ministerio de Industria, Energía y Minería - Poder Ejecutivo, Decreto número 143/013 - Fíjanse las especificaciones técnicas que deberán cumplir los aparatos receptores de Televisión Digital Terrestre para el estándar ISDB-T, Montevideo, Uruguay, 2013.         [ Links ]

[17] Edfors, O., Sandell, M., van de Beek, J.-J., Landström, D. and Sjöberg, F., An Introduction to Orthogonal Frequency-Division Multiplexing, Thesis, Luleå University of Technology, Luleå, Sweden, 1996.         [ Links ]

[18] Rodríguez-Díaz, B., Differential STBC for OFDM based wireless systems, Ph.D. Thesis, Technischen Universität Hamburg-Harburg, 2007.         [ Links ]

[19] Unidad Reguladora de Servicios de Comunicaciones, Pliego de Bases y Condiciones - Llamado a Interesados para obtener autorización destinada a brindar el Servicio de Radiodifusión de Televisión digital comercial con estación principal ubicada en el Departamento de Montevideo, Montevideo, Uruguay, 2013.         [ Links ]

[20] Middleton, J., PR & D. White, Paper: The Calibration of VHF/UHF field strength measuring equipment, British Broadcasting Corporation, 2008.         [ Links ]

[21] Ministerio de Industria, Energía y Minería - Poder Ejecutivo, Decreto Número 141/992 - Régimen de rotulaciones de productos envasados en ausencia del cliente, listo para ofrecerlo a los consumidores - Con las modificaciones introducidas por el Decreto 231/006, Montevideo, Uruguay, 2006.         [ Links ]

[22] Associação Brasileira de Normas Técnicas, ABNT NBR 15606 Digital terrestrial television - Data coding and transmission specification for digital broadcasting, Parts 1 to 9, Brazil.         [ Links ]

[23] TeleMidia Laboratory, Computer Science Department of Pontificial Catholic University of Rio de Janeiro, [online], [accessed on April 11^th of 2014]. Available at: http://www.telemidia.puc-rio.br.         [ Links ]

[24] Laboratorio de Investigación y Formación en Informática Avanzada (LIFIA), Universidad de La Plata, [online], [accessed on April 11^th of 2014]. Available at: http://lifia.info.unlp.edu.ar/es/,         [ Links ]

[25] Ginga.ar, [online], [accessed on: 11^th April of 2014]. Available at: http://tvd.lifia.info.unlp.edu.ar/ginga.ar/index.php/download.         [ Links ]

[26] Ginga-NCL Conformance Testing, [online], [accessed on October 20^th of 2014]. Available at: http://testsuite.gingancl.org.br.         [ Links ]

[27] Protocolo de Homologación de Receptores ISDB-Tb, [online], [accessed on October 28^th of 2014]. Available at: http://www.dinatel.gub.uy/-/publican-protocolo-de-homologacion-de-receptores-de-tvd.         [ Links ]

[28] Association of Radio Industries and Businesses, Operational Guidelines for Digital Terrestrial Television Broadcasting - Fascicle 1, Version 3.8, 2008.         [ Links ]

[29] Unidad Reguladora de Servicios de Comunicaciones, Resolución 111/2014 - Establecer criterios de determinación de los campos en las estaciones transmisoras de Televisión Digital Terrestre en el Territorio Nacional, Montevideo, Uruguay, 2014, [online], [accessed on October 27^th of 2014]. Available at: https://www.ursec.gub.uy/inicio/transparencia/resoluciones/2014,         [ Links ]

[30] Moreno-López, G.A. and Jiménez-Builes, J.A., Cycle of PDCA T-Learning model and its application on interactive digital TV, DYNA, 173 (2), pp. 61-70, 2012.         [ Links ]

P. Flores-Guridi, received his Electrical Engineering degree of from the Universidad de la República (UdelaR), Uruguay in 2012, and is currently working on his Master's degree thesis in the field of Telecommunications. Flores is a teaching assistant at the School of Engineering of UdelaR, Uruguay and has participated in various research projects on digital broadcasting. His main academic interests include digital broadcasting systems, channel coding and software defined radio.

G. Guimerans, received his Computer Engineering degree from the Universidad de la República (UdelaR), Uruguay in 2010. He works as a General Manager of the Software Testing Center (Centro de Ensayos de Software (CES)), and also as teaching assistant in computer science at UdelaR. Guimerans entered the IT industry in 2007, and since then he has worked as an auditor developer, systems administrator, trainer, consultant, manager, and speaker. He has worked with a variety of platforms and technologies, consulting for nonprofits, telecoms, financial services, and others companies with public and private partners. His main research interests include performance testing and enterprise software.

J.P. Garella, received his degree in Electrical Engineering from the Universidad de la República (UdelaR), Uruguay in 2011. He is currently undertaking his Master´s degree in Electrical Engineering at the same university. He has participated in research projects on digital television with emphasis in perceived video quality estimation, no reference metrics and QoS monitoring. Currently he is working as a consultant in the approval of ISDB-Tb receiver task undertaken at the Digital TV Lab, LATU, Uruguay. His research interests include: Perceived Video Quality; QoE; Digital TV; ISDB-Tb.

J. Baliosian, received his degree in Computer Engineering from the Universidad de la República, Uruguay (UdelaR) in 1998, and his PhD in 2005, from the Polytechnic University of Catalonia (UPC), Spain. Javier has been working professionally since 1994 as a developer, designer, architect and leader of diverse distributed systems developments. He has been involved in several collaborative research projects with different groups such as the Computer Laboratory of the University of Cambridge and the Laboratory of Communication Networks at KTH in Sweden and the Ericsson Ireland Research Centre, where he worked as a researcher and project coordinator from 2005 to 2007. Since then, Javier has been working in the Department of Computer Science at the Universidad de la República, Uruguay, where he is contributing to several network management research projects. Javier Baliosian is also in the TPC of the main network management conferences such as NOMS and IM.

E. Grampín, received his degree in Electrical Engineering from the Universidad de la República (UdelaR), Uruguay in 1995, and completed his PhD in 2005 at the Polytechnic University of Catalonia (UPC), Spain. He has been working as a part-time teaching assistant in Computer Science at UdelaR, Uruguay and also as a network engineer at the Uruguayan Public Telecommunication Corporation (ANTEL), and since 1994, in the planning and operation of IP networks. After completion of his PhD he obtained an associate professor tenure-track position in UdelaR, Uruguay and became a professor in 2014. Grampín has been involved in several collaborative research projects with both academic and industry partners. He has completed several visiting professorships in different universities, spending one year in a post-doctoral position in 2010, at the Carlos III University (UC3M) in Madrid, Spain. His main research interests include autonomic, policy-based network management of wired and wireless networks, interdomain routing, traffic engineering, software defined networks, network experimentation and reproducible research.

R. Sotelo, is the Director of ICT Dept. at the Universidad de Montevideo, Uruguay and associate professor at Universidad de la República, Uruguay. He completed his degree in Electrical Engineering in 1994 and his MBA in 1999. He received his PhD degree in Telematics Engineering in 2010, from Universidad de Vigo, Spain. He has worked in the broadcasting and cable TV industry since 1991. He is advisor at DINATEL/MIEM. His main research interests are in the areas of quality of experience in multimedia applications, transmission of DTV and recommender systems.