2.1 State of the Art

To evaluate the performance of a MANET supporting real-time distance education services, several indicators should be comparatively analyzed under certain factors. For this reason, the performance metrics and the factors that affect them must be established.

Statistical Design of Experiments (DoE) is a strategy implemented to quantify the main effects of factors and their interactions on the required indicators. The response surface factorial design methodology follows a Taguchi approach, which alludes to some DoE strategies frequently employed by many researchers to identify the effects of several factors on each metric. The DoE strategy is essentially used to improve the design process. Some studies in this research field are detailed below.

In ^[6], the impact of certain factors and their interactions on the provision of MANET services are analyzed. In such work, performance factors such as real-time performance, total performance, and mean delay are employed to measure the provided service considering aspects such as architecture, QoS, routing protocols, MAC protocols, offered load, and node mobility. Simulation data can be analyzed using some statistical techniques such as Analysis of Variance (ANOVA) to identify the main effects and interactions of the factors that explain the performance parameters. In ^[7], the authors found that mean delay, MAC protocols, and their bidirectional interaction with the routing protocols are the most significant factors.

In ^[8], a 2^k factorial design strategy (which makes it possible to analyze the efficiency of MANETs) like the one to be implemented in this paper is applied. The main interaction effects of five factors (node ​​mobility, access control to the medium, network size, network density, and traffic load) are evaluated and two performance metrics are calculated (end-to-end delay and package delivery ratio). Suitable results were obtained with the combinations of factors and the metrics under analysis.

In ^[9], the performance of the Dynamic Source Routing (DSR) protocol in a MANET was evaluated using a Taguchi DoE technique to quantify the most important effect of six factors (transmission rates, greeting message delivery time, terrain, network size, node speed, and number of sources) on two performance metrics (end-to-end usage and delay). To determine the best performance of the DSR protocol, the Analysis of Variance (ANOVA) method was employed. This method allowed the authors to identify the best combinations of factors for the two metrics under study. Transmission rate was found to be the most important parameter that contributes to the network’s performance, while number of sources was the most influential factor on end-to-end delay. With respect to performance, multiple metrics, delay, and end-to-end performance were simultaneously considered.

Regarding robustness and automation in the Extract, Transform, Load (ETL) process in data warehousing, the authors of ^[10] present a multi-agent model that gathers the strengths of other approaches such as wrappers and ad-hoc solutions. Such a model considers the heterogeneity and availability of the data sources, as well as their distributed nature. For its validation, an experiment was performed using simulated and real data. However, this study is not focused on education.

In ^[11], a virtual platform that allows the simulation and the local and remote command and control of a SCARA robot arm was developed. This platform, based on the engineering education methodology, consists of two applications that simulate the robot and enable its remote and local command and control. Although communication was established via TCP/IP, the MANET technology was not used.

Security is a critical issue in mobile networks. Nowadays, communication via mobile devices is necessary; however, their use is restricted due to the presence of attacks. Encrypted authentication and data encryption are techniques employed to protect devices against such threats. In ^[12], a strategy to analyze the most common and safe encryption algorithms is presented.

2.2 Theoretical Framework

2.2.2 Distance Education and Real-Time Services

Technological tools are used for educational activities in which students and teachers do not need to be in the same physical space at the same time ^[14].

Distance education can be defined as a combination between distance training and distance learning. The former involves a student-instructor interaction in which the instructor acts as a guide in students’ educational process, while the latter represents an information system in which students are the only responsible for their education, which is based on academic materials ^[15].

Distance education can be:

• -Asynchronous

• -E-mail messaging

• -Discussion forums

• -Synchronous (or real-time)

• -Videoconference

• -Chat rooms

Figure 1 illustrates a technology scenario of real-time distance education and services. Other Tele-education services may include:

• -Telecommunications networks

• -Telematic services

• -Applications

• -Content

• -Going beyond knowledge transmission

• -Complementing with cooperative learning

• -The Web as a tool

• -Shared spaces and cooperation

• -Multimedia servers

• -More interactivity

• -Synchronous applications

• -Audio and video

• -Virtual reality ^[16].

Source: ^[14].

Figure 1 Example of a MANET supporting distance education services 

3.1 Methodology

The methodology proposed to evaluate the QoS of real-time tele-education services, such as a video streaming supported in a MANET, which allows verifying whether this network can serve as a communication infrastructure, is made up of three phases.

In the first phase (the operational one), the MANET is implemented. The second phase (the experimental one) is based on an adaptation of the DoE methodology, which is one of the most widely used statistical techniques to optimize experimentation. This methodology is defined as a method to systematically apply statistics to the experimentation process ^[17]. More precisely, it can be defined as the application of a set of tests in which voluntary changes are made to the control parameters of a process or system in order to observe and identify the reasons for changes in the output or response variable of the process ^[18].

To obtain objective and subjective conclusions, an experimental evaluation must comprise two fundamental and interrelated components: 1) an experimental design, which refers to the planning process of the experiment so that the data can be collected, in a viable manner, for statistical analysis; and 2) an efficient statistical analysis of the data ^[19].

The third phase (the evaluative one) consists in the verification or evaluation of the QoS once the problem to be solved has been identified. Figure 2 presents the proposed methodology.

Source: Authors’ own work.

Figure 2 The proposed methodology 

3.2 Testbed, Design, and Implementation

The experiment was conducted in the Center of Advanced Manufacturing Technology (CTMA for its Spanish acronym) at SENA (the Colombian National Training Service). The scenario consisted of two spaces: an outdoor 100-square meter area on a second floor (called the Lobby & Cafeteria) and an indoor 50-square meter area on the third floor (called the ICT/Electronics Lab). The testbed comprised ten nodes equally distributed in the two spaces.

All the nodes shared the same technical features: HP ProBook 6475b laptops with an AMD A10-4600M quad-core 2.3-GHz processor, a 6-GB RAM memory, and an Atheros AR9485 802.11 b/g/n integrated Wi-Fi adapter operating in “n” mode. Additionally, a Samsung Galaxy S3 (GT-I9300) smartphone with a quad-core 1.4-GHz processor, a 1-GB RAM memory, and an 802.11 b/g/n integrated Wi-Fi adapter operating in “n” mode was used to manage the MANET.

In a testbed, there are two systematic traffic sources that cannot be eliminated: 1) interconnected wireless access points within the campus, which cause interference and occupy the available bandwidth (which is typical in an academic environment); and 2) traffic control due to the Secure Shell (SSH) program, which is used to remotely start and control the measurement software on the source node.

To generate traffic and emulate real-time distance education services supported by a MANET, a network personal digital video recorder (npDvr)software ^[20] was used to stream a video in all the nodes connected to the MANET from the main node.

3.2.1 Testbed Interface

The MANET was implemented and managed using a smartphone app called MANET MANAGER (shown in Figure 3) ^[21].

Source: Authors’ own work.

Figure 3 MANET MANAGER app 

The following setting was used for the other nodes:Open Windows 7 Network and Sharing Center (Figure 4) in each computer.

Source: Authors’ own work.

Figure 4 Windows 7 Network and Sharing Center 

• -Select the MANET.

• -Double click to connect.

• -Set a static IP address for the other devices in the network in the 192.168.1.0/24 range, as can be seen in Table 1, without using the 192.168.1.1 IP address, which is reserved for the network administrator device.

Table 1 IP addresses of the client devices in the MANET 

Source: Authors’ own work.

• -Install and run the Optimized Link State Routing (OLSR) software (OLSR Switch) ^[22] and configure the parameters established in the MANET experiment, as observed in Figure 5.

Source: Authors’ own work.

Figure 5 Program to configure the parameters of MANET routing protocol: OLSR 

• -Install the app MANET VISUALIZER in the administrator device ^[23] to view the network’s topology, as shown in Figure 6.

Source: Authors’ own work.

Figure 6 MANET topology 

• -Using the administrator device of the MANET, with MANET MANAGER previously installed, the user can view different tables of the MANET (e.g., routing, topology, and neighbor table, among others) and execute some evaluation parameters, as can be seen in Figure 7.

Source: Authors’ own work.

Figure 7 MANET MANAGER software with information tables 

3.3 Experimental Scenario

The testbed provides an experimental platform to evaluate different real-time distance education services supported by a MANET using actual parameters. Moreover, several network topologies and scenarios can be implemented to analyze and evaluate such services considering multiple metrics.

This work considers different considerations:

Source: Authors’ own work.

Figure 8 Floor plans of the scenarios in the experiment at the CTMA (SENA) 

3.4 Experiment

This project evaluates a single response variable or metric of a MANET supporting real-time distance education services: the throughput. The latter is defined as the total data sent and received over a certain period as shown in equation 1. An efficient network achieves a better throughput, which is measured in bits per second. Table 2 details the usual operating values of such metric.

Table 2 Response variable 

Source: Authors’ own work.

3.5 Throughput Equation

Calculate Bandwidth-Delay product and TCP buffer size BDP (Bits of data in transit between hosts) = bottleneck link capacity (1):

Figure 9 illustrates the system implemented to evaluate throughput. This system comprises three main components (factors, experimental unit, and response variable) and other secondary elements. A distance education service was provided through real-time video streaming using four cameras connected to a Digital Video Recorder (DVR) in the ICT/Electronics Lab. A lecture in each field was recorded, and the equipment (connected to a main computer with a UTP cable) generated the service, which was transmitted, through the network, to mobile devices located indoors and outdoors. Said devices were connected to the MANET and received the service via a web client application (npDvr).

Source: Authors’ own work.

Figure 9 Components of the experimental setup 

The online service in the testbed was transmitted considering the parameters presented in Table 3. Each transmission session was controlled at the nodes by the SSH program, and the packets were captured using WinDump ^[24]. In general, a great number of factors can affect the performance of a MANET supporting real-time distance education services. Three factors were selected in this study: network size, transmission rate, and node speed. Other aspects, considered constant, are listed in Table 3.

Table 3 Coded factors and levels 

Source: Authors’ own work.

3.6 Design of the Factorial Combinations

In this paper, a 2^k factorial design was implemented, where k = 3 factors and each factor had two levels (or values). The R statistical software enabled the creation of 2³ = 8 design points, which are presented in standard order in Table 4. Each design point was evaluated in a real network scenario (testbed) with a replication of r = 3 times or number of factors. Each design point reflects the change in the network size (from 5 to 10 nodes). Point pairs 1-2/3-4, 5-6/7-8, and so on correspond to the change in the transmission rate (from a low to a high level). Additionally, point pairs 1-4/5-8 and so on denote the change in the nodes’ mobility speed (from low to high). The response value is also shown in Table 4. Such values resulted from the evaluation of the performance of a MANET supporting real-time distance education services.

Table 4 Design matrix for the entire factorial design (k = 3) 

Source: Authors’ own work.

3.7 Data Collection

The experiment was conducted in a real environment (or testbed). To obtain the throughput results, each of the eight design points was executed five times, and five runs were calculated per design point. The selection of the design points for the execution was based on the execution order created by Microsoft Excel to randomize data. Thus, each of the resulting 96 runs of the experiment in the testbed (32 × 3) were executed for 300 seconds.

3.8 Data Analysis

Figure 10 presents the means of the main efects evaluated with the response variable, throughput. It can be inferred that the performance of the MANET is affected by network size and the transmission speed of the video streaming service it supports. Mobility speed has little or no impact because it is almost at the same level of the corresponding response variable, which can be seen in Figure 10.

Source: Authors’ own work.

Figure 10 Means of the main effects 

Figure 11 presents the interaction plots of the throughput results. Regarding two-way interactions, performance is affected by the combined effect of network size and transmission rate, as observed in Figure 11 and vice versa, as shown in Figure 11 The other interactions follow the same pattern.

Source: Authors’ own work.

Figure 11 Interaction plots 

These interaction plots depict the effect of the combination of factors on the response variable.

Based on these results, it can be concluded that, in most cases, there is a significant effect due to the interaction between the factors. This conclusion is supported by the fact that the lines of each factor cross at some point. On the contrary, when they run parallel, there is a null interaction between the factors in each case.

3.8.1. Analysis of Variance and Conclusion on the Significance of the Sources of Variation of the Model

Analysis of Variance (ANOVA) can be used to determine whether there are any statistically significant differences between two or more means in order to define whether the treatments have different effects on the response variable. The results obtained with this method can be employed to produce estimations and test hypotheses ^[18], which corresponds to an intermediate step in the analysis. The next step is finding different measures and estimating their difference using complementary techniques. Table 5 shows the results obtained with the R statistical package used to perform the ANOVA test in the testbed of the MANET.

To draw conclusions regarding significance, the p-value of each factor had to be evaluated based on the risk factor assumed by the researchers. The risk factor considered in the ANOVA model was 0.05. Therefore, it was necessary to verify whether the p-value of each factor was greater or lower than such value. If it was above such value, the effect of the factor was found to be NOT SIGNIFICANT; and if it was below such value, the effect of the factor was found to be SIGNIFICANT in terms of the variability of the response variable.

As observed in Table 5, the p-value of the Nodes factor is significantly lower than the risk value, which means that this factor has a significant effect on the variability of the response variable. The Nodes-Bitrate and Bitrate-MobSpeed interactions exhibit the same behavior. However, the effect of most factors and interactions is not significant, i.e., the variability of the response variable does not depend on them. In other words, modifying the levels of such factors and interactions at the same time will not produce a different result when compared to changing them one at a time. In ANOVA, the null hypothesis is rejected if the p-value is lower than 0.05.

Table 5 Result of the ANOVA for the objective variable: throughput 

Source: Authors’ own work.

After the model proposed in this study was tested through residual analysis, it can be concluded that:

The data obtained from the experiment were analyzed and validated using a statistical technique known as Analysis of Variance (ANOVA) to draw objective and subjective conclusions. A type I analysis was conducted in this study because the level of each factor applied to the experimental setup was previously selected by the researchers. However, to apply such methodology, the following conditions had to be satisfied: homoscedasticity (homogeneity of the variances), independence of the observations, and normal distribution of the error. This experiment passed all the corresponding tests.

To solve the problem addressed in this study, the model that describes the behavior of the response variable (throughput) was a balanced, completely randomized experimental design with three factors. In this design, the interactions between the different factors were assumed to be not null, and the DoE statistical model followed certain steps to confirm the corresponding hypotheses ^[18].

MANETs are efficient networks that can be very useful to train students in any type of environment (outdoors or indoors). Such efficiency was observed when the network size increased from 5 to 10 nodes, which resulted in a better performance, as can be seen in the plots of the main-effect means in Figure 10. This finding is supported by the hypothesis of the experiment and the validation of the model.