SciELO - Scientific Electronic Library Online

vol.37 issue2Development of a first 3D crustal velocity model for the region of Bogotá, ColombiaMechanism of interchange between the reflections in a channel of communications with the power spectral density author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand




Related links

  • On index processCited by Google
  • Have no similar articlesSimilars in SciELO
  • On index processSimilars in Google


Ingeniería e Investigación

Print version ISSN 0120-5609

Ing. Investig. vol.37 no.2 Bogotá May/Aug. 2017 

Artículos originales

Estimation of quality of experience (QoE) in e-Health Ecosystems

Estimación de calidad de experiencia (QoE) en ecosistemas de e-Salud

V. Rojas-Mendizabal1 

A. Serrano-Santoyo2 

R. Conte-Galvan3 

S. Villarreal-Reyes4 

R. Rivera-Rodriguez5 

1 M.S. in Telecommunication and Telematics. Affiliation: Doctoral Student in Electronics and Telecommunications, CICESE Research Center, B.C., Mexico. Affiliation: Doctoral Student, CICESE, Mexico. E-mail:

2 PhD in Electrical Engineering, National Polytechnic Institute in Mexico City. Mexico. Affiliation: Titular Researcher at CICESE Research Center in Ensenada, B.C. Mexico. E-mail:

3 PhD in Electric Engineering, Virginia Polytechnic Institute and State University, US. Affiliation: Titular Researcher at CICESE Research Center, Ensenada, B.C. Mexico. E-mail:

4 Ph.D. in Electrical and Electronics Engineering, Loughborough University, United Kingdom. Affiliation: Titular Researcher at CICESE Research Center in Ensenada, B.C. Mexico. E-mail:

5 PhD from Autonomous University of Baja California, B.C., Mexico. Affiliation: Director of the Telematics Division at CICESE Research Center, B.C., Mexico. E-mail:


This article proposes a framework to design and implement e- Health interventions in a comprehensive manner. We draw on complexity science to study the interplay of the ecosystem, the behavior and interactions among its agents. We provide a platform to estimate the Quality of Experience (QoE) to assess the relationship between technology and human factors involved in e-Health projects. Our aim is to estimate QoE in e-Health ecosystems from the perspective of complexity by adopting a methodology that uses fuzzy logic to study the behavior of the ecosystem's agents. We apply the proposed framework to a remote diagnosis case by means of an ultrasound probe through a satellite link. Despite the ambiguities for determining QoE, the experiment demonstrates the applicability of the framework and allows to stressing the importance of human factors in the implementation of e-Health projects.

Keywords: QoE; QoS; complexity science; e-health; fuzzy logic


En este artículo se propone un marco de referencia para diseñar e implementar intervenciones de e-Salud de una manera integral. Recurrimos a la ciencia de la complejidad para estudiar la interacción de los agentes del ecosistema. Para ello, proporcionamos una plataforma para estimar la Calidad de Experiencia (o QoE, por sus siglas en inglés) para evaluar la relación entre la tecnología y los factores humanos que intervienen en los proyectos de e-Salud. Nuestro objetivo es estimar la QoE en los ecosistemas de e-Salud desde la perspectiva de la complejidad mediante la adopción de una metodología que utiliza la lógica difusa para estudiar el comportamiento de los agentes del ecosistema. Aplicamos el marco propuesto a un caso de diagnóstico remoto por medio de una sonda de ultrasonido a través de un enlace por satélite. A pesar de las ambigüedades involucradas en la determinación de calidad de la experiencia, el experimento demuestra la aplicabilidad del marco de referencia propuesto y permite subrayar la importancia de los factores humanos en la implementación de proyectos de e-Salud.

Palabras clave: QoE; QoS; ciencia de la complejidad; e-salud; lógica difusa


The development and application of Information and Communication Technologies (ICT) in the health sector provide significant opportunities for creating efficient systems and processes aiming to improve the quality and coverage of the offered services. We argue that the analysis of the health ecosystem including its human and socio-economic factors is key to fully exploit the deployment and adoption of digital equipment, health information systems and medical devices. Hence, the estimation of the factor known as Quality of Experience (QoE) is relevant. QoE involves elements relating the expectations, needs and experiences and even emotional aspects of all the stakeholders regarding the use and adoption of technology in a particular context. Therefore, given the complexity and multidimensional nature of the e-Health ecosystem, the evaluation of QoE becomes challenging (De Marez & De Moor, 2007; K. De Moor 2010; Rojas-Mendizabal, Serrano-Santoyo, Castillo-Olea, Gomez-Gonzalez, & Conte-Galvan, 2014). In our proposed framework, we conceptualize the ecosystem as a socio-technical system in which many disciplines converge giving rise to a dynamic environment highly dependent on the context and the interactions among all its agents. Our objective is to provide a platform for the estimation of QoE to supporting the design and implementation of e-Health projects in an integral fashion, i.e., to establish a bridge between traditional approaches centered exclusively on technology variables and those focusing on interdisciplinary concerns that include human factors and the context. In this regard, we draw on the interrelation of QoE and the principles of complexity science to accomplish such objective.

The International Telecommunication Union (ITU) introduced the term QoE, defining it as: "The general acceptance of an application or service, as perceived subjectively by the end user" (ITU-T P.1080 Rec. Series G, 2008). Similarly, ITU argues that the Quality of Service (QoS) is "the collective effect of service performance which determines the degree of satisfaction for service usage" (ITU-T P.800 Rec. Series E, 2008). QoS depends on measurable parameters such as delay, jitter, throughput, bit error rate (BER), and packet loss rate (PLR). Figure 1 shows the old and the new paradigms for the design of telecommunications systems in terms of user satisfaction. The old paradigm focused only on the technology side, however, the new paradigm, shows a user acceptance influenced by expectations and context (Alreshoodi & Woods, 2013; ETSI, 2010; Le Callet, Möller & Perkis, 2012; Möller & Raake, 2014). In this regard, new models of technology adoption and use such as the unified theory of acceptance and use of technology (UTAUT) consider the aspects of culture and context as key factors of social influence (Venkatesh & Zhang, 2010). While the development of technology is important when deploying an application of e-Health, it is also important to determine the degree of acceptance by the user. This is the purpose of our framework regarding the estimation of QoE for a particular e-Health intervention.

Source: Authors

Figure 1 Old and new paradigms for the design of telecommunications systems. 

We employed the proposed framework for gynecological remote diagnosis using an ultrasound probe through a satellite link. Based upon data obtained in the remote diagnosis experience, we estimated QoE using neuro-fuzzy networks. This simulation strategy allows us to observe the interaction and impact of both technological and human factors providing useful information to estimate the magnitude of QoE.

Related work

According to M. Ullah . (Ullah, Fiedler, & Wac, 2012), the factors that contribute to success in the development of an e-Health system are: user satisfaction and acceptance of the system (37%), technology implementation (29 %), organizational aspects (13%), policies and legislation (11%) and financial aspects (10%). These authors emphasize that the acceptance factors are the most representative in the successful implementation of e-Health projects. On the other hand, Broens noted in (Broens ., 2007), that the attitudes of the users (patients and physicians) in relation to a particular service, as well as the usability of the systems employed, are also significant.

Gong Y. . (Gong, Yang, Huang, & Su, 2009) define five QoE factors: integrity, continuity, availability, ease of use and immediacy of service. These authors focus on the correlation between QoS and QoE for user acceptance. A. Perkis . (Perkis, Munkeby, & Hillestad, 2006) introduce measurable elements (objective) and not measurable (subjective) associated with QoS and QoE for user acceptance. Their contribution does not consider the involvement of a context. Meanwhile, Moller . (Moller, Engelbrecht, Kuhnel, Wechsung, & Weiss, 2009) propose a new taxonomy for QoS involving elements that interact and influence the performance of the network. Regarding QoE, they include both human judgment and perception processes without considering the context. Similarly, Laghari K.. (Laghari & Connelly, 2012) present a model based on a complete telecommunications ecosystem including the interactions of their agents, as well as, aspects regarding technology, business, human behavior and human-technology, technology-business and business-human interactions. They define the ecosystem as "the systematic interaction of living beings (human) and non-living (technology and economics) in a particular context". Finally, Kilkki (2008) proposes an ecosystem, which includes technology, business models and human behavior; these authors stress the importance of the participation of engineers, entrepreneurs, scientists and their interaction with the end users.

Framework for the estimation of QoE

As part of the proposed framework, the variables for estimating QoE are identified and shown in Figure 2. The degree of usability is defined by the effectiveness and efficiency of the device, the software and the Internet connectivity. The weight of the variables to estimate QoE would vary depending on the particular intervention. For example, for the transmission of an X ray file over the Internet, downloading the data in full is required, therefore the transmission rate and associated bandwidth are important variables, however, delay and jitter, while important, they are not critical parameters for successful downloading of these files. However, in order to perform a teleconsultation in which a dermatoscope and videoconferencing are employed, jitter and delay play a critical role, as they can significantly affect the quality of the communication link. (Malindi, 2011; Nanda & Fernandes, 2007). Ultimately, the precise values of the variables employed depend on the particular context and characteristics of the intervention.

Source: Authors

Figure 2 QoE attributes.  

The framework proposed to estimate QoE consists of the following seven steps:

Stepl 1. Study of the e-Health ecosystem: Analysis of the context and definition of the e-Health intervention purpose. Inasmuch as QoE is associated to the needs and expectations of end users, it is important to state clearly the type of e-Health service (teleconsultation, remote diagnosis, telesurgery, etc.). Likewise it is essential to clearly identify the problem to solve. The insights of health professionals regarding the requirements and limitations of each service are very valuable and must be considered from the inception of the e-Health service. The purpose of the specific e-Health service is defined according to the context, user needs, financial requirements, if any, and other factors of administrative nature. At this stage, we draw on the proposal of Rolando Garcia for interdisciplinary research which provides theoretical an empirical foundations for understanding the dynamics of the particular e-Health intervention. This approach was valuable also for identifying the specify problem to solve and for defining the elements of the subsystems in place through the construction of an epistemic framework from the inception of the project (García, 2006).

Step 2. Identifying the actors and their interactions within the specific context: The objective of this step is to identify the human, technology and socioeconomic variables for the particular e-Health intervention.

Step 3. Assigning weights to variables: In this step we identify the variables for each actor (medical specialist, patient, devices, software and communications network) and their corresponding weights.

Step 4. Planning and conducting a survey: The opinion of the users is very important for the assessment of QoE. For this purpose, it is recommended to develop a questionnaire that includes elements to estimate the levels of perception and user experience. In this respect, the support of medical specialists, psychologists and professionals with basic statistical background is very valuable. In some cases the survey is improved by in-situ observation of the e-Health intervention.

Step 5. QoE simulations: We run our simulation including the input variables, the results of the survey and the in-situ observations. We apply a neuro-fuzzy network for simulating the particular intervention.

Step 6. Analysis of results: The simulation results show the performance of the key variables involved in the particular e-Health intervention. The proposed scenarios provided important insights regarding the assessment of QoE.

Step 7. Refinement of simulation results: Iteration and adjustments of the program routines allow to fine-tuning the neuro-fuzzy network performance.

Estimation of QoE for telediagnosis using an ultrasound probe

In order to estimate QoE, we follow the steps described above. The scenario for the tele diagnosis intervention using an ultrasound probe is shown in Figure 3. We chose a gynecology application between two remote sites connected by a satellite link. Personal computers with videoconferencing software were installed in each of the remote sites. The objective was to conduct a remote diagnosis to assessing the condition of the patient and the embryo regarding: Static fetal growth, fetal vitality, detection of twin pregnancy, fetal biometry, neurological baseline study, cardiac assessment, evaluation of renal functions, skeletal malformations, sex fetal amniotic fluid pathology, and pathology of the placenta. This assessment is useful in rural settings, where the lack of hospital infrastructure and medical specialists challenges the childbirth care (Benito, 2014).

Source: Authors

Figure 3 Scenario for the telediagnosis intervention. 

According to step 2, the actors involved and their rules of behavior were identified. Figure 4 shows the main actors within the e-Health intervention. The QoE Parameters involved in our simulation are shown in figure 5. In step 3, we assigned, with the support of medical specialists, weights to the variables used in the simulation process. The survey was conducted in step 4 taking into account insights from an in-situ observer to identify additional subjective elements to be included in the simulation. In step 5 we carried out the simulation using the data obtained in the previous step. The neuro-fuzzy networks simulation was carried out using both linguistic and numerical values. The results of the simulation are obtained in step 6 and finally in step 7, we analyze different scenarios to refine the values of the variables used to estimate QoE.

Source: Authors

Figure 4 Main actors involved in the e-Health intervention. 

Source: Authors

Figure 5 QoE ParametersResults and discussion. 

The experiment was carried out varying the transmission rates of the satellite link in order to obtain information about the network performance. The transmission rates employed were 256 Kbps, 512 Kbps and 1 Mbps. It is worth mentioning that, in order to study the usability issues involved, the actors did not receive any explanation regarding the features of the software or the interfaces used. We used a Likert scale for our questionnaire, where the participants responded to issues related to the e-Health intervention.

In order to obtain QoE for the intervention studied, we employed neuro-fuzzy networks of the type shown in Figure 6. The neuro-fuzzy networks have been used in different contexts because they allow the incorporation of various actors including perception criteria (Kwong & Wong, 2008). The application of the proposed neuro-fuzzy network was very valuable to understand the role of each of the actor's attributes and the interrelation of technology, human and socio-economic aspects, which are key to estimate the relative magnitude of QoE.

Source: Authors

Figure 6 Neuro-Fuzzy Network. 

Table 1 QoS requirements for e-Health 

Source: Authors

As shown in Figure 7, the value of QoE depends on QoS, i.e. with a resulting low value of QoS, it is not possible to obtain a high QoE; whereas if QoS is high, QoE is not necessarily high. This is because, as shown in Figure 6, other variables of subjective nature affect and influence the final magnitude of QoE. Therefore, it should be realized that a high QoS will not determine the performance and efficiency of the e-Health intervention. It is also essential to consider the influence of QoDevice and QoSoftware, which depend on usability and context factors. Hence, the human factors are key for the success of e-Health interventions.

Source: Authors

Figure 7 QoE magnitude for different values of QoS; a) QoS =10%, b) QoS =50% y c) QoS =90%. 

Figure 7 b) shows that for an average value of QoS (50%), it is possible to obtain a QoE between 60% and 80%. This estimated value for QoE is considered satisfactory, since human and technology factors combine to achieve his or her acceptance. Whereby, devices with acceptable operation and friendly graphical interface are key for getting a high QoE. This indicates the importance of human factors even when the values of QoS are average.

It is noteworthy that, in addition to the simulation results, opinions gathered from medical specialists in the development of the survey, indicated some limitations in this particular telediagnosis intervention: a) The medical specialist was not able to perform a cardiac assessment because the received ultrasound video signal as seriously affected by jitter. b) It was not possible to observe skeletal malformations in-depth or carrying out a neurological baseline study due to technical limitations of the portable ultrasound probe. These limitations affected the overall performance of the intervention. Nevertheless, the use of telediagnosis using an ultrasound probe and a satellite communications link has been effective for e-Health applications in remote and isolated contexts lacking both hospital facilities and physicians.

The simulation results indicate the relevance of the variables regarding usability and context to accomplish a successful e-Health intervention (see Figure 2). The estimation of QoE represents a major challenge due to the ambiguous nature of this factor (De Marez & De Moor, 2007; K. De Moor., 2010). The proposed framework constitutes a feasible strategy to face this challenge.

In addition, the steps suggested in this paper provide a basic methodology to study the interrelationship and interdependence of the human and technology factors involved in the e-Health intervention. Our approach may contribute to formulating public policies focused on enhancing the coverage and quality of health services in underserved regions. Furthermore, the estimation of QoE using the proposed framework may provide new findings and additional elements for the definition of indicators for the development and financial sustainability of Telemedicine projects (Pan American Health Organization, 2014). Additionally, it has been reported that a systematic approach in e-Health interventions may conduce to replicability and sustainability of the projects (González, Fernández, & Moreno, 2012).

In order to refine the proposed framework, we recommend for future work to increase the number of participants in the survey. We also recommend to run the neuro-fuzzy networks simulation for other cases of e-Health, such as teleconsultation, telemonitoring, health education and access to medical records (Malindi, 2011) to understand more in-depth the role of QoE in e-Health ecosystems.

Likewise, the use of agent-based modeling (ABM) would provide other perspectives to study the behavior of actors in the ecosystem. Experiences with different connectivity infrastructure and in different contexts would also contribute to better understand the impact of human elements on e-Health interventions.


The purpose of this article has been to provide a framework for the design and implementation of e-Health interventions taking into account the human, technology and socioeconomic factors. We estimate QoE using neuro-fuzzy networks simulation to generate scenarios to supporting the integral design of e-Health projects. We argue that our framework provides an alternative tool for the understanding of the role and nature of all the actors involved in the e-Health ecosystem. The proposed framework would also contribute to identifying indicators for cost-benefit and cost-effectiveness evaluation of e-Health projects. In the same manner, our framework can also be used as a platform for the development of standards, which is essential to strengthen policy and regulatory issues of e-Health, particularly in emerging and developing countries.


The authors wish to thank the company Astrium in Madrid, Spain, for using their facilities and providing the satellite link to carry out the experiments. We express our appreciation to the Rey Juan Carlos University and the Enlace Hispanoamericano de Salud (EHAS) Foundation for providing the equipment and logistics support regarding the preparation of the tests and survey. Finally, we thank the National Council of Science and Technology of Mexico (CONACYT) and the Center for Scientific Research and Higher Education of Ensenada (CICESE) for funding our research and the development of this paper.


Alreshoodi, M., & Woods, J. (2013). Survey on QoE\ QoS correlation models for multimedia services. arXiv preprint arXiv:1306.0221. [ Links ]

Benito, R. (2014, March 15). En la selva peruana me di cuenta de que un ingeniero también puede salvar vidas. El Confidencial. Retrieved from [ Links ]

Broens, T. H., Huis in't Veld, R. M., Vollenbroek-Hutten, M. M., Hermens, H. J., van Halteren, A. T., & Nieuwenhuis, L. J. (2007). Determinants of successful telemedicine implementations: a literature study. Journal of telemedicine and telecare, 13(6), 303-309. [ Links ]

De Marez, L., & De Moor, K. (2007). The challenge of user-and QoE-centric research and product development in today's ICT-environment. Observatorio (OBS*), 1(3). [ Links ]

ETSI. (2010). Quality of Experience (QoE) requirements for real-time communication services (Technical Report No. V1.0.2(2010-01)). Francia: European Telecommunications Standards Institute. [ Links ]

García, R. (2007). Sistemas complejos: conceptos, métodos y fundamentación epistemológica de la investigación interdisciplinaria. Barcelona: Gedisa. [ Links ]

Gong, Y., Yang, F., Huang, L., & Su, S. (2009, October). Model-based approach to measuring quality of experience. In Emerging Network Intelligence, 2009 First International Conference on (pp. 29-32). IEEE. [ Links ]

González, I. B., Fernández, A. M., & Moreno, C. R. (2012, Enero). Guía de la Cooperación Española para la incorporación de las TIC en las intervenciones de Salud en la Cooperación para el Desarrollo. Ministerio de Asuntos Exteriores y de Cooperación. [ Links ]

ITU-T P.800 Rec. Series E. (2008). Exploración General de la Red, Servicio Telefónico, Explotación del Servicio y Factores Humanos - Calidad de los servicio en telecomunicación: conceptos, modelos, objetivos, planificación de la seguridad de funcionamiento - Términos y definiciones relativos a la calidad de los servicios de telecomunicación. Unión Internacional de Telecomunicaciones. [ Links ]

ITU-T P.1080 Rec. Series G. (2008). Transmission Systems and Media, Digital Systems and Networks - Multimedia quality of service and performance, Generic and user-related aspects. Unión Internacional de Telecomunicaciones. [ Links ]

De Moor, K., Joseph, W., Ketykó, I., Tanghe, E., Deryckere, T., Martens, L., & De Marez, L. (2010). Linking users' subjective QoE evaluation to signal strength in an IEEE 802.11 b/g wireless LAN environment. EURASIP Journal on Wireless Communications and Networking, 2010, 6. [ Links ]

Kilkki, K. (2008). Quality of experience in communications ecosystem. J. UCS, 14(5), 615-624. [ Links ]

Kwong, C. K., & Wong, T. C. (2008, December). A neuro-fuzzy approach to generating customer satisfaction model for new product development. In Industrial Engineering and Engineering Management, 2008. IEEM 2008. IEEE International Conference on (pp. 1804-1808). IEEE. [ Links ]

Laghari, K. U. R., & Connelly, K. (2012). Toward total quality of experience: A QoE model in a communication ecosystem. IEEE Communications Magazine, 50(4). [ Links ]

Le Callet, P., Möller, S., & Perkis, A. (2012). Qualinet white paper on definitions of quality of experience. European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003), 3. [ Links ]

Malindi, P. (2011). QoS in telemedicine. INTECH Open Access Publisher. [ Links ]

Moller, S., Engelbrecht, K. P., Kuhnel, C., Wechsung, I., & Weiss, B. (2009, July). A taxonomy of quality of service and quality of experience of multimodal human-machine interaction. In Quality of Multimedia Experience, 2009. [ Links ]

QoMEx 2009. International Workshop on (pp. 7-12). IEEE. [ Links ]

Möller, S., & Raake, A. (2014). Quality of experience. New York, US: Springer. [ Links ]

Nanda, P., & Fernandes, R. C. (2007, January). Quality of service in telemedicine. In Digital Society, 2007. ICDS'07. First International Conference on the (pp. 2-2). IEEE. [ Links ]

Pan American Health Organization. (2014, October). Strategy for Universal Access to Health and Universal Health Coverage. 66th Session of The Regional Committee of WHO for the Americas. [ Links ]

Perkis, A., Munkeby, S., & Hillestad, O. I. (2006, June). A model for measuring quality of experience. In Signal Processing Symposium, 2006. NORSIG 2006. Proceedings of the 7th Nordic (pp. 198-201). IEEE. [ Links ]

Rojas-Mendizabal, V., Serrano-Santoyo, A., Castillo-Olea, C., Gomez-Gonzalez, A., & Conte-Galvan, R. (2014). E-saúde e complexidade: uma proposta para o desenho de políticas públicas. Jornal Brasileiro de TeleSSaúde, 3(2), 33-44. [ Links ]

Ullah, M., Fiedler, M., & Wac, K. (2012, March). On the ambiguity of Quality of Service and Quality of Experience requirements for eHealth services. In medical information and communication technology (ISMICT), 2012 6th international symposium on (pp. 1-4). IEEE. [ Links ]

Venkatesh, V., & Zhang, X. (2010). Unified theory of acceptance and use of technology: US vs. China. Journal of Global Information Technology Management, 13(1), 5-27. [ Links ]

How to cite: Rojas-Mendizabal, V., Serrano-Santoyo, A., Conte-Galvan, R., Villarreal-Reyes, S., Rivera-Rodriguez, R. (2017). Estimation of quality of experience (QoE) in e-Health ecosystems .Ingeniería e Investigación, 37(2), 52-59. DOI: 10.15446/ing.investig.v37n2.57664

Received: May 25, 2016; Accepted: May 03, 2017

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License