SciELO - Scientific Electronic Library Online

 
vol.26 issue46Alinear programming model for the parallel non-related machines problem, in the drying area of a chilean sawmillCausal analysis procedure focused on small software development organizations author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Article

Indicators

Related links

  • On index processCited by Google
  • Have no similar articlesSimilars in SciELO
  • On index processSimilars in Google

Share


Revista Facultad de Ingeniería

Print version ISSN 0121-1129

Rev. Fac. ing. vol.26 no.46 Tunja Sep./Dec. 2017

http://dx.doi.org/10.19053/01211129.v26.n46.2017.7310 

Article

Emotion recognition techniques using physiological signals and video games -Systematic review-

Técnicas de reconocimiento emocional utilizando señales fisiológicas y videojuegos -revisión sistemática-

Técnicas de reconhecimento emocional utilizando sinais fisiológicos e videogames -revisão sistemática-

Mauro Callejas-Cuervo1  , Laura Alejandra Martínez-Tejada2  , Andrea Catherine Alarcón-Aldana3 

1 Ph. D. Universidad Pedagógica y Tecnológica en Colombia (Tunja-Boyacá, Colombia). mauro.callejas@uptc.edu.co. ORCID: 0000-00019894-8737.

2 M. Sc. Universidad Santo Tomás (Tunja-Boyacá, Colombia). laura.martinez@usantoto.edu.co. ORCID: 0000-0003-2373-6312.

3M. Sc. Universidad Pedagógica y Tecnológica en Colombia (Tunja-Boyacá, Colombia). andrea.alarcon@uptc.edu.co. ORCID: 0000-00021983-7375. Revista Facultad de Ingeniería (Rev. Fac. Ing.) Vol. 26 (46), pp. 19-28. Septiembre-Diciembre, 2017. Tunja-Boyacá, Colombia. ISSN Impreso 0121-1129, ISSN Online 2357-5328, DOI: http://doi.org/10.19053/01211129.v26.n46.2017.7310

Abstract

Emotion recognition systems from physiological signals are innovative techniques that allow studying the behavior and reaction of an individual when exposed to information that may evoke emotional reactions through multimedia tools, for example, video games. This type of approach is used to identify the behavior of an individual in different fields, such as medicine, education, psychology, etc., in order to assess the effect that the content has on the individual that is interacting with it. This article shows a systematic review of articles that report studies on emotion recognition with physiological signals and video games, between January 2010 and April 2016. We searched in eight databases, and found 15 articles that met the selection criteria. With this systematic review, we found that the use of video games as emotion stimulation tools has become an innovative field of study, due to their potential to involve stories and multimedia tools that can interact directly with the person in fields like rehabilitation. We detected clear examples where video games and physiological signal measurement became an important approach in rehabilitation processes, for example, in Posttraumatic Stress Disorder (PTSD) treatments.

Key words: Biofeedbac; Emotion recognition; Physiological signals; Video games

Resumen

Los sistemas de reconocimiento emocional a partir de señales fisiológicas son técnicas innovadoras que permiten estudiar el comportamiento y la reacción de un individuo cuando está expuesto a información que puede evocar reacciones emotivas por medio de herramientas multimedia como, por ejemplo, los videojuegos. Este tipo de enfoque se utiliza para identificar el comportamiento de un individuo en diferentes campos, entre los que están la medicina, la educación y la psicología, entre otros, para evaluar el efecto que tienen los contenidos con los que interactúa. La revisión sistemática que aquí se expone, de artículos que reportan trabajos de reconocimiento emocional con señales fisiológicas y videojuegos entre enero de 2010 y abril de 2016, se realizó con 15 artículos, hallados en ocho bases de datos, los cuales cumplían con los criterios de selección de la búsqueda. Esta revisión sistemática reveló que el uso de videojuegos como herramientas de estimulación emocional se ha convertido en un campo de estudio innovador, debido a su potencial para involucrar historias y herramientas multimedia que pueden interactuar directamente con la persona en campos como la rehabilitación. Se detectaron ejemplos claros donde los videojuegos y la medición de la señal fisiológica se convierten en un enfoque importante para el proceso de rehabilitación, por ejemplo, en los tratamientos de trastorno por estrés postraumático (TEPT).

Palabras-clave: Biofeedback; Reconocimiento de emociones; Señales fisiológicas; Videojuegos

Resumo

Os sistemas de reconhecimento emocional a partir de sinais fisiológicos são técnicas inovadoras que permitem estudar o comportamento e a reação de um indivíduo quando está exposto a informação que pode evocar reações emotivas por meio de ferramentas multimídia como, por exemplo, os videogames. Este tipo de enfoque utiliza-se para identificar o comportamento de um indivíduo em diferentes campos, entre os quais estão a medicina, a educação e a psicologia, entre outros, para avaliar o efeito que têm os conteúdos com os que interatuam. A revisão sistemática que aqui se expõe, de artigos que reportam trabalhos de reconhecimento emocional com sinais fisiológicos e videogames entre janeiro de 2010 e abril de 2016, realizou-se com 15 artigos, achados em oito bases de dados, os quais cumpriam com os critérios de seleção da busca. Esta revisão sistemática revelou que o uso de videogames como ferramentas de estimulação emocional tem se convertido em um campo de estudo inovador, devido a seu potencial para envolver histórias e ferramentas multimídia que podem interagir diretamente com a pessoa em campos como a reabilitação. Detectaram-se exemplos claros onde os videogames e a medição do sinal fisiológico convertem-se em um enfoque importante para o processo de reabilitação, por exemplo, nos tratamentos de transtorno por estresse pós-traumático (TEPT).

Palavras-Chave: Biofeedback; Reconhecimento de emoções; Sinais fisiológicos; Videogames

I. Introduction

The analysis and study of human emotions is a complex field of research that can be approached from different points of view depending on the mechanisms employed for the stimulation, measurement, and analysis of emotional reactions in the human body. Currently, video games are among the tools with the greatest potential to stimulate the emotions of an individual; their strength lies in immersing the player in a virtual world, where they can experience different emotional reactions with no need to move large distances, engaging their attention, concentration, and time, in order to achieve the goals established by the immersive story of the game. Video games use stories, challenges, objectives, characters, scenarios, music, animation, and other audio-visual multimedia tools to capture the players' attention, and generate emotions that allow the individual to feel identified with the game, and thus, promoting a more outstanding game experience for those who would like to take up the challenge and use their time for this kind of activities. Along with books, music, images, and movies, videogames constitute a group of audio-visual tools that generate emotional reactions in individuals, according to the level of empathy they generate in the person. However, in contrast with the previous examples, video games represent a greater source of information on reaction, given that they are essentially interactive 1.

Apart from being one of the largest industries of technological development nowadays, video games have become a great source of research in applications oriented toward entertainment, as well as in applications known as "Serious Games," which try to solve problems in fields such as medicine [2the new Medicine degrees of the European Higher Education Area (EHEA, 3], the military 4, education 5,6, business 7, politics, citizenship [8], etc. Throughout the last decade, the convenience and importance of using video games or "Serious Games" have been demonstrated by analyzing the response of the players and potentiating some of their skills, simulating different virtual environments, and objectives that involve the direct participation of the people with the digital world. This study was carried out starting from the premise that a videogame can generate empathy in the player, producing different emotional reactions during gameplay, and that these reactions can be captured, measured, and analyzed through emotion recognition techniques and affective computing 9, such as face recognition 10, speech recognition 11, and physiological signals 12,13. The objective was to identify evidence and current research that integrate techniques of emotion recognition, involving physiological signals, such as electrocardiography (ECG), Galvanic skin response (GSR), and electromyography (EMG), with the use of video games for stimulating emotions; for this, we conducted a systematic review between 2010 and 2016, in which we identified a growth in the number of articles focused on video games and emotion recognition systems, demonstrating the work that is developed in the field.

II. Materials and methods

A. Selection criteria

The articles were selected from eight different bibliographic and academic databases related to engineering and medicine: ScienceDirect, IEEEXplorer, SCOPUS, EngineeringVillage, ISI Web of knowledge, ACM, EMBASE, and PubMed. The articles corresponded to studies focused on emotion recognition from the measurement of physiological signals (ECG, GSR, or EMG), and the use of video games as tools for emotion stimulation, published between 2010 and 2016, due to the growth of these approaches employing these techniques.

B. Search strategy

The search and selection of articles that deal with developments in the mentioned field was based on two searches using different terms that refer to the field of study.

In the first search, the selected works contained in their title, abstract, or key words, the terms "emotion recognition" and "physiological signals", and were published between January 2010 and April 2016. The search was carried out in the databases previously mentioned and, as a result, the following number of papers was obtained: ScienceDirect (14), IEEEXplorer (162), SCOPUS (144), EngineeringVillage (125), ISI Web of knowledge (291), ACM (7), EMBASE (16), and PubMed (13). Given that we were looking for studies that used video games as emotion stimulation tools, the previously search was complemented including the word "games" in any section of the article. The new search was conducted in the same eight academic databases and produced the following results: ScienceDirect (4), IEEEXplorer (7), SCOPUS (19), EngineeringVillage (8), ISI Web of knowledge (7), ACM (4), EMBASE (0), and PubMed (0), obtaining 49 articles in total.

A second search was carried out to complement the results obtained from the first search. This time, the objective was to identify works published between January 2010 and April 2016 that contained in their title, abstract, or key words the term "affective gaming." This term refers to video games that interact with the player through commands or actions using specialized tools, and respond or modify according to the emotions experienced by the player 14. The consulted databases produced the following results: ScienceDirect (121), IEEEXplorer (196), SCOPUS (27), EngineeringVillage (27), ISI Web of knowledge (16), ACM (451), EMBASE (1), and PubMed (15). Then, along with the previous search, the term "physiological signals" was included to be found in any section of the article; this search produced the following results: ScienceDirect (23), IEEEXplorer (14), SCOPUS (10), EngineeringVillage (5), ISI Web of knowledge (3), ACM (7), EMBASE (1), and PubMed (1), obtaining 64 articles in total.

C. Selection study

In total, 113 articles were found in the consulted databases (49 in the first search and 64 in the second search) containing the stated selection criteria. After excluding the duplicated papers, the number of articles was reduced to 74. Out of them, 46 were available in full. Finally, the accepted articles were classified according to the type of physiological signals measured, and the tool used to stimulate the individuals' emotions. 15 articles reported to have measured physiological signals such as ECG, EMG, or GSR, along with video games as a stimulation tool for emotion recognition, and described the results of this process.

D. Data analysis

The search was conducted to determine state-of-the-art techniques of emotion recognition based on physiological signs and video games. Of the 46 articles, 50 % (23) used video games as emotion stimulation tools, 10.87 % (5) used videos or movies as emotion stimulation tools, 4.35 % (2) used official images from the International Affective Picture System (IAPS), and 4.35 % (2) used some kind of mechanical device to stimulate emotions (vibrations 15 yet it is largely unmeasured, modeled or characterized. A better understanding of user perception will aid the design of tactile behavior that engages touch, with an experience that satisfies rather than intrudes. We measured 30 subjects\u2019 affec- tive response to vibrations varying in rhythm and frequency, then examined how differences in demographic, everyday use of touch, and tactile processing abilities contribute to variations in affective response. To this end, we developed five affective and sensory rat- ing scales and two tactile performance tasks, and also employed a published \ u2018Need for Touch\u2019 (NFT, LEGOS 16). The remaining 30.43 % (14) corresponded to studies in the field of emotion simulation 17-20, the development and adaptation of tools for measuring emotions 21, literature revisions and editorials 6,14,22-24, and the interaction of the individual with their environment and other individuals 25-282009. ACII 2009: IEEE.

III. Results

Out of the 46 publications found in this work, 14 reported developments using physiological signals different from ECG, GSR, and EMG (Table 1). Of the other 32, seven articles reported having worked with emotion stimulation tools other than video games (Table 2). Finally, of the last 25, 10 corresponded to articles on developments in emotion simulation, revision of literature regarding machine learning algorithm techniques and serious games, and development of tools such as toolkits and editorials for technical events (Table 3).

Table 1 Articles excluded for not using physiological signal measurements, such as ecg, gsr or emg  

Measurement signal Number of articles Articles
EEG 4 [29-32]
Magnetic Resonance Images of the brain 2 [27, 33]
Interface/ Questionnaire 3 [15, 25, 34]
Behavior 3 [26, 28, 35]
Body posture/ Facial -speech recognition 2 [36, 37]

Table 2 Article excluded for not using video games as an emotion stimulation tool 

Stimulation tool Number of articles Articles
Movies/videos 5 [38-42]
Images 1 [43]
LEGO 1 [16]

Table 3 Articles excluded for not presenting the results of the measurements or the research in the delimited area 

Main topic Number of articles Articles
Systematic reviews and studies (Machine learning, Serious games) 4 [6, 14, 23, 24]
Development of computational tools 1 [21]
Editorials 1 [22]
Digital simulation of emotions 4 [17]-[20]

The 15 articles selected in the study are shown in Table 4, which briefly summarizes the most important aspects taken from each study. We can identify that these studies, mostly, carried out multimodal emotion recognition, in which they integrated signals from different sensors to measure the physiological responses of the individual interacting with the video game, and thus, identified the emotion that was being experienced during the game. GSR, along with heart rate (HR) from ECG or blood volume pulse (BVP), are the most used signals to identify emotions, due to the high efficiency that can be achieved in their detection and measurement, as well as the ease of access to the measurement equipment. We can also observe that 17.4 % of the referenced articles involved the measurement of signals through electroencephalograms (EEG) as the main measured signal, or as a complement, given that the information extracted from the behavior of the brain waves is relevant for detecting emotions. Including sensors to measure EMG signals allow to identify movements in the corrugator and zygomaticus major muscles, which are in charge of controlling the movement of the eyebrows and cheeks in the face, and therefore are directly related with the recognition of the facial expression.

It is important to mention that a large amount of the literature reported in academic databases does not clearly identify the key words or the indexed terms; therefore, the search protocol and the subsequent meta-analysis did not include all the studies that report innovative and important developments in the field of study.

Table 4 Articles selected for the systematic review 

Article Physiological signals Video game type Measured emotions Analysis technique Population
[44] EDA (SCL, SCR) Racing Arousal levels Open and closed loops control 20 students (between 18-33 years)
[45] Respiration, SC, EMG shoulder, TEMP, BVP Tetris 12 discrete positive and negative emotions Questionnaire and Visual analysis-correlation 32 subjects (11 female and 21 males) ages between 20-61
[46] EEG, GSR, BVP, Respiration belt, temp Tetris arousal and pleasure levels Questionnaire and ANOVA analysis 20 participants (mean ages 27; 13 males; all right handed)
[47] EMG to Eye-blink Fire emergency simulator VR Stress Manual analysis 36 participants (21 M, 15 F ages between 16 and 35
[48] GSR and blood volume pulse (BVP) Super Mario 64 (NINTENDO64) Stress ANOVA analysis 14 males and 4 females university students aged 19 to 31
[49] Heart rate variability (HRV) and low skin conductance level (SCL) Medical videogame: 'Virtual Remote Dome Controller' Positive and Negative Affect Scale (PANAS) Linear correlation analyses 28 male volunteers aged between 19 and 27 (mean age of 22)
[50] Skin conductance StartleMart Stress Event-based stress 13 veterans suffering from posttraumatic stress disorder
[51] EEG, ECG and keyboard-mouse activity Horror game Slender: The Eight Pages (STEP) Fear Machine learning algorithms and questionnaire 11 participants (6 Males, 5 Females) aged between 21 and 32 years old
[52] EDA (SCL, SCR), HR, HVR Three emotionally laden VR scenarios with Oculus rift Russells model: Pleasure, Arousal, and Dominance(PAD) Matlab and SPSS, oneway ANOVA, paired t-test 6 high functioning children and adolescents with ASD between the ages from 9 to 14
[53] EDA, BVP, Facial EMG Levels in which virtual character carrying out a task inside a realistic VE Stress EDA-only algorithm Multi-sensor algorithm 35 participants (26 Males, 9 Females)
[54] SC, HR and facial EMG Horror game Russell's AV dimensional theory of emotions Triangulation process between events and arousal space. Simple local maxima/ minima (LMM) detection algorithm 6 volunteers
[55] ECG, facial EMG, HR, SBP, and DBP MadWorld Wii (Nintendo wii) Stress, boredom, and relaxing Questionnaire and correlation signals Female (n= 62) and male (n=74), (M age = 20.00, SD = 2.25)
[56] SCL Ultimatum Game Sensitivity to offers ANOVA analysis Two groups: 12 students (5 females) The mean ages were 19.1 (SD = 0.49) years for the psychopathic group, 16 students (8 females). and 19.1 (SD = 1.27) years for the low psychopathic group
[57] EEG, Eye tracking, facial expressions, skin conductivity, Posture, finger pressure Guitar Hero engagement/boredom, frustration Agents 21 students
[58] EEG and ECG Survival horror anxiety, suspense, and fear Affect annotation tool (AAT) 12 participants (11 males and 1 female) aged between 22 and 36 years old (mean = 25.42, SD =3.75)

IV. Conclusions

From the systematic revision, we can conclude that video games have become an innovative approach as emotion stimulation tools, due to their potential to involve stories and multimedia tools that can interact directly with the person. However, video games still lack a classification that allows them to be linked with the emotions they may evoke. When using video games in studies of affective recognition, their selection is carried out according to the situation that will be stimulated, in order to obtain the same reaction from the player to the virtual environment and, as we suppose, to a real one. The use of commercial video games, paid or free (the most popular ones are riddles, jigsaw puzzles, simulators, and survival/horror, among others), is a common denominator among the studies referenced in this article. However, a selection methodology that allows for an initial classification of different video games that might be used in the same study is not reported in the different works, nor which game is more suitable to the requirements of the research being carried out.

Signals like GSR and ECG are widely used in this type of studies for detecting and measuring parameters related to emotions, such as HR, SCL and SCR, due to their high efficiency showing the physiological change connected to the emotions experienced by the individual. The EMG signals are used to recognize facial expressions and their changes; however, when considering the use of virtual reality tools, such as immersive glasses, this type of measurement is compromised due to the necessity of putting sensors on the face. For this reason, it is necessary to identify areas of the anatomy where it is possible to put sensors to measure the muscular activity when some kind of emotion is experienced, mainly stress, fear, and anxiety.

Finally, the field of affective video games is growing, with different possible action fronts; from the inclusion of avatars, which can demonstrate emotion through biofeedback, to the inclusion of emotion detection from players to improve the video game development process, as well as other fields of social, educational, and health. To obtain an accurate and robust technological system for the health and treatment field using videogames, it is necessary to develop new hardware and software tools that allow to measure and analyze signals in real time, without interfering with the normal behavior of the individual when is interacting with the video game. For example, biofeedback is a novel tool developed in the field of game design, with the potential to not only affect the players' game experience, but also create new branches in applied fields such the treatment of mental disorders. This kind of works are still in an experimental stage, focusing on the implementation of tools and protocols, validating not only the sensors and the acquisition system, but also, the process to acquire the physiological data to eventually apply them in the rehabilitation field.

References

[1] S. Rigby, and R. Ryan, Glued to games : how video games draw us in and hold us spellbound. PREAGER, 2011. [ Links ]

[2] O. Colteli, X. Grandi, R. Tosca, P. Latorre, J. S. Sanchez, L. V. Lizan, F. Ros-Bernal, and C. Martinez-Cadenas, "Designing serious games for learning support in medicine studies: A specific method to elicit and formalize requirements," in 2014 IEEE Front. Educ. Conf Proc. ., pp. 1-4, 2014. DOI: http://doi.org/10.1109/FIE.2014.7044156. Links ]

[3] N. Duarte, O. Postolache, and J. Sharcanski, "KSGphysio - Kinect Serious Game for Physiotherapy," in Int. Conf. Expo. Electr. Power Eng, pp. 16-18, 2014. DOI: http://doi.org/10.1109/ICEPE.2014.6969981. Links ]

[4] J. Lai, W. Tang, and Y. He, "Team tactics in military serious game," in 2011 4th Int. Symp. Comput. Intell. Des. Isc. 2011, vol. 1, pp. 75-78, 2011. DOI: http://doi.org/10.1109/ISCID.2011.28. [ Links ]

[5] T. Mitamura, Y. Suzuki, and T. Oohori, "Serious games for learning programming languages," in Conf. Proc.- IEEE Int. Conf. Syst. Man Cybern ., pp. 1812-1817, 2012. DOI: http://doi.org/10.1109/ICSMC.2012.6378001. [ Links ]

[6] P. Wilkinson, "Affective educational games: Utilizing emotions in game-based learning," in 5th Int. Conf. Games Virtual Worlds Serious Appl. VS-GAMES 2013, 2013. DOI: http://doi.org/10.1109/VS-GAMES.2013.6624219. [ Links ]

[7] J. C. K. H. Riedel, and J. B. Hauge, "State of the art of serious games for business and industry," in 2011 17th Int. Conf. Concurr. Enterprising, pp. 1-8, 2011. [ Links ]

[8] A. M. Ahmed, Q. H. Mehdi, R. Moreton, and A. Elmaghraby, "Serious Games Providing Opportunities to Empower Citizen Engagement and Participation in E-government Services," in The 20th International Conference on Computer Games - CGAMES, 2015, pp. 138-142. DOI: http://doi.org/10.1109/CGames.2015.7272971. Links ]

[9] R. D. Ward, and P. H. Marsden, "Affective computing: problems, reactions and intentions," Interact. Comput., vol. 16 (4), pp. 707-713, Aug. 2004. DOI: http://doi.org/10.1016/j.intcom.2004.06.002.Links ]

[10] C. Soladié, H. Salam, C. Pelachaud, N. Stoiber, and R. Séguier, "A Multimodal Fuzzy Inference System Using a Continuous Facial Expression Representation for Emotion Detection," in 14th ACM International Conference on Multimodal Interaction - ICMI, 2012. DOI: http://doi.org/10.1145/2388676.2388782.Links ]

[11] O.-A. Schipor, S.-G. Pentiuc, and M.-D. Schipor, "Towards a multimodal emotion recognition framework to be integrated in a Computer Based Speech Therapy System," in 2011 6th Conf. Speech Technol. Human-Computer Dialogue, pp. 1-6, May 2011. DOI: http://doi.org/10.1109/SPED.2011.5940727. [ Links ]

[12] J. Kortelainen, S. Tiinanen, X. Huang, X. Li, S. Laukka, M. Pietikäinen, and T. Seppänen, "Multimodal emotion recognition by combining physiological signals and facial expressions: A preliminary study," in 34th Annual International Conference of the IEEE EMBS, 2012, pp. 5238-5241. DOI: http://doi.org/10.1109/EMBC.2012.6347175.Links ]

[13] K. Takahashi, S. Namikawa, and M. Hashimoto, "Multimodal Physiological Signals : Elicited using Japanese Kanji Words," in 35th International Conference on Telecommunications and Signal Processing - TSP, 2012, pp. 615-620. [ Links ]

[14] I. Kotsia, S. Zafeiriou, and S. Fotopoulos, "Affective gaming: A comprehensive survey," in IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., pp. 663-670, 2013. DOI: http://doi.org/10.1109/CVPRW.2013.100. [ Links ]

[15] H. Seifi, and K. E. Maclean, "A first look at individuals' affective ratings of vibrations," in World Haptics Conf. WHC 2013, pp. 605-610, Apr. 2013. DOI: http://doi.org/10.1109/WHC.2013.6548477. [ Links ]

[16] P. Mitkidis, J. J. McGraw, A. Roepstorff, and S. Wallot, "Building trust: Heart rate synchrony and arousal during joint action increased by public goods game," Physiol. Behav., vol. 149, pp. 101-6, Oct. 2015. DOI: http://doi.org/10.1016/j.physbeh.2015.05.033.Links ]

[17] M. Spraragen, and A. M. Madni, "Modeling of emotional effects on decision-making by game agents," Procedia Comput. Sci. , vol. 28, pp. 736-743, 2014. DOI: http://doi.org/10.1016/j.procs.2014.03.088.Links ]

[18] Z. Cai, B. Goertzel, C. Zhou, D. Huang, S. Ke, G.Yu, and M. Jiang, "OpenPsi: A novel computational affective model and its application in video games,"Eng. Appl. Artif. Intell ., vol. 26 (1), pp. 1-12, Jan. 2013. DOI: http://doi.org/10.1016/j.engappai.2012.07.013.Links ]

[19] L. J. Chang, and A. Smith, "Social emotions and psychological games," Curr. Opin. Behav. Sci .,vol. 5, pp. 133-140, Oct. 2015. DOI: http://doi.org/10.1016/j.cobeha.2015.09.010. Links ]

[20] I. Leite, A. Pereira, S. Mascarenhas, C. Martinho, R. Prada, and A. Paiva, "The influence of empathy in human-robot relations," Int. J. Hum. Comput. Stud ., vol. 71 (3), pp. 250-260, Mar. 2013. DOI: http://doi.org/10.1016/j.ijhcs.2012.09.005. Links ]

[21] S. Rank, and C. Lu, "PhysSigTK : Enabling Engagement Experiments with Physiological Signals for Game Design," in 2015 Int. Conf. Affect. Comput. Intell. Interact., pp. 968-969, 2015. DOI: http://doi.org/10.1109/ACII.2015.7344692. Links ]

[22] V. Argyriou, I. Kotsia, S. Zafeiriou, and M. Petrou, "Guest editorial introduction to the special issue on modern control for computer games," IEEE Trans. Cybern ., vol. 43 (6), pp. 1516-1518, Dec. 2013. DOI: http://doi.org/10.1109/TCYB.2013.2283551.Links ]

[23] N. Suttie, S. Louchart, T. Lim, and J. Ritchie, "Towards a biocybernetic approach for serious games real-time psychophysiological inferences for adaptive agents in serious games," Procedia Comput. Sci ., vol. 15, pp. 316-317, 2012. DOI: http://doi.org/10.1016/j.procs.2012.10.092. Links ]

[24] Y. Bengio, and N. Y. Georgios, "Learning Deep Physiological Models of Affect," no. April, pp. 2033, 2013. [ Links ]

[25] J. Broekens, and W. P. Brinkman, "AffectButton: A method for reliable and valid affective self-report," Int. J. Hum. Comput. Stud ., vol. 71 (6), pp. 641-667, Jun. 2013. DOI: http://doi.org/10.1016/j.ijhcs.2013.02.003. Links ]

[26] L. W. Lam, K. W. Chan, D. Fong, and F. Lo, "Does the look matter? The impact of casino servicescape on gaming customer satisfaction, intention to revisit, and desire to stay," Int. J. Hosp. Manag., vol. 30 (3), pp. 558-567, Sep. 2011. DOI: http://doi.org/10.1016/j.ijhm.2010.10.003. Links ]

[27] S. Pichon, S. W. Rieger, and P. Vuilleumier, "Persistent affective biases in human amygdala response following implicit priming with negative emotion concepts," Neuroimage, vol. 62 (3), pp. 1610-1621, Sep. 2012. DOI: http://doi.org/10.1016/j.neuroimage.2012.06.004. Links ]

[28] F. Van Winden, "Political economy with affect: On the role of emotions and relationships in political economics," Eur. J. Polit. Econ., vol. 40, pp. 298-311, Dec. 2015. DOI: http://doi.org/10.1016/j.ejpoleco.2015.05.005. Links ]

[29] H. Yoon, S.-W. Park, Y.-K. Lee, and J.-H. Jang, "Emotion recognition of serious game players using a simple brain computer interface," in 2013 Int. Conf. Inf. Commun. Technol. Converg. ICTC 2013, pp. 783-786, 2013. [ Links ]

[30] D. P. O. Bos, B. Reuderink, B. Van De Laar, H. Gürkök, C. Mühl, M. Poel, D. Heylen, and A. Nijholt, "Human-computer interaction for BCI games: Usability and user experience," in Proc. - 2010 Int. Conf. Cyberworlds, pp. 277-281, Oct. 2010. DOI: http://doi.org/10.1109/CW.2010.22.Links ]

[31] T. McMahan, I. Parberry, and T. D. Parsons, "Evaluating Player Task Engagement and Arousal Using Electroencephalography," Procedia Manuf., vol. 3, pp. 2303-2310, 2015. DOI: http://doi.org/10.1016/j.promfg.2015.07.376. Links ]

[32] R. Jenke, A. Peer, and M. Buss, "A Comparison of Evaluation Measures for Emotion Recognition in Dimensional Space," in 2013 Hum. Assoc. Conf. Affect. Comput. Intell. Interact., pp. 822-826, 2013. [ Links ]

[33] K. M. Harlé, and A. G. Sanfey, "Social economic decision-making across the lifespan: An fMRI investigation," Neuropsychologia, vol. 50 (7), pp. 1416-1424, Jun. 2012. DOI: http://doi.org/10.1016/j.neuropsychologia.2012.02.026. Links ]

[34] K. Muñoz, P. M. Kevitt, T. Lunney, J. Noguez, and L. Neri, "An emotional student model for gameplay adaptation," Entertain. Comput., vol. 2 (2), pp. 133-141, Jan. 2011. DOI: http://doi.org/10.1016/j.entcom.2010.12.006. Links ]

[35] R. S. J. D Baker, S. K. D'Mello, M. M. T. Rodrigo, and A. C. Graesser, "Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments," Int. J. Hum. Comput. Stud ., vol. 68 (4), pp. 223-241, Apr. 2010. DOI: http://doi.org/10.1016/j.ijhcs.2009.12.003. Links ]

[36] I. Kotsia, I. Patras, and S. Fotopoulos, "Affective gaming: Beyond using sensors," in 5th Int. Symp. Commun. Control Signal Process. ISCCSP 2012, pp. 2-4, 2012. DOI: http://doi.org/10.1109/ISCCSP.2012.6217768.Links ]

[37] K. P. Truong, D. A. Van Leeuwen, and F. M. G. De Jong , "Speech-based recognition of self-reported and observed emotion in a dimensional space," Speech Commun, vol. 54 (9), pp. 1049-1063, Nov. 2012. DOI: http://doi.org/10.1016/j.specom.2012.04.006. [ Links ]

[38] C. Li, C. Xu, and Z. Feng, "Analysis of physiological for emotion recognition with the IRS model," Neurocomputing, vol. 178, pp. 103-111, Feb. 2016. DOI: http://doi.org/10.1016/j.neucom.2015.07.112. [ Links ]

[39] C.-Y. Chang, C.-W. Chang, J.-Y. Zheng, andP.C. Chung, "Physiological emotion analysis using support vector regression," Neurocomputing , vol. 122, pp. 79-87, Dec. 2013. DOI: http://doi.org/10.1016/j.neucom.2013.02.041. Links ]

[40] M. Khezri, M. Firoozabadi, and A. R. Sharafat, "Reliable emotion recognition system based on dynamic adaptive fusion of forehead biopotentials and physiological signals," Comput. Methods Programs Biomed, vol. 122 (2), pp. 149-164, Nov. 2015. DOI: http://doi.org/10.1016/j.cmpb.2015.07.006.Links ]

[41] C. Y. Nie, R. Li, and J. Wang, "Emotion recognition based on multi-variant correlation of physiological signals," Appl. Mech. Mater ., vol. 380-384 (2), pp. 3750-3753, Aug. 2013. DOI: http://doi.org/10.4028/ www.scientific.net/AMM.380-384.3750.Links ]

[42] W. Huang, G. Liu, and W. Wen, "MAPD: A Multi-subject Affective Physiological Database," in Proc. - 2014 7th Int. Symp. Comput. Intell. Des. Isc ., vol. 2, pp. 585-589, 2015. [ Links ]

[43] C. Maaoui, F. Abdat, and A. Pruski, "Physio-visual data fusion for emotion recognition," Irbm, vol. 35 (3), pp. 109-118, Jun. 2014. DOI: http://doi.org/10.1016/j.irbm.2014.03.001. Links ]

[44] A. Parnandi, Y. Son, and R. Gutierrez-Osuna, "A control-theoretic approach to adaptive physiological games," in Proc. - 2013 Hum. Assoc. Conf. Affect. Comput. Intell. Interact. ACII, pp. 7-12, 2013. DOI: http://doi.org/10.1109/ACII.2013.8.Links ]

[45] A. Landowska, and M. R. Wrobel, "Affective reactions to playing digital games," in Proc. - 2015 8th Int. Conf. Hum. Syst. Interact. HSI, pp. 264-270, 2015. DOI: http://doi.org/10.1109/HSI.2015.7170678.Links ]

[46] G. Chanel, C. Rebetez, M. Bétrancourt, and T. Pun, "Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty," IEEE Trans. Syst. Cybern. A Syst. HUMANS, vol. 41 (6), pp.1052-1063, Nov. 2011. DOI: http://doi.org/10.1109/TSMCA.2011.2116000.Links ]

[47] L. Chittaro, and R. Sioni, "Exploring eye-blink startle response as a physiological measure for affective computing," in Proc. - 2013 Hum. Assoc. Conf. Affect. Comput. Intell. Interact. ACII, pp. 227-232, 2013. DOI: http://doi.org/10.1109/ACII.2013.44.Links ]

[48] S. Wu, and T. Lin, "Exploring the use of physiology in adaptive game design," in Int. Conf. Consum. Electron. Commun. Networks, CECNet 2011 - Proc., pp. 1280-1283, 2011. DOI: http://doi.org/10.1109/CECNET.2011.5768186. Links ]

[49] S. F. Raaijmakers, F. W. Steel, M. de Goede, N. C. Van Wouwe, J. B. F. Van Erp, and A.-M. Brouwer, "Heart Rate Variability and Skin Conductance Biofeedback: A Triple-Blind Randomized Controlled Study," in 2013 Hum. Assoc. Conf. Affect. Comput. Intell. Interact., pp. 289-293, 2013. [ Links ]

[50] C. Holmg, H. P. Martínez, G. N. Yannakakis, and K.-I. Karstoft, "To Rank or to Classify ? Annotating Stress for Reliable PTSD Profiling," in International Conference on Affective Computing and Intelligent Interaction (ACII), 2015, pp. 719-725. [ Links ]

[51] V. Vachiratamporn, R. Legaspi, K. Moriyama, and M. Numao, "Towards the design of affective survival horror games: An investigation on player affect," in Proc. - 2013 Hum. Assoc. Conf. Affect. Comput. Intell. Interact. ACII, pp. 576-581, 2013. DOI: http://doi.org/10.1109/ACII.2013.101. Links ]

[52] Y. Li, A. S. Elmaghraby, A. El-Baz, and E. M. Sokhadze, "Using physiological signal analysis to design affective VR games," in 2015 IEEE Int. Symp. Signal Process. Inf. Technol., pp. 57-62, 2015. DOI: http://doi.org/10.1109/ISSPIT.2015.7394401.Links ]

[53] L. Chittaro, and R. Sioni, "Affective computing vs. Affective placebo: Study of a biofeedback- controlled game for relaxation training," Int. J. Hum. Comput. Stud ., vol. 72 (8-9), pp. 663-673, Aug. 2014. DOI: http://doi.org/10.1016/j.ijhcs.2014.01.007.Links ]

[54] P. A. Nogueira, V. Torres, R. Rodrigues, and E. Oliveira, "An annotation tool for automatically triangulating individuals' psychophysiological emotional reactions to digital media stimuli," Entertain. Comput ., vol. 9-10, pp. 19-27, Jun. 2015. DOI: http://doi.org/10.1016/j.entcom.2015.06.003.Links ]

[55] G. L. Read, M. Ballard, L. J. Emery, and D. G. Bazzini, "Examining desensitization using facial electromyography:Violent videogames, gender, and affective responding," Comput. Human Behav., vol. 62, pp. 201-211, Sep. 2016. DOI: http://doi.org/10.1016/j.chb.2016.03.074. [ Links ]

[56] T. Osumi, and H. Ohira, "The positive side of psychopathy: Emotional detachment in psychopathy and rational decision-making in the ultimatum game," Pers. Individ. Dif., vol. 49 (5), pp. 451-456, Oct. 2010. DOI: http://doi.org/10.1016/j.paid.2010.04.016. Links ]

[57] J. Gonzalez-Sanchez, M. E. Chavez-Echeagaray, R. Atkinson, and W. Burleson, "ABE: An Agent-Based Software Architecture for a Multimodal Emotion Recognition Framework," 2011 Ninth Work. IEEE/IFIP Conf. Softw. Archit., pp. 187-193, Jun. 2011. DOI: http://doi.org/10.1109/WICSA.2011.32.Links ]

[58] V. Vachiratamporn, K. Moriyama, K. I. Fukui, and M. Numao, "An implementation of affective adaptation in survival horror games," IEEE Conf. Comput. Intell. Games, CIG, no. i, 2014. DOI: http://doi.org/10.1109/CIG.2014.6932893.Links ]

Authors' contributions Martínez-Tejada performed the search, the recompilation and the analysis of the papers referenced in this article, and contribute to write the manuscript. Callejas-Cuevo and Alarcón-Aldana contributed to write the manuscript and reviewed it. All authors read and approved the final manuscript.

Received: March 01, 2017; Accepted: July 20, 2017

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License