SciELO - Scientific Electronic Library Online

 
vol.82 número193Minería de datos sin estructura para el soporte de la adquisición de requerimientos mediante el uso de vocabularios controlados: Un estudio de mapeo sistemáticoEstándares HMI/SCADA para el diseño de interfaces en los centros de datos: El centro de control y operaciones como caso de estudio índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google

Compartir


DYNA

versión impresa ISSN 0012-7353

Dyna rev.fac.nac.minas vol.82 no.193 Medellín sep./oct. 2015

https://doi.org/10.15446/dyna.v82n193.53493 

DOI: http://dx.doi.org/10.15446/dyna.v82n193.53493

Development of a concept-based EMG-based speller

Desarrollo de un deletreador basado en conceptos basados en EMG

 

Robertas Damaševičius, Mindaugas Vasiljevas & Tomas Šumskas

 

Software Engineering Department, Kaunas University of Technology, Kaunas, Lithuania, robertas.damasevicius@ktu.lt, mindaugas.vasiljevas@ktu.lt, tomas.sumskas@ktu.lt

 

Received: February 18th, 2015. Received in revised form: March 16th, 2015. Accepted: September 26th, 2015

 

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.


Abstract
Physiological computing is a paradigm of computing that treats users' physiological data as input during computing tasks in an Ambient Assisted Living (AAL) environment. By monitoring, analyzing and responding to such inputs, Physiological Computing Systems (PCS) are able to respond to the users' cognitive, emotional and physical states. A specific case of PCS is Neural Computer Interface (NCI), which uses electrical signals governing users' muscular activity (EMG data) to establish a direct communication pathway between the user and a computer. We present taxonomy of speller application parameters, propose a model of PCS, and describe the development of the EMG-based speller as a benchmark application. We analyze and develop an EMG-based speller application with a traditional letter-based as well as visual concept-based interface. Finally, we evaluate the performance and usability of the developed speller using empirical (accuracy, information transfer speed, input speed) metrics.

Keywords: Physiologic computing, NCI, EMG, speller, user interface.

Resumen
La computación fisiológica es un paradigma de la computación que usa los datos de los usuarios como entradas durante las tareas computacionales en un Ambiente de vidacotidianasoportado po rcomputadores (AAL). Monitoreando, analizando y respondiendo a dichas entradas, los Sistemas de Computación Fisiológica pueden responder al estado cognitivo, emocional y físico de los usuarios. Un caso particular es el de la interface de Computación Neuronal (NCI), que usa señales eléctricas para manejar la actividad muscular del usuario establecioendo una comunicación directa entre el usuario y el computador. Se present una taxonomía de parametros de aplicación de deletreo, proponiendo un modelo de PCS y describiendo el desarrollo de un deletreador basado en EMG. Se analiza y desarrolla unaaplicación con un sistema basado en letras tradicionales y una interfaz visual. Finalmente, se evalua el desempeño y usabilidad del sistemadesarrollado.

Keywords: Computaciónfisiológica, NCI, EMG, deletreador, interfaz de usuario.


 

1. Introduction

Today's computer systems are failing to satisfy the increasing expectations of everyday users. While computer systems output features multimedia communication channels, computer input is still fairly limited to mechanical (keyboard, mouse), audio (speech) and tactile inputs [1]. Such limitations raise barriers for people with major or minor disabilities such as elderly people with motor impairments. Considering the predicted demographic changes in society and the need to improve the quality of daily life for humans during different periods of their life, new concepts and methods of human-computer interaction (such as Ambient Assisted Living (AAL) [2] systems) must be researched. These efficiently address the accessibility problems in human interaction with software applications and services while meeting the individual requirements of the users in general, including disabled and elderly people.

Physiological computing is a paradigm of computing that treats user's' physiological data as input during computing tasks [3]. By monitoring, analyzing and responding to such inputs, physiological computing systems (PCS) are able to monitor and respond to users' cognitive, emotional and physical states in real time. Examples of such physiological data are Electroencephalography (EEG), Electrooculography (EOG), Electromyography (EMG), etc. A user's state is captured using sensors attached to the body, and they could be used to offer assistance if the user is frustrated or unable to perform the task due to excessive mental workload, to adapt the level of challenge in order to sustain or increase engagement, or to incorporate an emotional display element into the user interface [4].

Specific examples of PCS are the neural computer interface (NCI) and the brain computer interface (BCI). BCI manipulates users' brain activity (EEG) to establish a direct pathway between the brain and computer. The concept is particularly suited to the needs of the handicapped as well to smart environments. NCI is similar to BCI in methods it uses as well as its applications; however, it uses the surface EMG signals to establish an interface between human peripheral neural system and computers by recording electrical signals governing a subject's muscular movements.

A speller is a typical example of a PCS, which is a benchmark application for BCI and NCI methods [5]. The speller aims to help persons unable to activate muscles traditionally used in communication (for example in the hands and tongue) to spell words by utilizing their neural activity. Usually, spellers use signal amplitude information; however, integrating this with signal preprocessing methods such as noise reduction methods (e.g., shrinking functions [6], nonlinear filter operators [7]), and user intent prediction techniques can improve the results [8]. Due to the noise present in the physiological signals, the detection of control signals requires the use of efficient digital signal processing techniques [9-11]. Furthermore, the requirements of real-time systems prohibit the use of highly complex computations and demand that the result is delivered just in time for the user to be able to use it [12].

In this paper, we continue our previous work [13,14] and describe a PCS model for EMG-based applications, analyze the requirements for the development of interfaces for impaired users and visual interfaces of known speller applications, and describe the development of an EMG speller as a typical benchmark application.

 

2. Analysis of requirements for speller applications

The requirements for speller applications can be categorized as being on different levels depending upon the physical abilities of their users [15]: 1) Users with no physical disability, who may use NCI for entertainment or in other situations where physical movement is restricted. 2) Users with minor impairments (such as older persons). 3) Users with severe physical disabilities, who may wish to use NCI as a secondary input. 4) Users who have limited muscle control and may need to use NCI as a method for communication.

First, the speller must follow general requirements for smart systems to be integrated into the AAL environments. Next, the specific requirements for impaired users must be followed. Impaired users need assistance such as automatic learning of user's behavior to estimate his/her current needs. Humans often make mistakes in interacting with machines, so any human-operated system user interface should be designed such that errors are prevented whenever possible: the deactivation of invalid commands; making errors easy to detect and showing users what they have done; and easily allowing undoes, reverse, correct errors [16]. For smart systems, the following principles (also called "operational modes") of Humanistic Intelligence Framework [17] must be satisfied:

  1. Constancy: the interface should operate continuously to read signals from human to computer and to provide a constant user-interface.
  2. Augmentation: the primary task is increasing the intelligence of the system rather than computing tasks.
  3. Mediation: the interface mediates between human senses, emotions and perceptions and acts as an information filter by blocking or attenuating undesired input in order to decrease negative effects of interaction (such as fatigue, information overload, etc.) and to increase positive effects (such as user satisfaction) by amplifying or enhancing desired inputs.

According to Lopes [18], user interface for persons with disabilities must: support user variability, providing the means to adapt to user-specific requirements, support a wide range of input devices and output modes, provide minimal user interface design, promote interaction and retain user attention on the tasks, and establish strong feedback mechanisms that may provide reward schemes for correct behavior (results). The requirements for interfaces for impaired users can be formulated as follows [2]: 1) Limited access to details: complex and vital details of the system have to be hidden to avoid overwhelming and trapping users. 2) Self-learning: detected common patterns in the behavior of the user should be used to automatically create rules or shortcuts that speed and ease up the use of the system. 3) System interruption: Impaired users have, in most cases, no idea how the system works;, therefore, easy cancellation of system's activities must be ensured.

 

3. Overview of speller systems and interfaces

The research into developing and improving speller systems focuses on improving spelling accuracy, increasing speed of information transfer, developing usable and effective speller interfaces, and combining EEG/EMG based input with input automation techniques such as word complete and automatic correction of misspellings.

The speller implementations can be characterized by:

Type of data: EEG [19], EMG [20], ECoG [8], EOG [21].

Type of analyzed signal: P300 event-related potentials (ERPs), which are a series of peaks and troughs appearing in the EEG in response to the occurrence of a discrete event such as presentation of a stimulus or psychological reaction to a stimulus [8], Error-related Potentials (ErrPs) generated by the subject's perception of an error [22], steady-state visual evoked potential (SSVEP), which are signals that are natural responses to visual stimulation at the same (or multiples of) frequency as the visual stimulus [23].

Modality: Auditory: the rows and columns of the letter matrix are represented by different sounds such as spoken numbers [24] or environmental sounds. Visual: subjects direct their eye gaze toward the letter they want to select. There are two cases: overt attention when eye gaze is directed toward the target letter, and covert attention when eye gaze is directed at a central fixation point [25].

Interface:

Single character (or Linear) speller: all letters are shown and each letter is flashed individually until letter selection is done [26].

Matrix Speller: All letters are arranged in a matrix. First, speller flashes an entire column or row of characters. Then, single letters are flashed in a sequence, and can be selected [27]. Different matrix sizes can be used, e.g., a 6x6 matrix containing all 26 letters of the alphabet and 10 digits (0-9), or even a full QWERTY keyboard [23].

Lateral single-character is a single-character paradigm comprising of all letters in the alphabet and it follows an event strategy that significantly reduces the time for symbol selection [28].

Chekerboard Speller [29]: the 8x9 matrix is virtually superimposed on a checkerboard, which the participants never actually see. The items in white cells of the 8 x 9 matrix are segregated into a white 6 x 6 matrix and the items in the black cells are segregated into a black 6 x 6 matrix. The items in the first matrix) randomly populate the white or black matrices, and the users see random groups of six items flashing (as opposed to rows and columns in Matrix Speller). Such layout controls for adjacency-distraction errors, as the adjacent items cannot be included in the same flash group.

Hex-o-Spell: speller consists of six circles that all have the same distance to the point of fixation. The circles are flashed while users direct their attention to one of the circles. First, the circle with the desired group of letters is selected. Second, letters are redistributed over the circles and the target letter is selected [30].

Frequency-based layout accounts for the relative frequency of character occurrence in a language [31]. It has a virtual keyboard with 32 symbols surrounded by five boxes flickering at different frequencies. These boxes correspond to commands to navigate the cursor, and to select the intended character. The application starts with the cursor in the central position corresponding to the most frequent character in English (i.e., "E"). Letters with the higher frequency of occurrence are positioned closer to the center while the less frequent ones are further away. The user can navigate the cursor to the desired letter and confirm his/her choice with the "Select" command. The further the character is located from the center, the more command selections (cursor movements) are required.

Stimulus type: the way each individual character changes (e.g., flashing, color change, etc.). Rapid serial visual presentation (RSVP) is a method of displaying information (generally text or images) in which the text is displayed word-by-word in a fixed focal position [32].

Stimulus rate: the speed at which individual characters change.

Stimulus pattern: grouping of symbols in the interface (e.g., QUERTY or DVORAK layouts on a virtual keyboard).

Character set (alphabet): includes all letters of the alphabet as well as some additional symbols (numbers, separation marks, etc.).

Intelligence techniques: additional techniques for improving the system's accuracy and rate of communication such as using the language model [33], word autocomplete, spelling correction or word prediction.

The result of the analysis can be considered as a taxonomy of speller application parameters, which could be used for developing new speller applications. Next, we discuss the PCS model and its application in developing the EMG-based systems.

 

4. A physiological computing system model

The physiological computing system (PCS) model proposed in this paper is based on the "operational modes" of the Humanistic Intelligence Framework [17]:

  1. Constancy: PCS should operate continuously to read signals from humans and provide a constant user interface.
  2. Augmentation: the primary task is increasing the intelligence of the system rather than computing tasks.
  3. Mediation: PCS mediates between human senses, emotions and perceptions and acts as an information filter by blocking or attenuating undesired inputs to decrease negative effects of interaction (fatigue, information overload) as well as to increase positive effects (e.g., user satisfaction) by amplifying or enhancing desired inputs.

The proposed model is summarized in Fig. 1 and has three levels of information processing.

  1. On the lowest level, the physiological signal is sampled into a data stream of physiological data. Downsampling can be used to decrease the amount of data and increase information processing speed at higher levels.
  2. On the intermediate level, data is aggregated and events corresponding to specific patterns of data are generated. Machine learning techniques such as artificial neural networks may be used to recognize such events and generate decisions.
  3. On the highest level, decisions are processed and used to generate control commands for external systems.

 

5. Development of EMG speller

5.1. Components and architecture

A NCI system is generally comprised of the following components: (1) a device that records muscular activity signals; (2) a signal preprocessor that reduces noise and artifacts; (3) a decoder that classifies the de-noised signal into control commands for (4) an external device or application (e.g., a robotic actuator, a computer program etc.), which provides feedback to the user [34].

Our speller application has three layers: 1) on the lowest layer, the physiological signal is sampled into a data stream of physiological data. Downsampling can be used to decrease the amount of data and increase information processing speed at higher levels. 2) On the intermediate layer, data is aggregated and events corresponding to specific patterns of data are generated. Machine learning techniques such as artificial neural networks may be used to recognize such events and generate decisions. 3) On the highest layer, decisions are processed and used to generate control commands for external applications (systems).

The speller's architecture consists of 6 main components: MainReader - the system module responsible for control of data reader, which is selected to use. ReaderAPI - public external interface module. All third-party modules must implement this component for full system integration. MainController- system module responsible for selected control module (executes commands). NiaReader- third-party module implemented for the "OCZ NIA" data reader device. SpellingSquare- third-party module implemented for text input in the symbol matrix using EMG-based commands. The dashed rectangle separates system components (inside the dashed rectangle) from external components, which are either the sensor controllers (EMG readers) or actuator controllers (software or hardware applications such as robots). The speller system is based on JavaNetBeans framework. The speller was developed with its future extension and maintenance in consideration so that external components are easy to add or remove.

5.2. Control

The speller has two types of control commands: "Select" command - selects a column or types a symbol of that column. "Cancel" command - exits the selected column or deletes the selected symbol. These control commands can be initiated by the movements of facial muscles. In practice, blinks of the eye are used to generate each control command (left eye blink for "select" and right eye blink for "cancel"). The user can see the EMG signal feedback in EMG signal view area (see Fig. 2).

The particular control command is performed when the amplitude of the EMG signal is higher than the specified threshold value. The thresholds are marked as yellow horizontal lines in the EMG signal view area. The upper threshold indicates the "select" command, and the lower threshold indicates the "cancel" command. Threshold values can be adjusted using threshold setting sliders. The signal view of EMG, when spelling the word "hello", is presented in Fig. 2. In Fig. 2 (left), the word "hello" is spelled without mistakes. In Fig. 2 (right), the spelling contains a few mistakes. In order to correct these mistakes cancellation commands must be performed. The spikes indicate the "select" command. One trial (selection of one character) contains two positive signal spikes, the first spike is for column selection, the second for letter selection in the corresponding column.

5.3. Traditional speller interface

The developed EMG speller has two different user interfaces. One is a traditional matrix-based speller interface that is presented in Fig. 3a. The other one is a novel concept-based interface that is presented in Fig. 3b. The most important part of the visual interface is symbol matrix. The matrix is adaptable so that various symbols (including special or national) could be added into the matrix. The red-colored column indicates the current position of the speller cursor. The cursor moves coherently form column to column until the user activates the "select" command. Next, the cursor moves onto each symbol in the specific column. After another "select" command, the particular symbol is selected. That symbol appears in the output area (see Fig. 3). The speller cursor moves by step, which varies from 500 to 1500 milliseconds. The step value depends on the number of mistakes the user makes. A smaller number of mistakes mean the cursor movement will move at a faster speed. The mistake is considered to be the "cancel" command.

The first row of the speller symbol matrix contains a dictionary selection. This selection allows the user to enter the dictionary. When a few symbols or word stem is written, the dictionary provides the user with the opportunity to complete the particular word faster. The system logs dictionary selections, thus commonly used words are at the top of the dictionary and the dictionary adapts to the user.

5.4. Visual concept-based speller interface

We have also implemented a completely different EMG speller interface. Traditional spellers use common letters in the alphabet rearranged in different layouts. We have implemented a visual concept based interface that is based on graphical symbols (graphemes) of a visual communication language. Visual language is a form of communication that uses visual elements as opposed to a formal written (textual) language to convey meaning or an idea. Visual language uses pictograms or ideograms to symbolize the concepts that are to be communicated. Pictograms are pictures that resemble what they signify, and represent a concept, object, activity, place or event by graphical illustration. An ideogram is a graphical symbol that represents an idea, rather than a group of letters arranged according to the phonemes and grammar of a spoken language, as is the case in textual languages.

Visual symbols form part of our daily lives through their use in medication, transport, computers, etc., because they indicate in a concise and easily understandable form places, directions, actions or action constraints in either the real world or virtual space. Thus, visual symbols can be used in a number of situations in which textual messages are not possible or adequate due to context or user based constraints. The development of software for people with impaired cognitive or motor capabilities requires the developers to take into account the high-level cognitive processes and mental models of the solution to a domain-specific problem (such as expressing basing needs of an impaired user), which may contribute to the ease of use of the developed application by its potential users.

While textual (letter-based) languages are good for expressing all kinds of human communication, they require a great deal of typing effort for human-computer communication, which for specific groups of users such as users with impaired motor capabilities may be a tiresome burden. A visual concept-based language allows high-level concepts to be expressed succinctly using a notation tailored to a set of specific user problems. Such languages could be tailored towards a specific domain, and could be based only on the relevant concepts and features of that domain. Therefore, visual languages can be considered as a medium of communication that allows for the gap to be bridged between the mental models of users and the domain systems and, consequently, to cut the distance (both in terms of effort required and speed of communication) between communicating parties in AAL applications.

A snapshot of the visual concept-based interface that was developed for the EMG speller is shown in Fig. 3b. The interface is organized using a hierarchical structure. It consists of symbol matrixes connected with each other by references. Each reference is represented as an icon of a particular domain. Currently, we have included visual symbols from 8 main concept domains: emotion domain, location domain, action domain, time domain, object domain, body part domain, person domain and special symbol domain. Most of the visual icons we use are adopted from The Noun Project (http://thenounproject.com/), while the remaining ones are custom-built. Each domain matrix as well as the root matrix can be extended easily by adding new icons (concepts) to the particular domain matrixes. Also each icon (concept) can contain a reference to the specific subdomain matrix. The taxonomical tree of concept matrixes is summarized in Fig. 4.

The meaning of visual concepts is presented in Table 1.

 

6. Experimental results

6.1. Experiment with text-based interface

The experiments we performed with 5 subjects (3 males), aged 24-54 (mean = 33) years. Subjects did not have any neurological abnormalities, reported normal or corrected to normal vision, and did not use medication. All subjects gave informed consent prior to the experiment.

The EMG data was recorded using OCZ Neuro Impulse Actuator equipment. Visual stimuli were presented on a 13.3'' size TFT LCD screen with 1360 × 768 pixel resolution and a refresh rate of 60 Hz. Subjects were seated in front of a table. The screen was in the middle of the table at a distance of approximately 100 cm from the subject. The size of each character was 1.5 × 1.5 cm (0.86 × 0.86° visual angle) and the entire speller matrix was 9.5 × 13 cm (5.44 × 7.42° visual angle).

Stimuli consisted of intensifications of the rows and columns in sequential order. Intensification was achieved by increasing the size of all characters in the row or column with a factor 500 for 1500 ms.

A trial is defined here as spelling of one character. All trials started with the speller being displayed on the screen, together with an instruction indicating which letter to select. Each stimulation sequence was followed by feedback on the screen, showing which letter or group of letters had been selected.

Three text paragraphs were given to the experiment participants. Their task was to input the proposed text paragraphs using speller. All text paragraphs were presented in the Lithuanian language. The first text paragraph contained 126 characters and its content was that of a daily conversation. The second text paragraph contained 111 characters and its content was that of a scientific speech. The third text paragraph contained 120 characters and was that of a scientific speech with mathematical equations. Each experiment participant repeated the experiment 4 times. The average accuracy, input speed and bit rate values were calculated.

Quantitatively, the performance of speller application can be evaluated using accuracy, information transfer speed and input speed metrics. Accuracy is calculated as the percentage of correct decisions during the symbol selection process. Bit rate (or information transfer rate) indicates how much information can be communicated per time unit (calculated using the Wolpaw's formula [35]). Finally, input speed is measured as the average time required to enter a set of benchmark texts.

The experimental results are presented in Table 2 and in Figs. 5, 6 & 7. Our results are comparable with the results achieved by other authors.

Accuracy values of the BCI/NCI spellers achieved by other authors (Table 2) are within a range of 80-95% (82.77% using ECoG [8], 87.58% using SSVEP-based BCI [24], 87.8% for EOG-based speller [21], 91.80% [36], 94.8% for RVSP based speller [32]).

The information transfer rate (aka bit rate) of the BCI/NCI-based speller applications achieved by other authors is within 7-41 bits/min (7.43 bits/min [37], 17.13 bits/min [38], 19.18 bits/min [36], 11.58-37.57 bits/min [39], 40.72 using SSVEP based BCI [23], 41.02 using ECoG[8]).

The symbol input speed of the BCI/NCI-based speller applications achieved by other authors is within 1-12 CPM (1.43 CPM for RSVP based speller [32], 4.33 CPM [36], 4.91 CPM [39], 9.39 CPM using SSVEP based BCI [23], 12.75 CPM [40]).

6.2. Experiment with visual concept based interface

The experiment with the visual concept-based interface was performed under the same conditions as the experiment with text-based interface. The participants had to enter one paragraph of text (196 symbols) that was based on daily conversation topics. This paragraph contained simple formulations of basic user needs (see an example of message in Fig. 8). The experiment was performed with 2 subjects (both male, aged between 24-28 years). The duration of the experiment was measured, and the input speed is presented in Fig. 9.

Two metrics of input speed are presented in Fig. 10: concept input speed (number of valid concepts entered in a time unit) and input speed of letters in a textual language, which require to be entered in a time unit in order to convey the same message. The results show that using the visual concept-based interface can increase input speed to 26.03 sym/min (equivalent to 4.31 concepts/min) in comparison to 11.35 sym/min (see Table 2) using the EMG speller's traditional text-based interface.

 

7. Evaluation and conclusion

We have described the development of the EMG speller application for an Assisted Living Environment based on a three-layered model of a Physiological Computing system based on the "operational modes" of Mann's Humanistic Intelligence Framework. This system is controlled by voluntary muscular movements, particularly the orbicular ones (i.e., eye blinking), which are translated into text input commands.

The developed speller application is adaptive (input speed can be adapted dynamically in response to the user's state) and intelligence (it uses word complete and word frequency features). The application can aid people with reduced mobility by increasing their autonomy in their home environment.

Two types of interfaces were developed and evaluated: the traditional letter matrix-based interface and the novel visual concept-based interface. The speller's letter-matrix interface has been evaluated using accuracy, bit rate and input speed metrics. The empirical results obtained are within range of results also obtained by other authors. The visual concept-based interface was evaluated using concept input speed and compared with equivalent text input speed. The results show an improvement in input speed by a factor of 2.3 in comparison to the best results achieved using a letter matrix based interface with a dictionary and adaptable input speed, which could be explained by the conciseness of the visual language (both in terms of the size of dictionary and the length of visual "words") as well as by a reduced user effort required to communicate a message.

The advantages of using a visual language rather than a textual (alphabet-based) one are as follows: visual representation is easier to understand and communicate, common visual signs (such as computer icons) are universally known and understood, there is no need to implement multiple languages as the signs of a visual language are understandable for all users, there are fewer grammar constraints, there is immediate visual feedback, a smaller effort due to economy of concepts is required for communication, input speed is increased as the user has to select only one visual symbol (icon) instead of having to enter many the different letters in a word.

The disadvantages of the visual language are: ambiguity in interpreting sentences consisting of visual symbols and limited expression capabilities.

We plan to address these problems in future the by extending the dictionary of visual symbols as well as by integrating an automatic translation plugin for conversion between the entered sequence of visual symbols and its textual counterpart.

 

Acknowledgements

The work described in this paper has been carried out within the framework the Operational Programme for the Lithuanian Development of Human Resources 2007-2013 in the "Strengthening the capacities of researchers and scientists" project VP1-3.1-ŠMM-08-K-01-018 and the "Research and development of Internet technologies and their infrastructure for smart environments of things and services" (2012-2015) project, which were funded by European Social Fund (ESF). The authors also would like to acknowledge the contribution of the COST Action IC1303 - Architectures, Algorithms and Platforms for Enhanced Living Environments (AAPELE).

 

References

[1] Schroeder, A. and Wirsing, M. Developing physiological computing systems: Challenges and Solutions. Proc. of Software Engineering Conference, LNI 198, GI 2012, pp. 21-36, 2012.         [ Links ]

[2] Marinc, A., Stocklöw, C., Braun, A., Limberger, C., Hofmann, C. and Kuijper, A., Interactive personalization of ambient assisted living environments, In: Human interface and the Management of Information - vol. 6771, LNCS Springer-Verlag, Berlin, Heidelberg, pp. 567-576, 2011. DOI: 10.1007/978-3-642-21793-7_64        [ Links ]

[3] Allanson, J. and Fairclough, S.H., A research agenda for physio-logical computing. Interacting with Computers, 16 (5), pp. 857-878, 2004. DOI: 10.1016/j.intcom.2004.08.001        [ Links ]

[4] Fairclough, S.F., Fundamentals of physiological computing. Interacting with Computers, 21 (1-2), pp. 133-145, 2009. DOI: 10.1016/j.intcom.2008.10.011        [ Links ]

[5] Cecotti, H., Spelling with non-invasive Brain-Computer Interfaces - Current and future trends. J. of Physiology-Paris, 105 (1-3), pp. 106-114, 2011. DOI: 10.1016/j.jphysparis.2011.08.003        [ Links ]

[6] Martisius, I. and Damasevicius, R., Class-adaptive denoising for EEG data classification, In: Proc. of 11th Int. Conference on Artificial Intelligence and Soft Computing, ICAISC - vol. 7268, LNCS, Springer, Zakopane, Poland. pp. 302-309, 2012. DOI: 10.1007/978-3-642-29350-4_36        [ Links ]

[7] Martisius, I., Damasevicius, R., Jusas, V. and Birvinskas D., Using higher order nonlinear operators for SVM classification of EEG data. Electronics and Electrical Engineering, 3 (119), pp. 99-102, 2012. DOI: 10.5755/j01.eee.119.3.1373        [ Links ]

[8] Speier, W., Fried, I. and Pouratian, N., Improved P300 speller performance using electrocorticography, spectral features, and natural language processing. Clin.Neuroph. 124 (7), pp.1321-1328, 2013. DOI: 10.1016/j.clinph.2013.02.002        [ Links ]

[9] Birvinskas, D., Jusas, V., Martisius, I. and Damasevicius R., Data compression of EEG signals for artificial neural network classification. Information Technology and Control, 42 (3), pp. 238-241, 2013. DOI: 10.5755/j01.itc.42.3.1986        [ Links ]

[10] Martisius, I., Birvinskas, D., Damasevicius, R., and Jusas V., EEG Dataset reduction and classification using wave atom transform, Proc. of 23rd International Conference on Artificial Neural Networks (ICANN2013), Sofia, Bulgaria, pp. 208-215, 2013. DOI: 10.1007/978-3-642-40728-4_26        [ Links ]

[11] Birvinskas, D., Jusas, V., Martisius, I. and Damasevicius R., EEG Dataset reduction and feature extraction using discrete cosine transform, Proc. of UK 6th European Modelling Symposium on Mathematical Modelling and Computer Simulation (EMS2012), pp. 199-204, 2012. DOI: 10.1109/ems.2012.88        [ Links ]

[12] Martišius, I., Šidlauskas, K. and Damaševičius, R., Real-time training of voted perceptron for classification of EEG data. International Journal of Artificial Intelligence (IJAI), 10 (S13), pp. 41-50, 2013.         [ Links ]

[13] Vasiljevas, M., Turčinas, R.and Damaševičius R., EMG Speller with adaptive stimulus rate and dictionary support, Proc. of FeDCSIS'2014: Federated Conference on Computer Science and Information Systems, Warsaw, Poland, pp. 233-240, 2014. DOI: 10.15439/2014f338        [ Links ]

[14] Vasiljevas, M., Turcinas, R. and Damasevicius, R., Development of EMG-based speller, Proc. of INTERRACCION 2014: XV International Conference on Human Computer Interaction, 2014. DOI: 10.1145/2662253.2662260        [ Links ]

[15] Quek, M., Höhne, J., Murray-Smith, R. and Tangermann, M., Designing future BCIs: Beyond the bit rate, in: Allison, B., Dunne, S., Leeb, R., Millan, J.D.R. and Nijholt, A. (Eds.), Towards Practical Brain-Computer Interfaces: Bridging the Gap from Research to Real-world Applications, Springer, pp. 173-196, 2013.         [ Links ]

[16] Johnson, J., Designing with the mind in mind: A simple guide to understanding user interface design rules. Morgan Kaufmann, Burlington, 2011.         [ Links ]

[17] Mann, S., Wearable computing: Toward humanistic intelligence. IEEE Intelligent Systems, 16 (3), pp. 10-15, 2001. DOI: 10.1109/5254.940020        [ Links ]

[18] Lopes, J.B., Designing user interfaces for severely handicapped persons, Proc. of the 2001 EC/NSF Workshop on Universal accessibility of ubiquitous computing: providing for the elderly (WUAUC'01), ACM, New York, NY, USA, pp. 100-106, 2001. DOI: 10.1145/564526.564553        [ Links ]

[19] Tomioka, R. and Müller, K.-R., A regularized discriminative framework for EEG analysis with application to brain-computer interface. NeuroImage, 49 (1), pp. 415-432, 2010. DOI: 10.1016/j.neuroimage.2009.07.045        [ Links ]

[20] Lalitharatne, T.D., Teramoto, K., Hayashi, Y. and Kiguchi, K., Towards hybrid EEG-EMG-based control approaches to be used in bio-robotics applications: Current status, challenges and future directions. Journal of Behavioral Robotics, 4 (2), pp. 147-154, 2013. DOI: 10.2478/pjbr-2013-0009        [ Links ]

[21] Liu, Y., Zhou, Z., and Hu, D., Gaze independent brain-computer speller with covert visual search tasks. Clinical Neurophysiology, 122 (6), pp. 1127-36, 2011. DOI: 10.1016/j.clinph.2010.10.049        [ Links ]

[22] Combaz, A., Chumerin, N., Manyakov, N.V., Robben, A., Suykens, J.A.K. and Van Hulle, M.M., Towards the detection of error-related potentials and its integration in the context of a P300 speller brain-computer interface. Neurocomputing, 80, pp. 73-82, 2012. DOI: 10.1016/j.neucom.2011.09.013        [ Links ]

[23] Hwang, H.-J., Lim, J.-H., Jung, Y.-J., Choi, H., Lee, S.W. and Im, C.H. Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard. Journal of Neuroscience Methods, 208 (1), pp. 59-65, 2012. DOI: 10.1016/j.jneumeth.2012.04.011        [ Links ]

[24] Furdea, A., Halder, S., Krusienski, D.J., Bross, D., Nijboer, F., Birbaumer,N., Kübler, A., An auditory oddball (P300) spelling system for brain-computer interfaces. Psychophysiology, 46, pp. 617-25, 2009. DOI: 10.1111/j.1469-8986.2008.00783.x        [ Links ]

[25] Brunner, P. and Schalk, G., Toward a gaze-independent matrix speller brain-computer interface. Clinical Neurophysiology, 122 (6), pp. 1063-1064, 2011. DOI: 10.1016/j.clinph.2010.11.014        [ Links ]

[26] Ortner, R., Prueckl, R., Putz, V., Scharinger, J., Bruckner, M., Schnuerer, A. and Guger, C., Accuracy of a P300 speller for different conditions: A comparison. Proc. of the 5th Int. Brain-Computer Interface Conference, Graz, Austria, 2011, 196 P.         [ Links ]

[27] Farwell, L.A. and Donchin, E., Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and clinical neurophysiology, 70 (6), pp. 510-523, 1988. DOI: 10.1016/0013-4694(88)90149-6        [ Links ]

[28] Akram, F., Han, H.-S., Jeon, H.J., Park, K., Park, S-H., Cho, J. and Kim, T.-S., An efficient words typing P300-BCI system using a modified T9 interface and random forest classifier, Proc. of 35th Ann. Int. Conf. of the IEEE EMBS, pp. 2251-2254, 2013. DOI: 10.1109/embc.2013.6609985        [ Links ]

[29] Townsend, G., LaPallo, B.K., Boulay, C.B., Krusienski, D.J., Frye, G.E., Hauser, C.K., Schwartz, N.E., Vaughan, T.M., Wolpaw, J. R. and Sellers, E.W., A novel p300-based brain-computer interface stimulus presentation paradigm: moving beyond rows and columns.Clinical Neurophysiology, 121 (7), pp. 1109-1120, 2010. DOI: 10.1016/j.clinph.2010.01.030        [ Links ]

[30] Treder, M. and Blankertz, B., Covert attention and visual speller design in an ERP-based brain-computer interface. Behavioral and Brain Functions, 6, pp. 1-13, 2010. DOI: 10.1186/1744-9081-6-28        [ Links ]

[31] Volosyak, I., Cecotti, H., Valbuena, D. and Gräser A., Evaluation of the Bremen SSVEP based BCI in real world conditions, Proc. of the 11th Int. Conference on Rehabilitation Robotics, Kyoto, Japan, pp. 322-331, 2009. DOI: 10.1109/icorr.2009.5209543        [ Links ]

[32] Acqualagna, L. and Blankertz, B., Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP). Clinical Neurophysiology, 124 (5), pp. 901-908, 2013. DOI: 10.1016/j.clinph.2012.12.050        [ Links ]

[33] Ulas, C. and Cetin, M., Incorporation of a language model into a brain computer interface based speller through HMMs, Proc. of Int. Conf. on Acoustics, Speech and Signal Processing, pp. 1138-1142, 2013. DOI: 10.1109/icassp.2013.6637828        [ Links ]

[34] Mora-Cortes, A., Manyakov, N.V., Chumerin,N. and Van Hulle, M.M., Language model applications to spelling with brain-computer interfaces. Sensors 14, pp. 5967-5993, 2014. DOI: 10.3390/s140405967        [ Links ]

[35] Wolpaw, J.R., Birbaumer, N., Heetderks, W.J., McFaralnd, D.J., Peckham, P.H., Schalk, G., Donchin, E., Quatrano, L.A., Robinson, C.J. and Vaughan T.M.,Brain-computer interface technology: A review of the first international meeting. IEEE Transactions on Rehabilitation Engineering, 8, pp. 164-173, 2000. DOI: 10.1109/TRE.2000.847807        [ Links ]

[36] Pires, G., Nunes, U. and Castelo-Branco, M., Comparison of a row-column speller vs. a novel lateral single-character speller: Assessment of BCI for severe motor disabled patients. Clinical Neurophysiology, 123 (6), pp. 1168-1181, 2012. DOI: 10.1016/j.clinph.2011.10.040        [ Links ]

[37] Käthner, I., Ruf, C.A., Pasqualotto, E., Braun, C., Birbaumer, N. and Halder, S., A portable auditory P300 brain-computer interface with directional cues. Clinical Neurophysiology, 124 (2), pp. 327-338, 2013. DOI: 10.1016/j.clinph.2012.08.006        [ Links ]

[38] Shahriari, Y. and Erfanian, A., Improving the performance of P300-based brain-computer interface through subspace-based filtering. Neurocomputing, 121, pp. 434-441, 2013. DOI: 10.1016/j.neucom.2013.05.001        [ Links ]

[39] Vilic, A., Kjaer, T.W., Thomsen, C.E., Puthusserypady, S. and Sorensen, H.B., DTU BCI Speller: An SSVEP-based spelling system with dictionary support, 35th Annual Int. Conf. of the IEEE EMBS, pp. 2212-2215, 2013.         [ Links ]

[40] Wang, P.T., King, C.E., Do, A.H. and Nenadic, Z., Pushing the communication speed limit of a noninvasive bci speller, CoRR abs/1212.0469, 2012.         [ Links ]

 

R. Damaševičius, is graduated from the Faculty of Informatics, Kaunas University of Technology (KTU) in Kaunas, Lithuania in 1999, where he received a BSc. degree in Informatics. He finished his MSc. studies in 2001 (cum laude), and completed his PhD. Thesis at KTU in 2005. Currently, he is a Professor in the Software Engineering Department, KTU and lectures in bioinformatics and robot programming courses. His research interests include brain-computer interfaces, bioinformatics, data mining and machine learning. He is the author or co-author of over 120 papers as well as of a monograph published by Springer.

M. Vasiljevas, is graduated from the Faculty of Informatics, KTU in Kaunas, Lithuania in 2011. He received a BSc. in Informatics Engineering. He finished his MSc. at the same university in 2013. Currently he is a PhD. student at Kaunas University of Technology and works as an assistant at the same university. His PhD. research is based on physiological computing, neural computer interface and machine learning.

T. Šumskas, is graduated from KTU in 2013. He received his BSc. degree in Informatics. Currently, he is finishing his Masters studies in Software Engineering. His Masters project is related to human-computer interaction, and text autocorrection using machine learning algorithms.