English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Introduction
Touch is the sense that obtains information from physical objects using haptic exploratory activities (Bremner and Spence, 2017). This sense has nerve receptors distributed in the skin, transferring different types of energy stimuli to nerve impulses.
Some examples are thermal energy detected by thermoreceptors or mechanical energy detected by mechanoreceptors. Mechanoreceptors detect skin stimuli such as vibration, stretch, pressure, and texture (Fox, 2014; French and Torkkeli, 2009). Vibration stimuli can be detected in a range from 8 to 400 Hz (Griffin, 2012). Skin also contains nociceptors, which generate a biological alert when they detect damaging or potentially damaging stimuli. Another receptor type is the proprioceptor, located in muscles and tendons, which indicates the location of the body and other objects. This kind of receptor is related to kinesthesia (Proske and Gandevia, 2012).
Analogously, haptic interfaces (HI) are elements or devices that stimulate receptors related to touch. HI are divided into tactile and kinesthetic interfaces to replicate essential tangible and subjection characteristics of real objects, respectively. Both types of stimuli are necessary for reproducing the sensation of contact and manipulation of an object inside a virtual or a remote environment (Golledge et al., 2006; O'Malley and Gupta, 2008).
Some studies show a trend that involves combining both interface types to complement their effects (Fontana et al., 2012; Hoshi et al., 2010; Pacchierotti et al., 2013; Yang, 2013; MIRALab, 2021), producing greater realism and immersion for the user inside the virtual environment. This immersive experience makes it an ideal technology to be applied in digital entertainment systems (DES).
DES are digital devices that allow people to enjoy different forms of entertainment, such as music, cinema, radio, games, and gambling, converging on digital media and platforms. DES usually involve electronic devices such as smartphones, DVD recorders and players, MP3 players, game consoles (Sadiku et al., 2017), computers, and others. One part of digital entertainment refers to playing digital interactive games on different kinds of electronic media (Tan and Jansz, 2008).
Currently, some DES use virtual reality (VR). In these types of systems, mainly visual and auditory cues help create immersion, and the use of tactile displays contributes to the manipulation of virtual objects (Pacchierotti et al., 2017; Villa-Salazar et al., 2020). The closer the system stimuli are to the perception of the human senses, the higher the immersion potential. This means matching multimodal information channels to the user's head and body (Rosa et al., 2016).
Hardware devices used with DES and virtual reality such as joysticks or cellphones typically use vibration cues to generate alerts or indicate actions, as is the case of gunshots or explosions. Furthermore, some developments use vibration to produce touch sensations. The interaction consists of feeling the presence of an object by means of a vibration stimulus (Cyberglove Systems, 2019; Kim et al., 2017). Usually, haptic stimulations do not consider the surface texture in a DES.
However, it is possible to perceive the surface texture through vibration stimuli based on when a finger touches and explores a texturized surface. This action produces vibrations in the skin that are transduced by mechanoreceptors (Bensmaïa and Hollins, 2003).
This paper describes the development of a vibrotactile device called HaptTech, which is designed to generate tactile stimuli for textures in virtual objects. The device is coupled with Novint Falcon, a kinesthetic interface. This system complements the user's kinesthetic perception in order to increase the perceived degree of realism regarding the interaction with a virtual object by reproducing the feel of its surface texture on the user's hand.
The proof of concept is carried out with the addition of a vibrotactile texture reproducer mounted on the fingertip to a force-kinesthetic commercial interface coupled to the palm. This development aims to improve the realism of touch sensation when a user interacts with a virtual object in a DES, where the user can differentiate between surface textures by touch.
This paper is organized as follows: the Prototype description section describes the design criteria and the development of the vibrotactile reproducer, the Novint Falcon interface and its coupling with the vibrotactile reproducer, and the virtual environment utilized; the Methodology section describes the test applied to the subjects; the Results section contains the obtained results for each experiment conducted, as well as their interpretation; the Discussion section delves into the contribution of this work; and the Conclusions section presents our final remarks, along with proposals for future work.
Prototype description
This section describes the design criteria, the development of the vibrotactile reproducer, the Novint Falcon interface and its coupling with the vibrotactile reproducer, and the virtual environment utilized.
Design criteria for a texture reproducer vibrotactile system
The design parameters of the texture reproducer are as follows. Vibration is the stimulus to be used since it is innocuous and, at the adequate frequency, it stimulates the Pacinian corpuscles, which transmit the sensation of texture. The vibration actuator has a reduced size, low power consumption, and operates within the skin perception range.
The actuator case should be made from a lightweight material, inoffensive to contact with the skin. The control and driver circuit must be of reduced size. The expected result is a compact and portable device with the appropriate resistance to support the movements of the participant's hand during testing.
Considering these parameters, the outcome is a robust, compact, and portable device, with an appropriate resistance to support the weight and movements of the subject's limbs undergoing the tests.
Development of the texture reproducer vibrotactile system
As per these requirements, Figure 1 illustrates the proposed hardware design of the vibrotactile texture reproducer system. It utilizes a microcontroller system (CPU). The device receives the commands from a computer via USB input and is activated when the surface of an object is touched. The CPU sends the appropriate signal pattern to the driver device, delivering the current and voltage levels to the actuator in order to generate the appropriate vibrotactile stimuli.
The actuator is a low-consumption vibration motor (Linear Resonant Actuator, LRA), with an operation frequency of 175 ± 1 Hz, oscillating at frequencies close to this value like a voice coil speaker, and it is controlled by a low-voltage haptic driver (DRV2605L) for LRAs. The vibrotactile stimuli are homogeneously transferred from the LRA to the user's finger, using a thimble made from a hypoallergenic resistant polymer. Figure 2 shows the selected devices in the vibrotactile fingertip actuator.
To create the three levels of roughness in the texture using vibration, the total amplitude range of the actuator output was divided by three. To this effect, the DRV driver was used in open-loop LRA drive mode through internally generated PWM, programming the driver to generate an electrical signal in order to stimulate the actuator in the appropriate proportion in each case. For instance, a 33% driver activation induces 33% of the maximum possible amplitude, which is related to a low roughness sensation. In the same way, a 66% activation corresponds to a medium level of roughness and 100% to a high level of roughness. A preliminary study verified that said percentage levels simulate the perception of roughness (Santís et al., 2016).
Description of the Novint Falcon interface
Figure 3 shows the kinesthetic interface Novint Falcon®. It is composed of a grip, an arm, a body, and a base. The grip translates the force feedback to the user's hand, and it contains the control buttons. This device communicates with a personal computer (PC) via a USB port.
Coupling with the Novint Falcon interface
The grip was adjusted to monitor the force feedback on the palm. The grip connects with a piece embedded in a glove, which holds the thimble (Figure 4).
Developing the virtual environment
A software application synchronizes the interaction between the Novint Falcon and the HaptTech tactile interface in order to create an increased sensation of immersion in the virtual environment on a PC screen. The application simulates interaction with a virtual cube through the control movements of the Novint Falcon. When the cube is virtually manipulated, force and tactile stimuli are transmitted to the user's hand.
The application used the C+ + language and other computer graphic tools. The OpenGL library was useful for rendering the objects, as well as other complementary tools such as FreeGLUT for Windows (Freeglut, 2019), and GLEW for light distribution or shading on the objects (Glew, 2019).
Methodology
The selected design was quasi-experimental because it provided high-level evidence without randomization (Bärnighausen et al., 2017).
Three experiments were performed on the aforementioned device, evaluating the texture reproducer vibrotactile system and its coupling with the Novint Falcon interface.
The institutional Ethics Committee approved the study. The participants signed an informed consent document before testing and later received instructions on how to execute the experiment, as well as on the appropriate hand position for a correct evaluation of the interface. The researchers read scripted instructions in order to guarantee that each participant received the same information to perform the tests.
The exclusion criteria included a review of hand skin conditions that affect sensitivity (e.g., scars, burns, traumas), diabetes, neurological or immunological diseases, any surgery or neurological condition that affects hand sensitivity, the exposure to toxic substances, and the prolonged use of medication or stimulants.
Experiment 1. Discrimination of vibrotactile stimuli
A group of participants aged 18 to 60 assessed three textures simulated by the vibrotactile HaptTech system. The sample size of 60 subjects was obtained using the GPower 3.1 software, with Power=80, a basis of Effect=0,4, Measurement=3, and Confidence=95%. The sample was acquired using non-probabilistic sampling, maintaining heterogeneity in 77 participants who participated in the experiments, before excluding two of them. To meet the inclusion and exclusion criteria, a physician evaluated the skin condition and neurological response of the upper limbs at the beginning and end of the test.
The test participants were not associated with the project to avoid bias in the sample selection and result detection, and the order of applied stimuli was random (Álvarez and Álvarez, 2009; Glantz, 2006).
To evaluate the tactile interface's capability to reproduce differentiable stimuli on the subjects, three stimuli were classified as low-, medium-, and high-intensity. The 75 participants underwent the vibrotactile response perception experiment, indicating, on a scale from 1 to 10, the degree to which they perceived a difference between the stimuli, where 1 meant that there was no perceivable difference and 10 that they were completely different. There were three experiments: the first experiment compared textures 1 and 2, the second compared textures 2 and 3, and the third compared textures 1 and 3. Researchers asked each participant to make exploratory circles and zigzag movements on the two surfaces for comparison in each experiment. Afterwards, the participants were asked to rate the degree of difference between the textures.
Experiment 2. Comparison between kinesthetic and vibrotactile plus kinesthetic stimuli
For experiment 2, 31 people aged 18 to 60 were suitable to participate. One person who did not meet the inclusion criteria was excluded. A small sample size (less than 30) would skew the variables, but, as the sample size increases, the variables tend to take on a normal distribution (Delice, 2010). At the beginning and at the end of the experiment, a physician evaluated the same criteria mentioned in experiment 1.
In this experiment, the users wore the glove containing the HaptTech system coupled with the Novint Falcon kinesthetic interface. The subjects manipulated the coupled HaptTech-Novint interface's grip while interacting with a small sphere and cube in a virtual environment (Figure 5).
The virtual cube randomly displayed three different surface textures (Figure 6). The user switched between the three different surfaces by pressing a selector button on the grip. In the first part of the experiment, the vibrotactile stimuli were deactivated, thus, the user only perceived the kinesthetic stimuli generated by the Novint Falcon Interface.
Source: Authors
Figure 6 Textures for user interaction: a) surface texture 1, b) surface texture 2, c) surface texture 3
In the second part of the experiment, the vibrotactile stimuli were activated in the HaptTech device, and the user perceived the kinesthetic and tactile stimuli simultaneously.
Each subject then completed a survey assessing whether there was any perceived difference between the two parts of the experiment.
Experiment 3. Comparison between perceived and visualized stimuli
The same 31 participants of experiment 2 participated in experiment 3. Using the glove, the users manipulated the small virtual sphere with the HaptTech-Novint Falcon interface to touch the surface of the virtual cube.
Figure 5 shows that the exploration of the cube's surface was carried out naturally, with an open hand and outstretched fingers. This time, the user assessed the level of realism regarding the perceived sensation of the visualized surface on the palm and finger by assigning a score from 1 to 10; 1 indicated that there was no relationship between the visualized surface and the sensation on the hand, and 10 indicated a strong relationship between the visualized surface and the perceived sensation. The users randomly interacted with the same three surfaces.
Data analysis
In experiment 1, the descriptive measures of the differences between textures were calculated. The Kolmogorov-Smirnov Test for Normality (N>50), whose null hypothesis is that the variable has a normal distribution, showed a result of less than 0,05, indicating an uneven distribution. The Friedman test was implemented to evaluate significant differences between texture comparisons. This test's null hypothesis is that all median values are the same, which is rejected if at least one median is different, with a significance value of less than 0,05.
In experiment 2, a dichotomic scale (yes/no) was used to examine the frequencies of the answers after the second part of the experiment.
In experiment 3, descriptive statistics for the coupling test determined how users felt about the interactions with the system, looking for trends in the data distribution and the statistical values. Additionally, the Shapiro-Wilk Test for Normality was applied (N<50), whose null hypothesis is that the variable has a normal distribution. This is rejected if the significance value is less than 0,05. The Friedman test also evaluated the significant differences between the level of realism with regard to the perceived sensation on the palm and finger according to the virtual surface. In this test, the null hypothesis is the same as in experiment 1.
Results
This section presents the results of the three experiments.
Vibrotactile stimuli on the fingertip
Figure 7 shows the discrimination percentage for the participants of experiment 1, which valued the difference between each texture pair. The obtained results show a tendency towards a high valuation regarding texture differentiability. Considering that the rounded lowest average was 7 (Table 1), the ratings were considered to be high from this number. A low percentage of subjects gave a low score (1, 2, 3) for the three comparisons. The subjects reported the highest score (7, 8, 9, and 10) for the comparison of textures 1 and 3, which was followed by textures 1 and 2.
Table 1 summarizes the descriptive measures of the discrimination results. It is clear that the highest mean was attributed to texture comparison 1-3, while the lowest average for texture comparison 2-3. In general, the median values between texture comparisons are in the upper extreme, which confirms the tendency towards a high differentiability.
Table 1 Descriptive measures of collected data
* s: Standard Deviation; ** Interquartile Range
Source: Authors
A negative symmetry is evident in the valuation distribution. Nevertheless, the Kolmogorov-Smirnov test was applied to examine the normality of the data. The result obtained was the rejection of the null hypothesis. The results were the following: textures 1-2 (KS=0,222; p=0,000), textures 2-3 (KS=0,131; p=0,003), and textures 1-3 (KS=0,302; p=0,000).
Table 2 shows the significant difference values between the three textures according to the Friedman test. There is at least one difference in one of the textures (p<0,05). This result confirms that the comparison between textures 1 and 3 has the greatest differentiability. In addition, its median value is the highest.
Coupling tests between the vibrotactile and the ki-nesthetic interfaces
Figure 5 also shows the virtual environment, the virtual object, and both the kinesthetic and tactile interfaces. In this environment, a subject manipulated a rigid cube with fixed dimensions, whose surface texture image displayed three different textures (Figure 6).
The results of experiment 2 were subjective; all participants (100%) reported a difference between the two parts of the experiment.
Table 3 shows a part of the descriptive analysis for the results of experiment 3. The data have a negative symmetry. Moreover, the median values are in the upper extreme, thus confirming the tendency to a high level of realism regarding the perceived sensation on the palm and finger according to the visualized surface.
Table 3 Descriptive statistics of the coupling test
* s: Standard Deviation; ** Interquartile Range
Source: Authors
The results obtained from experiment 3 are summarized in a box and whiskers diagram (Figure 8). The median value of the subjects' ratings for the degree of correspondence between the sensation and the visual image was 8 for textures 1 and 2 and 9 for texture 3, thus indicating that the perception of realism was closer to the maximum possible value. According to Figure 8, most of the data for texture 1 were concentrated between 7 and 9; for texture 2, the data were concentrated between 6 and 9; and, for texture 3, the scores were concentrated between 7 and 10. In light of the high values of the medians, it is considered that a high perception of realism was achieved in each texture.
The Shapiro-Wilk test confirmed that the data does not have a normal distribution. The results were the following: texture 1 (SW=0,906; p=0,010), texture 2 (SW=0,909; p= 0,013), and texture 3 (SW=0,848; p= 0,000).
The Friedman test showed no significant differences (p>0,05) between the three realism comparisons (Table 4).
Source: Authors
Figure 8 Box and whiskers diagram for the evaluation of each tactile stimulus coupled with the kinesthetic interface
Discussion
The main contribution of this work was testing the performance of a low-cost vibrotactile system attachable to the Novint Falcon interface by means of physically reproducing three virtual textures. The system has some characteristics that improve the level of immersion into a virtual environment, which a DES can take advantage of in order to enhance interactivity.
Three fundamental elements support the above: a) the vibrotactile system HaptTech reproduces three different stimuli and is incorporable to a DES; b) there is an increased level of immersion regarding texture perception with the inclusion of a vibrotactile stimulus to the Novint Falcon kinesthetic interface; and c) there is a high correspondence between the vibrotactile pattern presented to the subjects and the texture displayed.
A review of the state of the art shows a representative European project that simulates the touch of virtual textiles. This system, Haptex, uses tactile and kinesthetic actuators with several contact points in every finger. These actuators, developed independently, are bulky, and the system is not easily transportable (MIRALab, 2021; Haptex, 2021). HaptTech adapted a tactile actuator to a commercial kinesthetic system, in which the vibration stimulus acts on the contact area of the fingertip. The main features of this system are its simplicity, safety, and portability.
Vibrotactile texture reproduction incorporable to a digital entertainment system
Regarding the HaptTech vibrotactile stimuli reproduction system, there are three stimuli varying in surface roughness, differentiable enough from each other, to be used in a DES. This was verified by conducting differentiability tests on 75 people and finding that a percentage higher than 65% of the sample considered the differentiability to be greater than 7 in all pairs of textures (on a 1-10 perception scale). For the best texture pair comparison, 92% of the population agreed with this level of differentiability (Figure 7).
The vibration stimuli received by the subjects are within the perception range of the skin and activate mechanoreceptors (Chouvardas et al., 2008). These receptors detect the variations in the applied stimulus.
DES could include a texture reproduction module, which would create a higher level of immersion for the user by differentiating object textures in virtual environments used in commercial or serious games.
These vibrotactile systems are safe, portable, and energy-efficient, which makes them useful for game applications, in addition to adding a tactile complement for interacting with texturized objects.
Only three texture stimuli were used in the tests for this study, but future research should use more. The degree of vibration intensity and its relationship with surface roughness in the three texture stimuli confirmed a relationship of differentiability. It is possible to improve tactile simulation, achieving a higher resolution by including additional stimuli to the vibration. However, this augmented effect may not be necessary for digital entertainment applications.
Increased level of immersion
As for the increase in the level of immersion when perceiving textures, by adding the device for reproducing vibrotactile stimuli to the kinesthetic stimulation provided by the Novint Falcon interface, a 100% differentiation (present vs. absent stimulus) was achieved in 31 participants.
By including tactile feedback in a VR environment while the participants interact with virtual objects, the mechanoreceptors are also activated, which favors immersion. The importance of this complement is that it can generate more complex interactions that are different from just identifying boundaries or obstacles in VR, as is the case of recognizing different texturized objects.
The coupling reproduces the texture perception directly on the fingertip, unlike other devices with pen-like grips (Culbertson and Kuchenbecker, 2017b; Ujitoko et al., 2019). These indirectly transfer the sensation of the surface to the hand.
The physical coupling of the vibrotactile reproducer is noninvasive. The electronic coupling was carried out through programming routines that synchronized the texture stimuli when the virtual object was touched, thus enhancing the level of immersion.
High correspondence between vibrotactile patterns and visualized textures
A high correspondence between vibrotactile patterns applied to the subject and the virtual surface textures makes for a more realistic perception (Culbertson and Kuchenbecker, 2017a). The test performed to assess perception correspondence yielded results with high medians. Based on the Box and Whiskers diagram (Figure 8), the data are mostly located at the upper end, indicating that users perceive a relationship between what they see with what they feel.
The above is useful for applications that require permanent correspondence between visual and manual perception, e.g., palpation procedures or others that involve texture identification.
Two possible phenomena can explain the instances of less coincidence: a) concerning vision and haptics in material perception, the disparity between the mental image that the person formed when observing the texture that formed when they received the tactile stimulus (Baumgartner et al., 2013; Klatzky and Lederman, 2010); and b), the prolonged use of vibrotactile actuators, which could block the user's discernability because this actuator only stimulates one type of tactile receptor (Pacinian, i.e., a Phasic type or of fast adaptation), and this receptor can be saturated or adapt when a stimulus is applied too often (Fox, 2014).
Conclusions
This research described the development of a vibrotactile device to generate surface texture stimuli while coupled with the Novint Falcon kinesthetic interface.
By testing the low-cost vibrotactile reproducer, HaptTech, which reproduces three textures while coupled to the interface, results were found which favor the level of realism of the perceived sensation, which suggests that the device could help increase the level of immersion in a VR environment, in addition to being useful in a DES.
Current applications of interactive digital entertainment do not feature texture differentiation, although they use vibrotactile actuators, commonly for contact, in order to indicate the presence of an object or signal an action such as a gunshot or bursts.
The addition of tactile feedback generates stimuli that activate mechanoreceptors, which could also improve the immersion and complement the visual sensation produced by the DES.
Vibrotactile stimulation has no risks or side effects. However, prolonged vibrotactile stimulation can block the perception of the same stimulus.
Complementing kinesthetic perception and visual feedback is suitable for application in a DES, especially for VR, serious games for rehabilitation, or training. Further studies on the differentiation of more than three textures would improve performance in different applications.
Future work
Future research could change the vibration component to a piezoelectric element, which maintains a reduced size and weight but generates high-resolution stimuli. Furthermore, testing could be conducted for greater degrees of realism by coupling the stimuli with other complementary stimuli (e.g., electro-cutaneous or electro-capacitive), while adapting the findings of previous research. A greater degree of realism means that the virtual stimulus simulates the real stimuli more closely.
More tests are necessary, with the addition of other textures for differentiation while discerning the influence of the visual texture expectation produced in the user. It is also necessary to implement a method to convert the roughness of a surface into a quantified, digitalized vibrotactile signal.
Finally, future applications of the HaptTech device to a DES may benefit from an additional hardware device that simultaneously manages the interaction with the virtual environment using both the texture reproducer and the Novint Falcon interface.
