Inglês (pdf)
Artigo em XML
Referências do artigo
Tradução automática
Enviar este artigo por email
Citado por SciELO 
Citado por Google 
Similares em
SciELO 
Similares em Google 
Permalink
1. Introduction
The vast changes in technology and the ubiquitous presence of high-speed internet have immensely changed people’s interaction with digital media. Due to digital reincarnation, cinema has transformed into a multi-modal visual experience and traveled across the world. Viewing cinema is an interaction between cinema and the viewers. Both top-down and bottom-up factors affect viewing a movie (Thompson & Bordwell, 2011). There are topdown or endogenous factors that come from within an individual, such as ideology, intelligence, memory, or comprehension, which matters while watching a movie. At the same time, exogenous factors or bottom-up factors that come from within the stimuli like the composition of the frame, lighting, editing, and sound also matter.
Subtitles refer to the text displayed on-screen simultaneously with the audio. The audience’s eyes are drawn towards subtitles not only because the text acts as a source of meaningful information (top-down), but also because its appearance brings about on-screen changes (bottom-up), as proposed by Kruger et al. (2015). Subtitles usually do not comprise more than two lines and appear horizontally at the bottom center of the screen, in synchrony with the image and dialogue (Diaz-Cintas, 2010). Subtitles attract the gaze of viewers, just like the face of characters or movements of the characters (Hershler & Hochstein, 2005; Yarbus, 1967). The subtitles have become a common exogenous factor in cinema nowadays. The time subtitle displayed on-screen depends on the speed at which original dialogues are delivered and the assumed reading speed of the audience. There are two main types of subtitles: interlingual and intralingual. Intralingual subtitles contain the textual version of dialogue in the same script, whereas interlingual subtitles contain a textual translation of foreign language dialogues (Polcz, 2008).
People who watch a movie with an intralingual subtitle skip the subtitles more often than while watching a movie with interlingual subtitles subject to their language linguistic/reading skills in the concerned languages (Szarkowska et al., 2016). It is reported that people who watch intralingual subtitled movies often find it difficult to keep visual attention on moving images due to the competition from the subtitles. The research established that viewers automatically attend to subtitles on screen without their volition (d’Ydewalle & De Bruycker, 2007). Previous research also suggests that viewers have a strong tendency to read subtitles almost spontaneously regardless of their importance or utility (Cerf et al., 2009).
Viewers have to engage in multimodal information processing such as reading subtitles, watching the video, and listening to soundtracks while watching a movie with subtitles (Krejtz et al., 2013). Subtitle reading is different from static text reading because subtitles appear against the background of moving images (Kruger et al., 2015). Also, subtitles appear on the screen only for a few seconds. The subtitle text is considered a part of the image. Hence, the subtitles are always in competition with moving visuals. When subtitles appear along with visuals, it affects the viewing behavior of viewers. They have to manage time to read the subtitles along with watching running movie images. Viewers find it easy to read subtitles if the verbatim of the subtitle is similar to the dialogue in the original soundtrack (Szarkowska & GerberMorón, 2018). While watching a foreign language movie, there can be two options for the audience: either watch the movie with subtitles or watch the dubbed version of the movie. People have different preferences on this topic. Even cultural preferences are also there in the case of dubbing and subtitling in cinema (Borell, 2000). However, the subtitle option is preferred often as it is cheaper and faster than dubbing (Gambier, 2003).
An earlier study using the eye tracking method reported that there is a general pattern in which viewers process subtitles on screen while viewing a video (Jensema et al., 2000). The viewers’ gaze starts looking at the center of the screen first and then within a fraction of a second the gaze will fixate at the beginning of the subtitle text only to return to the moving visual image after reading the subtitle. However, these deflections in gaze shift are not always smooth. Apart from the subtitle, editing style also matters for the smooth viewing experience of the audience. Our primary focus in this study is on the effect of subtitles on gaze during the shot change.
1.1 Editing Style
Editing is a process that integrates the various discontinuous shots or different points of view by using cuts. Editing helps to generate the rhythm and pace of a movie. It creates a perceived narrative continuity in space and time. Based on the spatiotemporal dimension, we can differentiate into two styles of editing: continuity editing style and discontinuity editing style (Bordwell & Thompson, 2008). After a long period of trial and error since the conception of cinema, filmmakers found an editing style to improve the attention and to reduce the disorientation of the viewers. This editing style is called continuity editing style (also called classical editing style). Under continuity editing style there are different editing techniques such as establishing shot, match action, 180-degree rule, 30degree rule, shot/reverse shot, and point of view editing. Filmmakers believe that by following a continuity editing style, cuts can be made “invisible” (Dmytryk, 1986). Over 90% of shots surveyed from Hollywood movies adhere to the continuity editing style (Levin & Wang, 2009). While watching continuity editing cinema, the gaze behavior of multiple viewers displays a significant degree of coordination. This is called attentional synchrony; it was first observed by Stelmach and Herdman (1991).
These editing rules of Hollywood studios were purposefully violated by certain filmmakers to create a different cinema than they exposed till then. They violated the spatial and temporal continuity of shots by introducing a new editing style collectively known as the discontinuity editing style. Filmmakers use various techniques such as flashback, flashforward, montage, jump cuts, violation of the 180-degree rule, etc., to create discontinuity. The new style generated a self-awareness among the viewers that they are watching a movie. Editing style in cinema can affect the visual perception of viewers (Shimamura, 2014; Smith, 2013). Kruger et al., (2015) studied the influence of shot changes on subtitle reading and found that the nature of editing has a strong influence on how people read the subtitle.
1.2 Subtitles and Shot Change
A shot is nothing but a continuous run of a camera (d’Ydewalle, 1998). Shot change, also known as ‘cut’, is a transition between the two consecutive shots. It was an accepted general norm that subtitles should not appear over a shot change as the viewers tend to re-read the text. Over the years, the shot duration of cinema has reduced drastically. Hence, several subtitles may be displayed across the shots in contemporary cinema (Bordwell, 2006; Shimamura, 2014). One research study established that most viewers do not re-read the subtitle over a shot change, whereas the participant’s gaze went back to the beginning of the subtitle after a shot change assuming it as a new subtitle (De Linde & Kay, 1999; Krejtz et al., 2013).
1.3 Overview of the Current Study
Appearance of subtitles on screen presents challenges to the viewers because the gaze must shift between visual image and text. Most of the previous studies related to subtitles and eye-tracking focused on the reading behavior of subtitles rather than the impact of subtitles on the gaze behavior of viewing cinema (Fresno & Sepielak, 2020; Hansen-Schirra et al., 2016; Krejtz et al., 2013; Kurzhals et al, 2020; Lång et al., 2021; Liao et al, 2021; Szarkowska et al., 2016, 2019). Here, we have explored the subtitle as an exogenous factor/ bottom-up in cinema and its impact on the gaze behavior of viewers. We specifically focused on the gaze behavior of viewers during the shot changes while watching movie clips with and without subtitles.
Previous studies found the difference in eye movement patterns between different editing styles (Germeys & d’Ydewalle, 2007; Smith, 2013), but they never studied the difference between editing styles while watching subtitled movies. In the present study, we also examined the gaze behaviour between continuity editing style and discontinuity editing style cinema with the presence or absence of subtitles.
The timing of subtitles and the shot changes also matter while seeing a movie. First category of subtitles which start before the cut and continue after the shot changes, whereas the second category of subtitles starts after the cut. We would like to know whether there is any difference between the gaze behavior of the participants while watching video clips with subtitles started before the cut and after the cut. Hence the third objective studied the effect of timing of the subtitle (before the cut and after the cut) on the gaze behavior of the participants.
2. Method
2.1 Participants
Twenty-five university students with normal or corrected vision participated in the study. Participants’ ages ranged from 20 to 35 years with a mean age of 24 (SD = 3.1). None of the participants had seen the movies used in the study before. All the participants were well versed in English, although it is a second language for them. All the participants preferred subtitled movies over dubbed movies.
2.2 Apparatus
Eye movements were recorded using an SMI Desktop Eye tracker with a frequency of 1250 Hz. Movie clips were displayed using an experimental setup at 1024 768 pixels and had different frame rates for different clips. Participants viewed the clips from a distance of 55 cm.
2.3 Stimuli
Stimuli were video clips, taken from real cinema with different editing styles. There were three video clips of continuity editing (CE) style and three video clips of discontinuity editing (DE) style presented to the participants. All the video clips followed the concerned editing rules. The details of the video clips used in the present study are presented in Tables 1 and 2. These video clips could be accessed using the following doi: https://doi.org/10.17632/ zb9gcvtm8f.1. The average duration of a video clip was 2-3 minutes. All the video clips used were in color and they varied on pixel ratio. All the cuts used in this study are ‘straight cuts’, meaning “instantaneous transitions between shots, not gradual effects such as dissolves or fades” (Smith & Henderson, 2008, p. 8).
Order of presentation of continuity and discontinuity editing video clips were randomized across the participants. Two versions of the video clips were constructed: with subtitles and without subtitles. The language of the movie as well as the subtitle displayed on the screen was in English.
When the characters utter a dialogue, the corresponding text is displayed on the screen. Subtitles appeared on the bottom center of the screen. Freesans font used for the subtitles text. Default color of the text was white with a black outline. Background of the text was transparent.
Table 1 Video Clips Used under Continuity Editing Style (CE) Condition
| Movies | About the scene | Shots |
|---|---|---|
| The Shining (Kubrick, 1980) | Two characters are arguing with each other | POV, Eye-line match |
| Rear Window (Hitchcock, 1954) | Two characters are having conversations with each other | Shot/reverse shot |
| L. A. Confidential (Hanson, 1997) | Famous interrogation scene | Intense continuity editing |
Table 2 Video Clips Used under Discontinuity Editing Style (DE) Condition
| Movies | About the scene | Shots |
|---|---|---|
| Dancer in the Dark (Trier, 2000) | Rehearsal of a play is staged | Violated the 180-degree rule and jump cuts |
| Gummo (Korine, 1997) | Two characters are going to meet another two characters | Violated 180-degree rule |
| Into the Wild (Penn, 2007) | Convocation day and later characters have a conversation in a hotel | Montage and flashback |
2.4 Procedure
Participants were seated in a sound-attenuated and dimly lit room. A chin rest was used to control the head movements and to maintain a constant distance from the screen throughout the experiment. A 9-point calibration was done before the presentation of stimuli in each session. Participants were instructed just to watch the video clips that appear on the screen. Recording of eye movements was done with subtitles and without subtitle conditions separately with a gap of 6-7 days. Although participants who watched the video clips were under the binocular condition, SMI eye-tracker considered the data from the left eye only for further analysis. The data collection for each condition was carried out in two sessions with a short break between the sessions. Since this was a within-group design, participants were shown the same video clips twice, once under subtitle condition and again under no-subtitle condition after about one week. The experiment duration was about 20 minutes each per condition per participant. After the experiment, a small interview was taken to know about the visual experience of participants.
2.5 Ethics
The study was approved by the Human Research Ethics Committee of the University. Informed consent was obtained from all the participants. The confidentiality and anonymity of the participants were ensured by removing all identifying information from the transcripts and assigning codes.
2.6 Data Analysis
For analysis, we selected a single shot change from each video clip that fulfilled the concerned editing style criterion. We created dynamic AOIs, which covered the main characters or the speaker’s face. For practical reasons, simple oval-shaped AOIs, which are tightly overlaid with the characters’ movements, were used. The AOI in the new shot was different from the previous shot in the concerned shot change. Another constraint for selecting a shot change for analysis was that subtitles in the new shot should appear immediately after the shot change. The Be-Gaze software was used to calculate the metrics. For further analysis, eye movement data were exported to SPSS and Excel software. The information on the visual experience of participants was collected by a qualitative approach.
2.7 Quantitative Analysis
We separated the data based on the onset time of subtitles. The data in which subtitles started before the cut and the data in which subtitles appeared after the cut was analyzed separately. Outliers were removed from the data before analysis. The normality test confirmed the normality assumption.
2.7.1 Subtitle started before the cut
Here subtitles had already started in the previous shot and displayed for around 1-second then continued in the next shot. We analyzed the eye movement pattern of participants for 2-seconds after the cut. To know whether the presence of subtitles during the shot change affects the eye movement patterns, we performed repeated measures of two-way analysis of variance (ANOVA) and paired ttest for further analysis.
2.7.2 Subtitle started after the cut
In contrast to the above, where the subtitles already started in the previous before the cut, subtitles in this case come along with the new shot. As a result, two different visual stimuli (text and video image) are simultaneously presented immediately after following a cut. We focused on how the participants’ gaze responded to it.
We analysed the Entry time. First fixation duration, dwell time, scan path, and the average fixation duration measures to infer the effect of subtitles on gaze behavior under the CE and DE styles of editing.
Table 3 Summary of Two-way Repeated ANOVA Results for Entry time, Dwell Time, and First Fixation Duration for the Condition Subtitles Appearing Before a cut
| Metrics | Source | df | F | Sig. | Partial Eta Squared | |
|---|---|---|---|---|---|---|
| Entry time | Subtitle | Editing | 23 | 8.113 | .009 | .261 |
| Editing | 23 | 3.802 | .063 | .142 | ||
| Subtitle* | 23 | 1.780 | .195 | .072 | ||
| Dwell time | Subtitle | Editing | 23 | 9.068 | .006 | .292 |
| Editing | 23 | 2.357 | .139 | .097 | ||
| Subtitle* | 23 | .356 | .557 | .016 | ||
| First | Subtitle | Editing | 21 | .661 | .425 | .031 |
| Fixation | Editing | 21 | 7.567 | .012 | .265 | |
| Duration | Subtitle* | 21 | .831 | .372 | .038 |
Note. Significance <.05.
The location of the target AOI in a new shot was different from the previous shot. Hence, during the shot change, participants had to move their eye positions to the AOI in the new frame, but the presence of subtitles at the bottom center of the screen affects the gaze behavior of the participants. The entry time is the time taken by the participants to reach the new AOI after the cut, which is measured in milliseconds. We calculated the entry time by recording the duration from the cut to the time when the first fixation in the new AOI started. First fixation duration is the duration of the first fixation of each participant, after entering the AOI in a new shot. Dwell time is the sum of fixation durations and saccadic movement inside the AOI. Scan path is the total length traveled by the participant’s gaze. It is the total of fixations and saccades and is measured in pixels. The average fixation duration and saccadic amplitude during the initial 4 seconds after the cut was calculated.
Subjective experiences also matter while watching cinema. Here, we were interested to know more about the impact of subtitled movies. Hence, we conducted a semi-structured interview with the participants to know their visual experience. After finishing the experiment, each participant was asked questions related to the visual experience while watching subtitled movies. Although most of them were closed-ended questions with yes or no answers, participants could answer the questions in a descriptive manner.
3. Results
3.1 Subtitle Starting before the Cut Condition
A summary of two-way ANOVA on measures of entry time, dwell time, and first fixation duration is presented in Table 3. The results suggest that the presence of subtitles in a video clip has a significant impact on the entry time to the AOI -F (1, 23) = 8.113, p = .009, η 2= .261. Participants took more time (M = 693.95, SE = 35.26) to reach the AOI under subtitle condition than under no-subtitle condition (M = 547.11, SE = 44.79).
The main effect of editing style was nonsignificant (p= .063) and the interaction effect between subtitle and editing style was also not significant (p = .195).
After reaching the AOI, how much time participants spent on AOI is measured by dwell time. Results reveal a significant difference in dwell time -F (1, 23) = 9.068, p = .006, η 2 = .292- between the subtitle condition and no-subtitle condition. Even though the subtitle already started before the cut, the participants spent more time reading the text in a new shot and dwell less time on the visual image (M =790.99, SE = 60.20) compared to the no subtitle condition (M = 1073.82, SE = 70.26).
In the case of first fixation duration, there was no significant effect of subtitles (p = .425), but the main effect of editing style was significant (p = .012). The first fixation duration under continuity editing (M =323.24, SE = 37.42) was more as compared to under discontinuity editing style (M = 232.23, SE = 20.33) condition. The interaction effect was not significant (p = 0.372). The editing styles per se did not have any impact on the gaze behavior, which was confirmed by the results of paired t-test. The results showed that the mean difference across the conditions for the entry time (p = .067), dwell time (.126) and first fixation duration (.277) were all statistically nonsignificant.
3.2 Subtitle Starting after the Cut
Under this condition, immediately after the previous shot, a new shot appeared along with a subtitle on it. We performed repeated measures two-way analysis of variance (ANOVA) to understand the effect of presence or absence of subtitles and of editing style on entry time, dwell time, and first fixation duration. The summary of the results is presented in Table 4.
The entry time of participant’s gaze to the AOI in subtitle video condition (M = 508.87, SE = 48.86) was more than that of under no-subtitle condition (M = 450.54, SE = 37.6). However, the mean difference was not statistically significant (p = .306). The entry time in the AOI under the continuity editing style (CE) condition (M = 352.17, SE = 27.90) was significantly lower than that under the discontinuity editing (DE) condition (M = 607.25, SE = 68.70; p = .006). There was no significant interaction between subtitle conditions and editing styles (p = .789).
Table 4 Summary of Repeated 2-way ANOVA for Subtitles Appearing after the cut Condition
| Metrics | Source | df | F | Sig. | Partial Eta Squared | |
|---|---|---|---|---|---|---|
| Entry time | Subtitle | Editing | 16 | 1.119 | .306 | .065 |
| Editing | 16 | 10.111 | .006 | .387 | ||
| Subtitle* | 16 | .074 | .789 | .005 | ||
| Dwell time | Subtitle | Editing | 22 | 12.976 | .002 | .371 |
| Editing | 22 | 3.756 | .066 | .146 | ||
| Subtitle* | 22 | .073 | .790 | .003 | ||
| First | Subtitle | Editing | 18 | 3.331 | .085 | .156 |
| Fixation | Editing | 18 | 1.162 | .295 | .061 | |
| Duration | Subtitle* | 18 | 4.255 | .054 | .191 |
Note. Significance <.05.
Table 5 Summary of Within-Subject Test Results Scan Path and Average Fixation Duration
| Metrics | Source | df | F | Sig. | Partial Eta Squared | |
|---|---|---|---|---|---|---|
| Scan path | Subtitle | 24 | 20.308 | .000 | .458 | |
| Editing | 24 | 8.955 | .006 | .272 | ||
| Subtitle* | Editing | 24 | 10.474 | .004 | .304 | |
| Average | Subtitle | 23 | 21.242 | .000 | .480 | |
| Fixation | Editing | 23 | 6.736 | .016 | .227 | |
| Duration | Subtitle* | Editing | 23 | 7.137 | .014 | .237 |
Note. Significance <.05.
The main effect of subtitle condition was significant for dwell time (p = .002). Gaze duration in AOI of the subtitle condition (M = 615.17, SE = 62.76) was less compared to the no subtitle condition (M = 1032.62, SE = 82.25). The main effect for editing style (p = .066) as well as the interaction effect (p= .79) between subtitle and editing style conditions was not significant. For the first fixation duration, the main effect of subtitle condition (p = .085), the main effect of editing style in cinema (p = .295), and the interaction effect (p = .054) were all not significant.
We performed a paired t-test to find out the difference between the above measures across editing conditions. The results showed that entry time under CE style (M = 350.44, SD = 131.73) was significantly lower than that of DE style (M = 609.14, SD = 383.63); t(22) = -2.877, p = .009). Dwell time in CE style (M = 697.7, SD = 409.8) was more than that of DE (M = 530.8, SD = 280.7) (t(23) = 2.05, p = .05).
3.2.1 Timing of the subtitle
We also performed a paired t-test to compare the impact of the timing of subtitle appearance before the cut and after the cut. The subtitles already started in the previous shot (before the cut) showed longer entry time to reach the AOI in the new stimulus (Mean (before the cut) = 679.88; Mean (after the cut) = 479.49; t(46) = 3.675, p = .001). The dwell time on AOI (calculated for two seconds) also showed a significant difference (Mean (before the cut) = 773.49; Mean (after the cut) =.606.9); t(48) = 2.648, p = .001). The difference in df is because of the removal of outliers.
3.3 Scan Path and Fixation Duration
We calculated the scan path length for a 4-second duration just after the cut. There was a significant main effect of subtitle condition and editing style condition. The interaction effect was also significant. The scan path length under the subtitle condition (M = 1349.73, SE = 112.63) was significantly longer than that under nosubtitle condition (M = 812.48, SE = 40.77). Scan path length under CE condition (M = 962.65, SE = 79.16) was significantly shorter than that under DE style (M = 1199.56, SE = 64.11). The interaction effect between subtitle condition and editing style was also significant. A similar analysis was carried out on average fixation duration. The average fixation duration in subtitle condition (M = 242.51, SE = 10.63) was significantly lower than that under no-subtitle condition (M = 418.94, SE = 37.62). Average Fixation duration under CE style (M = 376.37, SE = 36.20) was significantly higher than that under DE style (M = 284.076, SE = 10.43). The interaction between editing style in cinema and subtitle condition was also significant. The summary of two-way repeated ANOVA is presented in Table 5.
3.4 Interview
In addition to the above quantitative analysis, we conducted a semi-structured interview to understand the visual experience of the participants after watching subtitled movie clips with CE and DE styles of editing using the following questions.
1. While watching movies, do you always get completely involved in it?
2. While watching the movie, do you use subtitles?
3. Do you prefer the location of subtitles at the bottom center of the screen?
4. Do you prefer subtitled movies instead of no-subtitled movies?
5. Do you think visual information is lost while reading subtitles?
6. Do you think reading subtitles hinders visual experiences?
7. How would you rank the visual experience of movies with subtitles?
Figure 1 represented the visual experience of participants for the 1-6 items. Out of 25 participants, 92% of the participants think visual information is lost while reading subtitles and 72% think that subtitles hinder the visual experience. For the last question (7), participants had to rate the visual experience of cinema with subtitles on a 5-point scale. Out of 25 participants, 10 were completely distracted, 8 were distracted, 2 were no-distracted, 3 were involved, and 2 were completely involved.
4. Discussion
The central aim of the current study was to examine the gaze behavior of participants while seeing video clips with subtitles and without subtitles. Subtitles always help the viewers to comprehend the dialogues in a scene if the language of the movie is foreign to them. In the current study, we used video clips taken from English language movies, and the subtitles were also in English. Even though our participants were proficient in English, it was not their first language. The presence of subtitles at the bottom center of the screen at times helps the viewers to comprehend better the movie, but, at the same time, it can also act as a hindrance to the smooth visual perception of cinema. During the semi structured interview, many of the participants reported that it was a hindrance to their visual experiences.
The second goal of the study was to explore the interaction between editing style and the subtitle on viewers’ gaze behavior. In addition, we examined the impact of timing of the subtitle appearance during a shot change on gaze behavior. These are discussed below.
4.1 Subtitle versus No-subtitle
The onset of subtitles draws the attention of viewers at once, even though they are engaged in watching video stimuli (De Bruycker, 2003). The scan path data in the present study also reveals that the moment subtitles appear on the screen, participants move their visual attention towards the text. When the subtitles were displayed before the cut, participants took more time to reach the AOI on the next shot (as compared to the no-subtitle condition). Since participants were reading the subtitle before the cut, they continued to read the text on the next visual before moving their eyes towards the AOI, but this difference between subtitled and nosubtitled conditions disappeared when the subtitles appeared after the cut. The semi-structured interview also showed that most of the participants thought that subtitles hindered their visual experience of movie scenes.
Figure 2 Eye Positions of Participants in the Video Clip (The Shining) Without Subtitles. This is the 20th Frame after 01:18 Seconds of the Video Clip
The dwell time on the visual image (AOI) is shorter in the subtitle condition compared to the no-subtitle condition, because participants moved their gaze to the subtitled area. When subtitles appear after the cut, even if there was competition for attention between visual image and the text, participants’ attention was more towards the subtitles. Although participants were proficient in English and could listen to the audio alone, they still preferred to read the subtitles than simply look at the images. We do not know whether the changes will be larger if the subtitled language is the native language of the participants. The scan path length for the subtitle condition was longer compared to the nosubtitle condition. In the no-subtitle condition, participants’ gaze focused mostly on the character’s face or other interesting objects (AOI). When there were subtitles, participants were seen shifting their gaze between AOI and subtitle (see Figures 2 and 3). Reading a text is a distinct process when compared to scene perception (Rayner, 1992). The mean fixation duration of reading is shorter compared to the scene perception. In our study, we also found that the average fixation duration of participants in the subtitle condition is less compared to the no-subtitle condition.
4.2 Editing Style and Subtitle
The editing style also plays a significant role in influencing the eye movement pattern of viewers. We observed that when there were subtitles, the time taken by the participants to reach the AOI under CE style editing was shorter compared to that under DE style editing. Disorientation created by the discontinuity editing style may partly explain this. However, the editing style did not show any main effect vis a vis dwell time, scan path length, and average fixation duration while watching a subtitled clip. Thus the observed delay in entry time (into the AOI image) might be more due to the text interference in visual perception.
4.3 Timing of the Subtitle and Gaze Behavior
We considered two different timings of subtitles: the presentation of subtitles displayed before the cut and after the cut. The participants who watched video clips with subtitles displayed before the cut took a longer time to reach the AOI than when the subtitles appeared after the cut. Since the participants were already reading the subtitle before the cut, they probably continued to read the text before moving towards the visual image (AOI).
5. Conclusion
The present study attempted at providing a comprehensive picture of the effect of subtitles on the gaze behavior of viewers while watching continuity editing and discontinuity editing style cinema. Here we opted for intralingual (English) subtitled (and not subtitled) video clips as stimuli. The presence of the subtitles in cinema affects the gaze behavior of the participants. At the onset of the subtitle, the participants’ eyes automatically move towards the text even if the subtitles were in the same language as the audio. This leads to spending less time on visual images. The editing style did not have any major impact on the gaze behavior while watching subtitled video clips. Participants preferred to read the subtitles than the visual image, even if the subtitles presented during the shot changes (both before the cut and after the cut). Even though the participants were proficient in English, it was still a second language for them. We are not sure whether there would have been a larger effect of subtitles if the subtitles were in their first language. Future studies could probe the gaze behavior under interlingual subtitle conditions, how the gaze behavior interfaces with the audio and video streaming in a cinema, especially concerning the emotions expressed in cinema and the emotional experience in viewers.
