Scientific texts are texts with multiple representations (texts and diagrams) containing specific narrative styles, signal representations in pictures, and difficult academic words. Students may struggle with scientific texts—whether through having weak reading ability, over-focusing on text, or ignoring scientific pictures—as well as with graphic comparison and integration strategies. Therefore, students often face difficulties in understanding scientific texts. In particular, academic words are translated using different translation methods (such as paraphrasing and transliteration), and the understanding of academic words may be affected by semantic transparency. The application of eye-tracking technology is suitable for exploring how middle-school students read such content. Few eye movement studies have focused on academic words, with most exploring the cognitive process of adult readers recognizing academic words when reading plain text. Therefore, understanding how middle-school students read scientific texts containing academic words is a crucial topic.
This study used eye tracking to explore the cognitive processes and strategies of seventh-grade students with different reading abilities as they read illustrated scientific texts as well as to examine how readers cope with academic words with high (paraphrase) and low (transliteration) semantic transparency. The three specific research questions were as follows:
a. When seventh-grade students with different reading abilities read scientific texts with different levels of semantic transparency (paraphrased or transliterated text), is any difference observable in memory and comprehension performance? (Assessed through free recall and reading comprehension tests).
b. How do seventh-grade students with different reading abilities recognize academic words with different levels of semantic transparency (paraphrased or transliterated words)? What information can be used to assist their understanding? (Based on academic words as the unit of analysis and analyzing eye movement patterns).
c. On the basis of the analysis of eye movement patterns and cued retrospective think-aloud (CRTA), what strategies do seventh-grade students with different reading abilities adopt when reading illustrated scientific texts?
Participants. Seventh-grade students (N = 65) were divided into groups based on their reading ability through a pretest. The high, middle, and low reading ability groups comprised 23, 22, and 20 students, respectively. The average age was 13.12 years (SD = 0.27), and all study participants were native Chinese speakers with normal or corrected vision.
Materials. We designed four illustrated scientific texts by manipulating academic words (paraphrase and transliteration). Each text was presented on a single page, with the text placed on the left and two colorful pictures on the right. The number of texts related to physics, chemistry, and biology themes, and the number of characters and words in each text were evenly matched. Academic words were all low-frequency, and word length was consistent. Three types of diagrams were used, namely organizational and interpretational diagrams and tables. The paragraphs and illustrations of the four texts all had causal contexts. Texts on wireless charging and blood type had high causal connections, whereas texts on radar and penicillin had moderate causal connections. In terms of reading order, accounting for the order of experimental processing, a counterbalance design was implemented.
Apparatus. We used the EyeLink Portable Duo (SR Research Ltd., Ottawa, Canada) eye tracker with a sampling rate of 1000 Hz. A fixed chin rest was employed to minimize head movement and perform eye movement correction. If the average error of the gaze position was less than 0.5°, the angle of view did not require correction.
Procedure. The research procedure was divided into two stages, namely the pretest and eye movement experiments. The pretest consisted of an academic word familiarity questionnaire and standardized test of reading ability. In the eye movement experiment, relevant guidelines were first displayed on the screen to remind participants of the experimental procedure and precautions, and then the participants were requested to read the practice text to familiarize themselves with the experimental procedure. A 3-min reading time was set for each official text. After the participants had read the text, the text test was complete. After the experiment, the examiner asked the participants to perform free recall and played the participants’ own eye-tracking video to prompt CRTA.
Overall strategy use during reading was analyzed using one-way analysis of variance (ANOVA), and use of substrategies was analyzed using two-way ANOVA. Reading ability was the between-subjects variable, and word type (paraphrase or transliteration), diagram type (organizational diagram, interpretational diagram, or table), or text theme (four texts) were the within-subjects variables. The dependent variable was the performance of reading the texts or multiple eye movement indicators. The CRTA interview data were used to examine the connection between the three reading ability levels and usage of 17 substrategies, thus revealing whether seventh-grade students with different reading abilities apply different reading strategies to read illustrated scientific texts. The following are the explanations of the research results from post-test performance, eye movement indicators, and CRTA interview data:
a. Reading comprehension and free recall: The students with high reading ability outperformed students with middle and low reading abilities in reading comprehension and free recall. The students’ free recall score for the paraphrased version was significantly higher than that for the transliterated version.
b. Eye movement (academic words): The students adjusted their reading strategies according to the semantic transparency of academic words. The gaze duration for transliterated academic words was significantly higher than that for paraphrased academic words, indicating that transliterated academic words require longer to decode. The students relied on text messages for scientific reading, and the number of references to sentences in texts was significantly higher than that to scientific diagrams.
c. Eye movement (illustrated scientific texts): The high-ability group decoded words more rapidly than the middle-ability and low-ability groups. The mean fixation duration of the high-ability group was significantly shorter than that of the low-ability group. Furthermore, the students realized the critical role of the table, because the revisited fixation duration, total fixation duration, and proportion of fixation duration for the table were significantly longer than those for the other types of diagrams. Students with different reading abilities exhibited differences in processing various types of diagrams. The high-ability group devoted more time to analyzing and reanalyzing the diagrams and recognized the causal context with a high correlation between the two diagrams in the wireless charging text and blood type text. The middle-ability group recognized the correlation between the diagrams and texts, and therefore, for the wireless charging and blood type texts (high diagram–text association), the number of saccades between text and diagrams were both relatively high. The proportion of fixation duration for reading pictures in the low-ability group was significantly lower than that in the middle-ability and high-ability groups, and the low-ability students were unable to distinguish the correlation between the two diagrams.
d. In terms of the three reading levels, the high-ability group used the overall strategy significantly more than the low-ability group. For the 17 substrategies, the high-ability group exhibited the highest usage of illustrated information and integrated graphic information; the low-ability group exhibited the most frequent usage of key cognitive errors and unclear reading goals.
Studies of eye movement during reading with CRTA have been mostly conducted on adult research participants (Trevors, 2016; van Gog et al., 2005), with a distinct lack of research on young readers. This study may be the first to focus on analyzing the reading of illustrated scientific texts by middle-school students using this approach. In this study, obtaining the subjective interpretation data of the eye movement pattern of the reader, rather than indirectly inferring the reader’s reading process through eye movement data, assisted in deepening our understanding of the reading process. In terms of the interpretation of academic words, the results revealed that middle-school students, similar to adult readers, devote more time to decoding academic words in the initial processing stage of reading (Jian & Ko, 2014). This study also supports the theoretical views of cognitive theory of multimedia learning (CTML) (Mayer, 2005) and ITPC (Schnotz & Bannert, 2003). Representations of text and diagrams link and organize the meaning of the messages through dual-code representation, integrating the presented information with an individual’s prior knowledge. This is an essential step in the process for achieving the comprehension of diagrams and texts. The CRTA data revealed that high-ability students often use inference and integrated reading strategies (such as the use of illustrated information, integrated texts, and diagrams) to correctly identify the type of picture and amount of information and adjust their reading strategy accordingly. The middle-ability students often used information extraction strategies (such as finding key points and rereading), whereas the low-ability students often used negative reading processing methods (such as misunderstanding key points and unclear reading purposes); the low-ability students’ methods increased the difficultly of learning and understanding effectively when reading popular scientific texts.
In this study, we selected academic words for paraphrasing and transliteration. Future studies could conduct in-depth research on semantic transparency, such as designing academic words with high (paraphrase), medium (phonetic-meaning translation), and low (transliteration) semantic transparency to further compare the process of Chinese academic word recognition. Moreover, the amount of information in the illustrations of the experimental materials of this study was not controlled. In the future, expert validity evaluation, among others approaches, can be applied to evaluate and design the manipulation of the information content of the illustrations.