Text and diagrams are frequently presented together in science textbooks and popular science articles. Text is used to describe concepts, and diagrams contain visual and spatial information depicting conceptual relationships, object structures, and developmental processes. Two primary theoretical models have been widely adopted by researchers in the domain of text–diagram comprehension: Mayer’s cognitive theory of multimedia learning (CTML; Mayer, 2005, 2014) and Schnotz’s integrated model of text and picture comprehension (ITPC; Schnotz & Bannert, 2003; Schnotz et al., 2014). In contrast to findings of benefits associated with Mayer’s “multimedia principle” (Mayer, 2005, 2014), several studies have documented neutral or negative learning effects from studying images (McTigue, 2009; Segers et al., 2008). Previous literature reviews on text–diagram reading have either been published more than 10 years ago (Carney & Levin, 2002; Phillips et al., 2010; Vekiri, 2002) or mostly focused on offline outcome measures (Carney & Levin, 2002; Guo et al., 2020; Phillips et al., 2010; Schnotz, 2014), which may not reflect the complexity of text–diagram processing. Empirical evidence of eye movements during text– diagram reading has been obtained in several studies, but no systematic review has been conducted to synthesize these findings.
This systematic review was designed to synthesize the empirical research findings of eye-tracking studies in the domain of text–diagram science reading over the past 30 years. The three specific research questions were the following:
a. What are the readers’ eye movement patterns, and what is the relationship between these patterns and learning performance in text–diagram science reading?
b. How and under what conditions do potential influencing factors affect readers’ eye movement patterns and performance in text–diagram science reading?
c. Can interventions affect eye movement patterns and performance in text–diagram science reading?
A systematic literature review was conducted using a methodological three-step process (Guo et al., 2020).
a. Identifying and searching articles published between January 1990 and May 2020 in Scopus, Education Database (ProQuest), ERIC (ProQuest), PsycArticle, and Airiti Library (Chinese) by using a combination of multiple key terms: eye-tracking (eye-tracking OR eye-movement OR eye-fixation), diagram (figure* OR illustration* OR graph* OR picture* OR diagram*), and reading (reading OR text* OR multimedia). After duplicates were excluded, 579 English-language articles and 33 Chinese-language articles were identified during the initial search.
b. Screening and coding studies using predetermined selection criteria: (a) examined empirically; (b) published in a peer-reviewed SCIE, SSCI, TSSCI, or THCI journal; (c) measured participants’ eye movements and learning outcomes; (d) involved text–diagram reading in the science domain; (e) primarily focused on reading comprehension (studies involving problem-solving or reasoning tasks were excluded); (f) used static visual displays as materials (studies using video, audio, simulation, computer games, and interactive diagrams were excluded). After abstract and full-text screening, 50 English-language articles (including 54 studies) and one Chinese-language article were retained for inclusion in the analysis.
c. Analyzing the included studies and interpreting findings using inductive paradigmatic analysis (Polkinghorne, 1995). Based on Guo et al. (2020), categories were not predetermined by the researchers but rather were identified in an inductive manner. After descriptive information was extracted from the included studies and tentatively coded, we compared the similarities and differences of the codes and categorized them into themes to answer the research questions. Finally, the research findings were presented on the basis of these different themes.
Of the 55 studies, 32 were conducted with college students. Regarding content area, more than half of the studies involved biology and medicine (n = 34), and mechanics and physics (n = 13) and earth sciences and geography (n = 10) represented the second and third most common content areas. Articles were most commonly sourced from the journals Learning and Instruction (n = 6) and Computers & Education (n = 5). SMI RED 250 (n = 10), EyeLink 1000 (n = 10), and Tobii T120 (n = 9) were the most common models of eye-trackers used. The most commonly used eye movement indicators were total fixation duration (n = 41) and the transition between areas of interest (n = 36).
The qualitative synthesis of the studies was organized according to two major themes: cognitive processes and interventions.
a. Cognitive processes. (1) Eye movement patterns. The eye movement data support dual-coding theory (Paivio, 1986), which is the theoretical basis of CTML and ITPC and suggests that the human cognitive system is composed of two sets of subsystems: verbal and pictorial. Generally, reading is text-driven rather than diagram-driven, and readers may have different reading patterns that involve varying degrees of diagram processing and text–diagram integration. Those who more often inspect diagrams and refer to text and diagrams during reading usually performed better. (2) Reader characteristics. Higher-level readers were more likely to utilize diagram inspection and text–diagram referencing tactics compared with lower-level readers, and they generally had better learning performance; this applied to students in both academic-oriented schools (compared to non-academic-oriented schools) and high-ability students (compared to low-ability students). People with low prior knowledge (PK) tended to inspect more text than those with high PK, but no consistent pattern was discernible in diagram inspections, text–diagram referencing, and learning performance. In terms of cognitive style, visualizers fixated more on diagrams, and verbalizers fixated more on text, but the findings regarding learning performance were mixed. (3) Material characteristics and their interactions with reader characteristics. High-ability readers put more effort into difficult articles, and low-ability readers put more effort into articles with medium-low difficulty. High-ability and senior-grade readers increased diagram inspection and “diagram–item” references as the difficulty of the task increased. Readers in the second cycle of reading decreased text inspection and increased diagram inspection, reflecting adjustment by the reader. When inconsistent information appeared in material, readers typically performed more inspections, reinspections, and transitions in the beginning, but this did not affect reading performance.
b. Interventions. (1) Material manipulation. First, readers tended to fixate on signaled sections (i.e., key points were highlighted via colors, labels, and arrows) faster than on nonsignaled sections, and this effect was greater for readers with low PK. Readers performed better on signaled materials than on nonsignaled materials, but this pattern did not show among younger readers. Second, readers inspected explanatory diagrams more than decorative diagrams; however, the effect on performance by adding different types of diagrams was unclear. Third, readers transitioned more between text and diagrams when the two were physically integrated than when they were separated. The spatial contiguity effect on performance depended on whether the information was necessary. (2) Instructional interventions. Overall, instructional intervention usually affected readers’ eye movement patterns and was likely to enhance learning performance through mediators of visual behavior such as text–diagram references and diagram inspection. Reading ability and domain knowledge may be moderating variables.
Our results demonstrated that the inclusion of diagrams in science texts usually affects readers’ visual behavior but does not guarantee positive learning performance. The key to learning improvement is the effective operation of the dual channels of a reader’s cognitive system. Some reader characteristics (such as ability and PK) may play moderating roles in eye movement and learning performance. Several gaps in eye-tracking research on text–diagram science reading were identified in this review. First, more research studies should be conducted with preschool children, K-12 students, and adults. Second, future research studies should include spatial-scale eye movement indicators and distinguish between initial and late processing indicators. Third, other process measures, such as think-aloud protocols, should be included to help interpret eye movement data. Fourth, future research should focus on potential moderators (such as ability, PK, age, and content area) or the factors that contain complex research manipulations and were underrepresented in the studies in our review (such as cognitive style and type of diagram). Additionally, it is recommended that future research further explore readers’ self-regulatory processes by using eyetracking technology. Finally, the findings of this study suggest that instructors should also consider individual differences and modify their instructions to help readers master the skills of reading diagrams and text–diagram references rather than focusing exclusively on the course content.
First, the strict screening criteria used in this study may have limited the scope of this review. Second, some empirical studies were based on small samples or a single measurement, which may have limited the findings, and the data pertaining to some themes in the review came from a small number of studies; therefore, special care must be taken when interpreting the results. Third, the empirical research itself may have been somewhat biased when published, which may have inflated the effectiveness of the interventions. Fourth, this review focused on the domain of scientific reading materials, and consequently, the results may not be generalized to other content areas.