Integrating Technology in Mathematics Instruction on Grade School Academic Achievement in Taiwan: A Meta-Analysis Author:Yuen-Kuang Clif Liao, Yung-Hsin Chen
Research Article
Jonassen (2000) suggested that learning with information technology (IT) involves three stages: learning from computers, learning about computers, and learning with computers. The development of learning with IT in Taiwan, beginning with computer-assisted instruction, loosely follows these three stages (Chang, 2002). Integrating IT into instruction is the first stage of learning with computers. Previous meta-analyses (Cheung & Slavin, 2013; Demir & Basol, 2014; Hartley, 1977; Li & Ma, 2011; Slavin et al., 2008, 2009; Slavin & Lake, 2008; Rakes et al., 2010; Sokolowski1 et al., 2015; Young, 2017) regarding the effectiveness of integrating IT into mathematics instruction (ITMI) have reported positive effects compared with nonITMI classes; effect sizes (ESs) were 0.07–0.9. These meta-analyses also concluded that variables such as publication year, publication type, learning stage, research design, intervention duration, technology type, assessment tool, instructional approach, and mathematics topic might influence the overall ES. A total of 19 meta-analyses investigating the effects of integrating IT into student learning in Taiwan have been performed; however, none of these studies were specifically focused on mathematics.
In this study, we performed a meta-analysis to synthesize existing research regarding the effects of integrating IT with mathematics instruction on the academic achievement of elementary and secondary school students in Taiwan. We searched the National Digital Library of Theses and Dissertations in Taiwan, Airiti Library, Index to Taiwan Periodical Literature System, Scopus, EBSCOhost, ProQuest, ScienceDirect, and Web of Science databases for relevant studies by using keywords “math,” “technology,” “computer,” and “achievement” and gathered 282 studies (with 20,190 participants). We then transformed the quantitative data into ESs. After the calculation of the ES for each study, six studies with unusually large ESs were excluded in further analyses (Lipsey & Wilson, 2001). Thus, the total number of studies was 276.
We used the meta-analytic approach suggested by Borenstein et al. (2009), Hedge and Olkin (1985), and Lipsey and Wilson (2001). The ES was defined as the mean difference between the treatment and control groups divided by the pooled standard deviations. The criteria for inclusion of studies were as follows: (1) Studies must compare the effects of ITMI and traditional instruction (TI) on student academic achievement in mathematics; (2) participants must be elementary or secondary school students; (3) the research design must include treatment and control groups, and the treatment group must receive treatment that involved integrating IT into instruction; (4) studies must provide adequate quantitative data for both treatment and control groups so that the ES could be estimated; (5) the number of participants for both ITMI and TI groups must be over 15; studies were excluded if the overall participants were less than 30; (6) the study participants must be Taiwanese students; (7) studies must be published between 1993 and 2019.
On the basis of previous meta-analyses, the moderating effects of 13 variables were investigated. These variables were classified into three categories: (1) research characteristics, including learning stage, topic in mathematics, type of publication, and year of publication; (2) research methods, including study design, instructor bias, reliability of assessment tools, number of treatment class sessions, and sample size; and (3) research design, including the instructional approach for the treatment group, learning device for the student, method of integration, and timing of integration. Hedges’ g was applied for ES calculation. If studies provided only an F-ratio value or a t value, equivalent formulae were used. In addition, the homogeneity test presented by Borenstein et al. (2009) was used to aggregate and analyze the ESs for all 276 studies. The significance of the mean ES was evaluated by its 95% confidence interval (95% CI). A significantly positive (+) mean ES indicated that the results favored the ITMI group, whereas a significantly negative (−) ES indicated that the results favored the TI group. The results of this metaanalysis revealed that the overall mean ESs were 0.32 (95% CI = 0.30-0.35, z = 24.31, p < .0001) and 0.35 (95% CI = 0.310.39, z = 17.54, p < .0001) for the fixed-effects model and random-effects model, respectively. An effect is said to be small when ES < 0.2, medium when ES ≈ 0.5, and large when ES > 0.8 (Cohen, 1992). The results indicated that integrating technology into mathematics instruction had a significant small to medium positive effect compared with TI on the academic achievement of Taiwanese students. Moreover, the homogeneity test was significant (QT = 582.37, p < .0001), indicating that the findings did not share a common ES. A series of moderator analyses were then performed. The analysis results revealed that 10 of the 13 moderating variables selected in this study had statistically significant effects on the overall mean ES. The findings were as follows: (1) The mean ES was higher for elementary school students than high school students. (2) Elementary school students had a higher mean ES for the topics “Number and Quantity” and “Geometry” than did junior high students, but junior high students had a greater mean ES for “Algebra” than did elementary school students. (3) Journal articles had higher mean ES than did unpublished papers. (4) Studies published in 2004–2009 had higher mean ES than did those published in 2010–2014. (5) Studies that applied a pretest–posttest control group design had higher mean ES than did those that applied a quasiexperimental design. (6) Studies with small sample sizes had a higher mean ES than did those with large sample sizes. (7) Studies with less than 15 overall class sessions had a higher mean ES than did those with more than 15 class sessions. (8) Individual learning had a higher mean ES than did whole-class or small group learning. (9) Studies using immediate response system learning devices had a higher mean ES than did those using traditional paper and pencil or mixed devices. (10) Integrating technology in class had a higher ES compared with integrating technology before class or after class. In this meta-analysis, Funnel plot, Rosenthal’s (1979) fail-safe Ns and Orwin’s (1983) fail-safe Ns were applied to examine publication bias. The Funnel plot indicated that the studies were distributed symmetrically. Rosenthal’s and Orwin’s fail-safe Ns were 5580 and 8653, respectively—higher than the critical value of 5K+10. The results of all three methods suggest that there was no publication bias. On the basis of these findings, the implications of this meta-analysis are outlined as followings: (1) Education planners in Taiwan should provide adequate funding supporting ITMI and should encourage elementary and secondary school mathematics teachers to implement ITMI in their classes, particularly for students who require remedial instruction; (2) education planners in Taiwan should encourage mathematics educators to develop instructional programs, teaching methods, and learning materials for ITMI classes; (3) fewer than 15 ITMI class sessions have the strongest effects; and (4) future meta-analyses should examine the effects of varied instructional approaches (e.g., collaborative learning, problem-based learning, project-based learning, and self-regulated learning) alongside ITMI. Finally, this study is the first meta-analysis to focus on the effects of technology integration into mathematics instruction on the academic achievement of Taiwanese students. By examining empirical research on this topic, this meta-analysis provides research-based evidence of the positive outcomes of using technology in mathematics classes as well as how those effects are influenced by moderating variables. The findings provide education policymakers and mathematics teachers with valuable insights into methods of improving mathematics achievement.
Integrating Technology in Mathematics Instruction on Grade School Academic Achievement in Taiwan: A Meta-Analysis
下載
