National Taiwan Normal University
教育心理學報  回首頁
Apycom jQuery Menus
492 publication date:Dec, 2017
Developing and Validating a Scientific Multi-Text Reading Comprehension Assessment: Evidence from Texts Describing Relationships between Climate Changes and the Three Gorges Dam
    Author:Hsiao-Hui Lin, Yu-Htsuen Tzeng
Research Article

 This study aimed to advance the Scientific Multi-Text Reading Comprehension Assessment (SMTRCA), with a focus on the Rubric of Multi-Text Reading Comprehension Assessment (RMTRCA) designed to evaluate the extent of reading comprehension. To this end, we used scientific texts describing the dispute of the relationships between climate changes and the Three Gorges Dam and developed assessment items according to our rubric. Test items included 10 close-ended and 9 open-ended questions and were categorized into 4 subscales: information retrieval, information generalization, information interpretation, and information integration. The results of analysis showed that the cronbach’s α values were more than .9, indicating that the intra-rater consistency was well. Secondly, the Kendall’s coefficient of concordance was more than .8 and its P value was smaller than .001, denoting a consistent scoring pattern between raters. Additionally, the analysis of many-facet Rasch measurement (MFRM) and the comparison of the rating scale model (RSM) and the partial credit model (PCM) showed that the chi-square test of rater severity and threshold difficulty were significant. The infit and outfit MNSQ of the former are between 1±0.3, meaning that both severe and lenient raters can distinguish high-ability students from low-ability students more effectively. The latter means that the rating procedures involve human interpretation, evaluation and scoring processes so that it is difficult to reach a machine-like consistency level. However, this is in line with expectations of typical human judgment processes. Thirdly, most values of Cronbach’s α of test items were larger than .7 except those from information retrieval and information generalization but overall they were all within acceptable range. Finaly, confirmatory factor analysis showed that there was an acceptable goodness-of-fit among the SMTRCA. The SMTRCA accounts for .60, .66, .80, and .80 of the variance associated with the first order factors of 4 subscales.


下載


關鍵詞: Confirmatory factor analysis, Many-facet Rasch measurement, Partial credit model, Rating scale model, Scientific multiple text reading comprehension assessment


Integrating Christian Spirituality into Counseling: An analysis of family relationship counseling process

Copyright © 2024 Bulletin of Educational Psychology
Address: No. 162 Hoping E. Rd. Sec. 1, Taipei 10610, Taiwan, R.O.C.
Tel: (02) 77493757, Fax: (02) 23413865, Email: t05002@ntnu.edu.tw
All rights reserved. BEISU Design