This study aimed to advance the Scientific Multi-Text Reading Comprehension Assessment (SMTRCA), with a focus on the Rubric of Multi-Text Reading Comprehension Assessment (RMTRCA) designed to evaluate the extent of reading comprehension. To this end, we used scientific texts describing the dispute of the relationships between climate changes and the Three Gorges Dam and developed assessment items according to our rubric. Test items included 10 close-ended and 9 open-ended questions and were categorized into 4 subscales: information retrieval, information generalization, information interpretation, and information integration. The results of analysis showed that the cronbach’s α values were more than .9, indicating that the intra-rater consistency was well. Secondly, the Kendall’s coefficient of concordance was more than .8 and its P value was smaller than .001, denoting a consistent scoring pattern between raters. Additionally, the analysis of many-facet Rasch measurement (MFRM) and the comparison of the rating scale model (RSM) and the partial credit model (PCM) showed that the chi-square test of rater severity and threshold difficulty were significant. The infit and outfit MNSQ of the former are between 1±0.3, meaning that both severe and lenient raters can distinguish high-ability students from low-ability students more effectively. The latter means that the rating procedures involve human interpretation, evaluation and scoring processes so that it is difficult to reach a machine-like consistency level. However, this is in line with expectations of typical human judgment processes. Thirdly, most values of Cronbach’s α of test items were larger than .7 except those from information retrieval and information generalization but overall they were all within acceptable range. Finaly, confirmatory factor analysis showed that there was an acceptable goodness-of-fit among the SMTRCA. The SMTRCA accounts for .60, .66, .80, and .80 of the variance associated with the first order factors of 4 subscales.