National Taiwan Normal University
教育心理學報  回首頁
Apycom jQuery Menus
07 publication date:Jun, 1974
Truncation Error and the Selection of Statistical Formula in Computer Operation
    Author:盧欽銘 Ching-Ming Lu
Research Article

 This paper used eight groups of numerical data, which were composited by some different length of digits, to demonstrate the existance of truncation error in the computer operation. The IBM 360 computer was used as the instrument. It provided 7 significant digits in the single-precision floating point mode, and 16 in double-precision flotaing point mode. The findings show when the length of digits was long, using definition formula in both single and double precision floating point mode would get more precise outputs than using the computation formula; and there was no truncation error, if the length of digits was short.





下載




The Discrimination Learning in Mentally Retarded Children as a Function of Rigidity Tendency

Copyright © 2024 Bulletin of Educational Psychology
Address: No. 162 Hoping E. Rd. Sec. 1, Taipei 10610, Taiwan, R.O.C.
Tel: (02) 77493757, Fax: (02) 23413865, Email: t05002@ntnu.edu.tw
All rights reserved. BEISU Design