IDL double与float的精度检验

pro test
  compile_opt idl2
  a = 100.000000002
  b = 100.000000000
  help, a
  help, b
  if a gt b then $
    print, '100.000000002 gt 100.000000000'
  if a eq b then $
    print, '100.000000002 eq 100.000000000'
  if a lt b then $
    print, '100.000000002 lt 100.000000000'
  a = 100.000000002d
  b = 100.000000000d
  help, a
  help, b
  if a gt b then $
    print, '100.000000002d gt 100.000000000d'
  if a eq b then $
    print, '100.000000002d eq 100.000000000d'
  if a lt b then $
    print, '100.000000002d lt 100.000000000d'
  print, ''
  a = double(100.000000002)
  b = double(100.000000000)
  help, a
  help, b
  if a gt b then $
    print, 'double(100.000000002) gt double(100.000000000)'
  if a eq b then $
    print, 'double(100.000000002) eq double(100.000000000)'
  if a lt b then $
    print, 'double(100.000000002) lt double(100.000000000)'
  print, ''
  a = 100.000000002*1.0d
  b = 100.000000000*1.0d
  help, a
  help, b
  if a gt b then $
    print, '100.000000002*1.0d gt 100.000000000*1.0d'
  if a eq b then $
    print, '100.000000002*1.0d eq 100.000000000*1.0d'
  if a lt b then $
    print, '100.000000002*1.0d lt 100.000000000*1.0d'
  print, ''
 
  a = double('100.1234567892')
  b = double('100.1234567890')
  help, a
  help, b
  if a gt b then $
    print, '100.1234567892*1.0d gt 100.1234567890*1.0d'
  if a eq b then $
    print, '100.1234567892*1.0d eq 100.1234567890*1.0d'
  if a lt b then $
    print, '100.1234567892*1.0d lt 100.1234567890*1.0d'
  print, ''
end
 
运行结果
A               FLOAT     =       100.000
B               FLOAT     =       100.000
100.000000002 eq 100.000000000
 
A               DOUBLE    =        100.00000
B               DOUBLE    =        100.00000
100.000000002d gt 100.000000000d
 
A               DOUBLE    =        100.00000
B               DOUBLE    =        100.00000
double(100.000000002) eq double(100.000000000)
 
A               DOUBLE    =        100.00000
B               DOUBLE    =        100.00000
100.000000002*1.0d eq 100.000000000*1.0d
 
A               DOUBLE    =        100.12346
B               DOUBLE    =        100.12346
str100.1234567892 gt str100.1234567890
 
初始的数据如果不是double型的,后面转为double型是没有用的,因为精度已经丢失
从a=100.000000002 可知,虽然是小数点后9位, 但IDL认为是float型
a=100.000000002d 可知,加上d之后,IDL认为是double型
从a = double('100.1234567892') 可知,读入一个位数很多的字符串,将其转为double,可以最大限度的保持后续计算的精度
 
 
 
原文地址:https://www.cnblogs.com/lqqgis/p/13626557.html