Re: Problem about SGL with temperature unit

[Xu]

Thanks a lot, Jean-Pierre.
From what you posted above, I am quite sure that it is a defect of the
conversion routine for the temperature unit. The reference temperature
for degC (273.15) and/or degF is stored internally as a DBL.
273.15(SGL) - 273.15(DBL) = -6.10352E-6;
274.15(SGL) - 273.15(DBL) = 0.94;
.
To solve this problem, there should be two reference temperatures,
273.15(DBL) and 273.15(SGL), for DBL and SGL with temperature unit
respectively.

Re: Problem about SGL with temperature unit

[Xu]

Thank you all for reply.
You are right, 1 degC is represented as 274.15 internally.

BUT how the internal number is converted to degC?
It shows that 274.150 (SGL) - 273.150 (SGL) could have an exact binary
representation 1 (with exponent = 0 and mantissa = 1). For examples:

274.150 (SGL) - 273.150 (SGL) results 1,
exponent = 0
mantissa = 1

275.150 (SGL) - 273.150 (SGL) results 2,
exponent = 1
mantissa = 1

276.150 (SGL) - 273.150 (SGL) results 3,
exponent = 1
mantissa = 1.5

.

Re: Problem about SGL with temperature unit

[altenbach]

Right-click on each, -- format and precision and  select
- floating point
- 2 digits of precision (or whatever the quality of your data is, i
doubt it is microdegrees)
- unselect hide trailing zeroes if desired.

(A SGL number is rather inaccurate, and e.g. 1 cannot be represented
exactly in binary with the number of bits used for the mantissa.)

Re: Problem about SGL with temperature unit

[Jean-Pierre Drolet]

Quote of the year:i
 1 cannot be represented exactly in binary...
/i
That's too funny altenbach! How many bits of precision do you need to
represent 1 exactly in binary??
=A6=AC

Re: Problem about SGL with temperature unit

[altenbach]

Sorry, this did not come out quite right! Thanks J-P.

... Zero is even harder. :-)

You've probably seen the T-shirt: There are 10 kinds of people: those
who understand binary and those who don't or similar...

 back to the problem -

Anyway, the problem here is actually with the associated units, which
are internally stored as Kelvins and only converted to degC for
display.

Set it to Kelvin and small enough integers will be accurate. In your
case, 1 degC is actually represented as a fractional number (274.15)
internally and does not have an exact binary representation.