On 24 December 2010 14:25, Marnen Laibow-Koser <li...@ruby-forum.com> wrote:
> Colin Law wrote in post #970447:
>> On 24 December 2010 03:59, Marnen Laibow-Koser <li...@ruby-forum.com>
>> wrote:
>>>>
>>>> x = BigDecimal.new('1')
>>>> => #<BigDecimal:102c078,'0.1E1',4(8)>
>>>> y = BigDecimal.new('3')
>>>> => #<BigDecimal:10388f0,'0.3E1',4(8)>
>>>> z = x / y
>>>> => #<BigDecimal:103680c,'0.33333333E0',8(16)>
>>>>
>>>> In the case of float we have 16 decimal digits of precision, but in the
>>>> case of BigDecimal we have only eight digits of precision.
>>>
>>> Are you sure? Or are only eight digits displayed?
>>
>> $ rails console
>> Loading development environment (Rails 3.0.3)
>> ruby-1.9.2-p0 > x=BigDecimal('1')/BigDecimal('3')
>> => #<BigDecimal:ad823a4,'0.33333333E0',8(16)>
>> ruby-1.9.2-p0 > x*BigDecimal('3')
>> => #<BigDecimal:ad2b270,'0.99999999E0',8(20)>
>> ruby-1.9.2-p0 > x*BigDecimal('3') == 1
>> => false
>> ruby-1.9.2-p0 > x*BigDecimal('3') - 1
>> => #<BigDecimal:aa4fdf8,'-0.1E-7',4(20)>
>>
>
>
> Interesting. However, the BigDecimal constructor allows specification
> of the minimum number of significant figures, so in this case we could
> have done BigDecimal.new('1', 16).
ruby-1.9.2-p0 > x = BigDecimal.new('1', 16)/BigDecimal.new('3',16)
=> #<BigDecimal:a146bbc,'0.33333333E0',8(16)>
ruby-1.9.2-p0 > x*BigDecimal('3',16) - 1
=> #<BigDecimal:a114acc,'-0.1E-7',4(20)>
It doesn't make any difference how many digits you have, 1/3 as a
decimal number multiplied by 3 will never be equal to 1.
I am not sure why the above still gives only 8 digits accuracy
however, I would have thought it aught to be 16.
Colin
--
You received this message because you are subscribed to the Google Groups "Ruby
on Rails: Talk" group.
To post to this group, send email to rubyonrails-t...@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
