Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
565 views
in Technique[技术] by (71.8m points)

ruby - Why are my BigDecimal objects initialized with unexpected rounding errors?

In Ruby 2.2.0, why does:

BigDecimal.new(34.13985572755337, 9)

equal 34.0 but

BigDecimal.new(34.13985572755338, 9)

equal 34.1398557?

Note that I am running this on a 64 bit machine.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

Initialize with Strings Instead of Floats

In general, you can't get reliable behavior with Floats. You're making the mistake of initializing your BigDecimals with Float values instead of String values, which introduces some imprecision right at the beginning. For example, on my 64-bit system:

float1 = 34.13985572755337
float2 = 34.13985572755338

# You can use string literals here, too, if your Float can't be properly
# represented. For example:
#
#    BigDecimal.new("34.13985572755337", 9)
#
# would be safer, but Float#to_s works fine with the provided corpus.
bd1 = BigDecimal.new(float1.to_s, 9)
bd2 = BigDecimal.new(float2.to_s, 9)

bd1.to_s
#=> "0.3413985572755337E2"
bd2.to_s
#=> "0.3413985572755338E2"

bd1.to_f == float1
#=> true
bd2.to_f == float2
#=> true

This is one of those cases where the internal representation of the arguments matter. Therefore, your mileage will vary depending on how you initialize your object.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...