[go-nuts] Re: Inconsistent rounding with float printf ?
https://play.golang.org/p/j5HKxitS-Z6 See https://0.30004.com/ On Friday, December 6, 2019 at 1:25:01 AM UTC-8, Christophe Meessen wrote: > > I have noticed that printf performs an apparently inconsistent rounding of > floating point values. > > I divide a big number by 1000 and printf the resulting value with "%.1f". > Here is the code: https://play.golang.org/p/e7dD3c6IHq2 > > I would expect the rounding rule to be "round away from zero" as defined > here: https://math.stackexchange.com/a/2252888/33796 > In this case 0.5 is rounded to 1 (or 0.05 to 0.1) and -0.5 to -1 (or -0.05 > to -0.1). > > Here is what I see with golang 1.13.4: > > 999000/1000 = 999.0 > 999050/1000 = 999.0 // expected 999.1 > 999100/1000 = 999.1 > 999150/1000 = 999.1 // expected 999.2 > 999200/1000 = 999.2 > 999250/1000 = 999.2 // expected 999.3 > 999300/1000 = 999.3 > 999350/1000 = 999.4 > 999400/1000 = 999.4 > 999450/1000 = 999.5 > 999500/1000 = 999.5 > 999550/1000 = 999.5 // expected 999.6 > 999600/1000 = 999.6 > 999650/1000 = 999.6 // expected 999.7 > 999700/1000 = 999.7 > 999750/1000 = 999.8 > 999800/1000 = 999.8 > 999850/1000 = 999.9 > 00/1000 = 999.9 > 50/1000 = 1000.0 > -50/1000 = -1000.0 > -00/1000 = -999.9 > -999850/1000 = -999.9 > -999800/1000 = -999.8 > -999750/1000 = -999.8 > -999700/1000 = -999.7 > -999650/1000 = -999.6 // expected -999.7 > -999600/1000 = -999.6 > -999550/1000 = -999.5 // expected -999.6 > -999500/1000 = -999.5 > -999450/1000 = -999.5 > -999400/1000 = -999.4 > -999350/1000 = -999.4 > -999300/1000 = -999.3 > -999250/1000 = -999.2 // expected -999.3 > -999200/1000 = -999.2 > -999150/1000 = -999.1 // expected -999.2 > -999100/1000 = -999.1 > -999050/1000 = -999.0 // expected -999.1 > > > Is this actual rounding the result of a rule or is it pseudo random ? > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/138f55d2-e7a6-4f12-85fd-d14bc42699f7%40googlegroups.com.
Re: [go-nuts] Re: Inconsistent rounding with float printf ?
On Fri, Dec 6, 2019 at 6:04 PM Christophe Meessen wrote: > I can't change expectations. It is to convert a byte count into a human > readable byte count ( with kB, MB, ... units). So it was an XY problem? No floating point operations are necessary to do that. Also, check several existing libraries, for example this one: https://github.com/dustin/go-humanize -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-VnzYWABrP8ET7KkqeRrjNzH6kxoE8Sjwog8mcqbgTxqA%40mail.gmail.com.
[go-nuts] Re: Inconsistent rounding with float printf ?
I can't change expectations. It is to convert a byte count into a human readable byte count ( with kB, MB, ... units). I found out that I can produce the expected result by using math.Round. See here https://play.golang.org/p/UorDwbKlLj5 For my use case, I ended up converting "manually" the integer into a .1f float by using an array. The resulting code is 12 time faster than the one using the printf and float64 operations. It also need only one allocation to convert the byte array into a string. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/425dd75e-7d58-44cd-9e99-b197b81c078e%40googlegroups.com.
