Re: Suggestion needed on test failures due to double arithmetics
On Thu, Nov 25, 2021 at 01:45:49PM +0100, Giulio Paci wrote: > Moreover I am still wondering if the compiler behavior is correct in this > case and why it is so unstable. It's correct when you don't care about the amount of precision, and it's unstable for the reasons described in gcc(1) for the options you mentioned, e.g. "This option prevents undesirable excess precision on machines such as the 68000 where the floating registers (of the 68881) keep more precision than a "double" is supposed to have. Similarly for the x86 architecture. For most programs, the excess precision does only good, but a few programs rely on the precise definition of IEEE floating point.", as in different circumstances the temporary values will have or not have the x87 80-bit precision, leading to different calculation results. -- WBR, wRAR signature.asc Description: PGP signature
Re: Suggestion needed on test failures due to double arithmetics
Il gio 25 nov 2021, 13:21 Andrey Rahmatullin ha scritto: > On Thu, Nov 25, 2021 at 01:13:20PM +0100, Giulio Paci wrote: > > The double values refer to timing information. The specific format, > > known as CTM, stores information in seconds in decimals (e.g. "30.66" > > seconds) from the beginning of the stream. > > The failing tool reads this information into double variables > Sounds like it just shouldn't read this data into a float type but use > some fixed-point data type instead. > This is also my opinion (and already suggested upstream), although it would make the code a little bit less readable. Moreover I am still wondering if the compiler behavior is correct in this case and why it is so unstable. Apart from this corner case (which in my opinion should also work), I have not seen bad assumptions about double arithmetics in the rest of the failing tool. Bests, Giulio
Re: Suggestion needed on test failures due to double arithmetics
Hi Paul, On Thu, Nov 25, 2021 at 3:24 AM Paul Wise wrote: > Giulio Paci wrote: > > > 3) what is the most appropriate solution. > > As I understand it, floating point values should not be compared > without some kind of accuracy/precision factor. Zero idea about the > best reference for how to do it correctly, but here is a random one: > > https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/ Thanks for this link. It is a very great resource and summarizes very well what I already know about double/float and much more. Since the test case is dealing with timings, I think the most related article is [1]. However even after reading that article it seems to me that in this case it should be reasonable to expect stable behavior of those operators. I have uploaded simplified code that showcase the issue and some of the instabilities [2]. The code seems to behave as if the last value is different from the other 3, supposed equal values. I am not even sure what I am seeing in the debugger, since most of the values are optimized out (and I am not so skilled with debuggers). [1] https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/ [2] https://pastebin.com/embed_js/T3g560UV Bests, Giulio
Re: Suggestion needed on test failures due to double arithmetics
On Thu, Nov 25, 2021 at 01:13:20PM +0100, Giulio Paci wrote: > The double values refer to timing information. The specific format, > known as CTM, stores information in seconds in decimals (e.g. "30.66" > seconds) from the beginning of the stream. > The failing tool reads this information into double variables Sounds like it just shouldn't read this data into a float type but use some fixed-point data type instead. -- WBR, wRAR signature.asc Description: PGP signature
Re: Suggestion needed on test failures due to double arithmetics
On Thu, Nov 25, 2021 at 8:54 AM Andrey Rahmatullin wrote: > > On Wed, Nov 24, 2021 at 06:38:07PM +0100, Giulio Paci wrote: > > Dear mentors, > > while updating SCTK package I enabled the execution of the test suite > > which was previously disabled. The tests are working fine on x86_64 > > architecture, but a couple of them are failing on i386. > > After investigation [1] I found out that tests are failing because they > > rely on the assumptions that, when a and b have the same double value: > > 1) "a < b" is false; > > 2) "a - b" is 0.0. > What do they actually test, why do they use these assumptions? SCTK is a toolkit to evaluate speech recognition (and other related tasks) tools performance. These tools usually read audio streams and produce simple text files containing the transcriptions and time information (relative to the stream) to synchronize the transcription to the stream. These files are very similar to video subtitles files. The SCTK compares two textual files (usually one is a manually created file and the other is created by an automatic tool) to score how different these outputs are. The tests are checking that SCTK produces the same score reports when provided with the same input files. The double values refer to timing information. The specific format, known as CTM, stores information in seconds in decimals (e.g. "30.66" seconds) from the beginning of the stream. The failing tool reads this information into double variables and, to simplify, it compares "up to when the timings in one file is less than the timings in the other files. If it exceeds or is the same, it checks the difference". In this kind of application you are not usually going beyond what you can store uncompressed on a filesystem in PCM. So, even assuming audio samples of 1 byte, int64 should be a reasonable type to store timings (in samples, rather then seconds). But I understand that doing so would complicate the logic of the tool, especially since it is very unlikely that math approximation would be an issue. To be honest I did not expect the corner case above would fail since it is comparing a value against another value that should just be the same. I have uploaded simplified code that showcase the issue and some of the instabilities [1]. The code seems to behave as if the last value is different from the other 3, supposed equal values. [1] https://pastebin.com/embed_js/T3g560UV Bests, Giulio
Re: Suggestion needed on test failures due to double arithmetics
On Wed, Nov 24, 2021 at 06:38:07PM +0100, Giulio Paci wrote: > Dear mentors, > while updating SCTK package I enabled the execution of the test suite > which was previously disabled. The tests are working fine on x86_64 > architecture, but a couple of them are failing on i386. > After investigation [1] I found out that tests are failing because they > rely on the assumptions that, when a and b have the same double value: > 1) "a < b" is false; > 2) "a - b" is 0.0. What do they actually test, why do they use these assumptions? -- WBR, wRAR signature.asc Description: PGP signature
Re: Suggestion needed on test failures due to double arithmetics
Giulio Paci wrote: > 3) what is the most appropriate solution. As I understand it, floating point values should not be compared without some kind of accuracy/precision factor. Zero idea about the best reference for how to do it correctly, but here is a random one: https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/ -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Suggestion needed on test failures due to double arithmetics
Dear mentors, while updating SCTK package I enabled the execution of the test suite which was previously disabled. The tests are working fine on x86_64 architecture, but a couple of them are failing on i386. After investigation [1] I found out that tests are failing because they rely on the assumptions that, when a and b have the same double value: 1) "a < b" is false; 2) "a - b" is 0.0. Unfortunately, these assumptions do not always hold on i386 as the behavior of those operators seem to change according to compilation flags (e.g., optimization or debug flags) or the surrounding instructions (e.g., if a function is invoked nearby or not). I think the reason for this behavior is probably related to gcc options -mfpmath [2] and -ffloat-store [3]. Using -ffloat-store option indeed seems to fix the issue. However I am wondering: 1) if enabling -ffloat-store is an acceptable solution; 2) if this is a compiler bug (I always expect approximated results with double arithmetics, but the above assumptions seem very basic to me, considering that they cover the special case where a and b are equal); 3) what is the most appropriate solution. Do you have any suggestion? I just asked the same question upstream, but I would appreciate your opinion as well. Bests, Giulio [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981030#39 [2] https://gcc.gnu.org/onlinedocs/gcc-10.3.0/gcc/x86-Options.html#x86-Options [3] https://gcc.gnu.org/onlinedocs/gcc-10.3.0/gcc/Optimize-Options.html#Optimize-Options
