Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1 ID: 51396 Updated by: paj...@php.net Reported by: codeslinger at compsalot dot com Summary: Math is Unreliable Status: Feedback Type: Bug Package: Math related Operating System: any PHP Version: Irrelevant New Comment: There is no need to write a novel about this possible issue. Please ask the Mandriva guys to actually report a bug here (in this report for example), post their patch(es), a reproduce script with a description of their architecture (compiler, compiler versions, OS, etc.). Everything else are unnecessary information. Now, what Johannes said is still valid: - we don't support custom version of PHP (aka patched) - that includes suhoshin patch - Debugger/optimizer (zend, xdebug, etc.) has to be removed to reproduce a problem. Thanks for your feedback. Previous Comments: [2010-05-23 09:38:47] codeslinger at compsalot dot com In response to johan...@php.net above. Let me just very politely reiterate that I originally encountered this bug on the STOCK WINDOWS BUILD from php.net, therefore to dismiss this bug because it is ubuntu specific is not a valid reason. This bug is very hard to reproduce, but none-the-less occurs far too frequently under real-world conditions. The actual values required to reproduce this bug appear to vary depending on what version of php is being tested and what os/cpu it is running on. Given the randomly variant nature of this problem, I suspect that the only way you could properly test this bug and have confidence in the outcome is if you loop through every possible floating point value and convert it to a string. You would also need to do this on multiple platforms. Of course that would probably take a few years... so perhaps code inspection and some alerts would be a better option. Testing numbers selected at random is not likely to succeed, the number space is too large and you can't account for possible biases in the random number generator -- perhaps it only returns numbers which php sees as valid. Other people have reported this bug, see for instance #49764 and also the above comments about Mandriva Devs creating a patch to fix this bug. I do not regard millions of Ubuntu users as unimportant, or irrelevant. The severity of the consequences of this bug ought to be sufficient justification for a little bit of extra effort being expended -- even if the problem had been caused by ubuntu patches which it wasn't. People who are affected by this bug may not always realize what the problem is. This bug is probably underreported by quite a bit. Also as pointed out earlier the majority of php web pages do not do very much floating point math and therefore would not encounter this bug. In the discussion above it appears that there is some obscure case for which the number conversion is off-by-one. Pajoye thinks he has a fix. The fact that this afflicts Financial Transactions -- as reported by multiple people -- makes this a gravely serious bug... so why then is it so exasperatingly difficult to get the php dev community to take this problem seriously? In case you are wondering why it took me so long to respond to Johannes, it's because I had to cool off first I really am trying not to be overwrought, honest, ;-) [2010-04-14 04:37:08] john dot smith dot 1964 at gmail dot com The folk at Mandriva saw the same problem, figured it out and submitted a patch to PHP. I'm confused, because we're certainly still seeing the problem: https://qa.mandriva.com/show_bug.cgi?id=37171 "In PHP on Mandriva 2008, some float to string conversions return "0.0:" !! In critical software, this can lead to major loss of data or inconsistant results." [2010-04-13 03:36:47] john dot smith dot 1964 at gmail dot com I am seeing this bug consistently on standard Windows builds such as 5.2.4 and 5.2.13. Our Server is: Windows NT 5.2 build 3790 Sample code is simple: On 5.2.4 it will result in "1450.:0" every time. On 5.2.4, other such *funny* values are 1701,1821,1951,2091,2101,2111,2121,2341,2351... On 5.2.13,the numbers 1700 and 1900 consistently return "colon-ized" results. This is a especially problematic, because 1700 and 1900 occur more frequently in our eCommerce app! This issue is a real problem for us. It has been touched on (but not solved) in at least Bugs 46376, 47716, 47304 and 47418. [2010-03-27 14:19:44] johan...@php.net You are mentioned this version information: php -v PHP 5.2.4-2ubuntu5.10 with Suhosin-Patch 0.9.6.2 (cli) (built: Ja
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1 ID: 51396 Updated by: johan...@php.net Reported by: codeslinger at compsalot dot com Summary: Math is Unreliable Status: Feedback Type: Bug Package: Math related Operating System: any PHP Version: Irrelevant New Comment: You are mentioned this version information: php -v PHP 5.2.4-2ubuntu5.10 with Suhosin-Patch 0.9.6.2 (cli) (built: Jan 6 2010 22:01:14) Copyright (c) 1997-2007 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies with Xdebug v2.0.3, Copyright (c) 2002-2007, by Derick Rethans with Zend Debugger v5.2.15, Copyright (c) 1999-2008, by Zend Technologies This version is very different from the versions we provide. a) Ubuntu adds some custom patches b) Suhosin does major changes to the engine c) Xdebug as well as Zend Debugger do changes to our executor unit. All these components aren't supported here. Previous Comments: [2010-03-27 12:50:58] codeslinger at compsalot dot com One further note, in the repro above, it has to be exactly 16 nines. by adding or removing a 9, it does not fail. Also, as far as I know, all of the failures have been on 32bit Intel cpu's. This probably will not fail on a 64bit cpu. [2010-03-27 12:22:12] codeslinger at compsalot dot com well, it's hard to prove a negative. but I have run a bunch of tests on the Linux snapshot and it appears to be working fine. A VERY BIG THANKYOU I did not find any windows snapshots, the latest is 5.2.13 from Feb 24. based on the symptoms, my initial assumption was that this was caused by some kind of array corruption, this turned out to be wrong. the actual repro can be boiled down to one line... :-) echo (string) (double) -0.0; With the caveat that there are other values which also trigger this. [2010-03-26 21:37:12] paj...@php.net Please try using this snapshot: http://snaps.php.net/php5.2-latest.tar.gz For Windows: http://windows.php.net/snapshots/ [2010-03-26 21:32:51] codeslinger at compsalot dot com as far as the low incidence of occurrence and the millions of users not seeing it. I've said all along that it is hard to reproduce. But when dealing with financial transactions, that is not good enough, it only takes one mistake to have a huge problem on your hands. Out of all of those millions of users, I'd venture to say that the very overwhelming majority are using php for string processing not number crunching. And in many cases where it does show up such as positioning something on a web page, it would be easy to shrug off. So there is no way to know how often this happens in the wild, based on user feedback. [2010-03-26 21:02:47] codeslinger at compsalot dot com The billing program was failing on multiple computers at multiple locations, it failed on XP and Windows 2000 with various cpus. Those were customer sites! I reproduced the problem on a vmware setup with windows 2000. This snowflake program is failing on Ubuntu Hardy 8.0.4 with all of the updates. This is the stock php that comes with Ubuntu. This is a Pentium M 32bit Laptop. I've never experienced a memory error on this computer and the fact of it's consistency would argue against this being some kind of hardware issue. Here is what I get when I run the simplefail in the default config. php -v PHP 5.2.4-2ubuntu5.10 with Suhosin-Patch 0.9.6.2 (cli) (built: Jan 6 2010 22:01:14) Copyright (c) 1997-2007 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies with Xdebug v2.0.3, Copyright (c) 2002-2007, by Derick Rethans with Zend Debugger v5.2.15, Copyright (c) 1999-2008, by Zend Technologies == php simplefail.php Selected onepair.txt, Found 1 items (string)(double) -0.1 === -0.1 which of course is correct But when we convert the value in an array it fails Conversion Error: PHP Math idx = 0 || '-0.1' !== '-0.0:' = THANK YOU I REALLY APPRECIATE THE HELP The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/bug.php?id=51396 -- Edit this bug report at http://bugs.php.net/bug.php?id=51396&edit=1
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1 ID: 51396 Updated by: paj...@php.net Reported by: codeslinger at compsalot dot com Summary: Math is Unreliable Status: Feedback Type: Bug Package: Math related Operating System: any PHP Version: Irrelevant New Comment: Please try using this snapshot: http://snaps.php.net/php5.2-latest.tar.gz For Windows: http://windows.php.net/snapshots/ Previous Comments: [2010-03-26 21:32:51] codeslinger at compsalot dot com as far as the low incidence of occurrence and the millions of users not seeing it. I've said all along that it is hard to reproduce. But when dealing with financial transactions, that is not good enough, it only takes one mistake to have a huge problem on your hands. Out of all of those millions of users, I'd venture to say that the very overwhelming majority are using php for string processing not number crunching. And in many cases where it does show up such as positioning something on a web page, it would be easy to shrug off. So there is no way to know how often this happens in the wild, based on user feedback. [2010-03-26 21:02:47] codeslinger at compsalot dot com The billing program was failing on multiple computers at multiple locations, it failed on XP and Windows 2000 with various cpus. Those were customer sites! I reproduced the problem on a vmware setup with windows 2000. This snowflake program is failing on Ubuntu Hardy 8.0.4 with all of the updates. This is the stock php that comes with Ubuntu. This is a Pentium M 32bit Laptop. I've never experienced a memory error on this computer and the fact of it's consistency would argue against this being some kind of hardware issue. Here is what I get when I run the simplefail in the default config. php -v PHP 5.2.4-2ubuntu5.10 with Suhosin-Patch 0.9.6.2 (cli) (built: Jan 6 2010 22:01:14) Copyright (c) 1997-2007 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies with Xdebug v2.0.3, Copyright (c) 2002-2007, by Derick Rethans with Zend Debugger v5.2.15, Copyright (c) 1999-2008, by Zend Technologies == php simplefail.php Selected onepair.txt, Found 1 items (string)(double) -0.1 === -0.1 which of course is correct But when we convert the value in an array it fails Conversion Error: PHP Math idx = 0 || '-0.1' !== '-0.0:' = THANK YOU I REALLY APPRECIATE THE HELP [2010-03-26 17:25:40] ahar...@php.net Well, it is the next character after '9', and the character string is built up in zend_dtoa() by adding a value L (presumably intended to be in the range 0..9) to '0'. If L somehow ends up being 10, you'd get a colon. With the values that are apparently causing problems (the problematic value in onepair.txt is 0.0167...), it does kind of look like a rounding issue to me, although I've no idea why it's not being triggered by more than two or three users in that case. [2010-03-26 17:18:22] ras...@php.net Bad memory perhaps? But why consistently a ':' ? [2010-03-26 16:57:15] ahar...@php.net JFTR, I was also unable to reproduce the failure case from any of the data files on both 32-bit and 64-bit Linux builds and a 32-bit Windows build. I've got a couple of boxen crunching away generating random doubles and converting them to strings in what seems to be the sort of range that causes problems (one 32-bit Linux, one 64-bit Linux): nothing yet after a couple of billion iterations. In short, I don't have a clue what it could be either, but I'll keep the random double generation going a while longer just in case it hits paydirt. The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/bug.php?id=51396 -- Edit this bug report at http://bugs.php.net/bug.php?id=51396&edit=1
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1 ID: 51396 Updated by: ahar...@php.net Reported by: codeslinger at compsalot dot com Summary: Math is Unreliable Status: Feedback Type: Bug Package: Math related Operating System: any PHP Version: Irrelevant New Comment: Well, it is the next character after '9', and the character string is built up in zend_dtoa() by adding a value L (presumably intended to be in the range 0..9) to '0'. If L somehow ends up being 10, you'd get a colon. With the values that are apparently causing problems (the problematic value in onepair.txt is 0.0167...), it does kind of look like a rounding issue to me, although I've no idea why it's not being triggered by more than two or three users in that case. Previous Comments: [2010-03-26 17:18:22] ras...@php.net Bad memory perhaps? But why consistently a ':' ? [2010-03-26 16:57:15] ahar...@php.net JFTR, I was also unable to reproduce the failure case from any of the data files on both 32-bit and 64-bit Linux builds and a 32-bit Windows build. I've got a couple of boxen crunching away generating random doubles and converting them to strings in what seems to be the sort of range that causes problems (one 32-bit Linux, one 64-bit Linux): nothing yet after a couple of billion iterations. In short, I don't have a clue what it could be either, but I'll keep the random double generation going a while longer just in case it hits paydirt. [2010-03-26 16:39:33] paj...@php.net Rasmus, he is not the only person. There are two other reports about it. However he is the first one to experience it on non windows platform. [2010-03-26 16:26:05] ras...@php.net I ran your simplefail.php on all 4 data files. Never saw the failure case. Then I took just the huge rawpairs.txt file (had to increase the memory_limit) and modified your program slightly to do: for($i=0;$i<10;$i++) ConvertData($data); and ran that, which took quite a while. Still no failures. You are still the only person who has ever reported these weird ":" characters showing up, and you have yet to produce any sort of code that reproduces it for someone else. We'll keep trying, but something that happens for 1 person out of millions is suspicious. [2010-03-26 15:28:16] codeslinger at compsalot dot com it's still not 20 lines... it never will be but I found one surprising thing, if I save the array to disk and read it back in I still get the same failure. so essentially the snowflake is a Psudo-RNG and that's why trying to simplify it does not work, because it has to be able to generate all of those different numbers. but once we have the set of numbers the generator becomes irrelevant. Try this program: http://www.compsalot.com/phpbug/kochsnowflake/simplefail/ weirdly, every failure is a -0.1 but there are two things that I'm concerned about. 1) It seems unlikely that the billing program was only failing on that amount, it is a pretty unusual amount. Not one I'd normally expect to see and there were a lot of failures. 2) I'm concerned about the limitations of the detection method. I am only finding bad values because they have a colon in the string. I am not currently checking for mathematical correctness. We may not be seeing the whole story, there could be a lot more bad values that just are not blatant enough to be observed. But at least this should give you the starting point that you are asking for. The program is reduced to reading a file from disk and checking each number to see if it's valid. can't get any simpler than that. Of course if the corruption is happening when the value is created, looking at them after the fact may still not help much. But when I look at these values in the eclipse zend debugger, they look normal enough despite the fact that they can't be properly converted. -- The data files are as follows: rawpairs.txt is a 6.5meg file and contains both good and bad values, you don't need this file unless you are trying to validate the serialization. This is from a single run of the snowflake, $Nest = 3; $Depth = 5; The following were extracted from the raw data. onepair.txt contains a single failure item selectpairs.txt contains 8 failures which were from a single run of snowflake manypairs.txt contains about 40 failures which are the result of multiple runs of the snowflake using different Nest & Depth parameters.
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1 ID: 51396 Updated by: ras...@php.net Reported by: codeslinger at compsalot dot com Summary: Math is Unreliable Status: Feedback Type: Bug Package: Math related Operating System: any PHP Version: Irrelevant New Comment: Bad memory perhaps? But why consistently a ':' ? Previous Comments: [2010-03-26 16:57:15] ahar...@php.net JFTR, I was also unable to reproduce the failure case from any of the data files on both 32-bit and 64-bit Linux builds and a 32-bit Windows build. I've got a couple of boxen crunching away generating random doubles and converting them to strings in what seems to be the sort of range that causes problems (one 32-bit Linux, one 64-bit Linux): nothing yet after a couple of billion iterations. In short, I don't have a clue what it could be either, but I'll keep the random double generation going a while longer just in case it hits paydirt. [2010-03-26 16:39:33] paj...@php.net Rasmus, he is not the only person. There are two other reports about it. However he is the first one to experience it on non windows platform. [2010-03-26 16:26:05] ras...@php.net I ran your simplefail.php on all 4 data files. Never saw the failure case. Then I took just the huge rawpairs.txt file (had to increase the memory_limit) and modified your program slightly to do: for($i=0;$i<10;$i++) ConvertData($data); and ran that, which took quite a while. Still no failures. You are still the only person who has ever reported these weird ":" characters showing up, and you have yet to produce any sort of code that reproduces it for someone else. We'll keep trying, but something that happens for 1 person out of millions is suspicious. [2010-03-26 15:28:16] codeslinger at compsalot dot com it's still not 20 lines... it never will be but I found one surprising thing, if I save the array to disk and read it back in I still get the same failure. so essentially the snowflake is a Psudo-RNG and that's why trying to simplify it does not work, because it has to be able to generate all of those different numbers. but once we have the set of numbers the generator becomes irrelevant. Try this program: http://www.compsalot.com/phpbug/kochsnowflake/simplefail/ weirdly, every failure is a -0.1 but there are two things that I'm concerned about. 1) It seems unlikely that the billing program was only failing on that amount, it is a pretty unusual amount. Not one I'd normally expect to see and there were a lot of failures. 2) I'm concerned about the limitations of the detection method. I am only finding bad values because they have a colon in the string. I am not currently checking for mathematical correctness. We may not be seeing the whole story, there could be a lot more bad values that just are not blatant enough to be observed. But at least this should give you the starting point that you are asking for. The program is reduced to reading a file from disk and checking each number to see if it's valid. can't get any simpler than that. Of course if the corruption is happening when the value is created, looking at them after the fact may still not help much. But when I look at these values in the eclipse zend debugger, they look normal enough despite the fact that they can't be properly converted. -- The data files are as follows: rawpairs.txt is a 6.5meg file and contains both good and bad values, you don't need this file unless you are trying to validate the serialization. This is from a single run of the snowflake, $Nest = 3; $Depth = 5; The following were extracted from the raw data. onepair.txt contains a single failure item selectpairs.txt contains 8 failures which were from a single run of snowflake manypairs.txt contains about 40 failures which are the result of multiple runs of the snowflake using different Nest & Depth parameters. [2010-03-26 11:14:13] paj...@php.net Does it happen always? Can you try to make a script with only this function call with the values causing this effect? The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/bug.php?id=51396 -- Edit this bug report at http://bugs.php.net/bug.php?id=51396&edit=1
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1
ID: 51396
Updated by: ahar...@php.net
Reported by: codeslinger at compsalot dot com
Summary: Math is Unreliable
Status: Feedback
Type: Bug
Package: Math related
Operating System: any
PHP Version: Irrelevant
New Comment:
JFTR, I was also unable to reproduce the failure case from any of the
data files on both 32-bit and 64-bit Linux builds and a 32-bit Windows
build. I've got a couple of boxen crunching away generating random
doubles and converting them to strings in what seems to be the sort of
range that causes problems (one 32-bit Linux, one 64-bit Linux): nothing
yet after a couple of billion iterations.
In short, I don't have a clue what it could be either, but I'll keep the
random double generation going a while longer just in case it hits
paydirt.
Previous Comments:
[2010-03-26 16:39:33] paj...@php.net
Rasmus, he is not the only person. There are two other reports about it.
However he is the first one to experience it on non windows platform.
[2010-03-26 16:26:05] ras...@php.net
I ran your simplefail.php on all 4 data files. Never saw the failure
case.
Then I took just the huge rawpairs.txt file (had to increase the
memory_limit)
and modified your program slightly to do:
for($i=0;$i<10;$i++) ConvertData($data);
and ran that, which took quite a while. Still no failures.
You are still the only person who has ever reported these weird ":"
characters
showing up, and you have yet to produce any sort of code that reproduces
it for
someone else. We'll keep trying, but something that happens for 1
person out of
millions is suspicious.
[2010-03-26 15:28:16] codeslinger at compsalot dot com
it's still not 20 lines... it never will be but I found one
surprising thing, if I save the array to disk and read it back in I
still get the same failure.
so essentially the snowflake is a Psudo-RNG and that's why trying to
simplify it does not work, because it has to be able to generate all of
those different numbers. but once we have the set of numbers the
generator becomes irrelevant.
Try this program:
http://www.compsalot.com/phpbug/kochsnowflake/simplefail/
weirdly, every failure is a -0.1
but there are two things that I'm concerned about.
1) It seems unlikely that the billing program was only failing on that
amount, it is a pretty unusual amount. Not one I'd normally expect to
see and there were a lot of failures.
2) I'm concerned about the limitations of the detection method. I am
only finding bad values because they have a colon in the string. I am
not currently checking for mathematical correctness. We may not be
seeing the whole story, there could be a lot more bad values that just
are not blatant enough to be observed.
But at least this should give you the starting point that you are asking
for. The program is reduced to reading a file from disk and checking
each number to see if it's valid. can't get any simpler than that.
Of course if the corruption is happening when the value is created,
looking at them after the fact may still not help much. But when I look
at these values in the eclipse zend debugger, they look normal enough
despite the fact that they can't be properly converted.
--
The data files are as follows:
rawpairs.txt is a 6.5meg file and contains both good and bad values,
you don't need this file unless you are trying to validate the
serialization. This is from a single run of the snowflake, $Nest = 3;
$Depth = 5;
The following were extracted from the raw data.
onepair.txt contains a single failure item
selectpairs.txt contains 8 failures which were from a single run of
snowflake
manypairs.txt contains about 40 failures which are the result of
multiple runs of the snowflake using different Nest & Depth parameters.
[2010-03-26 11:14:13] paj...@php.net
Does it happen always? Can you try to make a script with only this
function call with the values causing this effect?
[2010-03-26 11:11:05] codeslinger at compsalot dot com
String conversion errors
There is a flag called $FloatType
You can use it to select the type of conversion that is done.
//this fails with string conversion error
case 0: $s = (string)$f;
//this fails with string conversion error
case 12: $s = is_float($f) ? sprintf('%.12F', $f) : (string)$f;
//this fails with string conversion error
case 14: $s = is_float($f) ? sprintf('%.14F', $f) : (string)$f;
//this W
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1
ID: 51396
Updated by: paj...@php.net
Reported by: codeslinger at compsalot dot com
Summary: Math is Unreliable
Status: Feedback
Type: Bug
Package: Math related
Operating System: any
PHP Version: Irrelevant
New Comment:
Rasmus, he is not the only person. There are two other reports about it.
However he is the first one to experience it on non windows platform.
Previous Comments:
[2010-03-26 16:26:05] ras...@php.net
I ran your simplefail.php on all 4 data files. Never saw the failure
case.
Then I took just the huge rawpairs.txt file (had to increase the
memory_limit)
and modified your program slightly to do:
for($i=0;$i<10;$i++) ConvertData($data);
and ran that, which took quite a while. Still no failures.
You are still the only person who has ever reported these weird ":"
characters
showing up, and you have yet to produce any sort of code that reproduces
it for
someone else. We'll keep trying, but something that happens for 1
person out of
millions is suspicious.
[2010-03-26 15:28:16] codeslinger at compsalot dot com
it's still not 20 lines... it never will be but I found one
surprising thing, if I save the array to disk and read it back in I
still get the same failure.
so essentially the snowflake is a Psudo-RNG and that's why trying to
simplify it does not work, because it has to be able to generate all of
those different numbers. but once we have the set of numbers the
generator becomes irrelevant.
Try this program:
http://www.compsalot.com/phpbug/kochsnowflake/simplefail/
weirdly, every failure is a -0.1
but there are two things that I'm concerned about.
1) It seems unlikely that the billing program was only failing on that
amount, it is a pretty unusual amount. Not one I'd normally expect to
see and there were a lot of failures.
2) I'm concerned about the limitations of the detection method. I am
only finding bad values because they have a colon in the string. I am
not currently checking for mathematical correctness. We may not be
seeing the whole story, there could be a lot more bad values that just
are not blatant enough to be observed.
But at least this should give you the starting point that you are asking
for. The program is reduced to reading a file from disk and checking
each number to see if it's valid. can't get any simpler than that.
Of course if the corruption is happening when the value is created,
looking at them after the fact may still not help much. But when I look
at these values in the eclipse zend debugger, they look normal enough
despite the fact that they can't be properly converted.
--
The data files are as follows:
rawpairs.txt is a 6.5meg file and contains both good and bad values,
you don't need this file unless you are trying to validate the
serialization. This is from a single run of the snowflake, $Nest = 3;
$Depth = 5;
The following were extracted from the raw data.
onepair.txt contains a single failure item
selectpairs.txt contains 8 failures which were from a single run of
snowflake
manypairs.txt contains about 40 failures which are the result of
multiple runs of the snowflake using different Nest & Depth parameters.
[2010-03-26 11:14:13] paj...@php.net
Does it happen always? Can you try to make a script with only this
function call with the values causing this effect?
[2010-03-26 11:11:05] codeslinger at compsalot dot com
String conversion errors
There is a flag called $FloatType
You can use it to select the type of conversion that is done.
//this fails with string conversion error
case 0: $s = (string)$f;
//this fails with string conversion error
case 12: $s = is_float($f) ? sprintf('%.12F', $f) : (string)$f;
//this fails with string conversion error
case 14: $s = is_float($f) ? sprintf('%.14F', $f) : (string)$f;
//this WORKS, does not cause a string conversion error
case 20: $s = is_float($f) ? sprintf('%.20F', $f) : (string)$f;
When I say conversion error, I mean that the result is "0.0:" not that
php gives any kind of error message.
[2010-03-26 10:23:27] codeslinger at compsalot dot com
quote: Btw, it could be due to bad initialized data as well
I don't understand the question/statement.
The program generates all of it's data.
If you look near the bottom of pov.pco.php you will see the following
line:
echo "FATAL ERROR: PHP Math is Corrupt Again!!! idx = $idx\n";
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1
ID: 51396
Updated by: ras...@php.net
Reported by: codeslinger at compsalot dot com
Summary: Math is Unreliable
Status: Feedback
Type: Bug
Package: Math related
Operating System: any
PHP Version: Irrelevant
New Comment:
I ran your simplefail.php on all 4 data files. Never saw the failure
case.
Then I took just the huge rawpairs.txt file (had to increase the
memory_limit)
and modified your program slightly to do:
for($i=0;$i<10;$i++) ConvertData($data);
and ran that, which took quite a while. Still no failures.
You are still the only person who has ever reported these weird ":"
characters
showing up, and you have yet to produce any sort of code that reproduces
it for
someone else. We'll keep trying, but something that happens for 1
person out of
millions is suspicious.
Previous Comments:
[2010-03-26 15:28:16] codeslinger at compsalot dot com
it's still not 20 lines... it never will be but I found one
surprising thing, if I save the array to disk and read it back in I
still get the same failure.
so essentially the snowflake is a Psudo-RNG and that's why trying to
simplify it does not work, because it has to be able to generate all of
those different numbers. but once we have the set of numbers the
generator becomes irrelevant.
Try this program:
http://www.compsalot.com/phpbug/kochsnowflake/simplefail/
weirdly, every failure is a -0.1
but there are two things that I'm concerned about.
1) It seems unlikely that the billing program was only failing on that
amount, it is a pretty unusual amount. Not one I'd normally expect to
see and there were a lot of failures.
2) I'm concerned about the limitations of the detection method. I am
only finding bad values because they have a colon in the string. I am
not currently checking for mathematical correctness. We may not be
seeing the whole story, there could be a lot more bad values that just
are not blatant enough to be observed.
But at least this should give you the starting point that you are asking
for. The program is reduced to reading a file from disk and checking
each number to see if it's valid. can't get any simpler than that.
Of course if the corruption is happening when the value is created,
looking at them after the fact may still not help much. But when I look
at these values in the eclipse zend debugger, they look normal enough
despite the fact that they can't be properly converted.
--
The data files are as follows:
rawpairs.txt is a 6.5meg file and contains both good and bad values,
you don't need this file unless you are trying to validate the
serialization. This is from a single run of the snowflake, $Nest = 3;
$Depth = 5;
The following were extracted from the raw data.
onepair.txt contains a single failure item
selectpairs.txt contains 8 failures which were from a single run of
snowflake
manypairs.txt contains about 40 failures which are the result of
multiple runs of the snowflake using different Nest & Depth parameters.
[2010-03-26 11:14:13] paj...@php.net
Does it happen always? Can you try to make a script with only this
function call with the values causing this effect?
[2010-03-26 11:11:05] codeslinger at compsalot dot com
String conversion errors
There is a flag called $FloatType
You can use it to select the type of conversion that is done.
//this fails with string conversion error
case 0: $s = (string)$f;
//this fails with string conversion error
case 12: $s = is_float($f) ? sprintf('%.12F', $f) : (string)$f;
//this fails with string conversion error
case 14: $s = is_float($f) ? sprintf('%.14F', $f) : (string)$f;
//this WORKS, does not cause a string conversion error
case 20: $s = is_float($f) ? sprintf('%.20F', $f) : (string)$f;
When I say conversion error, I mean that the result is "0.0:" not that
php gives any kind of error message.
[2010-03-26 10:23:27] codeslinger at compsalot dot com
quote: Btw, it could be due to bad initialized data as well
I don't understand the question/statement.
The program generates all of it's data.
If you look near the bottom of pov.pco.php you will see the following
line:
echo "FATAL ERROR: PHP Math is Corrupt Again!!! idx = $idx\n";
If you set a breakpoint there in a php debugger and examine the vaules
you will see that that array contains a float of 0.1
and that the string it was converted to is "0.0:"
[201
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1
ID: 51396
Updated by: paj...@php.net
Reported by: codeslinger at compsalot dot com
Summary: Math is Unreliable
Status: Feedback
Type: Bug
Package: Math related
Operating System: any
PHP Version: Irrelevant
Assigned To: aharvey
New Comment:
Does it happen always? Can you try to make a script with only this
function call with the values causing this effect?
Previous Comments:
[2010-03-26 11:11:05] codeslinger at compsalot dot com
String conversion errors
There is a flag called $FloatType
You can use it to select the type of conversion that is done.
//this fails with string conversion error
case 0: $s = (string)$f;
//this fails with string conversion error
case 12: $s = is_float($f) ? sprintf('%.12F', $f) : (string)$f;
//this fails with string conversion error
case 14: $s = is_float($f) ? sprintf('%.14F', $f) : (string)$f;
//this WORKS, does not cause a string conversion error
case 20: $s = is_float($f) ? sprintf('%.20F', $f) : (string)$f;
When I say conversion error, I mean that the result is "0.0:" not that
php gives any kind of error message.
[2010-03-26 10:23:27] codeslinger at compsalot dot com
quote: Btw, it could be due to bad initialized data as well
I don't understand the question/statement.
The program generates all of it's data.
If you look near the bottom of pov.pco.php you will see the following
line:
echo "FATAL ERROR: PHP Math is Corrupt Again!!! idx = $idx\n";
If you set a breakpoint there in a php debugger and examine the vaules
you will see that that array contains a float of 0.1
and that the string it was converted to is "0.0:"
[2010-03-26 10:17:47] codeslinger at compsalot dot com
Thank You for the response.
This program executes thousands of identical loops
it performs a number format conversion on each of those thousands of
numbers in it's array.
Out of all of those thousands of identical conversions about 4 of them
will produce an invalid result.
for this specific program, if I skip the array_merge (see $Nest) then
the bug does not occur.
Every time I make a change to the loops etc. the behavior changes, even
to where the bug is not triggered. I have already simplified this
program quite a bit and the result was that it now fails at a totally
different array index then when I first observed the problem. The
trigger is very finicky.
XP is not a factor here, even though the original problem was found on
XP, I am running this Snowflake program on Ubuntu Linux.
converting a number to a string should never result in that string
containing a non-numeric character.
I can try to reduce this program further, but really and truly, it is
hard to trigger this -- but once triggered it is totally consistent and
repeatable.
[2010-03-26 10:11:25] paj...@php.net
Btw, it could be due to bad initialized data as well (as you experience
it on unix as well).
Can you try to run your script under valgrind too?
[2010-03-26 09:54:33] paj...@php.net
It looks to me in a bug at the number formatting in the Windows API (not
php dependent). Other users met this problem on old XP systems. You
could reproduce it using number_format only. Or maybe only with a single
printf function.
Can you try using another windows box or using 5.3.2-vc9 builds?
Btw, it is not about bad or good attitude but about having something we
can actually use to help you. I can understand your worries after having
spent hours to debug this problem, but we will still use a small script
to debug this problem, if any.
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
http://bugs.php.net/bug.php?id=51396
--
Edit this bug report at http://bugs.php.net/bug.php?id=51396&edit=1
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1 ID: 51396 Updated by: paj...@php.net Reported by: codeslinger at compsalot dot com Summary: Math is Unreliable Status: Feedback Type: Bug Package: Math related Operating System: any PHP Version: Irrelevant Assigned To: aharvey New Comment: Btw, it could be due to bad initialized data as well (as you experience it on unix as well). Can you try to run your script under valgrind too? Previous Comments: [2010-03-26 09:54:33] paj...@php.net It looks to me in a bug at the number formatting in the Windows API (not php dependent). Other users met this problem on old XP systems. You could reproduce it using number_format only. Or maybe only with a single printf function. Can you try using another windows box or using 5.3.2-vc9 builds? Btw, it is not about bad or good attitude but about having something we can actually use to help you. I can understand your worries after having spent hours to debug this problem, but we will still use a small script to debug this problem, if any. [2010-03-26 09:40:23] codeslinger at compsalot dot com did you even LOOOKK at the script that I provided? it is not that complicated. a few loops and some array allocations. previous attempts to reduce this to anything simpler have been unsuccessful. this is exactly the attitude that causes me to be overwrought. [2010-03-26 09:30:30] paj...@php.net Thank you for this bug report. To properly diagnose the problem, we need a short but complete example script to be able to reproduce this bug ourselves. A proper reproducing script starts with , is max. 10-20 lines long and does not require any external resources such as databases, etc. If the script requires a database to demonstrate the issue, please make sure it creates all necessary tables, stored procedures etc. Please avoid embedding huge scripts into the report. [2010-03-26 08:51:34] codeslinger at compsalot dot com oops, fixed. various attempts to isolate this to a simple test case have not yet succeeded. This program is the closest I've come to a simple test case. after many hours of working on this, what I can tell you is that it requires lots of calculations and the use of arrays. - lots of memory allocations. yes, I confess to being overwrought. I've got 3 years of work, and the future of my software company at stake with this bug. and a lot of experience with bug reports that get ignored, including the original entry for this bug. [2010-03-26 04:31:56] ahar...@php.net It looks like the Web server you've posted the sample code to is interpreting the PHP files, so it's impossible to get the source. If you could upload the files with an extension that's not being handled by your PHP module, that would be handy. If there's any chance of a more minimal test case, that would also be rather useful. Finally, without wanting to sound overly snarky, that description was about 1200 words too long and infinitely too overwrought. Brevity is appreciated. The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/bug.php?id=51396 -- Edit this bug report at http://bugs.php.net/bug.php?id=51396&edit=1
Bug #51396 [Fbk]: Math is Unreliable
Edit report at http://bugs.php.net/bug.php?id=51396&edit=1 ID: 51396 Updated by: paj...@php.net Reported by: codeslinger at compsalot dot com Summary: Math is Unreliable Status: Feedback Type: Bug Package: Math related Operating System: any PHP Version: Irrelevant Assigned To: aharvey New Comment: It looks to me in a bug at the number formatting in the Windows API (not php dependent). Other users met this problem on old XP systems. You could reproduce it using number_format only. Or maybe only with a single printf function. Can you try using another windows box or using 5.3.2-vc9 builds? Btw, it is not about bad or good attitude but about having something we can actually use to help you. I can understand your worries after having spent hours to debug this problem, but we will still use a small script to debug this problem, if any. Previous Comments: [2010-03-26 09:40:23] codeslinger at compsalot dot com did you even LOOOKK at the script that I provided? it is not that complicated. a few loops and some array allocations. previous attempts to reduce this to anything simpler have been unsuccessful. this is exactly the attitude that causes me to be overwrought. [2010-03-26 09:30:30] paj...@php.net Thank you for this bug report. To properly diagnose the problem, we need a short but complete example script to be able to reproduce this bug ourselves. A proper reproducing script starts with , is max. 10-20 lines long and does not require any external resources such as databases, etc. If the script requires a database to demonstrate the issue, please make sure it creates all necessary tables, stored procedures etc. Please avoid embedding huge scripts into the report. [2010-03-26 08:51:34] codeslinger at compsalot dot com oops, fixed. various attempts to isolate this to a simple test case have not yet succeeded. This program is the closest I've come to a simple test case. after many hours of working on this, what I can tell you is that it requires lots of calculations and the use of arrays. - lots of memory allocations. yes, I confess to being overwrought. I've got 3 years of work, and the future of my software company at stake with this bug. and a lot of experience with bug reports that get ignored, including the original entry for this bug. [2010-03-26 04:31:56] ahar...@php.net It looks like the Web server you've posted the sample code to is interpreting the PHP files, so it's impossible to get the source. If you could upload the files with an extension that's not being handled by your PHP module, that would be handy. If there's any chance of a more minimal test case, that would also be rather useful. Finally, without wanting to sound overly snarky, that description was about 1200 words too long and infinitely too overwrought. Brevity is appreciated. [2010-03-26 00:36:48] codeslinger at compsalot dot com Description: is math accuracy important? no I am not talking about round-off errors. I am talking about the fact that most of the time, php correctly says that 2+2 = 4, but that sometimes it says that 2+2 = 0. What if your paycheck was being printed by a php program and you discovered that the math is inaccurate, would you care then? I ask these questions because I previously reported this bug because it occurred in a billing system that I wrote, but nobody considered that bug important enough to pursue it. Too complex, too hard to reproduce. whatever. Well, I have now encountered this bug again, with a much simpler program. and this program demonstrates that math in php is completely and totally untrustworthy. Does anyone care that fundamental functionality is unreliable? What good is a programming language that can't do math? php passes trivial math tests just fine; it's only when you use it in complex real-world ways that it starts failing. No these aren't binary base conversion errors, I'm guessing that this is memory corruption, so far I've only observed it with thousands of calculations. But the trivial programs I wrote in an attempt to reproduce the problem never succeeded in failing. I can only get it to fail when I'm doing real-world / complex stuff Characteristics: This is not a bug in the php program; under specific conditions - described below - the php program does run correctly. The main problem occurs when a float is converted to a string by a program that makes heavy use of arrays. It's not an out of memory condition, I have successfully stress tested a
