At 03:16 PM 6/13/03 -0400, George Woltman wrote: >So far I've come up with two possibilities. >[....] >2) This case results from the way my C compiler treats floating point NaN. >NaN stands for not a number. If NaN is converted to an integer, the integer >is zero. So if the FFT data is all NaNs, prime95 will report a prime. Prime95 >check for NaNs every iteration, but if every FFT data value becomes NaN >after the inverse FFT of the last LL iteration, then we get a false positive. >Furthermore, corrupting a single value (the initial carry input to rounding >and carry propogation code) could set every FFT data value to NaN.
I'm fishing.... What about +/- INF ? What about corruption of any of the pre-computed arrays? Is there a constant "-2" that could be corrupted? --Luke _________________________________________________________________________ Unsubscribe & list info -- http://www.ndatech.com/mersenne/signup.htm Mersenne Prime FAQ -- http://www.tasam.com/~lrwiman/FAQ-mers
