On 1/13/2019 11:32 AM, Derek Buitenhuis wrote: > On 13/01/2019 02:44, James Almer wrote: >> Michael suggested 1000*FLT_EPSILON but IMO that's too big and may hide >> errors in future implementations. >> The value i used is the smallest value i found that didn't fail after >> several runs. 6.1e-05 for example fails. > > I figured that's how it was chosen (though not documented). I thought there > be a way to calculate it properly instead of empirically.
If there is, i don't know it. Float based fate tests have been fine tuned before when different architectures proved a certain stddev value was not lax enough, so this one could be increased if needed as well, but if you prefer i can use a big enough multiple of FLT_EPSILON instead. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
