[libav-devel] [PATCH] lavr: handle clipping in the float to s32 conversion

2012-10-12 Thread Justin Ruggles
We cannot clip to INT_MAX because that value cannot be exactly represented by a float value and ends up overflowing during conversion anyway. We need to use a slightly smaller float value, which ends up with slightly inaccurate results for samples which clip or nearly clip, but it is close enough.

Re: [libav-devel] [PATCH] lavr: handle clipping in the float to s32 conversion

2012-10-12 Thread Luca Barbato
On 10/12/2012 09:46 PM, Justin Ruggles wrote: We cannot clip to INT_MAX because that value cannot be exactly represented by a float value and ends up overflowing during conversion anyway. We need to use a slightly smaller float value, which ends up with slightly inaccurate results for samples