Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On 02.12.2015 20:58, Andreas Cadhalpun wrote: > On 19.11.2015 01:01, Andreas Cadhalpun wrote: >> Subject: [PATCH] sbrdsp_fixed: assert that input values for sbr_sum_square_c >> are valid >> >> Larger values can cause overflows, leading to this function returning a >> negative value. >> >> Signed-o

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On 19.11.2015 01:01, Andreas Cadhalpun wrote: > Subject: [PATCH] sbrdsp_fixed: assert that input values for sbr_sum_square_c > are valid > > Larger values can cause overflows, leading to this function returning a > negative value. > > Signed-off-by: Andreas Cadhalpun > --- > libavcodec/sbrdsp_

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On Wed, Nov 18, 2015 at 7:01 PM, Andreas Cadhalpun wrote: > On 16.11.2015 15:39, Nedeljko Babic wrote: On 11.11.2015 13:46, Michael Niedermayer wrote: >>> Comments fro AAC and SBR experts very welcome! >> >> This code was developed a while ago, but based on informations that I have >> this pa

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On 16.11.2015 15:39, Nedeljko Babic wrote: >>> On 11.11.2015 13:46, Michael Niedermayer wrote: >> Comments fro AAC and SBR experts very welcome! > > This code was developed a while ago, but based on informations that I have > this part of code was analysed regarding possibility of overflow and >

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On 16.11.2015 02:59, Michael Niedermayer wrote: > On Fri, Nov 13, 2015 at 10:32:47PM +0100, Andreas Cadhalpun wrote: >> Because these samples aren't tested with the aac_fixed decoder: >> * aac/ct_faac-adts.aac is only used to test the aac demuxer. >> * aac/al07_96.mp4 is for some reason only test

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

>> On 11.11.2015 13:46, Michael Niedermayer wrote: >> > On Sun, Nov 08, 2015 at 09:26:21PM +0100, Andreas Cadhalpun wrote: >> >> On 08.11.2015 20:17, Michael Niedermayer wrote: >> >>> but the patch does not look like an optimal solution >> >> >> >> It's certainly not pretty, but it fixes the crashe

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On Fri, Nov 13, 2015 at 10:32:47PM +0100, Andreas Cadhalpun wrote: > On 13.11.2015 04:15, Michael Niedermayer wrote: > > On Thu, Nov 12, 2015 at 08:43:42PM +0100, Andreas Cadhalpun wrote: > >> Considering that the aac float decoder can decode such samples, I tend > >> to think that the aac fixed de

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On 13.11.2015 04:15, Michael Niedermayer wrote: > On Thu, Nov 12, 2015 at 08:43:42PM +0100, Andreas Cadhalpun wrote: >> Considering that the aac float decoder can decode such samples, I tend >> to think that the aac fixed decoder should be able to do that, too. > > IMO this reasoning is flawed >

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On Thu, Nov 12, 2015 at 08:43:42PM +0100, Andreas Cadhalpun wrote: > On 11.11.2015 22:52, Michael Niedermayer wrote: > > On Wed, Nov 11, 2015 at 09:09:51PM +0100, Andreas Cadhalpun wrote: > >> On 11.11.2015 13:46, Michael Niedermayer wrote: > >>> On Sun, Nov 08, 2015 at 09:26:21PM +0100, Andreas Ca

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On 11.11.2015 22:52, Michael Niedermayer wrote: > On Wed, Nov 11, 2015 at 09:09:51PM +0100, Andreas Cadhalpun wrote: >> On 11.11.2015 13:46, Michael Niedermayer wrote: >>> On Sun, Nov 08, 2015 at 09:26:21PM +0100, Andreas Cadhalpun wrote: On 08.11.2015 20:17, Michael Niedermayer wrote: > b

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On Wed, Nov 11, 2015 at 09:09:51PM +0100, Andreas Cadhalpun wrote: > On 11.11.2015 13:46, Michael Niedermayer wrote: > > On Sun, Nov 08, 2015 at 09:26:21PM +0100, Andreas Cadhalpun wrote: > >> On 08.11.2015 20:17, Michael Niedermayer wrote: > >>> but the patch does not look like an optimal solution

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On 11.11.2015 13:46, Michael Niedermayer wrote: > On Sun, Nov 08, 2015 at 09:26:21PM +0100, Andreas Cadhalpun wrote: >> On 08.11.2015 20:17, Michael Niedermayer wrote: >>> but the patch does not look like an optimal solution >> >> It's certainly not pretty, but it fixes the crashes/assertion failur

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On Sun, Nov 08, 2015 at 09:26:21PM +0100, Andreas Cadhalpun wrote: > On 08.11.2015 20:17, Michael Niedermayer wrote: > > On Sun, Nov 08, 2015 at 05:14:10PM +0100, Andreas Cadhalpun wrote: > >> If accu overflows, a negative value can be returned. > >> > >> Signed-off-by: Andreas Cadhalpun > >> ---

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On 08.11.2015 20:17, Michael Niedermayer wrote: > On Sun, Nov 08, 2015 at 05:14:10PM +0100, Andreas Cadhalpun wrote: >> If accu overflows, a negative value can be returned. >> >> Signed-off-by: Andreas Cadhalpun >> --- >> libavcodec/sbrdsp_fixed.c | 28 ++-- >> 1 file chan

Re: [FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

On Sun, Nov 08, 2015 at 05:14:10PM +0100, Andreas Cadhalpun wrote: > If accu overflows, a negative value can be returned. > > Signed-off-by: Andreas Cadhalpun > --- > libavcodec/sbrdsp_fixed.c | 28 ++-- > 1 file changed, 22 insertions(+), 6 deletions(-) CCing the author

[FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

If accu overflows, a negative value can be returned. Signed-off-by: Andreas Cadhalpun --- libavcodec/sbrdsp_fixed.c | 28 ++-- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/libavcodec/sbrdsp_fixed.c b/libavcodec/sbrdsp_fixed.c index 5b7b7a6..7ab6cc6 10064