[FFmpeg-devel] [PATCH] avcodec/proresenc_anatoliy: Fix invalid left shift of negative number

2020-01-08 Thread Andreas Rheinhardt
This fixes ticket #7997 as well as the vsynth*-prores_# FATE-tests
(where * ranges over { 1, 2, 3, _lena } and # over { , _int, _444,
_444_int }).

(Given that prev_dc is in the range -0xC000..0x3FFF, no overflow can
happen upon multiplication with 2.)

Signed-off-by: Andreas Rheinhardt 
---
Actually, #7997 ran into the same issue as #7979 and so it is no longer
reproducible with the given commandline since aef24efb (which fixed
#7979). But it is still reproducible with e.g. the null muxer.

 libavcodec/proresenc_anatoliy.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/proresenc_anatoliy.c b/libavcodec/proresenc_anatoliy.c
index 0fc79fc1de..1fcb0ae913 100644
--- a/libavcodec/proresenc_anatoliy.c
+++ b/libavcodec/proresenc_anatoliy.c
@@ -224,7 +224,7 @@ static void encode_codeword(PutBitContext *pb, int val, int 
codebook)
 }
 
 #define QSCALE(qmat,ind,val) ((val) / ((qmat)[ind]))
-#define TO_GOLOMB(val) (((val) << 1) ^ ((val) >> 31))
+#define TO_GOLOMB(val) (((val) * 2) ^ ((val) >> 31))
 #define DIFF_SIGN(val, sign) (((val) >> 31) ^ (sign))
 #define IS_NEGATIVE(val) val) >> 31) ^ -1) + 1)
 #define TO_GOLOMB2(val,sign) ((val)==0 ? 0 : ((val) << 1) + (sign))
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/proresenc_anatoliy: Fix invalid left shift of negative number

2020-01-08 Thread Michael Niedermayer
On Wed, Jan 08, 2020 at 07:29:13PM +0100, Andreas Rheinhardt wrote:
> This fixes ticket #7997 as well as the vsynth*-prores_# FATE-tests
> (where * ranges over { 1, 2, 3, _lena } and # over { , _int, _444,
> _444_int }).
> 
> (Given that prev_dc is in the range -0xC000..0x3FFF, no overflow can
> happen upon multiplication with 2.)
> 
> Signed-off-by: Andreas Rheinhardt 
> ---
> Actually, #7997 ran into the same issue as #7979 and so it is no longer
> reproducible with the given commandline since aef24efb (which fixed
> #7979). But it is still reproducible with e.g. the null muxer.
> 
>  libavcodec/proresenc_anatoliy.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

will apply

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I know you won't believe me, but the highest form of Human Excellence is
to question oneself and others. -- Socrates


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".