Re: [libav-devel] [PATCH] vp8: check for too large dimensions

2015-06-07 Thread Ronald S. Bultje
Hi, On Sun, Jun 7, 2015 at 10:05 AM, Andreas Cadhalpun < andreas.cadhal...@googlemail.com> wrote: > +#define MARGIN (16 << 2) > +#define MAX_MB_SIZE (((INT16_MAX - MARGIN) >> 6) + 1) > So this is roughly 9 bits. > +if (s->avctx->coded_width > MAX_MB_SIZE * 16 || > +s->avctx->coded

[libav-devel] [PATCH] vp8: check for too large dimensions

2015-06-07 Thread Andreas Cadhalpun
If the dimensions are too large, s->mb_width or s->mb_height can become too large, leading to an int16_t overflow of s->mv_max.{x,y}. Signed-off-by: Andreas Cadhalpun --- libavcodec/vp8.c | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/libavcodec/vp8.c b/libavcodec/