2011/7/4 Måns Rullgård <[email protected]>: > Jason Garrett-Glaser <[email protected]> writes: > >> 2011/7/4 Måns Rullgård <[email protected]>: >>> "Jason Garrett-Glaser " <[email protected]> writes: >>> >>>> Module: libav >>>> Branch: master >>>> Commit: 4320a309ce10a7eec93aef239a0776a33b1a5a34 >>>> >>>> Author: Jason Garrett-Glaser <[email protected]> >>>> Committer: Jason Garrett-Glaser <[email protected]> >>>> Date: Mon Jun 27 13:07:26 2011 -0700 >>>> >>>> H.264: make filter_mb_fast support the case of unavailable top mb >>>> >>>> Significantly faster deblocking in streams with lots of slices. >>>> >>>> --- >>>> >>>> libavcodec/h264_loopfilter.c | 42 >>>> ++++++++++++++++++++++++++++-------------- >>>> 1 files changed, 28 insertions(+), 14 deletions(-) >>> >>> This seems to have introduced some invalid reads per >>> http://fate.libav.org/x86_32-linux-gcc-valgrind/20110704043908 >> >> The valgrind log seems to suggest that qp0 = >> s->current_picture.qscale_table[mb_xy-1]; also causes an invalid read. >> This is interesting, as this line executed when we were on the left >> edge -- even before my patch. > > Valgrind is only able to catch it on the top row though. > >> Does qscale_table not have edge padding? > > Apparently not.
How's this? Jason
From a9608ec7e4795079753eb649eaffdd1d3e90e4dd Mon Sep 17 00:00:00 2001 From: Jason Garrett-Glaser <[email protected]> Date: Mon, 4 Jul 2011 06:05:34 -0700 Subject: [PATCH] H.264: fix overreads of qscale_table filter_mb_fast assumed that qscale_table was padded like many of the other tables. --- libavcodec/mpegvideo.c | 5 +++-- libavcodec/mpegvideo.h | 1 + 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/libavcodec/mpegvideo.c b/libavcodec/mpegvideo.c index 4978d28..56c1b35 100644 --- a/libavcodec/mpegvideo.c +++ b/libavcodec/mpegvideo.c @@ -285,9 +285,10 @@ int ff_alloc_picture(MpegEncContext *s, Picture *pic, int shared){ } FF_ALLOCZ_OR_GOTO(s->avctx, pic->mbskip_table , mb_array_size * sizeof(uint8_t)+2, fail) //the +2 is for the slice end check - FF_ALLOCZ_OR_GOTO(s->avctx, pic->qscale_table , mb_array_size * sizeof(uint8_t) , fail) + FF_ALLOCZ_OR_GOTO(s->avctx, pic->qscale_table_base , (big_mb_num + s->mb_stride) * sizeof(uint8_t) , fail) FF_ALLOCZ_OR_GOTO(s->avctx, pic->mb_type_base , (big_mb_num + s->mb_stride) * sizeof(uint32_t), fail) pic->mb_type= pic->mb_type_base + 2*s->mb_stride+1; + pic->qscale_table= pic->qscale_table_base + 2*s->mb_stride+1; if(s->out_format == FMT_H264){ for(i=0; i<2; i++){ FF_ALLOCZ_OR_GOTO(s->avctx, pic->motion_val_base[i], 2 * (b4_array_size+4) * sizeof(int16_t), fail) @@ -339,7 +340,7 @@ static void free_picture(MpegEncContext *s, Picture *pic){ av_freep(&pic->mc_mb_var); av_freep(&pic->mb_mean); av_freep(&pic->mbskip_table); - av_freep(&pic->qscale_table); + av_freep(&pic->qscale_table_base); av_freep(&pic->mb_type_base); av_freep(&pic->dct_coeff); av_freep(&pic->pan_scan); diff --git a/libavcodec/mpegvideo.h b/libavcodec/mpegvideo.h index e072aa7..d309fcb 100644 --- a/libavcodec/mpegvideo.h +++ b/libavcodec/mpegvideo.h @@ -88,6 +88,7 @@ typedef struct Picture{ * halfpel luma planes. */ uint8_t *interpolated[3]; + int8_t *qscale_table_base; int16_t (*motion_val_base[2])[2]; uint32_t *mb_type_base; #define MB_TYPE_INTRA MB_TYPE_INTRA4x4 //default mb_type if there is just one type -- 1.6.3.3
_______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
