On 06/02/2011 04:42 PM, Ronald S. Bultje wrote:
Hi John,
On Thu, Jun 2, 2011 at 4:39 PM, John Stebbins<[email protected]> wrote:
On 06/02/2011 03:31 PM, Ronald S. Bultje wrote:
On Thu, Jun 2, 2011 at 3:29 PM, John Stebbins<[email protected]>
wrote:
Fair enough. There appear to be some cases remaining where av_malloc is
called with a size of 0. I'll set a break point and find out where the
request is coming from.
Thanks, if you can just show me a backtrace (I just run commands and
set a if (size == 0) abort() in av_malloc), I'll gladly accept the
fix.
The first occurrence of the problem that I find is in vc1_decode_init.
avctx->mb_stride gets set (from avctx->width) when ff_msmpeg4_decode_init
is called. But at the point ff_msmpeg4_decode_init is called avctx->width
hasn't been set yet (i.e. it is 0). width gets set in
vc1_decode_sequence_header which is called after ff_msmpeg4_decode_init.
Since width is 0, mb_stride is 0 and av_malloc gets called with 0.
I don't know exactly how to untangle this because I don't know the
dependencies between these initialization functions. Can anyone make any
suggestions?
Feel free to send me a file that triggers the problem and I'll happily
fix it for you.
I think this would take more than just a sample file to reproduce. I am reproducing this with HandBrake which has it's
own TS demux. So we are not using libavformat in this case and are allocating and initializing AVCodecContext,
initializing extradata, and calling avcodec_open ourselves.
We've always noticed a little weirdness with vc1 in this bit of code. We have to call avcodec_open twice to get it to
initialize properly due to this order of operations problem in vc1_decode_init.
_______________________________________________
libav-devel mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-devel
