On 10/11/18, Jerome Martinez <jer...@mediaarea.net> wrote:
> On 11/10/2018 14:08, Paul B Mahol wrote:
>> On 10/11/18, Jerome Martinez <jer...@mediaarea.net> wrote:
>>> Hi,
>>>
>>> Testing FFmpeg FFV1 encoder on big frames (more than 4K: 4300x3956), I
>>> have a warning for each frame encoded (so a lot of warnings!):
>>> [ffv1 @ 0000024a6bcfe880] Cannot allocate worst case packet size, the
>>> encoding could fail
>>>
>>> Checking avcodec/ffv1enc.c, it is due to the following lines:
>>> int64_t maxsize = AV_INPUT_BUFFER_MIN_SIZE
>>> + avctx->width*avctx->height*37LL*4;
>>> [...]
>>> if (maxsize > INT_MAX - AV_INPUT_BUFFER_PADDING_SIZE - 32) {
>>> av_log(avctx, AV_LOG_WARNING, "Cannot allocate worst case
>>> packet size, the encoding could fail\n");
>>> maxsize = INT_MAX - AV_INPUT_BUFFER_PADDING_SIZE - 32;
>>> }
>>>
>>> The value of maxsize is 2517614784, more than INT_MAX, sure, but I don't
>>> understand:
>>> - why does FFmpeg need to allocate 148 times the size of a frame?
>>> - why is having a number > INT_MAX an issue? modern machines are 64-bit
>>> and have 8+ GB of RAM, and in practice I currently saw no encoding
>>> failure on thousands of frames.
>>>
>>> Additionally I didn't get why maxsize is reduced ("only" need of 12
>>> times the size of the frame) in case of FFV1 version > 3 (experimental
>>> right now):
>>> if (f->version > 3)
>>> maxsize = AV_INPUT_BUFFER_MIN_SIZE +
>>> avctx->width*avctx->height*3LL*4;
>>>
>>> maxsize is used for calling ff_alloc_packet2(), which accepts 64-bit
>>> numbers.
>>> if ((ret = ff_alloc_packet2(avctx, pkt, maxsize, 0)) < 0)
>>> return ret;
>>>
>>> Is it possible to reduce this "37x4" multiplier without risk and/or
>>> remove maxsize reduction in case of sizeof(size_t) == 8 and/or show the
>>> warning only once?
>>>
>>> Issue can be reproduced with:
>>> ./ffmpeg.exe -f lavfi -i testsrc2=size=4300x3956 -t 0.040 -pix_fmt
>>> rgba64 rgba64.dpx
>>> ./ffmpeg.exe -i rgba64.dpx -c:v ffv1 rgba64.mkv
>> You are not using latest version of FFv1.
>
>
> As far as I know, version 3 is the latest (stable) version of FFV1, and
> I use this one.
> version > 3 are experimental.
> I am not sure that advising to use an experimental version for "fixing"
> a warning is good.
>
> Additionally, in theory a 16kx16k content would still produce this
> warning with FFV1 (experimental) v4 encoding, even if there is 16
> Terabytes of RAM (more than enough for having the MAX_INT = 2 GiB raw
> frame in RAM), so I understand that the warning would still be there (in
> theory) with FFV1 (experimental) v4.
>
>
>> In latest version, encoder will not produce packets which are bigger than
>> input ones -- uncompressed raw. And instead if that happens it will encode
>> as raw video.
>>
>> Theoretically output packets for old versions of FFv1 could be several
>> times bigger than raw input frame.
>
>
> In this case, the ratio (148 times the size of input) is proportional to
> the input frame size, and the test is on a fixed value (MAX_INT).
> I get now the reason to reserve more than the size of the input frame.
> I don't get the relationship between your explanation and these hard
> coded values (148 times the size of input and MAX_INT limit).
>
> Why 148x (37x4) and not 147 or 149?
> Why a warning and maxsize forced to MAX_INT if maxsize > MAX_INT? Is it
> an internal constraint of FFmpeg which would be not able to store
> encoded frames bigger than MAX_INT?
>
> Can any of that be changed e.g. warning only once for avoiding "spam" of
> such warning which hides other warnings during encoding?
Real issue is in enforced limit of av_malloc size.
For answer to that question and others ask author who introduced such
limitation.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
