On Fri, Oct 12, 2018 at 04:23:43PM +0200, Jerome Martinez wrote: > On 12/10/2018 15:30, Jan Ekström wrote: > >And while it seems like we're focusing on the arbitrary limit in > >av_malloc (which is an issue of its own), I do note that for this > >specific case this "worst case packed frame size" allocation heuristic > >might have to be modified? Or at least explained if possible, so that > >things can be improved in a proper way. > > > Thank you for the feedback. > I was focused on ff_alloc_packet2() which accepts 64-bit integers, and I did > not check what is behind (limited to INT_MAX too). > I understand now that this is a global limit, not tied to FFV1 encoder (but > when this limit is removed from mem.h, there will be a review of ode which > test this limit before calling ff_alloc_packet2()...). > > As you said in the other answer, it was unrealistic to hit such limit in > 2005, but now... 2 GB by packet is still huge (the input frame is "only" 128 > MB), but the multiplier used in FFV1 encoder makes the limit hit for "basic" > 4K frames. 8K frames are approaching... > A bit complex for me to remove this 2 GB limit for sure everywhere :(, so I > try to find another solution.
you can increase the 2gb limit to whatever you like with av_max_alloc() / -max_alloc That will likely result in bugs appearing which you will need to fix also there may be other places hardcoding some limits that would need to be reviewed and updated If people feel the 2gb limit is an issue then we could attempt to disable it in the fuzzer and see what happens. (after some local testing first ...) > > > > > >For historical context, this limit has been inside ffv1enc as follows, > >number 1 being the current version: > >1. avctx->width*avctx->height*37LL*4 ("added support for > >AV_PIX_FMT_GBRP16" - ce2217b25eccda9f5c14022bd69792e71b417b73) > >2. avctx->width*avctx->height*35LL*4 ("avcodec/ffv1enc: fix size used > >for ff_alloc_packet2()" - 904a2864bdafe19d18db95ca54dfb36d72957a16) > >3. avctx->width*avctx->height*((8*2+1+1)*4)/8 ("ffv1enc: switch to > >encode2()." - 278d88689ba89c78f6c2667cf98025835567d78d) > >4. ??? Limit was probably somewhere elsewhere or didn't exist before this > > I see it but still don't catch the reason (and I am very curious to find an > input frame able to e.g. make an output frame 2x bigger than the input > frame). > Eager to have some background about it (Michael?). the limit is a result of the v3 version. Its a upper bound of the worst case. With sufficient work you can probably find a much more complex expression that would bound the size for large resolutions to a smaller value. You will have to consider the range coder states and how they would evolve during encoding in the worst case. Their evolution is bound to tend towards less bad in a large frame so there the expression for the max size will become more complex than a w*h*constant and you better arent scared of "math proofs" because this pretty much needs one. That is to decrease the max size you need to properly proof that that a new bound is large enough. The current bound should be reasonable easy to proof but the more tight the bound is the more work it will be also you can allocate a smaller buffer and reallocate before it becomes too small. Multithreaded encoding will make this more complex though. These problems are avoided in v4, this is what should be used really [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB "Nothing to hide" only works if the folks in power share the values of you and everyone you know entirely and always will -- Tom Scott
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel