Hi, On Tue, Oct 18, 2011 at 8:52 AM, Uoti Urpala <[email protected]> wrote: > On Tue, 2011-10-18 at 16:35 +0100, Måns Rullgård wrote: >> Uoti Urpala <[email protected]> writes: >> >> > On Tue, 2011-10-18 at 12:50 +0100, Måns Rullgård wrote: >> >> "Ronald S. Bultje" <[email protected]> writes: >> >> >> >> > + int64_t dd = d > SQRT_INT64_MAX ? ((d >> 1) * d) >> >> > >> 29 : (d * d) >> 30; >> >> > + int64_t ddd = d > SQRT_INT64_MAX || dd > >> >> > SQRT_INT64_MAX ? >> >> > + ((dd >> 2) * d) >> >> > >> 28 : (dd * d) >> 30; >> >> >> >> It is still overflowing. If d uses 32 bits, d*d >> 30 uses 34, and >> >> (dd >> 2) * d will still overflow. It needs >> 3 or some other >> >> equivalent change. >> > >> > Where did you get "32" from? I see no obvious reason why exactly that >> > would be the upper limit - the types are signed so 32 bits is not a >> > natural limit, and the values used look like it could be larger than 32 >> > bits unless there's something special which prevents that. >> >> I saw it use exactly 32 bits in testing. > > Well I just tested it and saw d = 134093824000 (37 bits). Sinc scaler > 1280x720->104x104.
That code only executes for the bicubic case, but you're right that there's no strict max, it can go all the way to 48bits for 192000 to 16 width scaling (idiotic example, I know). Ronald _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
