Hi! Attached patch fixes the following ubsan errors: libavcodec/motion_est.c:959:42: runtime error: left shift of negative value -16 libavcodec/motion_est.c:960:42: runtime error: left shift of negative value -16
I do not have a testcase for lines 961f. Please comment, Carl Eugen
From 4dd1b8d61c5dd2e77b7c855a1d2acf7d90277daf Mon Sep 17 00:00:00 2001 From: Carl Eugen Hoyos <ceho...@ag.or.at> Date: Tue, 18 Apr 2017 11:02:30 +0200 Subject: [PATCH] lavc/motion_est: Fix undefined negative left shifts. --- libavcodec/motion_est.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c index 25b606f..316d16a 100644 --- a/libavcodec/motion_est.c +++ b/libavcodec/motion_est.c @@ -956,10 +956,10 @@ void ff_estimate_p_frame_motion(MpegEncContext * s, P_TOPRIGHT[1] = s->current_picture.motion_val[0][mot_xy - mot_stride + 2][1]; if (P_TOP[1] > (c->ymax << shift)) P_TOP[1] = c->ymax << shift; - if (P_TOPRIGHT[0] < (c->xmin << shift)) - P_TOPRIGHT[0] = c->xmin << shift; - if (P_TOPRIGHT[1] > (c->ymax << shift)) - P_TOPRIGHT[1] = c->ymax << shift; + if (P_TOPRIGHT[0] < (c->xmin * (1 << shift))) + P_TOPRIGHT[0] = c->xmin * (1 << shift); + if (P_TOPRIGHT[1] > (c->ymax * (1 << shift))) + P_TOPRIGHT[1] = c->ymax * (1 << shift); P_MEDIAN[0] = mid_pred(P_LEFT[0], P_TOP[0], P_TOPRIGHT[0]); P_MEDIAN[1] = mid_pred(P_LEFT[1], P_TOP[1], P_TOPRIGHT[1]); -- 1.7.10.4
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel