PR #20864 opened by thomasdullien URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20864 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20864.patch
For transparency: I am experimenting with an AI-assisted patch process, where the AI agent attempts to help root-cause analyze a crash by means of reproducing the crash with ASAN, making a recording with 'rr', and documenting the analysis at a granular level with verbatim quotes from the 'rr' trace. This root-cause analysis is then iteratively reviewed (e.g. as a human reviewer I check it for accuracy and plausibility) before a patch is generated. The process generates a detailed analysis report, an 'rr' trace that can be shared with other to help with the verification, and a patch. Given that it is unlear how to best share the 'rr' trace, I have only attached the detailed root-cause analysis document. This patch attempts to address https://trac.ffmpeg.org/ticket/11691 It should be noted that I am very much not an expert in the ffmpeg codebase, so please bear with me if the proposed patch isn't what is needed. I've re-run the test suite, and it looks very plausible to me. --------- Description ----------------- Many YUV-to-RGB conversion functions in yuv2rgb.c process 2 scanlines per iteration. When sws_scale() is called with insufficient source rows available (e.g., srcSliceY=1 with only 2 total rows, leaving 1 row), these functions would read beyond the allocated buffer bounds. This occurred because the convert_unscaled path did not verify that at least 2 rows were available before calling the conversion function. The issue was particularly evident with formats like yuv422p when converting to rgb4 or similar pixel formats. Add bounds checking in scale_internal() before invoking convert_unscaled to ensure at least 2 source rows remain after accounting for the slice offset. If fewer than 2 rows are available, return 0 (no rows processed) instead of attempting conversion, preventing the out-of-bounds access. Fixes: https://trac.ffmpeg.org/ticket/11691 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> ------------------------------------------- From 6138573dd0364eee7051eaa1ba27cebee738bb5a Mon Sep 17 00:00:00 2001 From: "Thomas Dullien (AI assisted)" <[email protected]> Date: Sat, 8 Nov 2025 14:30:28 +0100 Subject: [PATCH] swscale: Fix heap-buffer-overflow in unscaled YUV-to-RGB conversion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Many YUV-to-RGB conversion functions in yuv2rgb.c process 2 scanlines per iteration. When sws_scale() is called with insufficient source rows available (e.g., srcSliceY=1 with only 2 total rows, leaving 1 row), these functions would read beyond the allocated buffer bounds. This occurred because the convert_unscaled path did not verify that at least 2 rows were available before calling the conversion function. The issue was particularly evident with formats like yuv422p when converting to rgb4 or similar pixel formats. Add bounds checking in scale_internal() before invoking convert_unscaled to ensure at least 2 source rows remain after accounting for the slice offset. If fewer than 2 rows are available, return 0 (no rows processed) instead of attempting conversion, preventing the out-of-bounds access. Fixes: https://trac.ffmpeg.org/ticket/11691 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> --- libswscale/swscale.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/libswscale/swscale.c b/libswscale/swscale.c index f4c7eccac4..7086b0f0e2 100644 --- a/libswscale/swscale.c +++ b/libswscale/swscale.c @@ -1140,6 +1140,23 @@ static int scale_internal(SwsContext *sws, int offset = srcSliceY_internal; int slice_h = srcSliceH; + /* Many YUV-to-RGB conversion functions process 2 scanlines per iteration + * and require at least 2 rows to be available in the source buffer. + * Check if we have sufficient rows after considering the offset. */ + int src_rows_remaining = sws->src_h - offset; + if (slice_h > src_rows_remaining) { + /* Clamp to available rows */ + slice_h = src_rows_remaining; + } + if (slice_h < 2) { + /* Cannot safely process: conversion functions that process 2 rows + * per iteration would read beyond buffer bounds. Return 0 to indicate + * no rows were processed. This is a safety check to prevent + * heap-buffer-overflow in yuv2rgb.c and similar conversion functions. */ + ret = 0; + goto unscaled_end; + } + // for dst slice scaling, offset the pointers to match the unscaled API if (scale_dst) { av_assert0(offset == 0); @@ -1160,6 +1177,7 @@ static int scale_internal(SwsContext *sws, ret = c->convert_unscaled(c, src2, srcStride2, offset, slice_h, dst2, dstStride2); +unscaled_end: if (scale_dst) dst2[0] += dstSliceY * dstStride2[0]; } else { -- 2.49.1 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
