PR #25899 opened by Marti2203 URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25899 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25899.patch
Follow-up to #25514, for the minor issue noted in the Forgejo_Fairy review there. This PR is on top of #25514 and includes its commit; only the second commit is new. With a cursor clipped at the left edge, `put_cursor()` starts reading the cursor image and mask at a column offset, so after the last row the `cd`/`msk` increments move the pointers past the end of the `cur_w * cur_h` buffers (e.g. a 4x4 cursor with hot spot (2,1) at (0,0) ends at offset 18 of 16). They are never dereferenced, but forming such a pointer is undefined. The patch keeps the pointers at the start of the buffers and advances the integer offset instead, in all three pixel-size branches. It adds no branch to the row loop. Tested on an ASan build: fate-vmnc-16bit and fate-vmnc-32bit pass, and the 4-frame clipping stream from #25514 (left, top, left+top and unclipped cursor, 32 bpp) gives the same framemd5 as #25514 alone. This issue was found during an LLM review (Forgejo_Fairy, #25514); the fix was verified manually. >From 13f87badba8e8db7b3b0f21475fc31a92f0120b7 Mon Sep 17 00:00:00 2001 From: Martin Mirchev <[email protected]> Date: Tue, 27 Oct 2026 10:47:50 +0800 Subject: [PATCH 1/2] avcodec/vmnc: fix cursor drawing when clipped at the left or top edge When the cursor is partially off the left or top edge of the screen, put_cursor() clips the destination rectangle but still reads the cursor image and mask from their first column and row. The visible part of the cursor is thus drawn with the pixels of the part that is off screen, i.e. shifted by the clipped amount. Start reading the cursor image and mask at the first visible column and row. Signed-off-by: Martin Mirchev <[email protected]> --- libavcodec/vmnc.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/libavcodec/vmnc.c b/libavcodec/vmnc.c index 162e8ee9bb7..b5198dd949a 100644 --- a/libavcodec/vmnc.c +++ b/libavcodec/vmnc.c @@ -131,6 +131,7 @@ static void put_cursor(uint8_t *dst, int stride, VmncContext *c, int dx, int dy) { int i, j; int w, h, x, y; + int skip = 0; w = c->cur_w; if (c->width < c->cur_x + c->cur_w) w = c->width - c->cur_x; @@ -140,12 +141,14 @@ static void put_cursor(uint8_t *dst, int stride, VmncContext *c, int dx, int dy) x = c->cur_x; y = c->cur_y; if (x < 0) { - w += x; - x = 0; + w += x; + skip = -x; + x = 0; } if (y < 0) { - h += y; - y = 0; + h += y; + skip -= y * c->cur_w; + y = 0; } if ((w < 1) || (h < 1)) @@ -153,7 +156,7 @@ static void put_cursor(uint8_t *dst, int stride, VmncContext *c, int dx, int dy) dst += x * c->bpp2 + y * stride; if (c->bpp2 == 1) { - uint8_t *cd = c->curbits, *msk = c->curmask; + uint8_t *cd = c->curbits + skip, *msk = c->curmask + skip; for (j = 0; j < h; j++) { for (i = 0; i < w; i++) dst[i] = (dst[i] & cd[i]) ^ msk[i]; @@ -162,7 +165,7 @@ static void put_cursor(uint8_t *dst, int stride, VmncContext *c, int dx, int dy) dst += stride; } } else if (c->bpp2 == 2) { - uint16_t *cd = (uint16_t*)c->curbits, *msk = (uint16_t*)c->curmask; + uint16_t *cd = (uint16_t*)c->curbits + skip, *msk = (uint16_t*)c->curmask + skip; uint16_t *dst2; for (j = 0; j < h; j++) { dst2 = (uint16_t*)dst; @@ -173,7 +176,7 @@ static void put_cursor(uint8_t *dst, int stride, VmncContext *c, int dx, int dy) dst += stride; } } else if (c->bpp2 == 4) { - uint32_t *cd = (uint32_t*)c->curbits, *msk = (uint32_t*)c->curmask; + uint32_t *cd = (uint32_t*)c->curbits + skip, *msk = (uint32_t*)c->curmask + skip; uint32_t *dst2; for (j = 0; j < h; j++) { dst2 = (uint32_t*)dst; -- 2.54.0 >From 0d9eae579a03dd6ec902dbd1b42f97552b4f45b5 Mon Sep 17 00:00:00 2001 From: Martin Mirchev <[email protected]> Date: Tue, 10 Nov 2026 11:58:24 +0800 Subject: [PATCH 2/2] avcodec/vmnc: index the cursor rows instead of advancing the pointers With a cursor clipped at the left edge, put_cursor() starts reading the image and mask at a column offset, and after the last row the source pointers are advanced past the end of the cursor buffers. They are not dereferenced, but forming such a pointer is undefined. Keep the pointers at the start of the buffers and advance the integer offset instead. Reported in the review of #25514. Signed-off-by: Martin Mirchev <[email protected]> --- libavcodec/vmnc.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/libavcodec/vmnc.c b/libavcodec/vmnc.c index b5198dd949a..86949abd32a 100644 --- a/libavcodec/vmnc.c +++ b/libavcodec/vmnc.c @@ -156,34 +156,31 @@ static void put_cursor(uint8_t *dst, int stride, VmncContext *c, int dx, int dy) dst += x * c->bpp2 + y * stride; if (c->bpp2 == 1) { - uint8_t *cd = c->curbits + skip, *msk = c->curmask + skip; + uint8_t *cd = c->curbits, *msk = c->curmask; for (j = 0; j < h; j++) { for (i = 0; i < w; i++) - dst[i] = (dst[i] & cd[i]) ^ msk[i]; - msk += c->cur_w; - cd += c->cur_w; + dst[i] = (dst[i] & cd[skip + i]) ^ msk[skip + i]; + skip += c->cur_w; dst += stride; } } else if (c->bpp2 == 2) { - uint16_t *cd = (uint16_t*)c->curbits + skip, *msk = (uint16_t*)c->curmask + skip; + uint16_t *cd = (uint16_t*)c->curbits, *msk = (uint16_t*)c->curmask; uint16_t *dst2; for (j = 0; j < h; j++) { dst2 = (uint16_t*)dst; for (i = 0; i < w; i++) - dst2[i] = (dst2[i] & cd[i]) ^ msk[i]; - msk += c->cur_w; - cd += c->cur_w; + dst2[i] = (dst2[i] & cd[skip + i]) ^ msk[skip + i]; + skip += c->cur_w; dst += stride; } } else if (c->bpp2 == 4) { - uint32_t *cd = (uint32_t*)c->curbits + skip, *msk = (uint32_t*)c->curmask + skip; + uint32_t *cd = (uint32_t*)c->curbits, *msk = (uint32_t*)c->curmask; uint32_t *dst2; for (j = 0; j < h; j++) { dst2 = (uint32_t*)dst; for (i = 0; i < w; i++) - dst2[i] = (dst2[i] & cd[i]) ^ msk[i]; - msk += c->cur_w; - cd += c->cur_w; + dst2[i] = (dst2[i] & cd[skip + i]) ^ msk[skip + i]; + skip += c->cur_w; dst += stride; } } -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
