PR #23490 opened by Bogdan Lisman (bogdanpydev)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23490
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23490.patch
ljpeg_encode_yuv_mb() reads the source samples of each macroblock at
ptr = data[i] + linesize * (v * mb_y + y) + (h * mb_x + x);
The macroblock counts are rounded up (mb_width = ceil(width / hsample[0]),
mb_height = ceil(height / vsample[0])), so when a plane dimension is not a
multiple of the sampling factor the last macroblock addresses one sample
past the right and/or bottom edge of the plane. For yuvj420p this triggers
for any odd width or height; encoding a 999x999 frame reads 1999 samples
out of bounds.
Clamp the read coordinates to the last valid sample (edge replication).
For dimensions that are a multiple of the sampling factor the clamp never
triggers, so the encoded output is bit-exact and there is no functional
change; the over-read only touched padding samples that the decoder crops
away, which is why it is not visible in the decoded output and escapes
valgrind (the read stays inside the pooled frame allocation).
Fixes: ticket #10052
Signed-off-by: Bogdan Lisman <[email protected]>
# Summary of changes
Briefly describe what this PR does and why.
<!--
If this PR requires new FATE test samples, attach them to the PR and
list their target paths below (relative to the fate-suite root).
Attached filenames must match the sample's filename:
```fate-samples
# e.g. vorbis/new-sample.ogg
```
-->
>From e1650dcd846eb86d172d378d5f467e95d7984759 Mon Sep 17 00:00:00 2001
From: Bogdan Lisman <[email protected]>
Date: Mon, 15 Jun 2026 00:08:07 +0300
Subject: [PATCH] avcodec/ljpegenc: fix out-of-bounds read for odd dimensions
ljpeg_encode_yuv_mb() reads the source samples of each macroblock at
ptr = data[i] + linesize * (v * mb_y + y) + (h * mb_x + x);
The macroblock counts are rounded up (mb_width = ceil(width / hsample[0]),
mb_height = ceil(height / vsample[0])), so when a plane dimension is not a
multiple of the sampling factor the last macroblock addresses one sample
past the right and/or bottom edge of the plane. For yuvj420p this triggers
for any odd width or height; encoding a 999x999 frame reads 1999 samples
out of bounds.
Clamp the read coordinates to the last valid sample (edge replication).
For dimensions that are a multiple of the sampling factor the clamp never
triggers, so the encoded output is bit-exact and there is no functional
change; the over-read only touched padding samples that the decoder crops
away, which is why it is not visible in the decoded output and escapes
valgrind (the read stays inside the pooled frame allocation).
Fixes: ticket #10052
Signed-off-by: Bogdan Lisman <[email protected]>
---
libavcodec/ljpegenc.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/libavcodec/ljpegenc.c b/libavcodec/ljpegenc.c
index ab37ab0d5c..a9bd2fd5a7 100644
--- a/libavcodec/ljpegenc.c
+++ b/libavcodec/ljpegenc.c
@@ -127,16 +127,18 @@ static inline void ljpeg_encode_yuv_mb(LJpegEncContext
*s, PutBitContext *pb,
if (mb_x == 0 || mb_y == 0) {
for (i = 0; i < 3; i++) {
const uint8_t *ptr;
- int x, y, h, v, linesize;
+ int x, y, h, v, linesize, last_x, last_y;
h = s->hsample[i];
v = s->vsample[i];
linesize = frame->linesize[i];
+ last_x = (frame->width * h - 1) / s->hsample[0];
+ last_y = (frame->height * v - 1) / s->vsample[0];
for (y = 0; y < v; y++) {
for (x = 0; x < h; x++) {
int pred;
- ptr = frame->data[i] + (linesize * (v * mb_y + y)) + (h *
mb_x + x); //FIXME optimize this crap
+ ptr = frame->data[i] + linesize * FFMIN(v * mb_y + y,
last_y) + FFMIN(h * mb_x + x, last_x); //FIXME optimize this crap
if (y == 0 && mb_y == 0) {
if (x == 0 && mb_x == 0)
pred = 128;
@@ -161,16 +163,18 @@ static inline void ljpeg_encode_yuv_mb(LJpegEncContext
*s, PutBitContext *pb,
} else {
for (i = 0; i < 3; i++) {
const uint8_t *ptr;
- int x, y, h, v, linesize;
+ int x, y, h, v, linesize, last_x, last_y;
h = s->hsample[i];
v = s->vsample[i];
linesize = frame->linesize[i];
+ last_x = (frame->width * h - 1) / s->hsample[0];
+ last_y = (frame->height * v - 1) / s->vsample[0];
for (y = 0; y < v; y++) {
for (x = 0; x < h; x++) {
int pred;
- ptr = frame->data[i] + (linesize * (v * mb_y + y)) + (h *
mb_x + x); //FIXME optimize this crap
+ ptr = frame->data[i] + linesize * FFMIN(v * mb_y + y,
last_y) + FFMIN(h * mb_x + x, last_x); //FIXME optimize this crap
PREDICT(pred, ptr[-linesize - 1], ptr[-linesize], ptr[-1],
predictor);
if (i == 0)
--
2.52.0
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
