PR #25898 opened by SomeComputerGuy
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25898
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25898.patch
Two MPEG-4 Part 2 decoder bugs found while checking the Allwinner H616
(cedrus) hardware decoder against the ISO/IEC 14496-4 conformance
streams, using the ISO/IEC 14496-5 reference decoder (vm_dec, Amd 5
2004) as the arbiter where FFmpeg and the hardware disagreed.
1/3 adds an encode/decode FATE test for interlaced B-VOPs in field
direct mode (no sample needed); 2/3 fixes the field direct vertical
vectors and updates its reference. The encoder shares
ff_mpeg4_set_direct_mv(), so its B-VOP reconstruction changes too;
the test bitstream itself is unchanged.
3/3 fixes the one point GMC average vector for
sprite_warping_accuracy 0 (RSHIFT(x, 0) != x for negative x).
A FATE test for it needs vcon-asp5L0 (38 KB) from the 14496-4
suite; I can send that test once the sample is in the FATE suite.
Numbers against vm_dec are in the commit messages. make fate passes.
Not fixed here: with frame threading, field direct B-VOPs read
p_field_mv_table of a thread that never decoded the co-located
P-VOP, so the field vectors are zero (output differs from -threads 1
on vcon_ge_1_asp_l4, vcon-asp14L4). The new test is pinned to
THREADS=1 for now; a fix will follow.
>From da2e99170d0d3585291ca6c77c2527f2e41c3d0c Mon Sep 17 00:00:00 2001
From: Arun Iyengar <[email protected]>
Date: Tue, 10 Nov 2026 02:05:00 +0000
Subject: [PATCH 1/3] fate/mpeg4: add an interlaced field direct mode test
Encodes a woven test pattern with field motion estimation and two
B-VOPs, so that direct mode B macroblocks find field predicted
co-located macroblocks, and decodes it again.
The reference is the current output; the next commit fixes the field
direct vertical vectors and updates it.
Assisted-by: LLM
Signed-off-by: Arun Iyengar <[email protected]>
---
tests/fate/mpeg4.mak | 11 ++++++++++-
tests/ref/fate/mpeg4-field-direct | 19 +++++++++++++++++++
2 files changed, 29 insertions(+), 1 deletion(-)
create mode 100644 tests/ref/fate/mpeg4-field-direct
diff --git a/tests/fate/mpeg4.mak b/tests/fate/mpeg4.mak
index d0c680bfc1e..82286736057 100644
--- a/tests/fate/mpeg4.mak
+++ b/tests/fate/mpeg4.mak
@@ -33,6 +33,15 @@ fate-mpeg4-reserved-color-tags: CMD = run
ffprobe$(PROGSSUF)$(EXESUF) -v error \
-show_entries frame=color_primaries,color_transfer,color_space -of
default=nw=1 \
$(TARGET_SAMPLES)/mpeg4/reserved-color-tags.m4v
+# Interlaced B-VOPs whose co-located P macroblocks use field prediction
+# (field direct mode). Single-threaded because frame threads do not share
+# the co-located field vectors yet.
+FATE_MPEG4_FFMPEG-$(call TRANSCODE, MPEG4, M4V, LAVFI_INDEV TESTSRC2_FILTER
WEAVE_FILTER MPEG4VIDEO_PARSER) += fate-mpeg4-field-direct
+fate-mpeg4-field-direct: override THREADS = 1
+fate-mpeg4-field-direct: CMD = transcode lavfi
"testsrc2=size=352x144:rate=50,weave" m4v \
+ "-c:v mpeg4 -qscale 5 -bf 2 -flags +ildct+ilme -mbd rd -frames:v 12"
+
FATE_SAMPLES_AVCONV += $(FATE_MPEG4-yes)
+FATE_FFMPEG += $(FATE_MPEG4_FFMPEG-yes)
FATE_SAMPLES_FFPROBE += $(FATE_MPEG4_FFPROBE-yes)
-fate-mpeg4: $(FATE_MPEG4-yes) $(FATE_MPEG4_FFPROBE-yes)
+fate-mpeg4: $(FATE_MPEG4-yes) $(FATE_MPEG4_FFPROBE-yes)
$(FATE_MPEG4_FFMPEG-yes)
diff --git a/tests/ref/fate/mpeg4-field-direct
b/tests/ref/fate/mpeg4-field-direct
new file mode 100644
index 00000000000..25ed9c67e9f
--- /dev/null
+++ b/tests/ref/fate/mpeg4-field-direct
@@ -0,0 +1,19 @@
+c10554fc04606c8bf1de280012f40bf4 *tests/data/fate/mpeg4-field-direct.m4v
+56579 tests/data/fate/mpeg4-field-direct.m4v
+#tb 0: 1/25
+#media_type 0: video
+#codec_id 0: rawvideo
+#dimensions 0: 352x288
+#sar 0: 1/1
+0, 0, 0, 1, 152064, 0xbd4707e9
+0, 1, 1, 1, 152064, 0x435f271c
+0, 2, 2, 1, 152064, 0x3a8335b6
+0, 3, 3, 1, 152064, 0x226448de
+0, 4, 4, 1, 152064, 0xd7888021
+0, 5, 5, 1, 152064, 0xffc2cd53
+0, 6, 6, 1, 152064, 0xac0208ec
+0, 7, 7, 1, 152064, 0x78ce673c
+0, 8, 8, 1, 152064, 0x12c1a292
+0, 9, 9, 1, 152064, 0x0644f093
+0, 10, 10, 1, 152064, 0xb9f74c17
+0, 11, 11, 1, 152064, 0x1f51546a
--
2.54.0
>From 9fc6fbbb28faf3ccbab01db5bd96df9600e5368e Mon Sep 17 00:00:00 2001
From: Arun Iyengar <[email protected]>
Date: Thu, 5 Nov 2026 06:07:38 +0000
Subject: [PATCH 2/3] avcodec/mpeg4video: scale field direct vertical vectors
at frame precision
In field direct mode (interlaced B-VOP, co-located macroblock field
predicted) the vertical component of each field vector was scaled in
field units:
mv = (field_mv * TRB) / TRD + MVD
The ISO/IEC 14496-5 reference decoder (sys/mc.cpp) keeps the co-located
field vector in frame units, i.e. twice the field vector, does the
division and adds the delta there, and only then drops the half line.
Because the division truncates, the two disagree by one field line
whenever the frame unit quotient is odd, and the error is visible as
whole macroblock rows shifted by one line in B-VOPs.
Do the computation at frame precision and halve the result, for both
the forward and the backward vector. The encoder uses the same
function, so its B-VOP reconstruction now matches what a conformant
decoder produces.
Compared with the reference decoder (vm_dec, 14496-5 Amd 5 2004),
luma macroblocks with a maximum absolute difference > 4, -threads 1:
vcon_ge_1_asp_l4 (14496-4, 1920x1088, 41 frames): 32571 -> 0 (max 2)
vcon-asp14L4 (14496-4, 352x576, 298 frames): 3082 -> 0 (max 3)
fate-mpeg4-field-direct stream (12 frames): 26 -> 0 (max 2)
The remaining differences are IDCT rounding. The Allwinner H616 video
engine (cedrus) decodes these streams the same way as the reference.
Assisted-by: LLM
Signed-off-by: Arun Iyengar <[email protected]>
---
libavcodec/mpeg4video.c | 15 +++++++++------
tests/ref/fate/mpeg4-field-direct | 14 +++++++-------
2 files changed, 16 insertions(+), 13 deletions(-)
diff --git a/libavcodec/mpeg4video.c b/libavcodec/mpeg4video.c
index a53ce72dfd5..c8fe1dc662a 100644
--- a/libavcodec/mpeg4video.c
+++ b/libavcodec/mpeg4video.c
@@ -147,16 +147,19 @@ int ff_mpeg4_set_direct_mv(MpegEncContext *s, int mx, int
my)
}
s->mv[0][i][0] = s->p_field_mv_table[i][0][mb_index][0] *
time_pb / time_pp + mx;
- s->mv[0][i][1] = s->p_field_mv_table[i][0][mb_index][1] *
- time_pb / time_pp + my;
+ /* ISO/IEC 14496-5 scales the vertical field vector in frame
+ * units (twice the field vector), then drops the half line. */
+ s->mv[0][i][1] = (2 * s->p_field_mv_table[i][0][mb_index][1] *
+ time_pb / time_pp + my) >> 1;
s->mv[1][i][0] = mx ? s->mv[0][i][0] -
s->p_field_mv_table[i][0][mb_index][0]
: s->p_field_mv_table[i][0][mb_index][0] *
(time_pb - time_pp) / time_pp;
- s->mv[1][i][1] = my ? s->mv[0][i][1] -
- s->p_field_mv_table[i][0][mb_index][1]
- : s->p_field_mv_table[i][0][mb_index][1] *
- (time_pb - time_pp) / time_pp;
+ s->mv[1][i][1] = (my ? 2 * s->p_field_mv_table[i][0][mb_index][1] *
+ time_pb / time_pp + my -
+ 2 *
s->p_field_mv_table[i][0][mb_index][1]
+ : 2 * s->p_field_mv_table[i][0][mb_index][1] *
+ (time_pb - time_pp) / time_pp) >> 1;
}
return MB_TYPE_DIRECT2 | MB_TYPE_16x8 |
MB_TYPE_BIDIR_MV | MB_TYPE_INTERLACED;
diff --git a/tests/ref/fate/mpeg4-field-direct
b/tests/ref/fate/mpeg4-field-direct
index 25ed9c67e9f..6ba7aea0f18 100644
--- a/tests/ref/fate/mpeg4-field-direct
+++ b/tests/ref/fate/mpeg4-field-direct
@@ -6,14 +6,14 @@ c10554fc04606c8bf1de280012f40bf4
*tests/data/fate/mpeg4-field-direct.m4v
#dimensions 0: 352x288
#sar 0: 1/1
0, 0, 0, 1, 152064, 0xbd4707e9
-0, 1, 1, 1, 152064, 0x435f271c
-0, 2, 2, 1, 152064, 0x3a8335b6
+0, 1, 1, 1, 152064, 0x09502603
+0, 2, 2, 1, 152064, 0x5be134b9
0, 3, 3, 1, 152064, 0x226448de
-0, 4, 4, 1, 152064, 0xd7888021
-0, 5, 5, 1, 152064, 0xffc2cd53
+0, 4, 4, 1, 152064, 0x1aeb8038
+0, 5, 5, 1, 152064, 0x1599cd5d
0, 6, 6, 1, 152064, 0xac0208ec
-0, 7, 7, 1, 152064, 0x78ce673c
-0, 8, 8, 1, 152064, 0x12c1a292
+0, 7, 7, 1, 152064, 0xa3716848
+0, 8, 8, 1, 152064, 0x4984a3c8
0, 9, 9, 1, 152064, 0x0644f093
-0, 10, 10, 1, 152064, 0xb9f74c17
+0, 10, 10, 1, 152064, 0x1a8d4c1b
0, 11, 11, 1, 152064, 0x1f51546a
--
2.54.0
>From 363d7f7538ab5486f8052bce1adaef7525039d74 Mon Sep 17 00:00:00 2001
From: Arun Iyengar <[email protected]>
Date: Thu, 5 Nov 2026 06:08:06 +0000
Subject: [PATCH 3/3] avcodec/mpeg4videodec: do not round the GMC1 average MV
with a zero shift
With one sprite warping point, get_amv() returns the sprite offset
rounded by RSHIFT(offset << quarter_sample, sprite_warping_accuracy).
For sprite_warping_accuracy 0 the shift is 0, and RSHIFT(a, 0) is not
a: its rounding term (1 << 0) >> 1 is 0, but the branch for negative
values still subtracts 1, so a negative offset comes out one unit too
small. The GMC macroblock MV then differs from the one the motion
compensation used, and every MV predicted from it is off by one unit:
whole pixel columns are wrong in inter macroblocks next to GMC
macroblocks, and the error propagates.
With a zero shift there is nothing to round, so use the value as is.
Compared with the ISO/IEC 14496-5 reference decoder (vm_dec, Amd 5
2004), macroblocks with a maximum absolute difference > 4 (luma /
chroma), -threads 1:
vcon-asp5L0 (14496-4, 176x144, 150 frames): 2953 / 149 -> 1 / 0
vcon-asp14L1 (14496-4, 176x144, 299 frames): 13186 / 994 -> 0 / 300
The chroma macroblocks left in vcon-asp14L1 are all in B-VOPs, where
the reference decoder also disagrees with the Allwinner H616 video
engine; the engine matches FFmpeg with this patch within 2 on both
streams.
Assisted-by: LLM
Signed-off-by: Arun Iyengar <[email protected]>
---
libavcodec/mpeg4videodec.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/libavcodec/mpeg4videodec.c b/libavcodec/mpeg4videodec.c
index 03b145379df..0418f89dc38 100644
--- a/libavcodec/mpeg4videodec.c
+++ b/libavcodec/mpeg4videodec.c
@@ -862,7 +862,8 @@ static inline int get_amv(Mpeg4DecContext *ctx, int n)
if (ctx->divx_version == 500 && ctx->divx_build == 413 && a >=
s->quarter_sample)
sum = ctx->sprite_offset[0][n] / (1 << (a - s->quarter_sample));
else
- sum = RSHIFT(ctx->sprite_offset[0][n] * (1 << s->quarter_sample),
a);
+ sum = a ? RSHIFT(ctx->sprite_offset[0][n] * (1 <<
s->quarter_sample), a)
+ : ctx->sprite_offset[0][n] * (1 << s->quarter_sample);
} else {
dx = ctx->sprite_delta[n][0];
dy = ctx->sprite_delta[n][1];
--
2.54.0
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]