PR #25492 opened by Jake Stebbings (ProductionVideo) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25492 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25492.patch
NotchLC codes alpha per 4x4 block: mode 0 transparent, 1 opaque, 2 interpolated. Each interpolated block stores its own 8 bytes (endpoints a0, a1 and sixteen 3-bit selectors, one per pixel), with alpha = a0 + (a1 - a0) * s / 7 on a 0..255 scale. The decoder reads one 8-byte block per 16x16 macroblock and reuses it for every interpolated block in it, applies one selector per 4x4 block instead of per pixel, and scales with << 4 without the division by 7. On Notch-exported files with feathered edges this gives wrong alpha on almost every partially transparent pixel, and values up to 28560 (= 255 * 7 * 16) in a 12-bit alpha plane. Transparent/opaque blocks and the Y/U/V planes are unaffected. Patch 1 fixes the decoder. Patch 2 adds `fate-notchlc-alpha`. Testing - On Notch-exported production files with feathered alpha (not shareable): 150 frames, about 20 million partially transparent pixels; alpha matches an independent decoder exactly at 12 bits, and Y/U/V are byte-identical to the previous output. - `make fate` (no samples) passes: 3072 tests. - `fate-notchlc-alpha` passes with patch 1 and fails without it. Sample for the FATE suite (synthetic, 1889 bytes, no third-party content), to be added as `notchlc/notchlc_alpha_interp.mov`: https://raw.githubusercontent.com/ProductionVideo/nitpix/main/tools/notchlc-sample/notchlc_alpha_interp.mov MD5 28a85c7887f6d759f2e0509441faf159. Its generator, which also writes the expected 12-bit alpha, is alongside it: https://github.com/ProductionVideo/nitpix/tree/main/tools/notchlc-sample Disclosure: these patches were prepared with the help of an AI assistant (Claude, by Anthropic). I have reviewed and tested them and take responsibility for them. >From e4c15c8f1f1c54cd58aae3c2f57216fb25ad41c6 Mon Sep 17 00:00:00 2001 From: Jake Stebbings <[email protected]> Date: Mon, 26 Oct 2026 15:38:39 +0000 Subject: [PATCH 1/2] avcodec/notchlc: fix decoding of interpolated alpha blocks Alpha in NotchLC is coded per 4x4 block with a 2-bit mode: 0 transparent, 1 opaque, 2 interpolated. Each interpolated block stores its own 8 bytes: two 8-bit endpoints a0, a1 and sixteen 3-bit selectors s, one per pixel, with alpha = a0 + (a1 - a0) * s / 7 on a 0..255 scale. The decoder got three things wrong in the interpolated case: - it read one 8-byte block per 16x16 macroblock and reused it for every interpolated 4x4 block in that macroblock, instead of reading the next 8 bytes for each interpolated block; - it applied a single selector to all 16 pixels of a block, advancing the selector once per block instead of once per pixel; - it scaled with << 4 and no division by 7, so a full ramp produced 255 * 7 * 16 = 28560 in a 12-bit plane whose maximum is 4095, and a falling ramp wrapped around. Feathered edges therefore decoded with wrong alpha, and values far outside the 12-bit range reached the output. Transparent and opaque blocks, and the luma and chroma planes, were not affected. Read each interpolated block's own data, use one selector per pixel, and scale (a0 * 7 + (a1 - a0) * s) / 1785 to 12 bits with rounding. Tested on Notch-exported files with feathered alpha (not shareable): over 150 frames and about 20 million partially transparent pixels, alpha now matches an independent decoder implementation exactly at 12 bits, and the Y, U and V planes are byte-identical to the previous output. This patch was prepared with the help of an AI assistant (Claude, by Anthropic); I have reviewed and tested it and take responsibility for it. Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Jake Stebbings <[email protected]> --- libavcodec/notchlc.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/libavcodec/notchlc.c b/libavcodec/notchlc.c index 396089254e6..d494053622a 100644 --- a/libavcodec/notchlc.c +++ b/libavcodec/notchlc.c @@ -308,10 +308,6 @@ static int decode_blocks(AVCodecContext *avctx, AVFrame *p, return AVERROR_INVALIDDATA; bytestream2_seek(&dgb, offset, SEEK_SET); - control = bytestream2_get_le64(&dgb); - alpha0 = control & 0xFF; - alpha1 = (control >> 8) & 0xFF; - control = control >> 16; for (int by = 0; by < 4; by++) { for (int bx = 0; bx < 4; bx++) { @@ -331,9 +327,20 @@ static int decode_blocks(AVCodecContext *avctx, AVFrame *p, } break; case 2: + /* Each interpolated 4x4 block stores its own 8 bytes: + * two 8-bit endpoints and sixteen 3-bit selectors, + * alpha = a0 + (a1 - a0) * s / 7, scaled from 8 to 12 bits. */ + if (bytestream2_get_bytes_left(&dgb) < 8) + return AVERROR_INVALIDDATA; + control = bytestream2_get_le64(&dgb); + alpha0 = control & 0xFF; + alpha1 = (control >> 8) & 0xFF; + control >>= 16; for (int i = 0; i < 4; i++) { for (int j = 0; j < 4; j++) { - dsta[x + (i + by * 4) * alinesize + bx * 4 + j] = (alpha0 + (alpha1 - alpha0) * (control & 7)) << 4; + int v = (int)alpha0 * 7 + ((int)alpha1 - (int)alpha0) * (int)(control & 7); + dsta[x + (i + by * 4) * alinesize + bx * 4 + j] = (v * 4095 + 892) / 1785; + control >>= 3; } } break; @@ -341,7 +348,6 @@ static int decode_blocks(AVCodecContext *avctx, AVFrame *p, return AVERROR_INVALIDDATA; } - control >>= 3; m >>= 2; } } -- 2.54.0 >From 1d446802727d1946a098072b4260616e332b734e Mon Sep 17 00:00:00 2001 From: Jake Stebbings <[email protected]> Date: Mon, 26 Oct 2026 15:38:39 +0000 Subject: [PATCH 2/2] fate/video: add NotchLC interpolated alpha test The sample is a synthetic 64x32, 2-frame NotchLC MOV (1889 bytes, LZ4 envelope, flat grey picture) whose alpha mixes transparent, opaque and interpolated 4x4 blocks: every interpolated block has its own endpoints, selectors vary per pixel, and full and reversed 0..255 ramps are included. It contains no third-party picture content. The sample needs to be added to the FATE suite as notchlc/notchlc_alpha_interp.mov. The output of the previous decoder fails this test. This patch was prepared with the help of an AI assistant (Claude, by Anthropic); I have reviewed and tested it and take responsibility for it. Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Jake Stebbings <[email protected]> --- tests/fate/video.mak | 3 +++ tests/ref/fate/notchlc-alpha | 7 +++++++ 2 files changed, 10 insertions(+) create mode 100644 tests/ref/fate/notchlc-alpha diff --git a/tests/fate/video.mak b/tests/fate/video.mak index cd5459db3e6..13a3e930fde 100644 --- a/tests/fate/video.mak +++ b/tests/fate/video.mak @@ -301,6 +301,9 @@ fate-mv-sgirle: CMD = framecrc -i $(TARGET_SAMPLES)/mv/pet-rle.movie -an FATE_VIDEO-$(call FRAMECRC, MXG, MXPEG) += fate-mxpeg fate-mxpeg: CMD = framecrc -idct simple -flags +bitexact -i $(TARGET_SAMPLES)/mxpeg/m1.mxg -an +FATE_VIDEO-$(call FRAMECRC, MOV, NOTCHLC) += fate-notchlc-alpha +fate-notchlc-alpha: CMD = framecrc -i $(TARGET_SAMPLES)/notchlc/notchlc_alpha_interp.mov -pix_fmt yuva444p12le + FATE_NUV += fate-nuv-rtjpeg fate-nuv-rtjpeg: CMD = framecrc -idct simple -i $(TARGET_SAMPLES)/nuv/Today.nuv -an -enc_time_base demux diff --git a/tests/ref/fate/notchlc-alpha b/tests/ref/fate/notchlc-alpha new file mode 100644 index 00000000000..da44023000a --- /dev/null +++ b/tests/ref/fate/notchlc-alpha @@ -0,0 +1,7 @@ +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 64x32 +#sar 0: 0/1 +0, 0, 0, 1, 16384, 0x54fef7b8 +0, 1, 1, 1, 16384, 0xa6aefe28 -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
