PR #24676 opened by Marcos Ashton (MarcosAsh) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24676 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24676.patch
Makes the x86 16x16 and 32x32 HEVC idct use col_limit, which they ignored. The decoder derives it from the last significant coefficient, and in two 1080p x265 streams 72% of the 32x32 and 94% of the 16x16 calls have col_limit <= 8, so most of the first pass was transforming zero columns. It now skips the 4 column strips at or beyond col_limit, which leaves one or two of them in the common case. This relies on the columns past col_limit being zero. The only caller is ff_hevc_hls_residual_coding(), 16x16 and 32x32 always use the diagonal scan there, and the C second pass already depends on the same thing. The decoder never passes less than 4, and anything <= 0 still does the whole block. The checkasm test only ever passed col_limit equal to the block size, so the first commit adds a sparse check. It fills the block the way the decoder does for every last significant position, derives col_limit as cabac.c does, and also compares against the C function run with the full size, so a wrong col_limit from the test itself would show up too. It fails on an asm version that drops a needed column and passes on the unchanged asm. The bench runs at col_limit 8. Decoding the same two 20 s streams single threaded takes 5% fewer instructions and 2 to 3% fewer cycles. Full blocks cost the same as before. checkasm passes with 30 seeds, fate-hevc passes, and the framemd5 of both streams is identical before and after. Core Ultra 7 155H, best of five, pinned, before and after: ``` hevc_idct_16x16_8_sparse_sse2: 409.2 325.0 (1.26x) hevc_idct_16x16_8_sparse_avx: 409.4 325.3 (1.26x) hevc_idct_16x16_10_sparse_sse2: 410.2 327.4 (1.25x) hevc_idct_16x16_10_sparse_avx: 409.4 325.3 (1.26x) hevc_idct_32x32_8_sparse_sse2: 2212.7 1510.8 (1.46x) hevc_idct_32x32_8_sparse_avx: 2212.9 1513.1 (1.46x) hevc_idct_32x32_10_sparse_sse2: 2262.2 1498.2 (1.51x) hevc_idct_32x32_10_sparse_avx: 2209.8 1531.3 (1.44x) ``` >From ee49ad5d6623011ed6fe8efbf3611597dd714a5d Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Thu, 24 Sep 2026 19:22:55 +0100 Subject: [PATCH 1/2] tests/checkasm/hevc_idct: test sparse blocks with a real col_limit Only col_limit == block size was tested. Fill blocks like the decoder does for every last significant position, derive col_limit as in cabac.c, and also compare against C with the full block size. Bench col_limit 8, the most common value. --- tests/checkasm/hevc_idct.c | 65 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/tests/checkasm/hevc_idct.c b/tests/checkasm/hevc_idct.c index 139ae81727..076153d1aa 100644 --- a/tests/checkasm/hevc_idct.c +++ b/tests/checkasm/hevc_idct.c @@ -20,6 +20,7 @@ #include <string.h> +#include "libavutil/common.h" #include "libavutil/intreadwrite.h" #include "libavutil/mem_internal.h" @@ -36,11 +37,51 @@ } \ } while (0) +/* Fill the block the way the decoder does: coefficient groups and the + * coefficients inside them in up-right diagonal order, up to scan position + * last. Returns col_limit derived as in hevc/cabac.c. */ +static int fill_sparse(int16_t *coeffs, int log2_size, int last) +{ + int size = 1 << log2_size, ncg = size >> 2; + int n = 0, last_x = 0, last_y = 0, max_xy, col_limit; + + memset(coeffs, 0, sizeof(*coeffs) * size * size); + for (int d = 0; d < 2 * ncg - 1; d++) { + for (int x_cg = FFMAX(0, d - ncg + 1); x_cg <= FFMIN(d, ncg - 1); x_cg++) { + for (int e = 0; e < 7; e++) { + for (int x_c = FFMAX(0, e - 3); x_c <= FFMIN(e, 3); x_c++) { + int x = 4 * x_cg + x_c; + int y = 4 * (d - x_cg) + e - x_c; + + if (n > last) + goto done; + coeffs[y * size + x] = n == last ? (rnd() | 1) : + rnd() & 1 ? rnd() : 0; + last_x = x; + last_y = y; + n++; + } + } + } + } +done: + max_xy = FFMAX(last_x, last_y); + col_limit = last_x + last_y + 4; + if (max_xy < 4) + col_limit = FFMIN(4, col_limit); + else if (max_xy < 8) + col_limit = FFMIN(8, col_limit); + else if (max_xy < 12) + col_limit = FFMIN(24, col_limit); + return col_limit; +} + static void check_idct(HEVCDSPContext *h, int bit_depth) { int i; LOCAL_ALIGNED(32, int16_t, coeffs0, [32 * 32]); LOCAL_ALIGNED(32, int16_t, coeffs1, [32 * 32]); + LOCAL_ALIGNED(32, int16_t, coeffs2, [32 * 32]); for (i = 2; i <= 5; i++) { int block_size = 1 << i; @@ -57,6 +98,30 @@ static void check_idct(HEVCDSPContext *h, int bit_depth) fail(); bench_new(coeffs1, col_limit); } + + if (check_func(h->idct[i - 2], "hevc_idct_%dx%d_%d_sparse", block_size, block_size, bit_depth)) { + int last, limit; + + for (last = 0; last < size; last++) { + limit = fill_sparse(coeffs0, i, last); + memcpy(coeffs1, coeffs0, sizeof(*coeffs0) * size); + memcpy(coeffs2, coeffs0, sizeof(*coeffs0) * size); + call_ref(coeffs0, limit); + call_new(coeffs1, limit); + /* also check that the block is zero beyond limit */ + call_ref(coeffs2, block_size); + if (memcmp(coeffs0, coeffs1, sizeof(*coeffs0) * size) || + memcmp(coeffs0, coeffs2, sizeof(*coeffs0) * size)) { + fail(); + break; + } + } + /* bench col_limit 8, the most common value in real streams */ + for (last = 0; last < size; last++) + if ((limit = fill_sparse(coeffs1, i, last)) >= FFMIN(8, block_size)) + break; + bench_new(coeffs1, limit); + } } } -- 2.52.0 >From a32f23f4715d0ae0d79bf1f067039e5ac1921028 Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Thu, 24 Sep 2026 19:22:55 +0100 Subject: [PATCH 2/2] avcodec/x86/hevc: skip zero columns in the 16x16 and 32x32 idct Columns at or beyond col_limit are zero, so skip their strips in the first pass. Most 16x16 and 32x32 calls in real streams have col_limit <= 8. checkasm --bench on a Core Ultra 7 155H, before and after: hevc_idct_16x16_8_sparse_sse2: 409.2 325.0 (1.26x) hevc_idct_16x16_8_sparse_avx: 409.4 325.3 (1.26x) hevc_idct_16x16_10_sparse_sse2: 410.2 327.4 (1.25x) hevc_idct_16x16_10_sparse_avx: 409.4 325.3 (1.26x) hevc_idct_32x32_8_sparse_sse2: 2212.7 1510.8 (1.46x) hevc_idct_32x32_8_sparse_avx: 2212.9 1513.1 (1.46x) hevc_idct_32x32_10_sparse_sse2: 2262.2 1498.2 (1.51x) hevc_idct_32x32_10_sparse_avx: 2209.8 1531.3 (1.44x) HEVC decode: 5% fewer instructions, 2-3% fewer cycles. --- libavcodec/x86/hevc/idct.asm | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/libavcodec/x86/hevc/idct.asm b/libavcodec/x86/hevc/idct.asm index cfa156c705..06b9cd4cfb 100644 --- a/libavcodec/x86/hevc/idct.asm +++ b/libavcodec/x86/hevc/idct.asm @@ -698,8 +698,14 @@ cglobal hevc_idct_transpose_16x16, 0, 0, 0 ; void ff_hevc_idct_16x16_{8,10}_<opt>(int16_t *coeffs, int col_limit) ; %1 = bitdepth %macro IDCT_16x16 1 -cglobal hevc_idct_16x16_%1, 1, 2, 16, coeffs - mov r1d, 3 +cglobal hevc_idct_16x16_%1, 2, 3, 16, coeffs, col_limit + ; columns at or beyond col_limit are zero, skip their 4-column strips + ; (unsigned clamp: col_limit <= 0 does the whole block) + dec r1d + shr r1d, 2 + mov r2d, 3 + cmp r1d, r2d + cmova r1d, r2d .loop16: TR_16x4 8 * r1, 7, [pd_64], 64, 2, 32, 8, 16, 1, 0 dec r1d @@ -860,8 +866,14 @@ cglobal hevc_idct_transpose_32x32, 0, 0, 0 ; void ff_hevc_idct_32x32_{8,10}_<opt>(int16_t *coeffs, int col_limit) ; %1 = bitdepth %macro IDCT_32x32 1 -cglobal hevc_idct_32x32_%1, 1, 6, 16, 256, coeffs - mov r1d, 7 +cglobal hevc_idct_32x32_%1, 2, 6, 16, 256, coeffs, col_limit + ; columns at or beyond col_limit are zero, skip their 4-column strips + ; (unsigned clamp: col_limit <= 0 does the whole block) + dec r1d + shr r1d, 2 + mov r2d, 7 + cmp r1d, r2d + cmova r1d, r2d .loop32: TR_32x4 8 * r1, %1, 1 dec r1d -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
