PR #23925 opened by Niklas Haas (haasn) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23925 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23925.patch
This breaks the design philosophy of SwsOp just a little bit, in that it's a fairly complex non-atomic operation involving multiple steps. However, the semantics are so tightly married to the exact `lut3d.h` implementation details, that I don't think it's worth intentionally obscuring that relationship just for the sake of conceptual purity. In theory, we could treat these as three separate meta-uops e.g. SWS_UOP_LUT_TETRAHEDRAL, SWS_UOP_TONE_MAP, and SWS_UOP_LUT_TRILINEAR; with an explicit SWS_UOP_SCALE in between for the intermediate step between the tone mapping and trilinear step, but that prevents some fusion (e.g. fusing the scale adjustment directly into the `bias` and `scale` values for the PT channel desaturation). If it ever comes to it, we can still roll this decision back and generalize this uop some more. This PR only implements the C reference. I suspect that a properly vectorized SIMD solution is much faster, but I will follow up with that in a separate PR as it is quite an undertaking. From 452f52b1066246d9a464ddc450dab96475123b20 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Sun, 21 Jun 2026 16:51:03 +0200 Subject: [PATCH 01/18] swscale/ops: fix merge_comp_flags() for SWS_COMP_SWAPPED This violates the documentation (monoid property). It's a bit arbitrary whether to consider this an OR-type or AND-type flag, since mixing swapped and non-swapped components is almost surely a bug, but keeping it as an OR-type makes sure such cases at least show up in the result. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/ops.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libswscale/ops.c b/libswscale/ops.c index 0582a32345..1f555e9479 100644 --- a/libswscale/ops.c +++ b/libswscale/ops.c @@ -281,7 +281,7 @@ enum { /* merge_comp_flags() forms a monoid with SWS_COMP_IDENTITY as the null element */ static SwsCompFlags merge_comp_flags(SwsCompFlags a, SwsCompFlags b) { - const SwsCompFlags flags_or = SWS_COMP_GARBAGE; + const SwsCompFlags flags_or = SWS_COMP_GARBAGE | SWS_COMP_SWAPPED; const SwsCompFlags flags_and = SWS_COMP_IDENTITY; return ((a & b) & flags_and) | ((a | b) & flags_or); } -- 2.52.0 From 173a6d72c737ad7e9bfd0c25e3ee28083e8661c6 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Thu, 25 Jun 2026 18:31:13 +0200 Subject: [PATCH 02/18] swscale/ops: refactor comp flags propagation slightly Introduce a unified FORWARD() helper macro that can be used for any type of op, whether it is independent per component or more complex. By initializing every op to the same IDENTITY state, we can leverage the monoid property to make this work for naive propagations as well. As an aside, we also properly zero out the unrelated fields when discarding a component (i.e. marking it as GARBAGE). This will make a couple of up-coming refactors a bit easier. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/ops.c | 73 ++++++++++++++++++++++++++++-------------------- 1 file changed, 42 insertions(+), 31 deletions(-) diff --git a/libswscale/ops.c b/libswscale/ops.c index 1f555e9479..be3e9853a9 100644 --- a/libswscale/ops.c +++ b/libswscale/ops.c @@ -331,6 +331,19 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) break; } + for (int i = 0; i < 4; i++) + op->comps.flags[i] = SWS_COMP_IDENTITY; + + #define FORWARD(I, J, EXPR) do { \ + SwsCompFlags flags = prev.flags[J]; \ + op->comps.flags[I] = merge_comp_flags(op->comps.flags[I], (EXPR)); \ + } while (0) + + #define RESET(I) do { \ + op->comps.flags[I] = SWS_COMP_GARBAGE; \ + op->comps.min[I] = op->comps.max[I] = (AVRational64) {0}; \ + } while (0) + switch (op->op) { case SWS_OP_READ: /* Active components are taken from the user-provided values, @@ -366,27 +379,27 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) break; case SWS_OP_SWAP_BYTES: for (int i = 0; i < 4; i++) { - op->comps.flags[i] = (prev.flags[i] ^ SWS_COMP_SWAPPED) & SWS_COMP_DIRTY; - op->comps.min[i] = prev.min[i]; - op->comps.max[i] = prev.max[i]; + FORWARD(i, i, (flags ^ SWS_COMP_SWAPPED) & SWS_COMP_DIRTY); + op->comps.min[i] = prev.min[i]; + op->comps.max[i] = prev.max[i]; } break; case SWS_OP_WRITE: for (int i = 0; i < op->rw.elems; i++) av_assert1(!(prev.flags[i] & SWS_COMP_GARBAGE)); for (int i = 0; i < 4; i++) - op->comps.flags[i] = prev.flags[i]; + FORWARD(i, i, flags); break; case SWS_OP_LSHIFT: case SWS_OP_RSHIFT: for (int i = 0; i < 4; i++) - op->comps.flags[i] = prev.flags[i] & SWS_COMP_DIRTY; + FORWARD(i, i, flags & SWS_COMP_DIRTY); break; case SWS_OP_MIN: case SWS_OP_MAX: { AVRational64 *bound = op->op == SWS_OP_MIN ? op->comps.max : op->comps.min; for (int i = 0; i < 4; i++) { - op->comps.flags[i] = prev.flags[i]; + FORWARD(i, i, flags); if (op->clamp.limit[i].den) op->comps.flags[i] &= SWS_COMP_DIRTY; if (!bound[i].den) /* reset undefined bounds to known range */ @@ -396,9 +409,9 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) } case SWS_OP_DITHER: for (int i = 0; i < 4; i++) { - op->comps.flags[i] = prev.flags[i]; - op->comps.min[i] = prev.min[i]; - op->comps.max[i] = prev.max[i]; + FORWARD(i, i, flags); + op->comps.min[i] = prev.min[i]; + op->comps.max[i] = prev.max[i]; if (op->dither.y_offset[i] < 0) continue; /* Strip zero flag because of the nonzero dithering offset */ @@ -412,24 +425,21 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) const int pattern = op->pack.pattern[i]; if (pattern) { av_assert1(pattern < 32); - op->comps.flags[i] = prev.flags[0] & SWS_COMP_DIRTY; - op->comps.min[i] = Q(0); - op->comps.max[i] = Q((1ULL << pattern) - 1); + FORWARD(i, 0, flags & SWS_COMP_DIRTY); + op->comps.min[i] = Q(0); + op->comps.max[i] = Q((1ULL << pattern) - 1); } else - op->comps.flags[i] = SWS_COMP_GARBAGE; + RESET(i); } break; - case SWS_OP_PACK: { - SwsCompFlags flags = SWS_COMP_IDENTITY; + case SWS_OP_PACK: for (int i = 0; i < 4; i++) { if (op->pack.pattern[i]) - flags = merge_comp_flags(flags, prev.flags[i]); + FORWARD(0, i, flags & SWS_COMP_DIRTY); if (i > 0) /* clear remaining comps for sanity */ - op->comps.flags[i] = SWS_COMP_GARBAGE; + RESET(i); } - op->comps.flags[0] = flags & SWS_COMP_DIRTY; break; - } case SWS_OP_CLEAR: for (int i = 0; i < 4; i++) { if (SWS_COMP_TEST(op->clear.mask, i)) { @@ -439,17 +449,17 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) if (op->clear.value[i].den == 1) op->comps.flags[i] |= SWS_COMP_EXACT; } else { - op->comps.flags[i] = prev.flags[i]; + FORWARD(i, i, flags); } } break; case SWS_OP_SWIZZLE: for (int i = 0; i < 4; i++) - op->comps.flags[i] = prev.flags[op->swizzle.in[i]]; + FORWARD(i, op->swizzle.in[i], flags); break; case SWS_OP_CONVERT: for (int i = 0; i < 4; i++) { - op->comps.flags[i] = prev.flags[i]; + FORWARD(i, i, flags); if (!(prev.flags[i] & SWS_COMP_EXACT) || op->convert.expand) op->comps.flags[i] &= SWS_COMP_DIRTY; if (ff_sws_pixel_type_is_int(op->convert.to)) @@ -458,7 +468,6 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) break; case SWS_OP_LINEAR: for (int i = 0; i < 4; i++) { - SwsCompFlags flags = SWS_COMP_IDENTITY; AVRational64 min = Q(0), max = Q(0); bool first = true; for (int j = 0; j < 4; j++) { @@ -466,33 +475,32 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) AVRational64 mink = av_mul_q64(prev.min[j], k); AVRational64 maxk = av_mul_q64(prev.max[j], k); if (k.num) { - flags = merge_comp_flags(flags, prev.flags[j]); + FORWARD(i, j, flags); if (k.den != 1) /* fractional coefficient */ - flags &= ~SWS_COMP_EXACT; + op->comps.flags[i] &= ~SWS_COMP_EXACT; if (k.num < 0) FFSWAP(AVRational64, mink, maxk); min = av_add_q64(min, mink); max = av_add_q64(max, maxk); if (!first || av_cmp_q64(k, Q(1))) - flags &= SWS_COMP_DIRTY; + op->comps.flags[i] &= SWS_COMP_DIRTY; first = false; } } if (op->lin.m[i][4].num) { /* nonzero offset */ - flags &= ~SWS_COMP_ZERO & SWS_COMP_DIRTY; + op->comps.flags[i] &= ~SWS_COMP_ZERO & SWS_COMP_DIRTY; if (op->lin.m[i][4].den != 1) /* fractional offset */ - flags &= ~SWS_COMP_EXACT; + op->comps.flags[i] &= ~SWS_COMP_EXACT; min = av_add_q64(min, op->lin.m[i][4]); max = av_add_q64(max, op->lin.m[i][4]); } - op->comps.flags[i] = flags; op->comps.min[i] = min; op->comps.max[i] = max; } break; case SWS_OP_SCALE: for (int i = 0; i < 4; i++) { - op->comps.flags[i] = prev.flags[i] & SWS_COMP_DIRTY; + FORWARD(i, i, flags & SWS_COMP_DIRTY); if (op->scale.factor.den != 1) /* fractional scale */ op->comps.flags[i] &= ~SWS_COMP_EXACT; if (op->scale.factor.num < 0) @@ -521,7 +529,7 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) for (int i = 0; i < 4; i++) { if (!need_out[i]) - op->comps.flags[i] = SWS_COMP_GARBAGE; + RESET(i); } switch (op->op) { @@ -575,6 +583,9 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) memcpy(need_out, need_in, sizeof(need_in)); } + + #undef FORWARD + #undef RESET } static void op_uninit(SwsOp *op) -- 2.52.0 From 31fbb2e5b50699e2260d3c93e3eb435f6ddb2bd3 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Thu, 25 Jun 2026 18:36:44 +0200 Subject: [PATCH 03/18] swscale/ops: solve for component dependencies Needed to determine reverse dependencies of different input planes, for appropriately splitting op lists involving subsampled planes. Generates benign diffs that just reflect the new addition, e.g.: yuva444p 16x16 -> rgb24 16x16: [ u8 ===X] SWS_OP_READ : 3 elem(s) planar >> 0 min: {0 0 0 _}, max: {255 255 255 _} + inputs: {x y z _}, outputs: {xyz yz xy _} [ u8 ===X] SWS_OP_CONVERT : u8 -> f32 min: {0 0 0 _}, max: {255 255 255 _} + inputs: {x y z _}, outputs: {xyz yz xy _} [f32 ...X] SWS_OP_LINEAR : matrix3+off3 [[85/73 0 1.596027 0 -222.921566] [85/73 -0.391762 -0.812968 0 135.575295] [85/73 2.017232 0 0 -276.835851] [0 0 0 1 0]] min: {-222.921566 -171.630839 -276.835851 _}, max: {480.983073 432.493103 534.476153 _} + inputs: {xz xyz xy _}, outputs: {x y z _} [f32 ...X] SWS_OP_DITHER : 16x16 matrix + {0 3 2 -1} min: {-222.919612 -171.628886 -276.833898 _}, max: {481.981120 433.491150 535.474200 _} + inputs: {xz xyz xy _}, outputs: {x y z _} [f32 ...X] SWS_OP_MAX : {0 0 0 _} <= x min: {0 0 0 _}, max: {481.981120 433.491150 535.474200 _} + inputs: {xz xyz xy _}, outputs: {x y z _} [f32 ...X] SWS_OP_MIN : x <= {255 255 255 _} min: {0 0 0 _}, max: {255 255 255 _} + inputs: {xz xyz xy _}, outputs: {x y z _} [f32 +++X] SWS_OP_CONVERT : f32 -> u8 min: {0 0 0 _}, max: {255 255 255 _} + inputs: {xz xyz xy _}, outputs: {x y z _} [ u8 XXXX] SWS_OP_WRITE : 3 elem(s) packed >> 0 ('X' unused, 'z' byteswapped, '=' copied, '$' const, '+' integer, '0' zero) Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/ops.c | 46 +++++++++++++++++++++++++++++++------ libswscale/ops.h | 3 +++ tests/ref/fate/sws-ops-list | 2 +- 3 files changed, 43 insertions(+), 8 deletions(-) diff --git a/libswscale/ops.c b/libswscale/ops.c index be3e9853a9..21462df386 100644 --- a/libswscale/ops.c +++ b/libswscale/ops.c @@ -292,7 +292,8 @@ static void apply_filter_weights(SwsComps *comps, const SwsComps *prev, const AVRational64 posw = { weights->sum_positive, SWS_FILTER_SCALE }; const AVRational64 negw = { weights->sum_negative, SWS_FILTER_SCALE }; for (int i = 0; i < 4; i++) { - comps->flags[i] = prev->flags[i] & SWS_COMP_DIRTY; + comps->flags[i] = prev->flags[i] & SWS_COMP_DIRTY; + comps->dep_in[i] = prev->dep_in[i]; /* Only point sampling preserves exactness */ if (weights->filter_size != 1) comps->flags[i] &= ~SWS_COMP_EXACT; @@ -331,17 +332,21 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) break; } - for (int i = 0; i < 4; i++) - op->comps.flags[i] = SWS_COMP_IDENTITY; + for (int i = 0; i < 4; i++) { + op->comps.flags[i] = SWS_COMP_IDENTITY; + op->comps.dep_in[i] = SWS_COMP_NONE; + } #define FORWARD(I, J, EXPR) do { \ SwsCompFlags flags = prev.flags[J]; \ op->comps.flags[I] = merge_comp_flags(op->comps.flags[I], (EXPR)); \ + op->comps.dep_in[I] |= prev.dep_in[J]; \ } while (0) #define RESET(I) do { \ op->comps.flags[I] = SWS_COMP_GARBAGE; \ op->comps.min[I] = op->comps.max[I] = (AVRational64) {0}; \ + op->comps.dep_in[I] = SWS_COMP_NONE; \ } while (0) switch (op->op) { @@ -360,6 +365,7 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) op->comps.flags[i] = ops->comps_src.flags[idx] & SWS_COMP_DIRTY; op->comps.min[i] = ops->comps_src.min[idx]; op->comps.max[i] = ops->comps_src.max[idx]; + op->comps.dep_in[i] = SWS_COMP(idx); /** * Don't mark packed or fractional reads as a copy, because the @@ -521,13 +527,15 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) prev = op->comps; } - /* Backwards pass, solves for component dependencies */ - bool need_out[4] = { false, false, false, false }; + /* Backwards pass, solves for output component dependencies */ + SwsCompMask need_out[4] = {0}; + for (int n = ops->num_ops - 1; n >= 0; n--) { SwsOp *op = &ops->ops[n]; - bool need_in[4] = { false, false, false, false }; + SwsCompMask need_in[4] = {0}; for (int i = 0; i < 4; i++) { + op->comps.dep_out[i] = need_out[i]; if (!need_out[i]) RESET(i); } @@ -536,7 +544,7 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) case SWS_OP_READ: case SWS_OP_WRITE: for (int i = 0; i < op->rw.elems; i++) - need_in[i] = op->op == SWS_OP_WRITE; + need_in[i] = (op->op == SWS_OP_WRITE) ? SWS_COMP(i) : 0; for (int i = op->rw.elems; i < 4; i++) need_in[i] = need_out[i]; break; @@ -789,6 +797,17 @@ static char describe_comp_flags(SwsCompFlags flags) return '.'; } +static void print_deps(AVBPrint *bp, const SwsCompMask *deps) +{ + av_bprintf(bp, "{"); + for (int i = 0; i < 4; i++) { + if (i) + av_bprintf(bp, " "); + av_bprintf(bp, "%s", deps[i] ? ff_sws_comp_mask_str(deps[i]) : "_"); + } + av_bprintf(bp, "}"); +} + static void print_q(AVBPrint *bp, const AVRational64 q) { if (!q.den) { @@ -976,6 +995,19 @@ void ff_sws_op_list_print(void *log, int lev, int lev_extra, av_log(log, lev_extra, "%s\n", bp.str); } + bool has_deps = false; + for (int i = 0; i < 4; i++) + has_deps |= op->comps.dep_in[i] || op->comps.dep_out[i]; + if (has_deps) { + av_bprint_clear(&bp); + av_bprintf(&bp, " inputs: "); + print_deps(&bp, op->comps.dep_in); + av_bprintf(&bp, ", outputs: "); + print_deps(&bp, op->comps.dep_out); + av_assert0(av_bprint_is_complete(&bp)); + av_log(log, lev_extra, "%s\n", bp.str); + } + } av_log(log, lev, " ('X' unused, 'z' byteswapped, '=' copied, '$' const, '+' integer, '0' zero)\n"); diff --git a/libswscale/ops.h b/libswscale/ops.h index 67dd04d227..02c122655a 100644 --- a/libswscale/ops.h +++ b/libswscale/ops.h @@ -85,6 +85,9 @@ typedef struct SwsComps { /* Keeps track of the known possible value range, or {0, 0} for undefined * or (unknown range) floating point inputs */ AVRational64 min[4], max[4]; + + /* Keeps track of input (forward) and output (reverse) dependencies */ + SwsCompMask dep_in[4], dep_out[4]; } SwsComps; typedef enum SwsReadWriteMode { diff --git a/tests/ref/fate/sws-ops-list b/tests/ref/fate/sws-ops-list index 0fb88d65da..8b92ff60b6 100644 --- a/tests/ref/fate/sws-ops-list +++ b/tests/ref/fate/sws-ops-list @@ -1 +1 @@ -3a8f3bccd7c1f8407ee7c40fd26d86cc +63448a9d7345ac61bedcb479045358f3 -- 2.52.0 From 5d301e5d811d4530a399cc250551622d242b97d4 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Fri, 17 Jul 2026 12:41:43 +0200 Subject: [PATCH 04/18] swscale/ops: remove SwsLinearOp.mask This was originally introduced to make matching linear ops against implementations faster. However, since this is now handled on the uops level, there is no more reason to carry this metadata on the ops level. Simplifies a lot of places in the code. It will simplify even more, once the linear optimizations are moved to the uops level. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/aarch64/ops_impl_conv.c | 3 ++- libswscale/format.c | 7 +------ libswscale/ops.h | 8 +------- libswscale/ops_optimizer.c | 17 +++++++---------- libswscale/uops.c | 3 ++- 5 files changed, 13 insertions(+), 25 deletions(-) diff --git a/libswscale/aarch64/ops_impl_conv.c b/libswscale/aarch64/ops_impl_conv.c index 21360e51c7..daebac9f09 100644 --- a/libswscale/aarch64/ops_impl_conv.c +++ b/libswscale/aarch64/ops_impl_conv.c @@ -268,8 +268,9 @@ static int convert_to_aarch64_impl(SwsContext *ctx, const SwsOpList *ops, int n, case SWS_UOP_LINEAR: case SWS_UOP_LINEAR_FMA: out->mask = 0; + const uint32_t lin_mask = ff_sws_linear_mask(&op->lin); for (int i = 0; i < 4; i++) { - if (!SWS_OP_NEEDED(op, i) || !(op->lin.mask & SWS_MASK_ROW(i))) { + if (!SWS_OP_NEEDED(op, i) || !(lin_mask & SWS_MASK_ROW(i))) { for (int j = 0; j < 5; j++) out->par.lin.zero |= SWS_MASK(i, j); continue; diff --git a/libswscale/format.c b/libswscale/format.c index 70cc813dd7..2531435dae 100644 --- a/libswscale/format.c +++ b/libswscale/format.c @@ -1283,7 +1283,6 @@ static SwsLinearOp fmt_encode_range(const SwsFormat *fmt, bool *incomplete) c.m[0][0] = av_neg_q64(c.m[0][0]); } - c.mask = ff_sws_linear_mask(&c); return c; } @@ -1302,7 +1301,6 @@ static SwsLinearOp fmt_decode_range(const SwsFormat *fmt, bool *incomplete) if (!(fmt->desc->flags & AV_PIX_FMT_FLAG_ALPHA)) c.m[3][4] = Q(1); - c.mask = ff_sws_linear_mask(&c); return c; } @@ -1464,15 +1462,12 @@ linear_mat3(const AVRational m00, const AVRational m01, const AVRational m02, const AVRational m10, const AVRational m11, const AVRational m12, const AVRational m20, const AVRational m21, const AVRational m22) { - SwsLinearOp c = {{ + return (SwsLinearOp) {{ { Q64(m00), Q64(m01), Q64(m02), Q(0), Q(0) }, { Q64(m10), Q64(m11), Q64(m12), Q(0), Q(0) }, { Q64(m20), Q64(m21), Q64(m22), Q(0), Q(0) }, { Q(0), Q(0), Q(0), Q(1), Q(0) }, }}; - - c.mask = ff_sws_linear_mask(&c); - return c; } int ff_sws_decode_colors(SwsContext *ctx, SwsPixelType type, diff --git a/libswscale/ops.h b/libswscale/ops.h index 02c122655a..83ef2b49df 100644 --- a/libswscale/ops.h +++ b/libswscale/ops.h @@ -192,17 +192,11 @@ typedef struct SwsLinearOp { * [ Out.y ] = [ F G H I J ] * [ x y z w 1 ] * [ Out.z ] = [ K L M N O ] * [ Out.w ] = [ P Q R S T ] - * - * The mask keeps track of which components differ from an identity matrix. - * There may be more efficient implementations of particular subsets, for - * example the common subset of {A, E, G, J, M, O} can be implemented with - * just three fused multiply-add operations. */ AVRational64 m[4][5]; - uint32_t mask; /* m[i][j] <-> 1 << (5 * i + j) */ } SwsLinearOp; -/* Helper function to compute the correct mask */ +/* m[i][j] <-> 1 << (5 * i + j) */ uint32_t ff_sws_linear_mask(const SwsLinearOp *c); typedef struct SwsFilterOp { diff --git a/libswscale/ops_optimizer.c b/libswscale/ops_optimizer.c index 5b22f3b387..3546d8bc9e 100644 --- a/libswscale/ops_optimizer.c +++ b/libswscale/ops_optimizer.c @@ -238,7 +238,7 @@ static bool extract_scalar(const SwsLinearOp *c, SwsScaleOp scale = {0}; /* There are components not on the main diagonal */ - if (c->mask & ~SWS_MASK_DIAG4) + if (ff_sws_linear_mask(c) & ~SWS_MASK_DIAG4) return false; for (int i = 0; i < 4; i++) { @@ -260,6 +260,7 @@ static bool extract_scalar(const SwsLinearOp *c, static bool extract_constant_rows(SwsLinearOp *c, const SwsComps *prev, SwsClearOp *out_clear) { + const uint32_t mask = ff_sws_linear_mask(c); SwsClearOp clear = {0}; bool ret = false; @@ -269,12 +270,11 @@ static bool extract_constant_rows(SwsLinearOp *c, const SwsComps *prev, const_row &= c->m[i][j].num == 0 || /* scalar is zero */ (prev->flags[j] & SWS_COMP_ZERO); /* input is zero */ } - if (const_row && (c->mask & SWS_MASK_ROW(i))) { + if (const_row && (mask & SWS_MASK_ROW(i))) { clear.mask |= SWS_COMP(i); clear.value[i] = c->m[i][4]; for (int j = 0; j < 5; j++) c->m[i][j] = Q(i == j); - c->mask &= ~SWS_MASK_ROW(i); ret = true; } } @@ -321,7 +321,6 @@ static bool extract_swizzle(SwsLinearOp *op, const SwsComps *prev, if (swiz.mask == SWS_SWIZZLE(0, 1, 2, 3).mask) return false; /* no swizzle was identified */ - c.mask = ff_sws_linear_mask(&c); *out_swiz = swiz; *op = c; return true; @@ -622,12 +621,13 @@ retry: break; case SWS_OP_LINEAR: { + const uint32_t mask = ff_sws_linear_mask(&op->lin); SwsSwizzleOp swizzle; SwsClearOp clear; SwsScaleOp scale; /* No-op (identity) linear operation */ - if (!op->lin.mask) { + if (!mask) { ff_sws_op_list_remove_at(ops, n, 1); goto retry; } @@ -646,7 +646,6 @@ retry: op->lin.m[i][j] = sum; } } - op->lin.mask = ff_sws_linear_mask(&op->lin); ff_sws_op_list_remove_at(ops, n + 1, 1); goto retry; } @@ -654,22 +653,20 @@ retry: /* Optimize away zero columns */ for (int j = 0; j < 4; j++) { const uint32_t col = SWS_MASK_COL(j); - if (!(prev->comps.flags[j] & SWS_COMP_ZERO) || !(op->lin.mask & col)) + if (!(prev->comps.flags[j] & SWS_COMP_ZERO) || !(mask & col)) continue; for (int i = 0; i < 4; i++) op->lin.m[i][j] = Q(i == j); - op->lin.mask &= ~col; goto retry; } /* Optimize away unused rows */ for (int i = 0; i < 4; i++) { const uint32_t row = SWS_MASK_ROW(i); - if (SWS_OP_NEEDED(op, i) || !(op->lin.mask & row)) + if (SWS_OP_NEEDED(op, i) || !(mask & row)) continue; for (int j = 0; j < 5; j++) op->lin.m[i][j] = Q(i == j); - op->lin.mask &= ~row; goto retry; } diff --git a/libswscale/uops.c b/libswscale/uops.c index 44000b35f1..fc9d609636 100644 --- a/libswscale/uops.c +++ b/libswscale/uops.c @@ -506,11 +506,12 @@ static int translate_linear_op(SwsContext *ctx, SwsUOpList *ops, .uop = SWS_UOP_LINEAR, }; + const uint32_t mask = ff_sws_linear_mask(&op->lin); const bool bitexact = ctx->flags & SWS_BITEXACT; uint32_t exact = 0; for (int i = 0; i < 4; i++) { - if (!SWS_OP_NEEDED(op, i) || !(op->lin.mask & SWS_MASK_ROW(i))) { + if (!SWS_OP_NEEDED(op, i) || !(mask & SWS_MASK_ROW(i))) { uop.par.lin.zero |= SWS_MASK_ROW(i); continue; } -- 2.52.0 From d030325e4d04fbd8dd1c1f2fd9c60d4bc68fadb3 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Thu, 23 Jul 2026 17:21:43 +0200 Subject: [PATCH 05/18] swscale/csputils: remove unused struct Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/csputils.h | 4 ---- 1 file changed, 4 deletions(-) diff --git a/libswscale/csputils.h b/libswscale/csputils.h index c28e4ac7ae..82eb0b68f4 100644 --- a/libswscale/csputils.h +++ b/libswscale/csputils.h @@ -65,10 +65,6 @@ SwsMatrix3x3 ff_sws_get_adaptation(const AVPrimaryCoefficients *prim, AVWhitepointCoefficients to); /* Integer math definitions / helpers */ -typedef struct v3u8_t { - uint8_t x, y, z; -} v3u8_t; - typedef struct v2u16_t { uint16_t x, y; } v2u16_t; -- 2.52.0 From f62351b0a628bc00f8dc462b554d5ac80917835c Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Wed, 22 Jul 2026 17:23:19 +0200 Subject: [PATCH 06/18] swscale/aarch64: reject unknown ops by default Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/aarch64/ops_impl_conv.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/libswscale/aarch64/ops_impl_conv.c b/libswscale/aarch64/ops_impl_conv.c index daebac9f09..fbd69c9c87 100644 --- a/libswscale/aarch64/ops_impl_conv.c +++ b/libswscale/aarch64/ops_impl_conv.c @@ -199,8 +199,7 @@ static int convert_to_aarch64_impl(SwsContext *ctx, const SwsOpList *ops, int n, : SWS_UOP_LINEAR_FMA; break; case SWS_OP_DITHER: out->uop = SWS_UOP_DITHER; break; - case SWS_OP_FILTER_H: - case SWS_OP_FILTER_V: + default: return AVERROR(ENOTSUP); } -- 2.52.0 From fac6b26466b4af1de9c67910110dcdf04a57ca04 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Thu, 23 Jul 2026 17:21:56 +0200 Subject: [PATCH 07/18] swscale/lut3d: alignment (cosmetic) Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/lut3d.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libswscale/lut3d.c b/libswscale/lut3d.c index d701ca2ba5..694b455dbc 100644 --- a/libswscale/lut3d.c +++ b/libswscale/lut3d.c @@ -221,9 +221,9 @@ int ff_sws_lut3d_generate(SwsLut3D *lut3d, enum AVPixelFormat fmt_in, if (lut3d->dynamic) { ret = ff_sws_color_map_generate_dynamic(&lut3d->input[0][0][0], - &lut3d->output[0][0][0], - INPUT_LUT_SIZE, OUTPUT_LUT_SIZE_I, - OUTPUT_LUT_SIZE_PT, map); + &lut3d->output[0][0][0], + INPUT_LUT_SIZE, OUTPUT_LUT_SIZE_I, + OUTPUT_LUT_SIZE_PT, map); if (ret < 0) return ret; @@ -232,7 +232,7 @@ int ff_sws_lut3d_generate(SwsLut3D *lut3d, enum AVPixelFormat fmt_in, return 0; } else { return ff_sws_color_map_generate_static(&lut3d->input[0][0][0], - INPUT_LUT_SIZE, map); + INPUT_LUT_SIZE, map); } } -- 2.52.0 From a176b1754ab5565ab4eea8abfc209dbdf1211056 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Thu, 23 Jul 2026 19:21:21 +0200 Subject: [PATCH 08/18] swscale/lut3d: pad 3DLUT to eliminate over-read Instead of clamping in the application function. Trivial simplicity gain for the SIMD code. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/lut3d.c | 4 ++-- libswscale/lut3d.h | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/libswscale/lut3d.c b/libswscale/lut3d.c index 694b455dbc..e521bc4439 100644 --- a/libswscale/lut3d.c +++ b/libswscale/lut3d.c @@ -195,10 +195,9 @@ static av_always_inline v3u16_t apply_tone_map(const SwsLut3D *lut3d, v3u16_t ip const int shift = 16 - TONE_LUT_BITS; const int Ix = ipt.x >> shift; const int If = ipt.x & ((1 << shift) - 1); - const int In = FFMIN(Ix + 1, TONE_LUT_SIZE - 1); const v2u16_t w0 = lut3d->tone_map[Ix]; - const v2u16_t w1 = lut3d->tone_map[In]; + const v2u16_t w1 = lut3d->tone_map[Ix + 1]; const v2u16_t w = lerp2u16(w0, w1, If, shift); const int base = (1 << 15) - w.y; @@ -245,6 +244,7 @@ void ff_sws_lut3d_update(SwsLut3D *lut3d, const SwsColor *new_src) lut3d->map.src.frame_avg = new_src->frame_avg; ff_sws_tone_map_generate(lut3d->tone_map, TONE_LUT_SIZE, &lut3d->map); + lut3d->tone_map[TONE_LUT_SIZE] = lut3d->tone_map[TONE_LUT_SIZE - 1]; } void ff_sws_lut3d_apply(const SwsLut3D *lut3d, const uint8_t *in, int in_stride, diff --git a/libswscale/lut3d.h b/libswscale/lut3d.h index d2de851604..9e8381a2c0 100644 --- a/libswscale/lut3d.h +++ b/libswscale/lut3d.h @@ -56,7 +56,7 @@ typedef struct SwsLut3D { v3u16_t output[OUTPUT_LUT_SIZE_PT][OUTPUT_LUT_SIZE_PT][OUTPUT_LUT_SIZE_I]; /* Split tone mapping LUT (for dynamic tone mapping) */ - v2u16_t tone_map[TONE_LUT_SIZE]; /* new luma, desaturation */ + v2u16_t tone_map[TONE_LUT_SIZE + 1]; /* new luma, desaturation */ } SwsLut3D; SwsLut3D *ff_sws_lut3d_alloc(void); -- 2.52.0 From 8047c01100545bbfb662af8fe2c2146c9c49921e Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Fri, 24 Jul 2026 14:34:46 +0200 Subject: [PATCH 09/18] swscale/lut3d: use refstruct for 3D LUT allocations Needed anyways for the upcoming SWS_OP_LUT_3D. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/graph.c | 6 +++--- libswscale/lut3d.c | 10 ++++------ libswscale/lut3d.h | 4 +++- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/libswscale/graph.c b/libswscale/graph.c index b9d5df3b48..4cfe87ea31 100644 --- a/libswscale/graph.c +++ b/libswscale/graph.c @@ -682,7 +682,7 @@ static int add_convert_pass(SwsGraph *graph, const SwsFormat *src, static void free_lut3d(void *priv) { SwsLut3D *lut = priv; - ff_sws_lut3d_free(&lut); + av_refstruct_unref(&lut); } static int setup_lut3d(const SwsFrame *out, const SwsFrame *in, const SwsPass *pass) @@ -753,14 +753,14 @@ static int adapt_colors(SwsGraph *graph, const SwsFormat *src_fmt, tmp.format = fmt_in; ret = add_convert_pass(graph, &src, &tmp, input, &input); if (ret < 0) { - ff_sws_lut3d_free(&lut); + av_refstruct_unref(&lut); return ret; } } ret = ff_sws_lut3d_generate(lut, fmt_in, fmt_out, &map); if (ret < 0) { - ff_sws_lut3d_free(&lut); + av_refstruct_unref(&lut); return ret; } diff --git a/libswscale/lut3d.c b/libswscale/lut3d.c index e521bc4439..1b1b11aa4c 100644 --- a/libswscale/lut3d.c +++ b/libswscale/lut3d.c @@ -24,6 +24,7 @@ #include "libavutil/attributes.h" #include "libavutil/avassert.h" #include "libavutil/mem.h" +#include "libavutil/refstruct.h" #include "cms.h" #include "csputils.h" @@ -31,19 +32,16 @@ SwsLut3D *ff_sws_lut3d_alloc(void) { - SwsLut3D *lut3d = av_malloc(sizeof(*lut3d)); + const int flags = AV_REFSTRUCT_FLAG_NO_ZEROING; + SwsLut3D *lut3d = av_refstruct_alloc_ext(sizeof(*lut3d), flags, NULL, NULL); if (!lut3d) return NULL; + lut3d->map = (SwsColorMap) {0}; lut3d->dynamic = false; return lut3d; } -void ff_sws_lut3d_free(SwsLut3D **plut3d) -{ - av_freep(plut3d); -} - bool ff_sws_lut3d_test_fmt(enum AVPixelFormat fmt, int output) { return fmt == AV_PIX_FMT_RGBA64; diff --git a/libswscale/lut3d.h b/libswscale/lut3d.h index 9e8381a2c0..660dc08f26 100644 --- a/libswscale/lut3d.h +++ b/libswscale/lut3d.h @@ -59,8 +59,10 @@ typedef struct SwsLut3D { v2u16_t tone_map[TONE_LUT_SIZE + 1]; /* new luma, desaturation */ } SwsLut3D; +/** + * Allocates a refstruct. Note that the LUT contents are not initialized. + */ SwsLut3D *ff_sws_lut3d_alloc(void); -void ff_sws_lut3d_free(SwsLut3D **lut3d); /** * Test to see if a given format is supported by the 3DLUT input/output code. -- 2.52.0 From a63a8e29f8c4f74e342b089eb55b4775ff65010a Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Fri, 24 Jul 2026 14:41:58 +0200 Subject: [PATCH 10/18] swscale/lut3d: simplify and hard-code 3DLUT format The ops-based 3DLUT approach will take care of appropriately normalizing the input to the expected domain; so the format choice no longer matters here except for the lut3d_apply() function, which will only be used by the legacy reference code path. So we can just continue hard-coding the format there. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/graph.c | 26 ++++++++++++-------------- libswscale/lut3d.c | 21 ++++----------------- libswscale/lut3d.h | 21 +++++---------------- 3 files changed, 21 insertions(+), 47 deletions(-) diff --git a/libswscale/graph.c b/libswscale/graph.c index 4cfe87ea31..1c3fa188d5 100644 --- a/libswscale/graph.c +++ b/libswscale/graph.c @@ -702,8 +702,8 @@ static void run_lut3d(const SwsFrame *out, const SwsFrame *in, int y, int h, frame_shift(in, y, in_data); frame_shift(out, y, out_data); - ff_sws_lut3d_apply(lut, in_data[0], in->linesize[0], out_data[0], - out->linesize[0], out->width, h); + ff_sws_lut3d_apply_rgba64(lut, in_data[0], in->linesize[0], out_data[0], + out->linesize[0], out->width, h); } static int adapt_colors(SwsGraph *graph, const SwsFormat *src_fmt, @@ -712,7 +712,6 @@ static int adapt_colors(SwsGraph *graph, const SwsFormat *src_fmt, { SwsFormat src = *src_fmt; SwsFormat dst = *dst_fmt; - enum AVPixelFormat fmt_in, fmt_out; SwsColorMap map = {0}; SwsLut3D *lut; int ret; @@ -746,11 +745,16 @@ static int adapt_colors(SwsGraph *graph, const SwsFormat *src_fmt, if (!lut) return AVERROR(ENOMEM); - fmt_in = ff_sws_lut3d_pick_pixfmt(&src, 0); - fmt_out = ff_sws_lut3d_pick_pixfmt(&dst, 1); - if (fmt_in != src.format) { + ret = ff_sws_lut3d_generate(lut, &map); + if (ret < 0) { + av_refstruct_unref(&lut); + return ret; + } + + const enum AVPixelFormat fmt = AV_PIX_FMT_RGBA64; + if (src.format != fmt) { SwsFormat tmp = src; - tmp.format = fmt_in; + tmp.format = fmt; ret = add_convert_pass(graph, &src, &tmp, input, &input); if (ret < 0) { av_refstruct_unref(&lut); @@ -758,13 +762,7 @@ static int adapt_colors(SwsGraph *graph, const SwsFormat *src_fmt, } } - ret = ff_sws_lut3d_generate(lut, fmt_in, fmt_out, &map); - if (ret < 0) { - av_refstruct_unref(&lut); - return ret; - } - - return ff_sws_graph_add_pass(graph, fmt_out, src.width, src.height, + return ff_sws_graph_add_pass(graph, fmt, src.width, src.height, input, 0, 1, run_lut3d, setup_lut3d, lut, free_lut3d, output); } diff --git a/libswscale/lut3d.c b/libswscale/lut3d.c index 1b1b11aa4c..973575e1df 100644 --- a/libswscale/lut3d.c +++ b/libswscale/lut3d.c @@ -42,16 +42,6 @@ SwsLut3D *ff_sws_lut3d_alloc(void) return lut3d; } -bool ff_sws_lut3d_test_fmt(enum AVPixelFormat fmt, int output) -{ - return fmt == AV_PIX_FMT_RGBA64; -} - -enum AVPixelFormat ff_sws_lut3d_pick_pixfmt(const SwsFormat *fmt, int output) -{ - return AV_PIX_FMT_RGBA64; -} - /** * v0 and v1 are 'black' and 'white' * v2 and v3 are closest RGB/CMY vertices @@ -205,14 +195,10 @@ static av_always_inline v3u16_t apply_tone_map(const SwsLut3D *lut3d, v3u16_t ip return ipt; } -int ff_sws_lut3d_generate(SwsLut3D *lut3d, enum AVPixelFormat fmt_in, - enum AVPixelFormat fmt_out, const SwsColorMap *map) +int ff_sws_lut3d_generate(SwsLut3D *lut3d, const SwsColorMap *map) { int ret; - if (!ff_sws_lut3d_test_fmt(fmt_in, 0) || !ff_sws_lut3d_test_fmt(fmt_out, 1)) - return AVERROR(EINVAL); - lut3d->dynamic = map->src.frame_peak.num > 0; lut3d->map = *map; @@ -245,8 +231,9 @@ void ff_sws_lut3d_update(SwsLut3D *lut3d, const SwsColor *new_src) lut3d->tone_map[TONE_LUT_SIZE] = lut3d->tone_map[TONE_LUT_SIZE - 1]; } -void ff_sws_lut3d_apply(const SwsLut3D *lut3d, const uint8_t *in, int in_stride, - uint8_t *out, int out_stride, int w, int h) +void ff_sws_lut3d_apply_rgba64(const SwsLut3D *lut3d, const uint8_t *in, + int in_stride, uint8_t *out, int out_stride, + int w, int h) { while (h--) { const uint16_t *in16 = (const uint16_t *) in; diff --git a/libswscale/lut3d.h b/libswscale/lut3d.h index 660dc08f26..02e1122cfa 100644 --- a/libswscale/lut3d.h +++ b/libswscale/lut3d.h @@ -64,16 +64,6 @@ typedef struct SwsLut3D { */ SwsLut3D *ff_sws_lut3d_alloc(void); -/** - * Test to see if a given format is supported by the 3DLUT input/output code. - */ -bool ff_sws_lut3d_test_fmt(enum AVPixelFormat fmt, int output); - -/** - * Pick the best compatible pixfmt for a given SwsFormat. - */ -enum AVPixelFormat ff_sws_lut3d_pick_pixfmt(const SwsFormat *fmt, int output); - /** * Recalculate the (static) 3DLUT state with new settings. This will recompute * everything. To only update per-frame tone mapping state, instead call @@ -81,8 +71,7 @@ enum AVPixelFormat ff_sws_lut3d_pick_pixfmt(const SwsFormat *fmt, int output); * * Returns 0 or a negative error code. */ -int ff_sws_lut3d_generate(SwsLut3D *lut3d, enum AVPixelFormat fmt_in, - enum AVPixelFormat fmt_out, const SwsColorMap *map); +int ff_sws_lut3d_generate(SwsLut3D *lut3d, const SwsColorMap *map); /** * Update the tone mapping state. This will only use per-frame metadata. The @@ -91,10 +80,10 @@ int ff_sws_lut3d_generate(SwsLut3D *lut3d, enum AVPixelFormat fmt_in, void ff_sws_lut3d_update(SwsLut3D *lut3d, const SwsColor *new_src); /** - * Applies a color transformation to a plane. The format must match the format - * provided during ff_sws_lut3d_update(). + * Applies a color transformation to a plane in RGBA64 format. */ -void ff_sws_lut3d_apply(const SwsLut3D *lut3d, const uint8_t *in, int in_stride, - uint8_t *out, int out_stride, int w, int h); +void ff_sws_lut3d_apply_rgba64(const SwsLut3D *lut3d, const uint8_t *in, + int in_stride, uint8_t *out, int out_stride, + int w, int h); #endif /* SWSCALE_LUT3D_H */ -- 2.52.0 From 418df2def806e2ed1995e097f0caf5cd9fcb15af Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Mon, 27 Jul 2026 13:51:10 +0200 Subject: [PATCH 11/18] swscale/cms: fix IPT rounding error av_round16f() was incorrectly scaling to 65534, which would correspond to a neutral PT offset of 32767 = (1 << 15) - 1, but the code was assuming a value of (1 << 15). To fix it, and make the PT channel correctly symmetric around the intended neutral value, we have to map PT = +0.5 to 65536, an unrepresentable value. This is not an issue because the PT channel values are strictly inside some subset of [-0.5, 0.5] in practice, for real in-gamut color values - it's already an envelope that includes quite a bit of safety margin. Similarly, av_round16f() is also the wrong tool for the I/RGB channels, because it incorrectly scaled those to 65534, an off by one of the intended unorm16 full range peak of 65535. This silently resulted in e.g. RGBA64 true white (65535) not round-tripping through the 3DLUT. Fix both by splitting this helper into two separate helpers, each of them fixed to the correct, intended value range. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/cms.c | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/libswscale/cms.c b/libswscale/cms.c index 1c9c874a55..73448ecebc 100644 --- a/libswscale/cms.c +++ b/libswscale/cms.c @@ -562,9 +562,14 @@ static IPT saturation(const CmsCtx * ctx, IPT ipt) return rgb2ipt(rgb, ctx->dst.content2lms); } -static av_always_inline av_const uint16_t av_round16f(float x) +static av_always_inline av_const uint16_t round_unorm16(float x) { - return av_clip_uint16(x * (UINT16_MAX - 1) + 0.5f); + return av_clip_uint16(x * UINT16_MAX + 0.5f); +} + +static av_always_inline av_const uint16_t round_pt16(float x) +{ + return av_clip_uint16(x * (1 << 16) + 0.5f); } /* Call this whenever the hue changes inside the loop body */ @@ -602,7 +607,6 @@ static void generate_slice(void *priv, int jobnr, int threadnr, int nb_jobs, const float I_scale = 1.0f / (ctx.src.Imax - ctx.src.Imin); const float I_offset = -ctx.src.Imin * I_scale; - const float PT_offset = (float) (1 << 15) / (UINT16_MAX - 1); const float input_scale = 1.0f / (ctx.size_input - 1); const float output_scale_PT = 1.0f / (ctx.size_output_PT - 1); @@ -625,9 +629,9 @@ static void generate_slice(void *priv, int jobnr, int threadnr, int nb_jobs, if (output) { /* Save intermediate value to 3DLUT */ *input++ = (v3u16_t) { - av_round16f(I_scale * ipt.I + I_offset), - av_round16f(ipt.P + PT_offset), - av_round16f(ipt.T + PT_offset), + round_unorm16(I_scale * ipt.I + I_offset), + round_pt16(ipt.P + 0.5f), + round_pt16(ipt.T + 0.5f), }; } else { update_hue_peaks(&ctx, ipt.P, ipt.T); @@ -641,9 +645,9 @@ static void generate_slice(void *priv, int jobnr, int threadnr, int nb_jobs, c[2] = rgb.B; ctx.dst.eotf_inv(ctx.dst.Lw, ctx.dst.Lb, c); *input++ = (v3u16_t) { - av_round16f(c[0]), - av_round16f(c[1]), - av_round16f(c[2]), + round_unorm16(c[0]), + round_unorm16(c[1]), + round_unorm16(c[2]), }; } } @@ -655,9 +659,9 @@ static void generate_slice(void *priv, int jobnr, int threadnr, int nb_jobs, /* Generate split gamut mapping LUT */ for (int Tx = output_start; Tx < output_end; Tx++) { - const float T = output_scale_PT * Tx - PT_offset; + const float T = output_scale_PT * Tx - 0.5f; for (int Px = 0; Px < ctx.size_output_PT; Px++) { - const float P = output_scale_PT * Px - PT_offset; + const float P = output_scale_PT * Px - 0.5f; update_hue_peaks(&ctx, P, T); for (int Ix = 0; Ix < ctx.size_output_I; Ix++) { @@ -667,9 +671,9 @@ static void generate_slice(void *priv, int jobnr, int threadnr, int nb_jobs, double c[3] = { rgb.R, rgb.G, rgb.B }; ctx.dst.eotf_inv(ctx.dst.Lw, ctx.dst.Lb, c); *output++ = (v3u16_t) { - av_round16f(c[0]), - av_round16f(c[1]), - av_round16f(c[2]), + round_unorm16(c[0]), + round_unorm16(c[1]), + round_unorm16(c[2]), }; } } @@ -759,7 +763,7 @@ void ff_sws_tone_map_generate(v2u16_t *lut, int size, const SwsColorMap *map) const float I = src_scale * i + src_offset; IPT ipt = tone_map_apply(&ctx, (IPT) { I, 1.0f }); lut[i] = (v2u16_t) { - av_round16f(dst_scale * ipt.I + dst_offset), + round_unorm16(dst_scale * ipt.I + dst_offset), av_clip_uint16(ipt.P * (1 << 15) + 0.5f), }; } -- 2.52.0 From 27c4dee1347d68623902d1e7147336c43fdf8bb6 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Thu, 23 Jul 2026 19:48:24 +0200 Subject: [PATCH 12/18] swscale/ops: add SWS_OP_LUT_3D and supporting code After extensive testing, prototyping and benchmarking across a range of systems, I determined that the optimal data layout for the SIMD 3DLUT is essentially exactly the one we have. 16-bit integers are near optimal for quality vs compactness, and crucially, x86 lets us load the entire packed 3DLUT entry with a single `vpgatherdq` instruction. This dwarfs the loss from needing to cast the resulting 16-bit integers back to f32 and renormalize. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/ops.c | 37 ++++++++++++++++++++++++++++++++++++- libswscale/ops.h | 31 +++++++++++++++++++++++++++++++ libswscale/ops_optimizer.c | 11 +++++++++++ 3 files changed, 78 insertions(+), 1 deletion(-) diff --git a/libswscale/ops.c b/libswscale/ops.c index 21462df386..8482f7c5ec 100644 --- a/libswscale/ops.c +++ b/libswscale/ops.c @@ -88,6 +88,7 @@ const char *ff_sws_op_type_name(SwsOpType op) case SWS_OP_DITHER: return "SWS_OP_DITHER"; case SWS_OP_FILTER_H: return "SWS_OP_FILTER_H"; case SWS_OP_FILTER_V: return "SWS_OP_FILTER_V"; + case SWS_OP_LUT_3D: return "SWS_OP_LUT_3D"; case SWS_OP_INVALID: return "SWS_OP_INVALID"; case SWS_OP_TYPE_NB: break; } @@ -266,6 +267,11 @@ void ff_sws_apply_op_q(const SwsOp *op, AVRational64 x[4]) /* Filters have normalized energy by definition, so they don't * conceptually modify individual components */ return; + case SWS_OP_LUT_3D: + /* 3D LUTs are treated as a black box, so set those values to NaN */ + for (int i = 0; i < 3; i++) + x[i] = (AVRational64) {0}; + return; } av_unreachable("Invalid operation type!"); @@ -323,6 +329,7 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) case SWS_OP_UNPACK: case SWS_OP_FILTER_H: case SWS_OP_FILTER_V: + case SWS_OP_LUT_3D: break; /* special cases, handled below */ default: memcpy(op->comps.min, prev.min, sizeof(prev.min)); @@ -518,7 +525,21 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) apply_filter_weights(&op->comps, &prev, op->filter.kernel); break; } - + case SWS_OP_LUT_3D: + for (int i = 0; i < 3; i++) { + /* 3x3 dependency matrix; strip all information except + * SWS_COMP_GARBAGE (for correctness validation) */ + for (int j = 0; j < 3; j++) + FORWARD(i, j, flags & SWS_COMP_GARBAGE); + /* LUT output domain is always scaled to full 16-bit range */ + op->comps.min[i] = Q(0); + op->comps.max[i] = Q(UINT16_MAX); + } + /* Pass through alpha channel untouched */ + FORWARD(3, 3, flags); + op->comps.min[3] = prev.min[3]; + op->comps.max[3] = prev.max[3]; + break; case SWS_OP_INVALID: case SWS_OP_TYPE_NB: av_unreachable("Invalid operation type!"); @@ -587,6 +608,11 @@ void ff_sws_op_list_update_comps(SwsOpList *ops) } } break; + case SWS_OP_LUT_3D: + for (int i = 0; i < 3; i++) + need_in[i] = need_out[0] | need_out[1] | need_out[2]; + need_in[3] = need_out[3]; + break; } memcpy(need_out, need_in, sizeof(need_in)); @@ -609,6 +635,9 @@ static void op_uninit(SwsOp *op) case SWS_OP_FILTER_V: av_refstruct_unref(&op->filter.kernel); break; + case SWS_OP_LUT_3D: + av_refstruct_unref(&op->lut3d.lut); + break; } *op = (SwsOp) {0}; @@ -672,6 +701,9 @@ SwsOpList *ff_sws_op_list_duplicate(const SwsOpList *ops) case SWS_OP_FILTER_V: av_refstruct_ref(op->filter.kernel); break; + case SWS_OP_LUT_3D: + av_refstruct_ref_c(op->lut3d.lut); + break; } } @@ -930,6 +962,9 @@ void ff_sws_op_desc(AVBPrint *bp, const SwsOp *op) kernel->name, kernel->filter_size); break; } + case SWS_OP_LUT_3D: + av_bprintf(bp, "%-20s: %s", name, op->lut3d.dynamic ? "dynamic" : "static"); + break; case SWS_OP_TYPE_NB: break; } diff --git a/libswscale/ops.h b/libswscale/ops.h index 83ef2b49df..008e8c5b31 100644 --- a/libswscale/ops.h +++ b/libswscale/ops.h @@ -29,6 +29,7 @@ #include "graph.h" #include "filters.h" +#include "lut3d.h" #include "rational64.h" #include "uops.h" @@ -62,6 +63,9 @@ typedef enum SwsOpType { SWS_OP_FILTER_H, /* horizontal filtering */ SWS_OP_FILTER_V, /* vertical filtering */ + /* Table-based operations. Defined for floating point types only. */ + SWS_OP_LUT_3D, /* apply a SwsLut3D */ + SWS_OP_TYPE_NB, } SwsOpType; @@ -204,6 +208,32 @@ typedef struct SwsFilterOp { SwsPixelType type; /* pixel type to store result as */ } SwsFilterOp; +typedef struct SwsLut3dOp { + /** + * Reference to the external LUT3D to apply. This is managed by the caller, + * and must remain valid for the lifetime of the SwsOp and any compiled + * functions derived from it. + * + * *lut is never dereferenced by the SwsOp code itself, only at runtime by + * the actual dispatched implementation, and may be freely modified even + * after op compilation to place new values for dynamic tone-mapping. + * + * The reference algorithm for this operation lives in lut3d.c, and + * includes a tetrahedral interpolation component for the input LUT, and + * then an optional linear tone mapping LUT plus trilinear output LUT + * (when lut->dynamic is true). + * + * All linear interpolations are performed in the pixel value's native + * representation, even though the LUTs themselves are stored as unsigned + * packed 16-bit integers. The input value range is assumed to be scaled + * and clamped to the LUT's domain (i.e. [0, INPUT_LUT_SIZE - 1]), and the + * output value range will be [0, 2^16-1], except for the alpha channel, + * which is passed through untouched. + */ + const SwsLut3D *lut; /* refstruct */ + bool dynamic; +} SwsLut3dOp; + typedef struct SwsOp { SwsOpType op; /* operation to perform */ SwsPixelType type; /* pixel type to operate on */ @@ -219,6 +249,7 @@ typedef struct SwsOp { SwsScaleOp scale; SwsDitherOp dither; SwsFilterOp filter; + SwsLut3dOp lut3d; }; /** diff --git a/libswscale/ops_optimizer.c b/libswscale/ops_optimizer.c index 3546d8bc9e..fd3b70deed 100644 --- a/libswscale/ops_optimizer.c +++ b/libswscale/ops_optimizer.c @@ -77,6 +77,7 @@ static bool op_commute_clear(SwsOp *op, SwsOp *next) case SWS_OP_PACK: case SWS_OP_UNPACK: case SWS_OP_CLEAR: + case SWS_OP_LUT_3D: return false; case SWS_OP_TYPE_NB: break; @@ -157,6 +158,7 @@ static bool op_commute_swizzle(SwsOp *op, SwsOp *next) case SWS_OP_LINEAR: case SWS_OP_PACK: case SWS_OP_UNPACK: + case SWS_OP_LUT_3D: return false; case SWS_OP_TYPE_NB: break; @@ -198,6 +200,7 @@ static bool op_commute_filter(SwsOp *op, SwsOp *prev) case SWS_OP_MAX: case SWS_OP_FILTER_H: case SWS_OP_FILTER_V: + case SWS_OP_LUT_3D: return false; case SWS_OP_TYPE_NB: break; @@ -737,6 +740,14 @@ retry: goto retry; } break; + + case SWS_OP_LUT_3D: + /* Eliminate unnecessary 3DLUT */ + if (!(needed & SWS_COMP_ELEMS(3))) { + ff_sws_op_list_remove_at(ops, n, 1); + goto retry; + } + break; } } -- 2.52.0 From c55050230459e23938ec723ec0e5cf7da92e9762 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Thu, 23 Jul 2026 21:26:52 +0200 Subject: [PATCH 13/18] swscale/uops: add SWS_UOP_LUT_3D reference implementation The structure for the tetrahedral interpolation deviates slightly from the naive formulation in lut3d.c; instead of branching into every separate case, we sort the weights and offsets using a series of conditional swaps. This actually performs identically on my end, but results in code that is much closer to what SIMD will be doing. That should hopefully serve as a better reference for future SIMD implementors. (Myself included) I also reordered the dynamic tone-mapping code a bit to better indicate the sources of live register pressure that will manifest in the real SIMD kernel. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/csputils.h | 4 + libswscale/uops.c | 11 +++ libswscale/uops.h | 8 ++ libswscale/uops_backend.c | 1 + libswscale/uops_list.h | 1 + libswscale/uops_macros.h | 8 ++ libswscale/uops_tmpl.c | 159 +++++++++++++++++++++++++++++++++++++- 7 files changed, 191 insertions(+), 1 deletion(-) diff --git a/libswscale/csputils.h b/libswscale/csputils.h index 82eb0b68f4..5437276ead 100644 --- a/libswscale/csputils.h +++ b/libswscale/csputils.h @@ -73,6 +73,10 @@ typedef struct v3u16_t { uint16_t x, y, z; } v3u16_t; +typedef struct v3f32_t { + float x, y, z; +} v3f32_t; + /* Fast perceptual quantizer */ static const float PQ_M1 = 2610./4096 * 1./4, PQ_M2 = 2523./4096 * 128, diff --git a/libswscale/uops.c b/libswscale/uops.c index fc9d609636..95ce6c719d 100644 --- a/libswscale/uops.c +++ b/libswscale/uops.c @@ -159,6 +159,9 @@ void ff_sws_uop_name(const SwsUOp *op, char buf[SWS_UOP_NAME_MAX]) const unsigned size = 1u << par->dither.size_log2; av_bprintf(&bp, "_%ux%u", size, size); break; + case SWS_UOP_LUT_3D: + av_bprintf(&bp, "_%s", par->lut3d.dynamic ? "dynamic" : "static"); + break; } av_assert0(av_bprint_is_complete(&bp)); @@ -175,6 +178,9 @@ static void uop_uninit(SwsUOp *uop) case SWS_UOP_READ_PLANAR_FV_FMA: av_refstruct_unref(&uop->data.kernel); break; + case SWS_UOP_LUT_3D: + av_refstruct_unref(&uop->data.lut3d); + break; } *uop = (SwsUOp) {0}; @@ -655,6 +661,11 @@ static int translate_op(SwsContext *ctx, SwsUOpList *uops, SwsUOpFlags flags, uop.uop = SWS_UOP_SWAP_BYTES; uop.type = pixel_type_to_int(op->type); break; + case SWS_OP_LUT_3D: + uop.uop = SWS_UOP_LUT_3D; + uop.par.lut3d.dynamic = op->lut3d.dynamic; + uop.data.lut3d = av_refstruct_ref_c(op->lut3d.lut); + break; default: return AVERROR(ENOTSUP); } diff --git a/libswscale/uops.h b/libswscale/uops.h index 198e6caea9..feb8744865 100644 --- a/libswscale/uops.h +++ b/libswscale/uops.h @@ -33,6 +33,7 @@ typedef struct SwsContext SwsContext; typedef struct SwsFilterWeights SwsFilterWeights; +typedef struct SwsLut3D SwsLut3D; typedef struct SwsOpList SwsOpList; typedef enum SwsPixelType { @@ -175,6 +176,7 @@ typedef enum SwsUOpType { SWS_UOP_LINEAR, /* mask = non-trivial output rows */ SWS_UOP_LINEAR_FMA, /* with SWS_UOP_FLAG_FMA */ SWS_UOP_DITHER, /* mask = components to dither */ + SWS_UOP_LUT_3D, /* mask = needed output components */ /* Platform-specific uops would go here */ SWS_UOP_TYPE_NB, @@ -237,6 +239,10 @@ typedef struct SwsDitherUOp { uint8_t size_log2; } SwsDitherUOp; +typedef struct SwsLut3DUOp { + int dynamic; +} SwsLut3DUOp; + /** * Computes (1 << size_log2) + MAX(y_offset). The dither matrix attached to * the SwsUOp is always pre-padded to this number of lines. @@ -252,6 +258,7 @@ typedef union SwsUOpParams { SwsClearUOp clear; SwsLinearUOp lin; SwsDitherUOp dither; + SwsLut3DUOp lut3d; } SwsUOpParams; typedef struct SwsUOp { @@ -269,6 +276,7 @@ typedef struct SwsUOp { SwsPixel vec4[4]; SwsPixel mat4[4][5]; /* row major */ SwsShuffleMask shuffle; /* for SWS_UOP_RW_SHUFFLE */ + const SwsLut3D *lut3d; /* for SWS_UOP_LUT_3D; refstruct */ void *opaque; /* reserved for internal use */ } data; } SwsUOp; diff --git a/libswscale/uops_backend.c b/libswscale/uops_backend.c index d0d05ee42f..5b35c1b585 100644 --- a/libswscale/uops_backend.c +++ b/libswscale/uops_backend.c @@ -92,6 +92,7 @@ SWS_FOR(TYPE, CLEAR, REF_ENTRY) \ SWS_FOR(TYPE, LINEAR, REF_ENTRY) \ SWS_FOR(TYPE, DITHER, REF_ENTRY) \ + SWS_FOR(TYPE, LUT_3D, REF_ENTRY) \ /* end of macro */ static const SwsUOpTable uop_table = { diff --git a/libswscale/uops_list.h b/libswscale/uops_list.h index a35af90360..31632dfcc0 100644 --- a/libswscale/uops_list.h +++ b/libswscale/uops_list.h @@ -57,5 +57,6 @@ ENTRY(SWS_UOP_RSHIFT, "rshift") \ ENTRY(SWS_UOP_CLEAR, "clear") \ ENTRY(SWS_UOP_DITHER, "dither") \ + ENTRY(SWS_UOP_LUT_3D, "lut_3d") \ #endif /* SWSCALE_UOPS_LIST_H */ diff --git a/libswscale/uops_macros.h b/libswscale/uops_macros.h index 6d5c8facf9..3a21efe206 100644 --- a/libswscale/uops_macros.h +++ b/libswscale/uops_macros.h @@ -343,6 +343,8 @@ #define SWS_FOR_STRUCT_U8_LINEAR_FMA(MACRO, ...) #define SWS_FOR_U8_DITHER(MACRO, ...) #define SWS_FOR_STRUCT_U8_DITHER(MACRO, ...) +#define SWS_FOR_U8_LUT_3D(MACRO, ...) +#define SWS_FOR_STRUCT_U8_LUT_3D(MACRO, ...) #define SWS_FOR_U16_READ_PLANAR(MACRO, ...) \ MACRO(__VA_ARGS__, u16_read_planar_x , SWS_PIXEL_U16, SWS_UOP_READ_PLANAR , 0x1) \ MACRO(__VA_ARGS__, u16_read_planar_xy , SWS_PIXEL_U16, SWS_UOP_READ_PLANAR , 0x3) \ @@ -625,6 +627,8 @@ #define SWS_FOR_STRUCT_U16_LINEAR_FMA(MACRO, ...) #define SWS_FOR_U16_DITHER(MACRO, ...) #define SWS_FOR_STRUCT_U16_DITHER(MACRO, ...) +#define SWS_FOR_U16_LUT_3D(MACRO, ...) +#define SWS_FOR_STRUCT_U16_LUT_3D(MACRO, ...) #define SWS_FOR_U32_READ_PLANAR(MACRO, ...) \ MACRO(__VA_ARGS__, u32_read_planar_x , SWS_PIXEL_U32, SWS_UOP_READ_PLANAR , 0x1) \ MACRO(__VA_ARGS__, u32_read_planar_xyz , SWS_PIXEL_U32, SWS_UOP_READ_PLANAR , 0x7) \ @@ -845,6 +849,8 @@ #define SWS_FOR_STRUCT_U32_LINEAR_FMA(MACRO, ...) #define SWS_FOR_U32_DITHER(MACRO, ...) #define SWS_FOR_STRUCT_U32_DITHER(MACRO, ...) +#define SWS_FOR_U32_LUT_3D(MACRO, ...) +#define SWS_FOR_STRUCT_U32_LUT_3D(MACRO, ...) #define SWS_FOR_F32_READ_PLANAR(MACRO, ...) #define SWS_FOR_STRUCT_F32_READ_PLANAR(MACRO, ...) #define SWS_FOR_F32_READ_PLANAR_FH(MACRO, ...) \ @@ -1157,5 +1163,7 @@ MACRO(__VA_ARGS__, f32_dither_xyzw_3_2_0_5_16x16 , .type = SWS_PIXEL_F32, .uop = SWS_UOP_DITHER , .mask = 0xf, .par.dither = { .y_offset = {3, 2, 0, 5}, .size_log2 = 4 }) \ MACRO(__VA_ARGS__, f32_dither_xyzw_5_0_3_2_16x16 , .type = SWS_PIXEL_F32, .uop = SWS_UOP_DITHER , .mask = 0xf, .par.dither = { .y_offset = {5, 0, 3, 2}, .size_log2 = 4 }) \ MACRO(__VA_ARGS__, f32_dither_xyzw_5_2_3_0_16x16 , .type = SWS_PIXEL_F32, .uop = SWS_UOP_DITHER , .mask = 0xf, .par.dither = { .y_offset = {5, 2, 3, 0}, .size_log2 = 4 }) +#define SWS_FOR_F32_LUT_3D(MACRO, ...) +#define SWS_FOR_STRUCT_F32_LUT_3D(MACRO, ...) #endif /* SWSCALE_UOPS_MACROS_H */ diff --git a/libswscale/uops_tmpl.c b/libswscale/uops_tmpl.c index 172b7fb06b..52152bcdd5 100644 --- a/libswscale/uops_tmpl.c +++ b/libswscale/uops_tmpl.c @@ -30,6 +30,7 @@ # define PIXEL_TYPE SWS_PIXEL_F32 # define pixel_t float # define inter_t float +# define vec3_t v3f32_t # define PX F32 # define px f32 #elif BIT_DEPTH == 32 @@ -839,10 +840,166 @@ DECL_FUNC(linear, const SwsCompMask mask, const uint32_t one, const uint32_t zer SWS_FOR(PX, LINEAR, DECL_IMPL, linear) SWS_FOR_STRUCT(PX, LINEAR, DECL_ENTRY, .setup = fn(setup_linear) ) +/****************** + * Look-up tables * + ******************/ + +DECL_SETUP(setup_lut3d, params, out) +{ + const SwsLut3D *lut = params->uop->data.lut3d; + out->priv.ptr = (void *) av_refstruct_ref_c(lut); + out->free = ff_op_priv_unref; + return 0; +} + +#if IS_FLOAT +av_always_inline static vec3_t fn(vec3)(v3u16_t v) +{ + return (vec3_t) { v.x, v.y, v.z }; +} + +#define lerp(a, b, w) ((a) + (w) * ((pixel_t) (b) - (a))) + +av_always_inline static +vec3_t fn(lerp3)(vec3_t a, vec3_t b, pixel_t w) +{ + return (vec3_t) { + lerp(a.x, b.x, w), + lerp(a.y, b.y, w), + lerp(a.z, b.z, w), + }; +} + +av_always_inline static +vec3_t fn(lut3d_static)(const SwsLut3D *restrict lut3d, vec3_t rgb) +{ + const int r_base = (int) rgb.x; + const int g_base = (int) rgb.y; + const int b_base = (int) rgb.z; + + int off0 = (r_base < INPUT_LUT_SIZE - 1); + int off1 = (g_base < INPUT_LUT_SIZE - 1) * INPUT_LUT_SIZE; + int off2 = (b_base < INPUT_LUT_SIZE - 1) * INPUT_LUT_SIZE * INPUT_LUT_SIZE; + pixel_t f0 = rgb.x - r_base; + pixel_t f1 = rgb.y - g_base; + pixel_t f2 = rgb.z - b_base; + + /* Sort offsets descending by relative weight */ + if (f0 < f1) { + FFSWAP(pixel_t, f0, f1); + FFSWAP(int, off0, off1); + } + if (f0 < f2) { + FFSWAP(pixel_t, f0, f2); + FFSWAP(int, off0, off2); + } + if (f1 < f2) { + FFSWAP(pixel_t, f1, f2); + FFSWAP(int, off1, off2); + } + + /* Tetrahedral interpolation */ + const pixel_t w0 = 1 - f0; + const pixel_t w1 = f0 - f1; + const pixel_t w2 = f1 - f2; + const pixel_t w3 = f2; + + const v3u16_t *restrict base = &lut3d->input[b_base][g_base][r_base]; + const vec3_t v0 = fn(vec3)(base[0]); + const vec3_t v1 = fn(vec3)(base[off0]); + const vec3_t v2 = fn(vec3)(base[off0 + off1]); + const vec3_t v3 = fn(vec3)(base[off0 + off1 + off2]); + + return (vec3_t) { + w0 * v0.x + w1 * v1.x + w2 * v2.x + w3 * v3.x, + w0 * v0.y + w1 * v1.y + w2 * v2.y + w3 * v3.y, + w0 * v0.z + w1 * v1.z + w2 * v2.z + w3 * v3.z, + }; +} + +av_always_inline static +vec3_t fn(lut3d_dynamic)(const SwsLut3D *restrict lut3d, vec3_t rgb) +{ + rgb.x *= (TONE_LUT_SIZE - 1) / (pixel_t) UINT16_MAX; + + /* Linear interpolation */ + const int Ix = (int) rgb.x; + const pixel_t If = rgb.x - Ix; + + const v2u16_t a = lut3d->tone_map[Ix]; + const v2u16_t b = lut3d->tone_map[Ix + 1]; + + const pixel_t k = lerp(a.y, b.y, If); + const pixel_t bias = (1 << 15) - k; + const pixel_t scale = k / (pixel_t) (1 << 15); + + rgb.x = lerp(a.x, b.x, If); + rgb.y = bias + scale * rgb.y; + rgb.z = bias + scale * rgb.z; + + /* Re-scale to output LUT size */ + rgb.x *= (OUTPUT_LUT_SIZE_I - 1) / (pixel_t) UINT16_MAX; + rgb.y *= (OUTPUT_LUT_SIZE_PT - 1) / (pixel_t) UINT16_MAX; + rgb.z *= (OUTPUT_LUT_SIZE_PT - 1) / (pixel_t) UINT16_MAX; + + /* Trilinear interpolation */ + const int lo0 = (int) rgb.x; + const int lo1 = (int) rgb.y; + const int lo2 = (int) rgb.z; + + const int hi0 = FFMIN(lo0 + 1, OUTPUT_LUT_SIZE_I - 1); + const int hi1 = FFMIN(lo1 + 1, OUTPUT_LUT_SIZE_PT - 1); + const int hi2 = FFMIN(lo2 + 1, OUTPUT_LUT_SIZE_PT - 1); + + const pixel_t w0 = rgb.x - lo0; + const vec3_t c000 = fn(vec3)(lut3d->output[lo2][lo1][lo0]); + const vec3_t c001 = fn(vec3)(lut3d->output[lo2][lo1][hi0]); + const vec3_t c00 = fn(lerp3)(c000, c001, w0); + const vec3_t c010 = fn(vec3)(lut3d->output[lo2][hi1][lo0]); + const vec3_t c011 = fn(vec3)(lut3d->output[lo2][hi1][hi0]); + const vec3_t c01 = fn(lerp3)(c010, c011, w0); + const vec3_t c100 = fn(vec3)(lut3d->output[hi2][lo1][lo0]); + const vec3_t c101 = fn(vec3)(lut3d->output[hi2][lo1][hi0]); + const vec3_t c10 = fn(lerp3)(c100, c101, w0); + const vec3_t c110 = fn(vec3)(lut3d->output[hi2][hi1][lo0]); + const vec3_t c111 = fn(vec3)(lut3d->output[hi2][hi1][hi0]); + const vec3_t c11 = fn(lerp3)(c110, c111, w0); + + const pixel_t w1 = rgb.y - lo1; + const vec3_t c0 = fn(lerp3)(c00, c01, w1); + const vec3_t c1 = fn(lerp3)(c10, c11, w1); + + const pixel_t w2 = rgb.z - lo2; + return fn(lerp3)(c0, c1, w2); +} + +DECL_FUNC(lut3d, const SwsCompMask mask, const int dynamic) +{ + const SwsLut3D *restrict lut3d = impl->priv.ptr; + + SWS_LOOP + for (int i = 0; i < SWS_BLOCK_SIZE; i++) { + vec3_t c = { x[i], y[i], z[i] }; + c = fn(lut3d_static)(lut3d, c); + if (dynamic) + c = fn(lut3d_dynamic)(lut3d, c); + + x[i] = c.x; + y[i] = c.y; + z[i] = c.z; + } + + CONTINUE(x, y, z, w); +} +#endif /* IS_FLOAT */ + +SWS_FOR(PX, LUT_3D, DECL_IMPL, lut3d) +SWS_FOR_STRUCT(PX, LUT_3D, DECL_ENTRY, .setup = fn(setup_lut3d) ) + #undef PIXEL_MAX #undef PIXEL_SWAP #undef pixel_t #undef inter_t -#undef block_t +#undef vec3_t #undef PX #undef px -- 2.52.0 From 18434659613e765402a16252fbe21e46662bae79 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Fri, 24 Jul 2026 15:41:28 +0200 Subject: [PATCH 14/18] swscale/graph: lift SwsLut3D to SwsGraph top level Since this depends on the overall picture parameters anyways, updating it directly from the private setup() function was always a bit hacky. More importantly, this needed for the ops-based 3DLUT implementation, which is already using the pass priv pointer to store the compiled function. Overall, simpler to just lift it to the top level and update it directly as needed. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/graph.c | 40 ++++++++++++---------------------------- libswscale/graph.h | 6 ++++++ 2 files changed, 18 insertions(+), 28 deletions(-) diff --git a/libswscale/graph.c b/libswscale/graph.c index 1c3fa188d5..1514272c95 100644 --- a/libswscale/graph.c +++ b/libswscale/graph.c @@ -679,25 +679,10 @@ static int add_convert_pass(SwsGraph *graph, const SwsFormat *src, * Gamut and tone mapping * **************************/ -static void free_lut3d(void *priv) -{ - SwsLut3D *lut = priv; - av_refstruct_unref(&lut); -} - -static int setup_lut3d(const SwsFrame *out, const SwsFrame *in, const SwsPass *pass) -{ - SwsLut3D *lut = pass->priv; - - /* Update dynamic frame metadata from the original source frame */ - ff_sws_lut3d_update(lut, &pass->graph->src.color); - return 0; -} - static void run_lut3d(const SwsFrame *out, const SwsFrame *in, int y, int h, const SwsPass *pass) { - SwsLut3D *lut = pass->priv; + const SwsLut3D *lut = pass->graph->lut3d; uint8_t *in_data[4], *out_data[4]; frame_shift(in, y, in_data); frame_shift(out, y, out_data); @@ -713,7 +698,6 @@ static int adapt_colors(SwsGraph *graph, const SwsFormat *src_fmt, SwsFormat src = *src_fmt; SwsFormat dst = *dst_fmt; SwsColorMap map = {0}; - SwsLut3D *lut; int ret; /** @@ -741,30 +725,25 @@ static int adapt_colors(SwsGraph *graph, const SwsFormat *src_fmt, if (src.hw_format != AV_PIX_FMT_NONE || dst.hw_format != AV_PIX_FMT_NONE) return AVERROR(ENOTSUP); - lut = ff_sws_lut3d_alloc(); - if (!lut) + graph->lut3d = ff_sws_lut3d_alloc(); + if (!graph->lut3d) return AVERROR(ENOMEM); - ret = ff_sws_lut3d_generate(lut, &map); - if (ret < 0) { - av_refstruct_unref(&lut); + ret = ff_sws_lut3d_generate(graph->lut3d, &map); + if (ret < 0) return ret; - } const enum AVPixelFormat fmt = AV_PIX_FMT_RGBA64; if (src.format != fmt) { SwsFormat tmp = src; tmp.format = fmt; ret = add_convert_pass(graph, &src, &tmp, input, &input); - if (ret < 0) { - av_refstruct_unref(&lut); + if (ret < 0) return ret; - } } return ff_sws_graph_add_pass(graph, fmt, src.width, src.height, - input, 0, 1, run_lut3d, setup_lut3d, lut, - free_lut3d, output); + input, 0, 1, run_lut3d, NULL, NULL, NULL, output); } /*************************************** @@ -825,6 +804,8 @@ static void graph_uninit(SwsGraph *graph) pass_free(graph->passes[i]); av_free(graph->passes); + av_refstruct_unref(&graph->lut3d); + memset(graph, 0, sizeof(*graph)); } @@ -936,6 +917,9 @@ void ff_sws_graph_update_metadata(SwsGraph *graph, const SwsColor *color) return; ff_color_update_dynamic(&graph->src.color, color); + + if (graph->lut3d) + ff_sws_lut3d_update(graph->lut3d, &graph->src.color); } static void get_field(SwsGraph *graph, const SwsFormat *fmt, diff --git a/libswscale/graph.h b/libswscale/graph.h index 68c6a0b9eb..4399346d46 100644 --- a/libswscale/graph.h +++ b/libswscale/graph.h @@ -28,6 +28,7 @@ #include "swscale.h" #include "format.h" +#include "lut3d.h" static av_always_inline av_const int ff_fmt_vshift(enum AVPixelFormat fmt, int plane) { @@ -144,6 +145,11 @@ typedef struct SwsGraph { */ SwsFormat src, dst; + /** + * 3DLUT state used for gamut/tone mapping. (Optional) + */ + SwsLut3D *lut3d; /* refstruct */ + /** * Temporary execution state inside ff_sws_graph_run(); used to pass * data to worker threads. -- 2.52.0 From 73696ddbcdc4cc5c2f23a92eee489a59a3edb4f8 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Fri, 24 Jul 2026 16:48:07 +0200 Subject: [PATCH 15/18] swscale/graph: move 3DLUT pass to the legacy sws wrapper Instead of generating the 3DLUT and applying it right away, init_passes() now just generates it and passes it as a parameter to add_convert_pass(), which will forward it to the underlying implementation. For the existing ff_sws_lut3d_apply_rgba64() fallback, we relegate this to the legacy sws wrapper explicitly. This does require a temporary fix to force the use of the legacy backend when using a LUT3D, otherwise this would regress the unstable backend. That said, the unstable backend was already not handling 3DLUTs correctly, as it failed to normalize the SwsFormat attributes after the conversion to RGBA64, leading to e.g. double application of the YUV->RGB matrix, so this commit in isolation is technically a bug fix. (In theory, there is the edge case here of previously-unsupported pixel formats which only the ops backend handles, that will now never see the LUT3D application, but I don't think it's even possible to combine such formats with HDR colorspaces, so I think this is a moot point. Either way, we're still cowardly hiding behinde SWS_UNSTABLE so a bit of temporary breakage is fine. The next commit will clean it up properly.) Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/graph.c | 127 ++++++++++++++++++++++++++------------------- 1 file changed, 74 insertions(+), 53 deletions(-) diff --git a/libswscale/graph.c b/libswscale/graph.c index 1514272c95..cce11d6d0a 100644 --- a/libswscale/graph.c +++ b/libswscale/graph.c @@ -391,6 +391,18 @@ static void run_legacy_swscale(const SwsFrame *out, const SwsFrame *in, sws->src_h, out_data, out->linesize, y, h); } +static void run_legacy_lut3d(const SwsFrame *out, const SwsFrame *in, + int y, int h, const SwsPass *pass) +{ + const SwsLut3D *lut = pass->graph->lut3d; + uint8_t *in_data[4], *out_data[4]; + frame_shift(in, y, in_data); + frame_shift(out, y, out_data); + + ff_sws_lut3d_apply_rgba64(lut, in_data[0], in->linesize[0], out_data[0], + out->linesize[0], out->width, h); +} + static void legacy_chr_pos(SwsGraph *graph, int *chr_pos, int override, int *warned) { if (override == -513 || override == *chr_pos) @@ -518,9 +530,12 @@ static int init_legacy_subpass(SwsGraph *graph, SwsContext *sws, return 0; } +static int add_legacy_3dlut_pass(SwsGraph *graph, const SwsFormat *src, + SwsPass *input, SwsPass **output); + static int add_legacy_sws_pass(SwsGraph *graph, const SwsFormat *src, - const SwsFormat *dst, SwsPass *input, - SwsPass **output) + const SwsFormat *dst, const SwsLut3D *lut3d, + SwsPass *input, SwsPass **output) { int ret, warned = 0; SwsContext *const ctx = graph->ctx; @@ -535,6 +550,18 @@ static int add_legacy_sws_pass(SwsGraph *graph, const SwsFormat *src, if (!sws_isSupportedInput(src->format) || !sws_isSupportedOutput(dst->format)) return AVERROR(ENOTSUP); + /* If we need to apply a 3D LUT, add it as an explicit input prepass */ + if (lut3d) { + ret = add_legacy_3dlut_pass(graph, src, input, &input); + if (ret < 0) + return ret; + + SwsFormat tmp = *src; + tmp.format = input->format; + tmp.color = lut3d->map.dst; + return add_legacy_sws_pass(graph, &tmp, dst, NULL, input, output); + } + SwsContext *sws = sws_alloc_context(); if (!sws) return AVERROR(ENOMEM); @@ -609,6 +636,33 @@ static int add_legacy_sws_pass(SwsGraph *graph, const SwsFormat *src, return init_legacy_subpass(graph, sws, input, output); } +static int add_legacy_3dlut_pass(SwsGraph *graph, const SwsFormat *src, + SwsPass *input, SwsPass **output) +{ + int ret; + + const SwsLut3D *lut3d = graph->lut3d; + if (!lut3d) + return 0; + + const enum AVPixelFormat fmt = AV_PIX_FMT_RGBA64; + if (src->format != fmt) { + SwsFormat tmp = *src; + tmp.format = fmt; + ret = add_legacy_sws_pass(graph, src, &tmp, NULL, input, &input); + if (ret < 0) + return ret; + } + + ret = ff_sws_graph_add_pass(graph, fmt, src->width, src->height, + input, 0, 1, run_legacy_lut3d, NULL, NULL, NULL, + output); + if (ret < 0) + return ret; + + return 0; +} + /********************************* * Format conversion and scaling * *********************************/ @@ -656,18 +710,18 @@ static bool prefer_ops_backend(SwsContext *ctx, const SwsFormat *src, const SwsF } static int add_convert_pass(SwsGraph *graph, const SwsFormat *src, - const SwsFormat *dst, SwsPass *input, - SwsPass **output) + const SwsFormat *dst, const SwsLut3D *lut3d, + SwsPass *input, SwsPass **output) { SwsContext *ctx = graph->ctx; int ret; - if (prefer_ops_backend(ctx, src, dst)) { + if (prefer_ops_backend(ctx, src, dst) && !lut3d) { ret = add_ops_convert_pass(graph, src, dst, input, output); if (ret == AVERROR(ENOTSUP)) - ret = add_legacy_sws_pass(graph, src, dst, input, output); + ret = add_legacy_sws_pass(graph, src, dst, lut3d, input, output); } else { - ret = add_legacy_sws_pass(graph, src, dst, input, output); + ret = add_legacy_sws_pass(graph, src, dst, lut3d, input, output); if (ret == AVERROR(ENOTSUP)) ret = add_ops_convert_pass(graph, src, dst, input, output); } @@ -679,26 +733,9 @@ static int add_convert_pass(SwsGraph *graph, const SwsFormat *src, * Gamut and tone mapping * **************************/ -static void run_lut3d(const SwsFrame *out, const SwsFrame *in, int y, int h, - const SwsPass *pass) +static int generate_3dlut(SwsGraph *graph, SwsFormat *src, SwsFormat *dst) { - const SwsLut3D *lut = pass->graph->lut3d; - uint8_t *in_data[4], *out_data[4]; - frame_shift(in, y, in_data); - frame_shift(out, y, out_data); - - ff_sws_lut3d_apply_rgba64(lut, in_data[0], in->linesize[0], out_data[0], - out->linesize[0], out->width, h); -} - -static int adapt_colors(SwsGraph *graph, const SwsFormat *src_fmt, - const SwsFormat *dst_fmt, SwsPass *input, - SwsPass **output) -{ - SwsFormat src = *src_fmt; - SwsFormat dst = *dst_fmt; SwsColorMap map = {0}; - int ret; /** * Grayspace does not really have primaries, so just force the use of @@ -706,44 +743,30 @@ static int adapt_colors(SwsGraph *graph, const SwsFormat *src_fmt, * this does affect the weights used for the Grayscale conversion, but * in practise, that should give the expected results more often than not. */ - if (isGray(dst.format)) { - dst.color = src.color; - } else if (isGray(src.format)) { - src.color = dst.color; + if (isGray(dst->format)) { + dst->color = src->color; + } else if (isGray(src->format)) { + src->color = dst->color; } /* Fully infer color spaces before color mapping logic */ - graph->incomplete |= ff_infer_colors(&src.color, &dst.color); + graph->incomplete |= ff_infer_colors(&src->color, &dst->color); map.intent = graph->ctx->intent; - map.src = src.color; - map.dst = dst.color; + map.src = src->color; + map.dst = dst->color; if (ff_sws_color_map_noop(&map)) return 0; - if (src.hw_format != AV_PIX_FMT_NONE || dst.hw_format != AV_PIX_FMT_NONE) + if (src->hw_format != AV_PIX_FMT_NONE || dst->hw_format != AV_PIX_FMT_NONE) return AVERROR(ENOTSUP); graph->lut3d = ff_sws_lut3d_alloc(); if (!graph->lut3d) return AVERROR(ENOMEM); - ret = ff_sws_lut3d_generate(graph->lut3d, &map); - if (ret < 0) - return ret; - - const enum AVPixelFormat fmt = AV_PIX_FMT_RGBA64; - if (src.format != fmt) { - SwsFormat tmp = src; - tmp.format = fmt; - ret = add_convert_pass(graph, &src, &tmp, input, &input); - if (ret < 0) - return ret; - } - - return ff_sws_graph_add_pass(graph, fmt, src.width, src.height, - input, 0, 1, run_lut3d, NULL, NULL, NULL, output); + return ff_sws_lut3d_generate(graph->lut3d, &map); } /*************************************** @@ -757,14 +780,12 @@ static int init_passes(SwsGraph *graph) SwsPass *pass = NULL; /* read from main input image */ int ret; - ret = adapt_colors(graph, &src, &dst, pass, &pass); + ret = generate_3dlut(graph, &src, &dst); if (ret < 0) return ret; - src.format = pass ? pass->format : src.format; - src.color = dst.color; - if (!ff_fmt_equal(&src, &dst)) { - ret = add_convert_pass(graph, &src, &dst, pass, &pass); + if (!ff_fmt_equal(&src, &dst) || graph->lut3d) { + ret = add_convert_pass(graph, &src, &dst, graph->lut3d, pass, &pass); if (ret < 0) return ret; } -- 2.52.0 From 4a8e36205d82b3d83a69a9b3d6032d39e850e91f Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Fri, 24 Jul 2026 16:54:02 +0200 Subject: [PATCH 16/18] swscale/format: add SwsLut3D support to the ops list generator The value range normalization and input range clamp will normally be optimized away. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/format.c | 62 ++++++++++++++++++++++++++++++- libswscale/format.h | 18 +++++++-- libswscale/graph.c | 12 +++--- libswscale/op_list_gen_template.c | 2 +- 4 files changed, 82 insertions(+), 12 deletions(-) diff --git a/libswscale/format.c b/libswscale/format.c index 2531435dae..cbd7e0263e 100644 --- a/libswscale/format.c +++ b/libswscale/format.c @@ -1745,9 +1745,62 @@ int ff_sws_add_filters(SwsContext *ctx, SwsPixelType type, SwsOpList *ops, return add_filter(ctx, type, ops, SWS_OP_FILTER_V, src->height, dst->height); } +int ff_sws_apply_lut3d(SwsContext *ctx, SwsPixelType type, SwsOpList *ops, + const SwsLut3D *lut3d) +{ + /* Unnormalize to LUT input domain and clamp */ + const AVRational64 domain = Q(INPUT_LUT_SIZE - 1); + + RET(ff_sws_op_list_append(ops, &(SwsOp) { + .type = type, + .op = SWS_OP_LINEAR, + .lin = {{ + { domain, Q(0), Q(0), Q(0), Q(0) }, + { Q(0), domain, Q(0), Q(0), Q(0) }, + { Q(0), Q(0), domain, Q(0), Q(0) }, + { Q(0), Q(0), Q(0), Q(1), Q(0) }, + }}, + })); + + RET(ff_sws_op_list_append(ops, &(SwsOp) { + .op = SWS_OP_MAX, + .type = type, + .clamp = {{ Q(0), Q(0), Q(0) }}, + })); + + RET(ff_sws_op_list_append(ops, &(SwsOp) { + .op = SWS_OP_MIN, + .type = type, + .clamp = {{ domain, domain, domain }}, + })); + + /* Apply the 3DLUT itself */ + RET(ff_sws_op_list_append(ops, &(SwsOp) { + .op = SWS_OP_LUT_3D, + .type = type, + .lut3d.lut = av_refstruct_ref_c(lut3d), + .lut3d.dynamic = lut3d->dynamic, + })); + + /* Normalize back to [0, 1] */ + const AVRational64 inv = av_inv_q64(Q(UINT16_MAX)); + RET(ff_sws_op_list_append(ops, &(SwsOp) { + .type = type, + .op = SWS_OP_LINEAR, + .lin = {{ + { inv, Q(0), Q(0), Q(0), Q(0) }, + { Q(0), inv, Q(0), Q(0), Q(0) }, + { Q(0), Q(0), inv, Q(0), Q(0) }, + { Q(0), Q(0), Q(0), Q(1), Q(0) }, + }}, + })); + + return 0; +} + int ff_sws_op_list_generate(SwsContext *ctx, const SwsFormat *src, - const SwsFormat *dst, SwsOpList **out_ops, - bool *incomplete) + const SwsFormat *dst, const SwsLut3D *lut3d, + SwsOpList **out_ops, bool *incomplete) { /* The new code does not yet support alpha blending */ if (src->desc->flags & AV_PIX_FMT_FLAG_ALPHA && @@ -1770,6 +1823,11 @@ int ff_sws_op_list_generate(SwsContext *ctx, const SwsFormat *src, ret = ff_sws_add_filters(ctx, type, ops, src, dst); if (ret < 0) goto fail; + if (lut3d) { + ret = ff_sws_apply_lut3d(ctx, type, ops, lut3d); + if (ret < 0) + goto fail; + } ret = ff_sws_encode_colors(ctx, type, ops, src, dst, incomplete); if (ret < 0) goto fail; diff --git a/libswscale/format.h b/libswscale/format.h index ea2ab7dc41..35e93faf7b 100644 --- a/libswscale/format.h +++ b/libswscale/format.h @@ -201,13 +201,25 @@ int ff_sws_add_filters(SwsContext *ctx, SwsPixelType type, SwsOpList *ops, const SwsFormat *src, const SwsFormat *dst); /** - * Generate an SwsOpList defining a conversion from `src` to `dst`. + * Append a set of operations for applying a gamut/tone mapping 3D LUT to + * the pixels. The input and output domain are assumed to be normalized + * floating point RGBA in the range [0, 1]. + * + * Returns 0 on success, or a negative error code on failure. + */ +typedef struct SwsLut3D SwsLut3D; +int ff_sws_apply_lut3d(SwsContext *ctx, SwsPixelType type, SwsOpList *ops, + const SwsLut3D *lut3d); + +/** + * Generate an SwsOpList defining a conversion from `src` to `dst`, with an + * optional 3DLUT for converting between gamuts. * * Returns 0 on success, or a negative error code on failure. */ int ff_sws_op_list_generate(SwsContext *ctx, const SwsFormat *src, - const SwsFormat *dst, SwsOpList **out_ops, - bool *incomplete); + const SwsFormat *dst, const SwsLut3D *lut3d, + SwsOpList **out_ops, bool *incomplete); /** * Represents a view into a single field of frame data. diff --git a/libswscale/graph.c b/libswscale/graph.c index cce11d6d0a..4363862d1e 100644 --- a/libswscale/graph.c +++ b/libswscale/graph.c @@ -668,8 +668,8 @@ static int add_legacy_3dlut_pass(SwsGraph *graph, const SwsFormat *src, *********************************/ static int add_ops_convert_pass(SwsGraph *graph, const SwsFormat *src, - const SwsFormat *dst, SwsPass *input, - SwsPass **output) + const SwsFormat *dst, const SwsLut3D *lut3d, + SwsPass *input, SwsPass **output) { #if CONFIG_UNSTABLE SwsContext *ctx = graph->ctx; @@ -683,7 +683,7 @@ static int add_ops_convert_pass(SwsGraph *graph, const SwsFormat *src, return AVERROR(ENOTSUP); SwsOpList *ops; - int ret = ff_sws_op_list_generate(ctx, src, dst, &ops, &graph->incomplete); + int ret = ff_sws_op_list_generate(ctx, src, dst, lut3d, &ops, &graph->incomplete); if (ret < 0) return ret; @@ -716,14 +716,14 @@ static int add_convert_pass(SwsGraph *graph, const SwsFormat *src, SwsContext *ctx = graph->ctx; int ret; - if (prefer_ops_backend(ctx, src, dst) && !lut3d) { - ret = add_ops_convert_pass(graph, src, dst, input, output); + if (prefer_ops_backend(ctx, src, dst)) { + ret = add_ops_convert_pass(graph, src, dst, lut3d, input, output); if (ret == AVERROR(ENOTSUP)) ret = add_legacy_sws_pass(graph, src, dst, lut3d, input, output); } else { ret = add_legacy_sws_pass(graph, src, dst, lut3d, input, output); if (ret == AVERROR(ENOTSUP)) - ret = add_ops_convert_pass(graph, src, dst, input, output); + ret = add_ops_convert_pass(graph, src, dst, lut3d, input, output); } return ret; diff --git a/libswscale/op_list_gen_template.c b/libswscale/op_list_gen_template.c index 0030fddec2..bc7696c51e 100644 --- a/libswscale/op_list_gen_template.c +++ b/libswscale/op_list_gen_template.c @@ -52,7 +52,7 @@ static int enum_ops_fmt(SwsContext *ctx, void *opaque, dst.width = dst_sizes[i][0]; dst.height = dst_sizes[i][1]; - ret = ff_sws_op_list_generate(ctx, &src, &dst, &ops, &incomplete); + ret = ff_sws_op_list_generate(ctx, &src, &dst, NULL, &ops, &incomplete); if (ret == AVERROR(ENOTSUP)) return 0; /* silently skip unsupported formats */ else if (ret < 0) -- 2.52.0 From 9196edf40562055116d1e8bbfbff128577aadce1 Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Fri, 24 Jul 2026 17:12:48 +0200 Subject: [PATCH 17/18] swscale/uops_macros: also generate op lists involving 3DLUTs These may differ from the regular op lists in nontrivial ways, due to e.g. different optimization steps being taken. In practice, it seems this just adds the extra LUT_3D uops, but we don't know that for sure, so better to brute force the list. That said, I do think we can safely skip the extra backend flags in this case, at least. Even in the worst case scenario, that would just force a fallback to the C reference backend. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- libswscale/op_list_gen_template.c | 8 ++++---- libswscale/tests/sws_ops.c | 2 +- libswscale/tests/sws_ops_aarch64.c | 2 +- libswscale/uops_macros.h | 12 ++++++++++-- libswscale/uops_macros_gen.c | 26 ++++++++++++++++++++++++-- 5 files changed, 40 insertions(+), 10 deletions(-) diff --git a/libswscale/op_list_gen_template.c b/libswscale/op_list_gen_template.c index bc7696c51e..19d07fba0a 100644 --- a/libswscale/op_list_gen_template.c +++ b/libswscale/op_list_gen_template.c @@ -29,7 +29,7 @@ #define DUMMY_SIZE 16 -static int enum_ops_fmt(SwsContext *ctx, void *opaque, +static int enum_ops_fmt(SwsContext *ctx, void *opaque, const SwsLut3D *lut3d, enum AVPixelFormat src_fmt, enum AVPixelFormat dst_fmt, int (*cb)(SwsContext *ctx, void *opaque, SwsOpList *ops)) { @@ -52,7 +52,7 @@ static int enum_ops_fmt(SwsContext *ctx, void *opaque, dst.width = dst_sizes[i][0]; dst.height = dst_sizes[i][1]; - ret = ff_sws_op_list_generate(ctx, &src, &dst, NULL, &ops, &incomplete); + ret = ff_sws_op_list_generate(ctx, &src, &dst, lut3d, &ops, &incomplete); if (ret == AVERROR(ENOTSUP)) return 0; /* silently skip unsupported formats */ else if (ret < 0) @@ -86,7 +86,7 @@ fail: * @note `ops` belongs to sws_enum_op_lists(), but may be mutated by `cb`. */ static inline -int ff_sws_enum_op_lists(SwsContext *ctx, void *opaque, +int ff_sws_enum_op_lists(SwsContext *ctx, void *opaque, const SwsLut3D *lut3d, enum AVPixelFormat src_fmt, enum AVPixelFormat dst_fmt, int (*cb)(SwsContext *ctx, void *opaque, SwsOpList *ops)) { @@ -102,7 +102,7 @@ int ff_sws_enum_op_lists(SwsContext *ctx, void *opaque, const enum AVPixelFormat src_f = av_pix_fmt_desc_get_id(src); for (dst = dst_start; dst; dst = av_pix_fmt_desc_next(dst)) { const enum AVPixelFormat dst_f = av_pix_fmt_desc_get_id(dst); - int ret = enum_ops_fmt(ctx, opaque, src_f, dst_f, cb); + int ret = enum_ops_fmt(ctx, opaque, lut3d, src_f, dst_f, cb); if (ret < 0) return ret; if (dst_fmt != AV_PIX_FMT_NONE) diff --git a/libswscale/tests/sws_ops.c b/libswscale/tests/sws_ops.c index bb494b1f4e..677dc4db6e 100644 --- a/libswscale/tests/sws_ops.c +++ b/libswscale/tests/sws_ops.c @@ -173,7 +173,7 @@ bad_option: av_log_set_callback(log_stdout); - ret = ff_sws_enum_op_lists(ctx, graph, src_fmt, dst_fmt, print_passes); + ret = ff_sws_enum_op_lists(ctx, graph, NULL, src_fmt, dst_fmt, print_passes); if (ret < 0) goto fail; diff --git a/libswscale/tests/sws_ops_aarch64.c b/libswscale/tests/sws_ops_aarch64.c index cb47f42037..2155319a33 100644 --- a/libswscale/tests/sws_ops_aarch64.c +++ b/libswscale/tests/sws_ops_aarch64.c @@ -460,7 +460,7 @@ int main(int argc, char *argv[]) graph->ctx = ctx; ctx->opaque = &root; - ret = ff_sws_enum_op_lists(ctx, graph, AV_PIX_FMT_NONE, AV_PIX_FMT_NONE, + ret = ff_sws_enum_op_lists(ctx, graph, NULL, AV_PIX_FMT_NONE, AV_PIX_FMT_NONE, register_op); /** diff --git a/libswscale/uops_macros.h b/libswscale/uops_macros.h index 3a21efe206..84fbd3b073 100644 --- a/libswscale/uops_macros.h +++ b/libswscale/uops_macros.h @@ -1163,7 +1163,15 @@ MACRO(__VA_ARGS__, f32_dither_xyzw_3_2_0_5_16x16 , .type = SWS_PIXEL_F32, .uop = SWS_UOP_DITHER , .mask = 0xf, .par.dither = { .y_offset = {3, 2, 0, 5}, .size_log2 = 4 }) \ MACRO(__VA_ARGS__, f32_dither_xyzw_5_0_3_2_16x16 , .type = SWS_PIXEL_F32, .uop = SWS_UOP_DITHER , .mask = 0xf, .par.dither = { .y_offset = {5, 0, 3, 2}, .size_log2 = 4 }) \ MACRO(__VA_ARGS__, f32_dither_xyzw_5_2_3_0_16x16 , .type = SWS_PIXEL_F32, .uop = SWS_UOP_DITHER , .mask = 0xf, .par.dither = { .y_offset = {5, 2, 3, 0}, .size_log2 = 4 }) -#define SWS_FOR_F32_LUT_3D(MACRO, ...) -#define SWS_FOR_STRUCT_F32_LUT_3D(MACRO, ...) +#define SWS_FOR_F32_LUT_3D(MACRO, ...) \ + MACRO(__VA_ARGS__, f32_lut_3d_xyz_static , SWS_PIXEL_F32, SWS_UOP_LUT_3D , 0x7, 0) \ + MACRO(__VA_ARGS__, f32_lut_3d_xyz_dynamic , SWS_PIXEL_F32, SWS_UOP_LUT_3D , 0x7, 1) \ + MACRO(__VA_ARGS__, f32_lut_3d_xyzw_static , SWS_PIXEL_F32, SWS_UOP_LUT_3D , 0xf, 0) \ + MACRO(__VA_ARGS__, f32_lut_3d_xyzw_dynamic , SWS_PIXEL_F32, SWS_UOP_LUT_3D , 0xf, 1) +#define SWS_FOR_STRUCT_F32_LUT_3D(MACRO, ...) \ + MACRO(__VA_ARGS__, f32_lut_3d_xyz_static , .type = SWS_PIXEL_F32, .uop = SWS_UOP_LUT_3D , .mask = 0x7, .par.lut3d.dynamic = 0) \ + MACRO(__VA_ARGS__, f32_lut_3d_xyz_dynamic , .type = SWS_PIXEL_F32, .uop = SWS_UOP_LUT_3D , .mask = 0x7, .par.lut3d.dynamic = 1) \ + MACRO(__VA_ARGS__, f32_lut_3d_xyzw_static , .type = SWS_PIXEL_F32, .uop = SWS_UOP_LUT_3D , .mask = 0xf, .par.lut3d.dynamic = 0) \ + MACRO(__VA_ARGS__, f32_lut_3d_xyzw_dynamic , .type = SWS_PIXEL_F32, .uop = SWS_UOP_LUT_3D , .mask = 0xf, .par.lut3d.dynamic = 1) #endif /* SWSCALE_UOPS_MACROS_H */ diff --git a/libswscale/uops_macros_gen.c b/libswscale/uops_macros_gen.c index 2df9a9c2c2..ad77050f0c 100644 --- a/libswscale/uops_macros_gen.c +++ b/libswscale/uops_macros_gen.c @@ -117,6 +117,9 @@ static int generate_entry_struct(void *opaque, void *key) par->dither.y_offset[2], par->dither.y_offset[3], par->dither.size_log2); break; + case SWS_UOP_LUT_3D: + av_bprintf(bp, ", .par.lut3d.dynamic = %d", par->lut3d.dynamic); + break; } av_bprintf(bp, ")"); @@ -179,6 +182,9 @@ static int generate_entry_args(void *opaque, void *key) par->dither.y_offset[2], par->dither.y_offset[3], par->dither.size_log2); break; + case SWS_UOP_LUT_3D: + av_bprintf(bp, ", %d", par->lut3d.dynamic); + break; } av_bprintf(bp, ")"); @@ -320,11 +326,27 @@ static int sws_uops_macros_gen(char **out_str) ctx->opaque = &root; ctx->scaler = SWS_SCALE_BILINEAR; /* cheaper to generate filter kernels */ + /* Allocate dummy 3DLUT to force generation of SWS_UOP_LUT_3D */ + SwsLut3D *lut3d = ff_sws_lut3d_alloc(); + ret = ff_sws_enum_op_lists(ctx, graph, lut3d, AV_PIX_FMT_NONE, + AV_PIX_FMT_NONE, register_all_uops); + if (ret < 0) { + av_refstruct_unref(&lut3d); + goto fail; + } + + lut3d->dynamic = true; + ret = ff_sws_enum_op_lists(ctx, graph, lut3d, AV_PIX_FMT_NONE, + AV_PIX_FMT_NONE, register_all_uops); + av_refstruct_unref(&lut3d); + if (ret < 0) + goto fail; + /* Register all unique uops over every relevant combination of flags */ for (int i = 0; i < FF_ARRAY_ELEMS(flags_list); i++) { ctx->flags = flags_list[i]; - ret = ff_sws_enum_op_lists(ctx, graph, AV_PIX_FMT_NONE, AV_PIX_FMT_NONE, - register_all_uops); + ret = ff_sws_enum_op_lists(ctx, graph, NULL, AV_PIX_FMT_NONE, + AV_PIX_FMT_NONE, register_all_uops); if (ret < 0) goto fail; } -- 2.52.0 From 027efd2372c0f9f80857c7f0cc5d442bfac2a27a Mon Sep 17 00:00:00 2001 From: Niklas Haas <[email protected]> Date: Sun, 26 Jul 2026 00:04:59 +0200 Subject: [PATCH 18/18] checkasm/sw_ops: add SWS_UOP_LUT_3D test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit checkasm: - CPU: AMD Ryzen 9 9950X3D 16-Core Processor (00B40F40) - Timing source: x86 (rdtsc) - Bench duration: 100000 µs per function (448667793 cycles) - Random seed: 3773883393 Benchmark results: name cycles (vs ref) f32_lut_3d_xyz_dynamic_c: 125193.9 f32_lut_3d_xyz_static_c: 25792.1 f32_lut_3d_xyzw_dynamic_c: 123807.1 f32_lut_3d_xyzw_static_c: 25739.0 This is roughly ~50% faster than the existing code in lut3d.c, from a quick test. Sponsored-by: Sovereign Tech Fund Signed-off-by: Niklas Haas <[email protected]> --- tests/checkasm/sw_ops.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/tests/checkasm/sw_ops.c b/tests/checkasm/sw_ops.c index df81caf469..1e4c1abf7a 100644 --- a/tests/checkasm/sw_ops.c +++ b/tests/checkasm/sw_ops.c @@ -634,6 +634,27 @@ static void check_dither(const char *name, SwsUOp *uop) av_refstruct_unref(&matrix); } +static void check_lut_3d(const char *name, SwsUOp *uop) +{ + SwsLut3D *lut3d = ff_sws_lut3d_alloc(); + checkasm_init(&lut3d->input, sizeof(lut3d->input)); + + if (uop->par.lut3d.dynamic) { + checkasm_init(&lut3d->tone_map, sizeof(lut3d->tone_map)); + checkasm_init(&lut3d->output, sizeof(lut3d->output)); + lut3d->dynamic = true; + + /* Prevent out-of-bounds read from IPT values with abs(PT) > 0.5 */ + for (int i = 0; i < FF_ARRAY_ELEMS(lut3d->tone_map); i++) + lut3d->tone_map[i].y = FFMIN(lut3d->tone_map[i].y, 1 << 15); + } + + uop->data.lut3d = lut3d; + check_range(name, uop, MK_RANGES(INPUT_LUT_SIZE - 1)); + + av_refstruct_unref(&lut3d); +} + #define CHECK_FUNCTION(CHECK, NAME, ...) \ CHECK(#NAME, &(SwsUOp) { __VA_ARGS__ }); @@ -678,4 +699,5 @@ void checkasm_check_sw_ops(void) CHECK_FOR(CLEAR, check_vec4); CHECK_FOR(LINEAR, check_linear); CHECK_FOR(DITHER, check_dither); + CHECK_FOR(LUT_3D, check_lut_3d); } -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
