PR #25867 opened by Kilian (lusoris) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25867 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25867.patch
vf_zscale drops the content light level and the mastering display metadata on every transfer change. That includes a change to linear light in the same primaries, so a tone mapper further down the graph loses the peak it reads from that side data; vf_tonemap then falls back to a fixed peak. This change keeps those two side data types when the output is linear light in the same primaries, and drops the other colour-dependent side data as before. ## What changes - `libavfilter/vf_zscale.c`: side data pruning moves into `prune_side_data()`. A transfer change to `linear` with unchanged primaries removes the colour-dependent side data except `AV_FRAME_DATA_CONTENT_LIGHT_LEVEL` and `AV_FRAME_DATA_MASTERING_DISPLAY_METADATA`. Every other transfer or primaries change, and every size change, prunes as before. - The primaries count as unchanged only when they match both as tagged on the frames and as converted by zimg. With `primariesin` set, the output keeps the input tag although zimg converts from the option's primaries, so that case still drops the light levels. - `libavfilter/tests/zscale_light_levels.c`: new test program. It runs zscale on a PQ BT.2020 frame carrying a content light level, a mastering display and an ICC profile, and prints which of the three survive. - `tests/fate/filter-video.mak`, `tests/ref/fate/filter-zscale-light-levels`: new FATE test `fate-filter-zscale-light-levels`. The test covers five cases: | zscale options | light level | mastering display | ICC profile | | :--- | :--- | :--- | :--- | | `t=linear` | kept | kept | dropped | | `t=linear:p=bt709` | dropped | dropped | dropped | | `t=linear:primariesin=bt709` | dropped | dropped | dropped | | `t=bt709` | dropped | dropped | dropped | | `w=8:h=4` | kept | kept | kept | ## Testing Built on current master with GCC 16.2.1 and zimg 3.0.6 (`--enable-libzimg`). - `make fate-filter-zscale-light-levels fate-filter-zscale-passthrough fate-filter-zscale-chroma-input-override`: all pass. - With the `vf_zscale.c` change reverted and the new test kept, `fate-filter-zscale-light-levels` differs on the first case only (`t=linear: light level dropped, mastering display dropped`), so the test detects the old behaviour. - Without the check on the converted primaries, the `t=linear:primariesin=bt709` case keeps the light level and the mastering display, so the test also covers that check. - `ffmpeg` and `libavfilter/tests/zscale_light_levels` build with no warnings. >From 678b45a95831e2f0d3d4f322fc684b860a8af89f Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Sun, 8 Nov 2026 16:42:06 +0100 Subject: [PATCH] avfilter/vf_zscale: keep the light levels when linearising A transfer change drops all colour-dependent side data. When the output is linear light in the same primaries, the content light level and the mastering display still describe the frame, and a tone mapper further down the graph reads its peak from them; without them vf_tonemap falls back to a fixed peak. Keep those two in that case and drop the other colour-dependent side data as before. The primaries must match both as tagged on the frames and as converted by zimg: with primariesin set, the output keeps the input tag although zimg converts from the option's primaries. The test runs zscale on a PQ BT.2020 frame with light levels and an ICC profile, and checks which side data survive a change to linear, a change of primaries, a change of input primaries, a change to BT.709 and a plain resize. Signed-off-by: lusoris <[email protected]> --- libavfilter/Makefile | 1 + libavfilter/tests/zscale_light_levels.c | 139 ++++++++++++++++++++++ libavfilter/vf_zscale.c | 45 ++++++- tests/fate/filter-video.mak | 4 + tests/ref/fate/filter-zscale-light-levels | 5 + 5 files changed, 188 insertions(+), 6 deletions(-) create mode 100644 libavfilter/tests/zscale_light_levels.c create mode 100644 tests/ref/fate/filter-zscale-light-levels diff --git a/libavfilter/Makefile b/libavfilter/Makefile index dc5c083f013..bc2493fb5b3 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -692,6 +692,7 @@ TOOLS = graph2dot TESTPROGS = drawutils filtfmts formats integral TESTPROGS-$(CONFIG_DRAWVG_FILTER) += drawvg +TESTPROGS-$(CONFIG_ZSCALE_FILTER) += zscale_light_levels TOOLS-$(CONFIG_LIBZMQ) += zmqsend diff --git a/libavfilter/tests/zscale_light_levels.c b/libavfilter/tests/zscale_light_levels.c new file mode 100644 index 00000000000..c7ceceaefc1 --- /dev/null +++ b/libavfilter/tests/zscale_light_levels.c @@ -0,0 +1,139 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* Which colour-dependent side data zscale keeps on a transfer change. */ + +#include <stdio.h> + +#include "libavutil/frame.h" +#include "libavutil/log.h" +#include "libavutil/mastering_display_metadata.h" + +#include "libavfilter/avfilter.h" +#include "libavfilter/buffersink.h" +#include "libavfilter/buffersrc.h" + +#define FW 16 +#define FH 8 + +/* Runs buffer -> zscale=<args> -> buffersink on one frame. */ +static int run_filter(AVFrame *in, AVFrame *out, const char *zargs) +{ + AVFilterGraph *g = avfilter_graph_alloc(); + AVFilterContext *src = NULL, *zs = NULL, *sink = NULL; + char args[128]; + int ret; + + if (!g) + return AVERROR(ENOMEM); + snprintf(args, sizeof(args), "video_size=%dx%d:pix_fmt=%d:time_base=1/25:colorspace=%d", + in->width, in->height, in->format, in->colorspace); + ret = avfilter_graph_create_filter(&src, avfilter_get_by_name("buffer"), "in", args, NULL, g); + if (ret >= 0) + ret = avfilter_graph_create_filter(&zs, avfilter_get_by_name("zscale"), "zs", zargs, NULL, g); + if (ret >= 0) + ret = avfilter_graph_create_filter(&sink, avfilter_get_by_name("buffersink"), "out", NULL, NULL, g); + if (ret >= 0) + ret = avfilter_link(src, 0, zs, 0); + if (ret >= 0) + ret = avfilter_link(zs, 0, sink, 0); + if (ret >= 0) + ret = avfilter_graph_config(g, NULL); + if (ret >= 0) + ret = av_buffersrc_add_frame_flags(src, in, AV_BUFFERSRC_FLAG_KEEP_REF); + if (ret >= 0) + ret = av_buffersrc_add_frame(src, NULL); + if (ret >= 0) + ret = av_buffersink_get_frame(sink, out); + avfilter_graph_free(&g); + return ret; +} + +static int add_side_data(AVFrame *f) +{ + AVContentLightMetadata *cll = av_content_light_metadata_create_side_data(f); + AVMasteringDisplayMetadata *md = av_mastering_display_metadata_create_side_data(f); + + if (!cll || !md || !av_frame_new_side_data(f, AV_FRAME_DATA_ICC_PROFILE, 16)) + return AVERROR(ENOMEM); + cll->MaxCLL = 1000; + md->max_luminance = av_make_q(1000, 1); + md->min_luminance = av_make_q(1, 10000); + md->has_luminance = 1; + return 0; +} + +/* A PQ BT.2020 frame with light levels and an ICC profile. */ +static AVFrame *make_frame(void) +{ + AVFrame *f = av_frame_alloc(); + + if (!f) + return NULL; + f->format = AV_PIX_FMT_GBRPF32; + f->width = FW; + f->height = FH; + f->color_primaries = AVCOL_PRI_BT2020; + f->color_trc = AVCOL_TRC_SMPTE2084; + f->colorspace = AVCOL_SPC_RGB; + if (av_frame_get_buffer(f, 0) < 0 || add_side_data(f) < 0) { + av_frame_free(&f); + return NULL; + } + for (int p = 0; p < 3; p++) + for (int y = 0; y < FH; y++) + for (int x = 0; x < FW; x++) + ((float *)(f->data[p] + y * f->linesize[p]))[x] = 0.5f; + return f; +} + +static const char *has(const AVFrame *f, enum AVFrameSideDataType type) +{ + return av_frame_get_side_data(f, type) ? "kept" : "dropped"; +} + +/* Prints which side data survive zscale=<zargs>; returns 0 when the run worked. */ +static int report(const char *zargs) +{ + AVFrame *in = make_frame(), *out = av_frame_alloc(); + int ret = in && out ? run_filter(in, out, zargs) : AVERROR(ENOMEM); + + if (ret >= 0) + printf("%s: light level %s, mastering display %s, ICC profile %s\n", zargs, + has(out, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL), + has(out, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA), + has(out, AV_FRAME_DATA_ICC_PROFILE)); + else + printf("%s: failed\n", zargs); + av_frame_free(&in); + av_frame_free(&out); + return ret < 0; +} + +int main(void) +{ + int fails = 0; + + av_log_set_level(AV_LOG_QUIET); + fails += report("t=linear"); + fails += report("t=linear:p=bt709"); + fails += report("t=linear:primariesin=bt709"); + fails += report("t=bt709"); + fails += report("w=8:h=4"); + return fails != 0; +} diff --git a/libavfilter/vf_zscale.c b/libavfilter/vf_zscale.c index 8f600cbf035..02ac0c6c6d7 100644 --- a/libavfilter/vf_zscale.c +++ b/libavfilter/vf_zscale.c @@ -754,6 +754,43 @@ static int filter_slice(AVFilterContext *ctx, void *data, int job_nr, int n_jobs return 0; } +/* + * Linear light in the same primaries keeps the light levels of its source, so + * a tone mapper further down can still read the peak from them. + */ +static void remove_color_dependent_keep_light(AVFrame *out) +{ + for (int i = out->nb_side_data - 1; i >= 0; i--) { + const AVSideDataDescriptor *desc; + enum AVFrameSideDataType type; + + if (i >= out->nb_side_data) + continue; + type = out->side_data[i]->type; + desc = av_frame_side_data_desc(type); + if (type == AV_FRAME_DATA_CONTENT_LIGHT_LEVEL || + type == AV_FRAME_DATA_MASTERING_DISPLAY_METADATA) + continue; + if (desc && desc->props & AV_SIDE_DATA_PROP_COLOR_DEPENDENT) + av_frame_side_data_remove(&out->side_data, &out->nb_side_data, type); + } +} + +static void prune_side_data(const ZScaleContext *s, AVFrame *out, const AVFrame *in) +{ + int changed = 0; + + if (out->width != in->width || out->height != in->height) + changed |= AV_SIDE_DATA_PROP_SIZE_DEPENDENT; + if (out->color_trc != in->color_trc && out->color_trc == AVCOL_TRC_LINEAR && + out->color_primaries == in->color_primaries && + s->src_format.color_primaries == s->dst_format.color_primaries) + remove_color_dependent_keep_light(out); + else if (out->color_trc != in->color_trc || out->color_primaries != in->color_primaries) + changed |= AV_SIDE_DATA_PROP_COLOR_DEPENDENT; + av_frame_side_data_remove_by_props(&out->side_data, &out->nb_side_data, changed); +} + static int filter_frame(AVFilterLink *link, AVFrame *in) { AVFilterContext *ctx = link->dst; @@ -762,7 +799,7 @@ static int filter_frame(AVFilterLink *link, AVFrame *in) const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(link->format); const AVPixFmtDescriptor *odesc = av_pix_fmt_desc_get(outlink->format); char buf[32]; - int ret = 0, changed = 0; + int ret = 0; AVFrame *out = NULL; ThreadData td; @@ -841,11 +878,7 @@ static int filter_frame(AVFilterLink *link, AVFrame *in) (int64_t)in->sample_aspect_ratio.den * outlink->w * link->h, INT_MAX); - if (out->width != in->width || out->height != in->height) - changed |= AV_SIDE_DATA_PROP_SIZE_DEPENDENT; - if (out->color_trc != in->color_trc || out->color_primaries != in->color_primaries) - changed |= AV_SIDE_DATA_PROP_COLOR_DEPENDENT; - av_frame_side_data_remove_by_props(&out->side_data, &out->nb_side_data, changed); + prune_side_data(s, out, in); td.in = in; td.out = out; diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 82ab48dc68d..44f470cb9ab 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -648,6 +648,10 @@ fate-filter-scale-zero-dim: CMP = null FATE_FILTER_FFPROBE-$(call ALLYES, FFPROBE LAVFI_INDEV WRAPPED_AVFRAME_DECODER TESTSRC2_FILTER FORMAT_FILTER SETPARAMS_FILTER ZSCALE_FILTER) += fate-filter-zscale-passthrough fate-filter-zscale-passthrough: CMD = probe -print_format compact -show_entries frame=color_primaries,color_transfer -f lavfi "testsrc2=s=64x64:r=2:d=2,format=gbrp,setparams=color_primaries=bt709:color_trc=bt709,zscale=primariesin=bt470bg:primaries=bt470bg:transferin=smpte170m:transfer=smpte170m" +FATE_FILTER-$(CONFIG_ZSCALE_FILTER) += fate-filter-zscale-light-levels +fate-filter-zscale-light-levels: libavfilter/tests/zscale_light_levels$(EXESUF) +fate-filter-zscale-light-levels: CMD = run libavfilter/tests/zscale_light_levels$(EXESUF) + FATE_FILTER_VSYNTH_VIDEO_FILTER-$(CONFIG_VFLIP_FILTER) += fate-filter-vflip fate-filter-vflip: CMD = video_filter "vflip" diff --git a/tests/ref/fate/filter-zscale-light-levels b/tests/ref/fate/filter-zscale-light-levels new file mode 100644 index 00000000000..35959a5570b --- /dev/null +++ b/tests/ref/fate/filter-zscale-light-levels @@ -0,0 +1,5 @@ +t=linear: light level kept, mastering display kept, ICC profile dropped +t=linear:p=bt709: light level dropped, mastering display dropped, ICC profile dropped +t=linear:primariesin=bt709: light level dropped, mastering display dropped, ICC profile dropped +t=bt709: light level dropped, mastering display dropped, ICC profile dropped +w=8:h=4: light level kept, mastering display kept, ICC profile kept -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
