PR #25867 opened by Kilian (lusoris)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25867
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25867.patch

vf_zscale drops the content light level and the mastering display metadata on 
every transfer change. That includes a change to linear light in the same 
primaries, so a tone mapper further down the graph loses the peak it reads from 
that side data; vf_tonemap then falls back to a fixed peak. This change keeps 
those two side data types when the output is linear light in the same 
primaries, and drops the other colour-dependent side data as before.

## What changes

- `libavfilter/vf_zscale.c`: side data pruning moves into `prune_side_data()`. 
A transfer change to `linear` with unchanged primaries removes the 
colour-dependent side data except `AV_FRAME_DATA_CONTENT_LIGHT_LEVEL` and 
`AV_FRAME_DATA_MASTERING_DISPLAY_METADATA`. Every other transfer or primaries 
change, and every size change, prunes as before.
- The primaries count as unchanged only when they match both as tagged on the 
frames and as converted by zimg. With `primariesin` set, the output keeps the 
input tag although zimg converts from the option's primaries, so that case 
still drops the light levels.
- `libavfilter/tests/zscale_light_levels.c`: new test program. It runs zscale 
on a PQ BT.2020 frame carrying a content light level, a mastering display and 
an ICC profile, and prints which of the three survive.
- `tests/fate/filter-video.mak`, `tests/ref/fate/filter-zscale-light-levels`: 
new FATE test `fate-filter-zscale-light-levels`.

The test covers five cases:

| zscale options | light level | mastering display | ICC profile |
| :--- | :--- | :--- | :--- |
| `t=linear` | kept | kept | dropped |
| `t=linear:p=bt709` | dropped | dropped | dropped |
| `t=linear:primariesin=bt709` | dropped | dropped | dropped |
| `t=bt709` | dropped | dropped | dropped |
| `w=8:h=4` | kept | kept | kept |

## Testing

Built on current master with GCC 16.2.1 and zimg 3.0.6 (`--enable-libzimg`).

- `make fate-filter-zscale-light-levels fate-filter-zscale-passthrough 
fate-filter-zscale-chroma-input-override`: all pass.
- With the `vf_zscale.c` change reverted and the new test kept, 
`fate-filter-zscale-light-levels` differs on the first case only (`t=linear: 
light level dropped, mastering display dropped`), so the test detects the old 
behaviour.
- Without the check on the converted primaries, the 
`t=linear:primariesin=bt709` case keeps the light level and the mastering 
display, so the test also covers that check.
- `ffmpeg` and `libavfilter/tests/zscale_light_levels` build with no warnings.



>From 678b45a95831e2f0d3d4f322fc684b860a8af89f Mon Sep 17 00:00:00 2001
From: lusoris <[email protected]>
Date: Sun, 8 Nov 2026 16:42:06 +0100
Subject: [PATCH] avfilter/vf_zscale: keep the light levels when linearising

A transfer change drops all colour-dependent side data. When the output
is linear light in the same primaries, the content light level and the
mastering display still describe the frame, and a tone mapper further
down the graph reads its peak from them; without them vf_tonemap falls
back to a fixed peak. Keep those two in that case and drop the other
colour-dependent side data as before.

The primaries must match both as tagged on the frames and as converted
by zimg: with primariesin set, the output keeps the input tag although
zimg converts from the option's primaries.

The test runs zscale on a PQ BT.2020 frame with light levels and an ICC
profile, and checks which side data survive a change to linear, a
change of primaries, a change of input primaries, a change to BT.709
and a plain resize.

Signed-off-by: lusoris <[email protected]>
---
 libavfilter/Makefile                      |   1 +
 libavfilter/tests/zscale_light_levels.c   | 139 ++++++++++++++++++++++
 libavfilter/vf_zscale.c                   |  45 ++++++-
 tests/fate/filter-video.mak               |   4 +
 tests/ref/fate/filter-zscale-light-levels |   5 +
 5 files changed, 188 insertions(+), 6 deletions(-)
 create mode 100644 libavfilter/tests/zscale_light_levels.c
 create mode 100644 tests/ref/fate/filter-zscale-light-levels

diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index dc5c083f013..bc2493fb5b3 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -692,6 +692,7 @@ TOOLS     = graph2dot
 TESTPROGS = drawutils filtfmts formats integral
 
 TESTPROGS-$(CONFIG_DRAWVG_FILTER) += drawvg
+TESTPROGS-$(CONFIG_ZSCALE_FILTER) += zscale_light_levels
 
 TOOLS-$(CONFIG_LIBZMQ) += zmqsend
 
diff --git a/libavfilter/tests/zscale_light_levels.c 
b/libavfilter/tests/zscale_light_levels.c
new file mode 100644
index 00000000000..c7ceceaefc1
--- /dev/null
+++ b/libavfilter/tests/zscale_light_levels.c
@@ -0,0 +1,139 @@
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/* Which colour-dependent side data zscale keeps on a transfer change. */
+
+#include <stdio.h>
+
+#include "libavutil/frame.h"
+#include "libavutil/log.h"
+#include "libavutil/mastering_display_metadata.h"
+
+#include "libavfilter/avfilter.h"
+#include "libavfilter/buffersink.h"
+#include "libavfilter/buffersrc.h"
+
+#define FW 16
+#define FH 8
+
+/* Runs buffer -> zscale=<args> -> buffersink on one frame. */
+static int run_filter(AVFrame *in, AVFrame *out, const char *zargs)
+{
+    AVFilterGraph *g = avfilter_graph_alloc();
+    AVFilterContext *src = NULL, *zs = NULL, *sink = NULL;
+    char args[128];
+    int ret;
+
+    if (!g)
+        return AVERROR(ENOMEM);
+    snprintf(args, sizeof(args), 
"video_size=%dx%d:pix_fmt=%d:time_base=1/25:colorspace=%d",
+             in->width, in->height, in->format, in->colorspace);
+    ret = avfilter_graph_create_filter(&src, avfilter_get_by_name("buffer"), 
"in", args, NULL, g);
+    if (ret >= 0)
+        ret = avfilter_graph_create_filter(&zs, 
avfilter_get_by_name("zscale"), "zs", zargs, NULL, g);
+    if (ret >= 0)
+        ret = avfilter_graph_create_filter(&sink, 
avfilter_get_by_name("buffersink"), "out", NULL, NULL, g);
+    if (ret >= 0)
+        ret = avfilter_link(src, 0, zs, 0);
+    if (ret >= 0)
+        ret = avfilter_link(zs, 0, sink, 0);
+    if (ret >= 0)
+        ret = avfilter_graph_config(g, NULL);
+    if (ret >= 0)
+        ret = av_buffersrc_add_frame_flags(src, in, 
AV_BUFFERSRC_FLAG_KEEP_REF);
+    if (ret >= 0)
+        ret = av_buffersrc_add_frame(src, NULL);
+    if (ret >= 0)
+        ret = av_buffersink_get_frame(sink, out);
+    avfilter_graph_free(&g);
+    return ret;
+}
+
+static int add_side_data(AVFrame *f)
+{
+    AVContentLightMetadata *cll = 
av_content_light_metadata_create_side_data(f);
+    AVMasteringDisplayMetadata *md = 
av_mastering_display_metadata_create_side_data(f);
+
+    if (!cll || !md || !av_frame_new_side_data(f, AV_FRAME_DATA_ICC_PROFILE, 
16))
+        return AVERROR(ENOMEM);
+    cll->MaxCLL = 1000;
+    md->max_luminance = av_make_q(1000, 1);
+    md->min_luminance = av_make_q(1, 10000);
+    md->has_luminance = 1;
+    return 0;
+}
+
+/* A PQ BT.2020 frame with light levels and an ICC profile. */
+static AVFrame *make_frame(void)
+{
+    AVFrame *f = av_frame_alloc();
+
+    if (!f)
+        return NULL;
+    f->format = AV_PIX_FMT_GBRPF32;
+    f->width  = FW;
+    f->height = FH;
+    f->color_primaries = AVCOL_PRI_BT2020;
+    f->color_trc       = AVCOL_TRC_SMPTE2084;
+    f->colorspace      = AVCOL_SPC_RGB;
+    if (av_frame_get_buffer(f, 0) < 0 || add_side_data(f) < 0) {
+        av_frame_free(&f);
+        return NULL;
+    }
+    for (int p = 0; p < 3; p++)
+        for (int y = 0; y < FH; y++)
+            for (int x = 0; x < FW; x++)
+                ((float *)(f->data[p] + y * f->linesize[p]))[x] = 0.5f;
+    return f;
+}
+
+static const char *has(const AVFrame *f, enum AVFrameSideDataType type)
+{
+    return av_frame_get_side_data(f, type) ? "kept" : "dropped";
+}
+
+/* Prints which side data survive zscale=<zargs>; returns 0 when the run 
worked. */
+static int report(const char *zargs)
+{
+    AVFrame *in = make_frame(), *out = av_frame_alloc();
+    int ret = in && out ? run_filter(in, out, zargs) : AVERROR(ENOMEM);
+
+    if (ret >= 0)
+        printf("%s: light level %s, mastering display %s, ICC profile %s\n", 
zargs,
+               has(out, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL),
+               has(out, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA),
+               has(out, AV_FRAME_DATA_ICC_PROFILE));
+    else
+        printf("%s: failed\n", zargs);
+    av_frame_free(&in);
+    av_frame_free(&out);
+    return ret < 0;
+}
+
+int main(void)
+{
+    int fails = 0;
+
+    av_log_set_level(AV_LOG_QUIET);
+    fails += report("t=linear");
+    fails += report("t=linear:p=bt709");
+    fails += report("t=linear:primariesin=bt709");
+    fails += report("t=bt709");
+    fails += report("w=8:h=4");
+    return fails != 0;
+}
diff --git a/libavfilter/vf_zscale.c b/libavfilter/vf_zscale.c
index 8f600cbf035..02ac0c6c6d7 100644
--- a/libavfilter/vf_zscale.c
+++ b/libavfilter/vf_zscale.c
@@ -754,6 +754,43 @@ static int filter_slice(AVFilterContext *ctx, void *data, 
int job_nr, int n_jobs
     return 0;
 }
 
+/*
+ * Linear light in the same primaries keeps the light levels of its source, so
+ * a tone mapper further down can still read the peak from them.
+ */
+static void remove_color_dependent_keep_light(AVFrame *out)
+{
+    for (int i = out->nb_side_data - 1; i >= 0; i--) {
+        const AVSideDataDescriptor *desc;
+        enum AVFrameSideDataType type;
+
+        if (i >= out->nb_side_data)
+            continue;
+        type = out->side_data[i]->type;
+        desc = av_frame_side_data_desc(type);
+        if (type == AV_FRAME_DATA_CONTENT_LIGHT_LEVEL ||
+            type == AV_FRAME_DATA_MASTERING_DISPLAY_METADATA)
+            continue;
+        if (desc && desc->props & AV_SIDE_DATA_PROP_COLOR_DEPENDENT)
+            av_frame_side_data_remove(&out->side_data, &out->nb_side_data, 
type);
+    }
+}
+
+static void prune_side_data(const ZScaleContext *s, AVFrame *out, const 
AVFrame *in)
+{
+    int changed = 0;
+
+    if (out->width != in->width || out->height != in->height)
+        changed |= AV_SIDE_DATA_PROP_SIZE_DEPENDENT;
+    if (out->color_trc != in->color_trc && out->color_trc == AVCOL_TRC_LINEAR 
&&
+        out->color_primaries == in->color_primaries &&
+        s->src_format.color_primaries == s->dst_format.color_primaries)
+        remove_color_dependent_keep_light(out);
+    else if (out->color_trc != in->color_trc || out->color_primaries != 
in->color_primaries)
+        changed |= AV_SIDE_DATA_PROP_COLOR_DEPENDENT;
+    av_frame_side_data_remove_by_props(&out->side_data, &out->nb_side_data, 
changed);
+}
+
 static int filter_frame(AVFilterLink *link, AVFrame *in)
 {
     AVFilterContext *ctx = link->dst;
@@ -762,7 +799,7 @@ static int filter_frame(AVFilterLink *link, AVFrame *in)
     const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(link->format);
     const AVPixFmtDescriptor *odesc = av_pix_fmt_desc_get(outlink->format);
     char buf[32];
-    int ret = 0, changed = 0;
+    int ret = 0;
     AVFrame *out = NULL;
     ThreadData td;
 
@@ -841,11 +878,7 @@ static int filter_frame(AVFilterLink *link, AVFrame *in)
                   (int64_t)in->sample_aspect_ratio.den * outlink->w * link->h,
                   INT_MAX);
 
-        if (out->width != in->width || out->height != in->height)
-            changed |= AV_SIDE_DATA_PROP_SIZE_DEPENDENT;
-        if (out->color_trc != in->color_trc || out->color_primaries != 
in->color_primaries)
-            changed |= AV_SIDE_DATA_PROP_COLOR_DEPENDENT;
-        av_frame_side_data_remove_by_props(&out->side_data, 
&out->nb_side_data, changed);
+        prune_side_data(s, out, in);
 
         td.in = in;
         td.out = out;
diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak
index 82ab48dc68d..44f470cb9ab 100644
--- a/tests/fate/filter-video.mak
+++ b/tests/fate/filter-video.mak
@@ -648,6 +648,10 @@ fate-filter-scale-zero-dim: CMP = null
 FATE_FILTER_FFPROBE-$(call ALLYES, FFPROBE LAVFI_INDEV WRAPPED_AVFRAME_DECODER 
TESTSRC2_FILTER FORMAT_FILTER SETPARAMS_FILTER ZSCALE_FILTER) += 
fate-filter-zscale-passthrough
 fate-filter-zscale-passthrough: CMD = probe -print_format compact 
-show_entries frame=color_primaries,color_transfer -f lavfi 
"testsrc2=s=64x64:r=2:d=2,format=gbrp,setparams=color_primaries=bt709:color_trc=bt709,zscale=primariesin=bt470bg:primaries=bt470bg:transferin=smpte170m:transfer=smpte170m"
 
+FATE_FILTER-$(CONFIG_ZSCALE_FILTER) += fate-filter-zscale-light-levels
+fate-filter-zscale-light-levels: libavfilter/tests/zscale_light_levels$(EXESUF)
+fate-filter-zscale-light-levels: CMD = run 
libavfilter/tests/zscale_light_levels$(EXESUF)
+
 FATE_FILTER_VSYNTH_VIDEO_FILTER-$(CONFIG_VFLIP_FILTER) += fate-filter-vflip
 fate-filter-vflip: CMD = video_filter "vflip"
 
diff --git a/tests/ref/fate/filter-zscale-light-levels 
b/tests/ref/fate/filter-zscale-light-levels
new file mode 100644
index 00000000000..35959a5570b
--- /dev/null
+++ b/tests/ref/fate/filter-zscale-light-levels
@@ -0,0 +1,5 @@
+t=linear: light level kept, mastering display kept, ICC profile dropped
+t=linear:p=bt709: light level dropped, mastering display dropped, ICC profile 
dropped
+t=linear:primariesin=bt709: light level dropped, mastering display dropped, 
ICC profile dropped
+t=bt709: light level dropped, mastering display dropped, ICC profile dropped
+w=8:h=4: light level kept, mastering display kept, ICC profile kept
-- 
2.54.0

_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]

Reply via email to