PR #25645 opened by JuliusBairaktaris URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25645 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25645.patch
The AMF FRC runtime stamps every output with the newest input's pts, while the picture it shows lags that input: by two input frames with `use_future_frame` (the default), by one without. At 24 fps, video from `frc_amf` is shown 83 ms late in every player that uses it. I reported the runtime side as GPUOpen-LibrariesAndSDKs/AMF#622, which has no reply yet. This works around it with a TODO naming that issue: each output gets the pts and properties of the input it shows, and the warm-up repeats before the first input are dropped. Measured on Windows 11, RX 9070 XT, driver 32.0.32015.7029, by matching each output of a 1080p24 clip to the source frame with identical plane averages: | | master | this PR | |---|---|---| | `use_future_frame=1` | 117 frames 83.3 ms late | 119 frames on time | | `use_future_frame=0` | 118 frames 41.7 ms late | 119 frames on time | Interpolated frames still lie between their neighbours. 10-bit HEVC and D3D11VA input give 4 fewer frames at the start: the dropped repeats. Written with Claude Opus 5.5 in Claude Code; `Assisted-by` trailer on the commit. >From ccfe4ac66eed6c0fff1bb3bbcd5f9cabe5350157 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sat, 31 Oct 2026 17:41:14 +0100 Subject: [PATCH] avfilter/vf_frc_amf: stamp outputs with the time of their picture The AMF FRC component stamps every output surface with the pts of the newest input, but its pictures lag that input: by two input frames with use_future_frame (the default), by one without. At 24 fps the video is shown 83 ms late, and A/V sync is off by that much in every player that uses the filter. The first outputs are repeats of the first input. Keep the inputs' properties for that many more frames, and give each output the pts and properties of the input it shows, shifted from the runtime's pts by the distance between the newest input and that one. Interpolated frames keep their position between the two. Outputs that would land before the first input, the warm-up repeats, are dropped. This works around a runtime defect (GPUOpen-LibrariesAndSDKs/AMF#622), marked with a TODO. Tested on Windows 11, RX 9070 XT, driver 32.0.32015.7029, by matching each output of a 1080p24 clip to the source frame with the same plane averages: on master all 117 pass-through frames are 83.3 ms late with use_future_frame=1 and 41.7 ms late with use_future_frame=0; with this change all 119 are on time in both modes, and the interpolated frames still lie between their neighbours. 10-bit HEVC and D3D11VA input give 4 fewer frames at the start, the dropped warm-up repeats. Assisted-by: Claude Opus 5.5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_amf_common.c | 23 +++++++++++++++++++---- libavfilter/vf_amf_common.h | 1 + libavfilter/vf_frc_amf.c | 5 +++++ 3 files changed, 25 insertions(+), 4 deletions(-) diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c index e1d0b42ee37..03b427361c4 100644 --- a/libavfilter/vf_amf_common.c +++ b/libavfilter/vf_amf_common.c @@ -151,16 +151,31 @@ static int amf_deliver_output(AVFilterContext *avctx) return AVERROR(ENOMEM); if (ctx->outputs_per_input > 1) { - while (av_fifo_can_read(ctx->pending) > 1) { - av_fifo_peek(ctx->pending, &props, 1, 1); + AVFrame *newest; + size_t idx; + + while (av_fifo_can_read(ctx->pending) > ctx->output_delay + 1) { + av_fifo_peek(ctx->pending, &props, 1, ctx->output_delay + 1); if (props->pts > pts) break; av_fifo_read(ctx->pending, &props, 1); av_frame_free(&props); } props = NULL; - if (av_fifo_can_read(ctx->pending)) - av_fifo_peek(ctx->pending, &props, 1, 0); + if (av_fifo_can_read(ctx->pending)) { + for (idx = 0; idx + 1 < av_fifo_can_read(ctx->pending); idx++) { + av_fifo_peek(ctx->pending, &newest, 1, idx + 1); + if (newest->pts > pts) + break; + } + if (idx < ctx->output_delay) { + av_frame_free(&out); + continue; + } + av_fifo_peek(ctx->pending, &newest, 1, idx); + av_fifo_peek(ctx->pending, &props, 1, idx - ctx->output_delay); + pts -= newest->pts - props->pts; + } ret = props ? av_frame_copy_props(out, props) : 0; } else { props = NULL; diff --git a/libavfilter/vf_amf_common.h b/libavfilter/vf_amf_common.h index 9af3d2ddda9..911300a5871 100644 --- a/libavfilter/vf_amf_common.h +++ b/libavfilter/vf_amf_common.h @@ -39,6 +39,7 @@ typedef struct AMFFilterContext { int64_t status_pts; AVFifo *pending; int outputs_per_input; + int output_delay; int scale_type; int in_color_range; int in_primaries; diff --git a/libavfilter/vf_frc_amf.c b/libavfilter/vf_frc_amf.c index c7ef58b4294..1637b4580ef 100644 --- a/libavfilter/vf_frc_amf.c +++ b/libavfilter/vf_frc_amf.c @@ -117,6 +117,11 @@ static int amf_frc_filter_config_output(AVFilterLink *outlink) if (frc_ctx->enable) { ol->frame_rate.num *= 2; amf_ctx->outputs_per_input = 2; + /* TODO: the runtime stamps each output with the newest input's pts + * while showing the picture one input (two with the future frame) + * earlier (GPUOpen-LibrariesAndSDKs/AMF#622). Drop this delay when + * a fixed runtime is required. */ + amf_ctx->output_delay = frc_ctx->use_future_frame ? 2 : 1; } // Possible bug: FRC must be initialized enabled to be toggleable on the fly after init. -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
