PR #24324 opened by JuliusBairaktaris URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24324 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24324.patch
Depends on #23351, whose thirteen patches this is stacked on; only the last commit belongs to this pull request. The AMF filters run off a filter_frame callback, which has no way to tell a component that no more input is coming. Anything a component still holds when the input link ends is lost, and there is nowhere to call AMFComponent::Drain(). The scaler, the enhancer and the converter return one output per input today, so nothing is dropped in practice, but the call sequence is only correct as long as that stays true, and vf_frc_amf already shows an AMF component that emits several frames per input. Convert the three filters sharing amf_filter_filter_frame() to an activate callback: consume one input frame at a time, acknowledge the input status, and on EOF drain the pre-converter and the component before forwarding the status downstream. The output side of filter_frame moves into amf_deliver_output(), which activate reuses for the frames the drain produces, with the timestamp taken from the surface since there is no input frame to copy properties from. This is a no-op for current AMF components by design, so it is proposed separately rather than inside #23351, which fixes user visible bugs. Tested on Windows 11, RX 9070 XT, driver 32.0.31041.1004: sr_amf, vpp_amf and vqe_amf return 60 frames for a 60 frame clip, bit identical to the parent branch by framecrc, over AMF and D3D11VA surfaces; a truncated read with -frames:v still terminates; and the filter test matrix of #23351 passes unchanged. vf_frc_amf.c keeps its own filter_frame. It emits several frames per input and warrants the same treatment, with its own testing. >From bb329a7b045029f155c6d576ae16b29c9f01e015 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 28 Jun 2026 17:58:44 +0200 Subject: [PATCH 01/14] avfilter/vf_sr_amf: fix solid green frame with algorithm=sr1-1 Per AMD's AMF_HQ_Scaler_API.md, the HQ Scaler's VideoSR1.1 algorithm is only supported on a DX11/DX12 engine and with input/output formats other than NV12 or P010 (it accepts packed RGB: BGRA, RGBA, R10G10B10A2 or RGBA_F16). The filter however advertised NV12/P010 as valid input and passed the input format straight to AMFHQScaler::Init(), which returns AMF_OK even for the unsupported format. The existing result check therefore never fired and the filter silently emitted a solid green frame. When algorithm=sr1-1, restrict the negotiated formats to the packed 8-bit RGB formats RGBA and BGRA, which round-trip cleanly through the AMF hwcontext (both upload and hwdownload), so a converter is auto-inserted for YUV input, and reject an unsupported hardware surface explicitly in config_output instead of producing green. The other SR1.1-capable formats (R10G10B10A2, RGBA_F16) are not exposed: they are absent from the AMF hwcontext's supported_formats[]/supported_transfer_formats[], so an AMF frames context cannot use them and hwdownload would fail with ENOSYS; exposing them belongs in a separate hwcontext_amf change. Note that AMF's VideoSR algorithms (sr1-0 and sr1-1) leave the alpha channel zeroed and expose no property to control it; drop alpha downstream, e.g. with format=rgb24, for an opaque result. Refs AMD AMF_HQ_Scaler_API.md and mpv-player/mpv#18068. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- doc/filters.texi | 7 +++++++ libavfilter/vf_sr_amf.c | 31 ++++++++++++++++++++++++++++--- 2 files changed, 35 insertions(+), 3 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index de1aca5257..56730cc909 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -23268,6 +23268,13 @@ Point @item sr1-1 Video SR1.1 +This algorithm only supports packed RGB formats and requires a DirectX 11 or +DirectX 12 device, so it is available on Windows only. Inputs in other formats +such as @code{nv12} or @code{p010} are converted to a packed RGB format +(@code{rgba} or @code{bgra}) automatically. +Note that the VideoSR algorithms (@code{sr1-0} and @code{sr1-1}) do not +preserve the alpha channel (it is left zeroed); append e.g. @code{format=rgb24} +downstream if an opaque result is required. @end table diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c index 17bfdc2a03..9ab5b8c2f2 100644 --- a/libavfilter/vf_sr_amf.c +++ b/libavfilter/vf_sr_amf.c @@ -54,8 +54,9 @@ static int amf_filter_query_formats(AVFilterContext *avctx) { - const enum AVPixelFormat *output_pix_fmts; - static const enum AVPixelFormat input_pix_fmts[] = { + AMFFilterContext *ctx = avctx->priv; + const enum AVPixelFormat *input_pix_fmts, *output_pix_fmts; + static const enum AVPixelFormat input_pix_fmts_default[] = { AV_PIX_FMT_NV12, AV_PIX_FMT_P010, AV_PIX_FMT_BGRA, @@ -75,7 +76,21 @@ static int amf_filter_query_formats(AVFilterContext *avctx) AV_PIX_FMT_RGBAF16, AV_PIX_FMT_NONE, }; - output_pix_fmts = output_pix_fmts_default; + // VideoSR1.1 needs packed RGB on DX11/DX12 + static const enum AVPixelFormat pix_fmts_sr1_1[] = { + AV_PIX_FMT_RGBA, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_NONE, + }; + + if (ctx->algorithm == AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1) { + input_pix_fmts = pix_fmts_sr1_1; + output_pix_fmts = pix_fmts_sr1_1; + } else { + input_pix_fmts = input_pix_fmts_default; + output_pix_fmts = output_pix_fmts_default; + } return amf_setup_input_output_formats(avctx, input_pix_fmts, output_pix_fmts); } @@ -95,6 +110,16 @@ static int amf_filter_config_output(AVFilterLink *outlink) if (err < 0) return err; + if (ctx->algorithm == AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 && + (in_format == AV_PIX_FMT_NV12 || in_format == AV_PIX_FMT_P010)) { + av_log(avctx, AV_LOG_ERROR, + "sr1-1 (VideoSR1.1) requires a packed RGB format (rgba); " + "%s is not supported. Convert the input first (e.g. format=rgba) or " + "select another algorithm.\n", + av_get_pix_fmt_name(in_format)); + return AVERROR(EINVAL); + } + // HQ scaler should be used for upscaling only if (inlink->w > outlink->w || inlink->h > outlink->h) { av_log(avctx, AV_LOG_ERROR, "AMF HQ scaler should be used for upscaling only.\n"); -- 2.52.0 >From a08b2cd23ede4b44ce5fb9f3ed056145f5418354 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Fri, 31 Jul 2026 12:57:58 +0200 Subject: [PATCH 02/14] avfilter/vf_sr_amf: fix solid green frame with algorithm=point AMF returns an unwritten surface for AMF_HQ_SCALER_ALGORITHM_POINT when the surface format is NV12 or P010, exactly as it does for VideoSR1.1 on YUV: Init(), SubmitInput() and QueryOutput() all report AMF_OK and the output is a solid green frame. Unlike VideoSR1.1 this is not a documented limitation - point is listed as a plain scaling algorithm with no format restriction - and it has been open on AMD's tracker since 2023. Reuse the sr1-1 handling: negotiate the packed 8-bit RGB formats for point as well, so a converter is auto-inserted for YUV input, and reject an NV12/P010-backed hardware surface in config_output instead of emitting green. Reproduced with: ffmpeg -init_hw_device amf -f lavfi -i testsrc2=size=1280x720 \ -vf "format=nv12,hwupload,sr_amf=w=2560:h=1440:algorithm=point,\ hwdownload,format=nv12,format=rgb24" -frames:v 1 out.png Before the fix the output is a single flat green frame; after it matches bilinear/bicubic/sr1-0. Tested on Windows 11, RX 9070 XT, driver 32.0.31035.1003. Refs https://github.com/GPUOpen-LibrariesAndSDKs/AMF/issues/427 Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- doc/filters.texi | 3 +++ libavfilter/vf_sr_amf.c | 26 +++++++++++++++++--------- 2 files changed, 20 insertions(+), 9 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 56730cc909..23218e7571 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -23265,6 +23265,9 @@ This is a default value @item point Point +AMF returns an unwritten surface for this algorithm when the format is +@code{nv12} or @code{p010}, so inputs in those formats are converted to a packed +RGB format (@code{rgba} or @code{bgra}) automatically. @item sr1-1 Video SR1.1 diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c index 9ab5b8c2f2..e4d56c24a1 100644 --- a/libavfilter/vf_sr_amf.c +++ b/libavfilter/vf_sr_amf.c @@ -52,6 +52,12 @@ #endif +static int amf_hq_scaler_needs_packed_rgb(int algorithm) +{ + return algorithm == AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 || + algorithm == AMF_HQ_SCALER_ALGORITHM_POINT; +} + static int amf_filter_query_formats(AVFilterContext *avctx) { AMFFilterContext *ctx = avctx->priv; @@ -76,17 +82,17 @@ static int amf_filter_query_formats(AVFilterContext *avctx) AV_PIX_FMT_RGBAF16, AV_PIX_FMT_NONE, }; - // VideoSR1.1 needs packed RGB on DX11/DX12 - static const enum AVPixelFormat pix_fmts_sr1_1[] = { + // sr1-1 and point produce a blank surface on YUV input + static const enum AVPixelFormat pix_fmts_packed_rgb[] = { AV_PIX_FMT_RGBA, AV_PIX_FMT_BGRA, AV_PIX_FMT_AMF_SURFACE, AV_PIX_FMT_NONE, }; - if (ctx->algorithm == AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1) { - input_pix_fmts = pix_fmts_sr1_1; - output_pix_fmts = pix_fmts_sr1_1; + if (amf_hq_scaler_needs_packed_rgb(ctx->algorithm)) { + input_pix_fmts = pix_fmts_packed_rgb; + output_pix_fmts = pix_fmts_packed_rgb; } else { input_pix_fmts = input_pix_fmts_default; output_pix_fmts = output_pix_fmts_default; @@ -110,12 +116,14 @@ static int amf_filter_config_output(AVFilterLink *outlink) if (err < 0) return err; - if (ctx->algorithm == AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 && + if (amf_hq_scaler_needs_packed_rgb(ctx->algorithm) && (in_format == AV_PIX_FMT_NV12 || in_format == AV_PIX_FMT_P010)) { av_log(avctx, AV_LOG_ERROR, - "sr1-1 (VideoSR1.1) requires a packed RGB format (rgba); " - "%s is not supported. Convert the input first (e.g. format=rgba) or " - "select another algorithm.\n", + "%s requires a packed RGB format (rgba); %s is not supported. " + "Convert the input first (e.g. format=rgba) or select another " + "algorithm.\n", + ctx->algorithm == AMF_HQ_SCALER_ALGORITHM_POINT ? "point" + : "sr1-1 (VideoSR1.1)", av_get_pix_fmt_name(in_format)); return AVERROR(EINVAL); } -- 2.52.0 >From 3344e173751a1de495ae9b320a23aee084768561 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Fri, 31 Jul 2026 13:01:57 +0200 Subject: [PATCH 03/14] avutil/hwcontext_amf: allow X2BGR10 and RGBAF16 frames contexts Both formats are already in format_map[] and map to AMF_SURFACE_R10G10B10A2 and AMF_SURFACE_RGBA_F16, but they were missing from supported_formats[] and supported_transfer_formats[]. An AMF frames context could therefore not use them as sw_format, and hwupload/hwdownload rejected them, which in turn kept the AMF filters from offering the two formats AMF's HQ Scaler needs for HDR. Both are single-plane packed formats, so the existing transfer paths handle them unchanged: amf_transfer_data_to/from derive plane count from the surface and copy with av_image_copy2(). Verified on Windows 11, RX 9070 XT, driver 32.0.31035.1003: ffmpeg -init_hw_device amf -f lavfi -i testsrc2=size=640x360 \ -vf "format=x2bgr10le,hwupload,hwdownload,format=x2bgr10le" -f null - passes where it previously failed with "Invalid output format x2bgr10le for hwframe download", and a rawvideo rgbaf16le round-trip through hwupload,hwdownload comes back bit-exact. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavutil/hwcontext_amf.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libavutil/hwcontext_amf.c b/libavutil/hwcontext_amf.c index 505424af74..d67193a3c5 100644 --- a/libavutil/hwcontext_amf.c +++ b/libavutil/hwcontext_amf.c @@ -317,6 +317,8 @@ static const enum AVPixelFormat supported_formats[] = { AV_PIX_FMT_RGBA, AV_PIX_FMT_BGR0, AV_PIX_FMT_P010, + AV_PIX_FMT_X2BGR10, + AV_PIX_FMT_RGBAF16, #if CONFIG_D3D11VA AV_PIX_FMT_D3D11, #endif @@ -334,6 +336,8 @@ static const enum AVPixelFormat supported_transfer_formats[] = { AV_PIX_FMT_BGRA, AV_PIX_FMT_RGBA, AV_PIX_FMT_P010, + AV_PIX_FMT_X2BGR10, + AV_PIX_FMT_RGBAF16, AV_PIX_FMT_NONE, }; -- 2.52.0 >From 08509700a45f18f53cb71b0167815b3a10e6550d Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Fri, 31 Jul 2026 13:01:57 +0200 Subject: [PATCH 04/14] avfilter/vf_sr_amf: offer 10-bit and f16 packed RGB AMF's HQ Scaler accepts R10G10B10A2 and RGBA_F16 for the algorithms that require packed RGB, but the filter only offered RGBA and BGRA because an AMF frames context could not carry the other two. Now that hwcontext_amf supports them, add them to the negotiated list so a 10-bit or half-float source is no longer forced down to 8-bit RGB. Tested on Windows 11, RX 9070 XT, driver 32.0.31035.1003: sr1-1, point, sr1-0 and bilinear all produce correct output for x2bgr10le and rgbaf16le input at 1280x720 -> 2560x1440. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- doc/filters.texi | 5 +++-- libavfilter/vf_sr_amf.c | 2 ++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 23218e7571..4f21dcf7db 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -23267,14 +23267,15 @@ This is a default value Point AMF returns an unwritten surface for this algorithm when the format is @code{nv12} or @code{p010}, so inputs in those formats are converted to a packed -RGB format (@code{rgba} or @code{bgra}) automatically. +RGB format (@code{rgba}, @code{bgra}, @code{x2bgr10le} or @code{rgbaf16le}) +automatically. @item sr1-1 Video SR1.1 This algorithm only supports packed RGB formats and requires a DirectX 11 or DirectX 12 device, so it is available on Windows only. Inputs in other formats such as @code{nv12} or @code{p010} are converted to a packed RGB format -(@code{rgba} or @code{bgra}) automatically. +(@code{rgba}, @code{bgra}, @code{x2bgr10le} or @code{rgbaf16le}) automatically. Note that the VideoSR algorithms (@code{sr1-0} and @code{sr1-1}) do not preserve the alpha channel (it is left zeroed); append e.g. @code{format=rgb24} downstream if an opaque result is required. diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c index e4d56c24a1..1758c94ea9 100644 --- a/libavfilter/vf_sr_amf.c +++ b/libavfilter/vf_sr_amf.c @@ -86,6 +86,8 @@ static int amf_filter_query_formats(AVFilterContext *avctx) static const enum AVPixelFormat pix_fmts_packed_rgb[] = { AV_PIX_FMT_RGBA, AV_PIX_FMT_BGRA, + AV_PIX_FMT_X2BGR10, + AV_PIX_FMT_RGBAF16, AV_PIX_FMT_AMF_SURFACE, AV_PIX_FMT_NONE, }; -- 2.52.0 >From 8315c82c61da69cd2584dcb0239c768f97199d8a Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Fri, 31 Jul 2026 13:25:20 +0200 Subject: [PATCH 05/14] avfilter/vf_sr_amf: convert YUV input on the GPU for sr1-1 and point sr1-1 and point need a packed RGB surface. Software input was handled by restricting the negotiated formats so libavfilter inserts a converter, but a hardware surface backed by NV12 or P010 could only be rejected, which made both algorithms unusable with any AMF or D3D11VA producer upstream - including every player that hands libavfilter its decoder frames. Chain an AMFVideoConverter ahead of the scaler in that case instead. The conversion stays on the GPU, so no download is introduced, and the component is driven from the shared filter_frame path through a new pre_converter field. The HQ scaler has no output format property and emits the format it was initialised with, so one packed RGB target drives the converter output format, AMFHQScaler::Init() and the output frames context alike. P010 converts to X2BGR10 and NV12 to RGBA by default; an explicit format= is kept when it names a packed RGB format and rejected otherwise. Where no conversion is needed the scaler cannot change the format at all, so an explicit format= differing from the input is rejected and the target is pinned to the input format, rather than letting the output link negotiate a frames context that disagrees with the surface the scaler produces. The converter is asked for full range output, and the converted frame is tagged AVCOL_SPC_RGB with AVCOL_RANGE_JPEG rather than inheriting the input's YUV matrix. The output frames context follows the converted format, so the filter emits packed RGB where it previously emitted NV12; with format=same this is only reachable for the two algorithms that cannot produce YUV anyway. Tested on Windows 11, RX 9070 XT, driver 32.0.31041.1004, with -hwaccel amf -hwaccel_output_format amf: point and sr1-1 measure YMIN=YAVG=YMAX=0 before this change and match the plain decode to within 0.6/255 after, while bilinear, bicubic and sr1-0 keep their NV12 output. point at 2x scores 36.5 dB PSNR against a software neighbour reference, the converted frame is tagged pc/gbr, P010 reaches X2BGR10, the accepted and rejected format= values behave as described, and 300 frames run through the converter for both algorithms. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- doc/filters.texi | 13 +++++-- libavfilter/vf_amf_common.c | 24 ++++++++++++ libavfilter/vf_amf_common.h | 1 + libavfilter/vf_sr_amf.c | 73 +++++++++++++++++++++++++++++++------ 4 files changed, 96 insertions(+), 15 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 4f21dcf7db..9e79399881 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -23267,15 +23267,16 @@ This is a default value Point AMF returns an unwritten surface for this algorithm when the format is @code{nv12} or @code{p010}, so inputs in those formats are converted to a packed -RGB format (@code{rgba}, @code{bgra}, @code{x2bgr10le} or @code{rgbaf16le}) -automatically. +RGB format automatically. This is a driver bug rather than a documented +restriction; the conversion can be dropped once a driver that scales +@code{nv12} and @code{p010} correctly with @code{point} is in wide use. @item sr1-1 Video SR1.1 This algorithm only supports packed RGB formats and requires a DirectX 11 or DirectX 12 device, so it is available on Windows only. Inputs in other formats such as @code{nv12} or @code{p010} are converted to a packed RGB format -(@code{rgba}, @code{bgra}, @code{x2bgr10le} or @code{rgbaf16le}) automatically. +automatically. Note that the VideoSR algorithms (@code{sr1-0} and @code{sr1-1}) do not preserve the alpha channel (it is left zeroed); append e.g. @code{format=rgb24} downstream if an opaque result is required. @@ -23289,6 +23290,12 @@ Control hq scaler sharpening. The value is a float in the range of [0.0, 2.0] Controls the output pixel format. By default, or if none is specified, the input pixel format is used. +The @code{point} and @code{sr1-1} algorithms emit the packed RGB format the +scaler receives, so for them this option must name one of @code{rgba}, +@code{bgra}, @code{x2bgr10le} or @code{rgbaf16le}. When the input is converted +automatically, @code{x2bgr10le} is selected for @code{p010} input and +@code{rgba} otherwise, and the result is tagged as full range RGB. + @item keep-ratio Force the scaler to keep the aspect ratio of the input image when the output size has a different aspect ratio. Default value is false. diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c index 40c6fbceea..d99d4d98ca 100644 --- a/libavfilter/vf_amf_common.c +++ b/libavfilter/vf_amf_common.c @@ -66,6 +66,12 @@ void amf_filter_uninit(AVFilterContext *avctx) ctx->component = NULL; } + if (ctx->pre_converter) { + ctx->pre_converter->pVtbl->Terminate(ctx->pre_converter); + ctx->pre_converter->pVtbl->Release(ctx->pre_converter); + ctx->pre_converter = NULL; + } + if (ctx->master_display) av_freep(&ctx->master_display); @@ -100,6 +106,22 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) if (ret < 0) goto fail; + if (ctx->pre_converter) { + AMFGuid guid = IID_AMFSurface(); + AMFData *data_conv = NULL; + AMFSurface *surface_conv = NULL; + + res = ctx->pre_converter->pVtbl->SubmitInput(ctx->pre_converter, (AMFData*)surface_in); + surface_in->pVtbl->Release(surface_in); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter SubmitInput() failed with error %d\n", res); + res = ctx->pre_converter->pVtbl->QueryOutput(ctx->pre_converter, &data_conv); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK && data_conv, AVERROR_UNKNOWN, "Converter QueryOutput() failed with error %d\n", res); + res = data_conv->pVtbl->QueryInterface(data_conv, &guid, (void**)&surface_conv); + data_conv->pVtbl->Release(data_conv); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter QueryInterface(IID_AMFSurface) failed with error %d\n", res); + surface_in = surface_conv; + } + res = ctx->component->pVtbl->SubmitInput(ctx->component, (AMFData*)surface_in); surface_in->pVtbl->Release(surface_in); // release surface after use AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); @@ -158,6 +180,8 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) if (ctx->out_trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) out->color_trc = ctx->out_trc; + if (ctx->pre_converter) + out->colorspace = AVCOL_SPC_RGB; if (ret < 0) goto fail; diff --git a/libavfilter/vf_amf_common.h b/libavfilter/vf_amf_common.h index 0290a52c45..7879403d06 100644 --- a/libavfilter/vf_amf_common.h +++ b/libavfilter/vf_amf_common.h @@ -58,6 +58,7 @@ typedef struct AMFFilterContext { int reset_sar; AMFComponent *component; + AMFComponent *pre_converter; AVBufferRef *amf_device_ref; AVBufferRef *hwframes_in_ref; diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c index 1758c94ea9..deb020809d 100644 --- a/libavfilter/vf_sr_amf.c +++ b/libavfilter/vf_sr_amf.c @@ -35,6 +35,7 @@ #include "libavutil/hwcontext_amf_internal.h" #include "AMF/components/HQScaler.h" +#include "AMF/components/VideoConverter.h" #include "AMF/components/ColorSpace.h" #include "vf_amf_common.h" @@ -52,12 +53,27 @@ #endif +static enum AVPixelFormat amf_inlink_sw_format(AVFilterLink *inlink) +{ + FilterLink *inl = ff_filter_link(inlink); + + if (inl->hw_frames_ctx) + return ((AVHWFramesContext*)inl->hw_frames_ctx->data)->sw_format; + return inlink->format; +} + static int amf_hq_scaler_needs_packed_rgb(int algorithm) { return algorithm == AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 || algorithm == AMF_HQ_SCALER_ALGORITHM_POINT; } +static int amf_is_packed_rgb(enum AVPixelFormat format) +{ + return format == AV_PIX_FMT_RGBA || format == AV_PIX_FMT_BGRA || + format == AV_PIX_FMT_X2BGR10 || format == AV_PIX_FMT_RGBAF16; +} + static int amf_filter_query_formats(AVFilterContext *avctx) { AMFFilterContext *ctx = avctx->priv; @@ -82,7 +98,7 @@ static int amf_filter_query_formats(AVFilterContext *avctx) AV_PIX_FMT_RGBAF16, AV_PIX_FMT_NONE, }; - // sr1-1 and point produce a blank surface on YUV input + // sr1-1 and point need packed RGB; YUV is converted on the GPU static const enum AVPixelFormat pix_fmts_packed_rgb[] = { AV_PIX_FMT_RGBA, AV_PIX_FMT_BGRA, @@ -113,21 +129,54 @@ static int amf_filter_config_output(AVFilterLink *outlink) AMF_RESULT res; enum AVPixelFormat in_format; enum AMF_MEMORY_TYPE mem_type = AMF_MEMORY_UNKNOWN; + enum AVPixelFormat in_sw_format; + int needs_conversion; + + in_sw_format = amf_inlink_sw_format(inlink); + needs_conversion = amf_hq_scaler_needs_packed_rgb(ctx->algorithm) && + (in_sw_format == AV_PIX_FMT_NV12 || in_sw_format == AV_PIX_FMT_P010); + + if (amf_hq_scaler_needs_packed_rgb(ctx->algorithm)) { + if (!needs_conversion) { + if (ctx->format != AV_PIX_FMT_NONE && ctx->format != in_sw_format) { + av_log(avctx, AV_LOG_ERROR, "The HQ scaler does not convert formats, format must be same or %s.\n", + av_get_pix_fmt_name(in_sw_format)); + return AVERROR(EINVAL); + } + ctx->format = in_sw_format; + } else if (ctx->format == AV_PIX_FMT_NONE) { + ctx->format = in_sw_format == AV_PIX_FMT_P010 ? AV_PIX_FMT_X2BGR10 : AV_PIX_FMT_RGBA; + } else if (!amf_is_packed_rgb(ctx->format)) { + av_log(avctx, AV_LOG_ERROR, "This algorithm only outputs packed RGB, format=%s is not supported.\n", + av_get_pix_fmt_name(ctx->format)); + return AVERROR(EINVAL); + } + } err = amf_init_filter_config(outlink, &in_format); if (err < 0) return err; - if (amf_hq_scaler_needs_packed_rgb(ctx->algorithm) && - (in_format == AV_PIX_FMT_NV12 || in_format == AV_PIX_FMT_P010)) { - av_log(avctx, AV_LOG_ERROR, - "%s requires a packed RGB format (rgba); %s is not supported. " - "Convert the input first (e.g. format=rgba) or select another " - "algorithm.\n", - ctx->algorithm == AMF_HQ_SCALER_ALGORITHM_POINT ? "point" - : "sr1-1 (VideoSR1.1)", - av_get_pix_fmt_name(in_format)); - return AVERROR(EINVAL); + if (needs_conversion) { + AMFSize in_size = { inlink->w, inlink->h }; + + res = ctx->amf_device_ctx->factory->pVtbl->CreateComponent(ctx->amf_device_ctx->factory, ctx->amf_device_ctx->context, AMFVideoConverter, &ctx->pre_converter); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->pre_converter, AMF_VIDEO_CONVERTER_OUTPUT_FORMAT, (amf_int32)av_av_to_amf_format(ctx->format)); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + AMF_ASSIGN_PROPERTY_SIZE(res, ctx->pre_converter, AMF_VIDEO_CONVERTER_OUTPUT_SIZE, in_size); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->pre_converter, AMF_VIDEO_CONVERTER_OUTPUT_COLOR_RANGE, AMF_COLOR_RANGE_FULL); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + res = ctx->pre_converter->pVtbl->Init(ctx->pre_converter, av_av_to_amf_format(in_format), inlink->w, inlink->h); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, + "AMFConverter-Init() failed with error %d, %s to %s is not supported by this device%s\n", + res, av_get_pix_fmt_name(in_format), av_get_pix_fmt_name(ctx->format), + ctx->format == AV_PIX_FMT_X2BGR10 ? ", try format=rgba" : ""); + + in_format = ctx->format; } // HQ scaler should be used for upscaling only @@ -161,7 +210,7 @@ static int amf_filter_config_output(AVFilterLink *outlink) ctx->in_primaries = AMF_COLOR_PRIMARIES_UNDEFINED; ctx->in_trc = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED; ctx->color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; - ctx->out_color_range = AMF_COLOR_RANGE_UNDEFINED; + ctx->out_color_range = needs_conversion ? AMF_COLOR_RANGE_FULL : AMF_COLOR_RANGE_UNDEFINED; ctx->out_primaries = AMF_COLOR_PRIMARIES_UNDEFINED; ctx->out_trc = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED; -- 2.52.0 >From ce64c2829cf45e8b8501706c1a2f4abe8b6fb201 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 30 Aug 2026 15:15:28 +0200 Subject: [PATCH 06/14] avfilter/amf: fix leaks and a NULL dereference in the filter frame path amf_filter_filter_frame() overwrote out->hw_frames_ctx with a second reference to the output frames context. av_hwframe_get_buffer(), called from amf_amfsurface_to_avframe() for every output, has already attached one, so each frame leaked a reference and the frames context never reached zero. Its AMF surface pool therefore survived filter teardown, which matters to anything that rebuilds a filter graph. vf_frc_amf.c had the same line. The function also takes ownership of the input frame but returned without freeing it on two paths, the QueryInterface() failure after QueryOutput() and the case with no output, and it passed the result of amf_amfsurface_to_avframe() to av_frame_copy_props() unchecked, so an allocation failure or an unsupported surface memory type dereferenced NULL. Route those through the existing fail label. amf_amfsurface_to_avframe() only takes ownership of the surface once it is attached to the frame, so its fail path leaked the surface for both callers; release it there, and check the av_buffer_create() that performs the attach. amf_setup_input_output_formats() leaked the input list when allocating the output list failed. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_amf_common.c | 16 ++++++++-------- libavfilter/vf_frc_amf.c | 8 +------- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c index d99d4d98ca..fa5f35dae4 100644 --- a/libavfilter/vf_amf_common.c +++ b/libavfilter/vf_amf_common.c @@ -132,12 +132,14 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) AMFGuid guid = IID_AMFSurface(); res = data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface_out); // query for buffer interface data_out->pVtbl->Release(data_out); - AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryInterface(IID_AMFSurface) failed with error %d\n", res); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryInterface(IID_AMFSurface) failed with error %d\n", res); } else { - return AVERROR(EAGAIN); + ret = AVERROR(EAGAIN); + goto fail; } out = amf_amfsurface_to_avframe(avctx, surface_out); + AMF_GOTO_FAIL_IF_FALSE(avctx, out != NULL, AVERROR(ENOMEM), "Failed to convert AMFSurface to AVFrame\n"); ret = av_frame_copy_props(out, in); av_frame_unref(in); @@ -186,12 +188,6 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) if (ret < 0) goto fail; - out->hw_frames_ctx = av_buffer_ref(ctx->hwframes_out_ref); - if (!out->hw_frames_ctx) { - ret = AVERROR(ENOMEM); - goto fail; - } - av_frame_free(&in); return ff_filter_frame(outlink, out); fail: @@ -255,6 +251,7 @@ int amf_setup_input_output_formats(AVFilterContext *avctx, } output_formats = ff_make_pixel_format_list(output_pix_fmts); if (!output_formats) { + ff_formats_unref(&input_formats); return AVERROR(ENOMEM); } @@ -432,6 +429,8 @@ AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface) amf_free_amfsurface, (void*)avctx, AV_BUFFER_FLAG_READONLY); + if (!frame->buf[1]) + goto fail; } else { // FIXME: add processing of other hw formats av_log(ctx, AV_LOG_ERROR, "Unknown pixel format\n"); goto fail; @@ -480,6 +479,7 @@ AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface) return frame; fail: + pSurface->pVtbl->Release(pSurface); av_frame_free(&frame); return NULL; } diff --git a/libavfilter/vf_frc_amf.c b/libavfilter/vf_frc_amf.c index f5ea93619b..1b914908a0 100644 --- a/libavfilter/vf_frc_amf.c +++ b/libavfilter/vf_frc_amf.c @@ -227,7 +227,7 @@ static int amf_frc_filter_avframe(AVFilterLink *inlink, AVFrame *in) res = AMF_IFACE_CALL(data_out, QueryInterface, &guid, (void**)&surface_out); AMF_IFACE_CALL(data_out, Release); data_out = NULL; - AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryInterface(IID_AMFSurface) failed with error %d\n", res); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryInterface(IID_AMFSurface) failed with error %d\n", res); out = amf_amfsurface_to_avframe(avctx, surface_out); AMF_GOTO_FAIL_IF_FALSE(avctx, out != NULL, AVERROR(ENOMEM), "Failed to convert AMFSurface to AVFrame\n"); @@ -240,12 +240,6 @@ static int amf_frc_filter_avframe(AVFilterLink *inlink, AVFrame *in) if (frc_ctx->enable) out->duration /= 2; - out->hw_frames_ctx = av_buffer_ref(amf_ctx->hwframes_out_ref); - if (!out->hw_frames_ctx) { - ret = AVERROR(ENOMEM); - goto fail; - } - ret = ff_filter_frame(outlink, out); out = NULL; if (ret < 0) -- 2.52.0 >From 2beab6c77ebb57a5340297746be54411c6ff4650 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 30 Aug 2026 15:30:45 +0200 Subject: [PATCH 07/14] avfilter/amf: make the input format lists describe what AMF accepts amf_setup_input_output_formats() builds both negotiated lists from output_pix_fmts and ignores its input_pix_fmts argument, so the input lists have never been used and have drifted from what the filters handle. None of them lists AV_PIX_FMT_D3D11 or AV_PIX_FMT_DXVA2_VLD even though amf_avframe_to_amfsurface() wraps both. sr_amf and vpp_amf carry the two in their output lists, so they accept such input today only because the input list is dead; vqe_amf and frc_amf carry them nowhere and gain the input here. vpp_amf meanwhile lists three formats AMF will not take: YUV420P10 and 0RGB have no format_map[] entry at all, and the video converter rejects GRAY8 with AMF_NOT_SUPPORTED. Those three only appear to work today because the dead input list leaves libavfilter converting them in software first. Correct the lists in both directions so they describe reality before the helper is made to use them. No functional change on its own. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_frc_amf.c | 2 ++ libavfilter/vf_sr_amf.c | 4 ++++ libavfilter/vf_vpp_amf.c | 5 ++--- libavfilter/vf_vqe_amf.c | 2 ++ 4 files changed, 10 insertions(+), 3 deletions(-) diff --git a/libavfilter/vf_frc_amf.c b/libavfilter/vf_frc_amf.c index 1b914908a0..5b93f65ad1 100644 --- a/libavfilter/vf_frc_amf.c +++ b/libavfilter/vf_frc_amf.c @@ -70,6 +70,8 @@ static int amf_filter_query_formats(AVFilterContext *avctx) const enum AVPixelFormat *output_pix_fmts; static const enum AVPixelFormat input_pix_fmts[] = { AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, AV_PIX_FMT_NV12, AV_PIX_FMT_P010, AV_PIX_FMT_BGRA, diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c index deb020809d..d9ed1dcd8a 100644 --- a/libavfilter/vf_sr_amf.c +++ b/libavfilter/vf_sr_amf.c @@ -84,6 +84,8 @@ static int amf_filter_query_formats(AVFilterContext *avctx) AV_PIX_FMT_BGRA, AV_PIX_FMT_RGBA, AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, AV_PIX_FMT_RGBAF16, AV_PIX_FMT_NONE, }; @@ -105,6 +107,8 @@ static int amf_filter_query_formats(AVFilterContext *avctx) AV_PIX_FMT_X2BGR10, AV_PIX_FMT_RGBAF16, AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, AV_PIX_FMT_NONE, }; diff --git a/libavfilter/vf_vpp_amf.c b/libavfilter/vf_vpp_amf.c index 839a075ea7..43277224ad 100644 --- a/libavfilter/vf_vpp_amf.c +++ b/libavfilter/vf_vpp_amf.c @@ -49,16 +49,15 @@ static int amf_filter_query_formats(AVFilterContext *avctx) const enum AVPixelFormat *output_pix_fmts; static const enum AVPixelFormat input_pix_fmts[] = { AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, AV_PIX_FMT_NV12, AV_PIX_FMT_P010, - AV_PIX_FMT_0RGB, AV_PIX_FMT_BGR0, AV_PIX_FMT_BGRA, AV_PIX_FMT_RGB0, AV_PIX_FMT_RGBA, - AV_PIX_FMT_GRAY8, AV_PIX_FMT_YUV420P, - AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUYV422, AV_PIX_FMT_NONE, }; diff --git a/libavfilter/vf_vqe_amf.c b/libavfilter/vf_vqe_amf.c index 70b533e475..d53703ee58 100644 --- a/libavfilter/vf_vqe_amf.c +++ b/libavfilter/vf_vqe_amf.c @@ -63,6 +63,8 @@ static int amf_filter_query_formats(AVFilterContext *avctx) const enum AVPixelFormat *output_pix_fmts; static const enum AVPixelFormat input_pix_fmts[] = { AV_PIX_FMT_AMF_SURFACE, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, AV_PIX_FMT_NV12, AV_PIX_FMT_P010, AV_PIX_FMT_BGRA, -- 2.52.0 >From 382a449a3e0c7379f7ac79ca2aa46f0f406e36c6 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 30 Aug 2026 15:32:47 +0200 Subject: [PATCH 08/14] avfilter/amf: negotiate the input link from the input format list amf_setup_input_output_formats() built both negotiated lists from output_pix_fmts, so every filter's input_pix_fmts argument was dead and the input link only ever offered what the filter could output. For vpp_amf, whose input list is deliberately wider than its output list, libavfilter therefore inserted a software conversion in front of the filter for any input format missing from the output list. P010 is the damaging case: 10-bit software input was silently reduced to 8-bit YUV420P before being uploaded, and the filter was then asked to convert that back to P010. Build input_formats from input_pix_fmts, and apply the D3D11VA and DXVA2 device overrides to both lists, which the old code got for free by sharing one list. Fixes the FFmpeg side of #21620. With -f lavfi -i "smptehdbars=d=1,format=p010le" -vf "vpp_amf=format=p010le" signalstats YAVG moves from 432.141 to 413.631, matching the untouched P010 source exactly, and the p010le -> yuv420p conversion no longer appears in the filter graph. The 8-bit-to-P010 half of that report is an AMF limitation and is unaffected. Tested on Windows 11, RX 9070 XT, driver 32.0.31041.1004: vpp_amf, vqe_amf, frc_amf and every sr_amf algorithm, with AMF hardware surfaces and with NV12, YUV420P, YUV420P10, YUYV422, GRAY8, 0RGB, RGB0, BGR0, P010, RGBA and BGRA software input. No case that worked before stopped working; software P010 and RGBA into vpp_amf now work where they previously failed. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_amf_common.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c index fa5f35dae4..9cb1231991 100644 --- a/libavfilter/vf_amf_common.c +++ b/libavfilter/vf_amf_common.c @@ -215,22 +215,24 @@ int amf_setup_input_output_formats(AVFilterContext *avctx, #if CONFIG_D3D11VA case AV_HWDEVICE_TYPE_D3D11VA: { - static const enum AVPixelFormat output_pix_fmts_d3d11[] = { + static const enum AVPixelFormat pix_fmts_d3d11[] = { AV_PIX_FMT_D3D11, AV_PIX_FMT_NONE, }; - output_pix_fmts = output_pix_fmts_d3d11; + input_pix_fmts = pix_fmts_d3d11; + output_pix_fmts = pix_fmts_d3d11; } break; #endif #if CONFIG_DXVA2 case AV_HWDEVICE_TYPE_DXVA2: { - static const enum AVPixelFormat output_pix_fmts_dxva2[] = { + static const enum AVPixelFormat pix_fmts_dxva2[] = { AV_PIX_FMT_DXVA2_VLD, AV_PIX_FMT_NONE, }; - output_pix_fmts = output_pix_fmts_dxva2; + input_pix_fmts = pix_fmts_dxva2; + output_pix_fmts = pix_fmts_dxva2; } break; #endif @@ -245,7 +247,7 @@ int amf_setup_input_output_formats(AVFilterContext *avctx, } } - input_formats = ff_make_pixel_format_list(output_pix_fmts); + input_formats = ff_make_pixel_format_list(input_pix_fmts); if (!input_formats) { return AVERROR(ENOMEM); } -- 2.52.0 >From d7954aec1ef36821575d8b1aeacc77a0c627d66c Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 30 Aug 2026 15:34:30 +0200 Subject: [PATCH 09/14] avfilter/amf: pin the output format for components that cannot convert amf_init_filter_config() falls back to outlink->format for the output frames context when no format is requested. That is right for vpp_amf, whose converter is initialised from hwframes_out->sw_format and therefore makes the frames context true. The HQ scaler, the VQ enhancer and the frame rate converter have no output format property at all and emit the format they were initialised with, so whatever the output link happened to negotiate was attached to a surface that never had that layout. ffmpeg -init_hw_device amf -f lavfi -i testsrc2 \ -vf "format=nv12,hwupload,sr_amf=w=2560:h=1440,format=rgba" is enough to reach it: the link negotiates RGBA, the scaler emits NV12, and hwdownload then reads a two-plane surface as one packed plane. Take the input software format as the output format for those three filters. sr_amf keeps its explicit format= handling: a packed RGB target is still selectable for the algorithms that convert, and anything the scaler cannot emit is rejected rather than silently mislabelled. amf_inlink_sw_format() moves to vf_amf_common.c so all three can use it. sr_amf compares against the format the user asked for rather than the one a previous configuration resolved, so reconfiguring the graph with a different input format does not reject itself. Tested on Windows 11, RX 9070 XT, driver 32.0.31041.1004: for every sr_amf algorithm and for vqe_amf and frc_amf, with NV12, P010 and RGBA input, hwdownload now accepts exactly the format the component emits and rejects every other, where before the frames context could claim any format the output link negotiated. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_amf_common.c | 10 ++++++++++ libavfilter/vf_amf_common.h | 2 ++ libavfilter/vf_frc_amf.c | 2 ++ libavfilter/vf_sr_amf.c | 30 +++++++++++------------------- libavfilter/vf_vqe_amf.c | 2 ++ 5 files changed, 27 insertions(+), 19 deletions(-) diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c index 9cb1231991..98ce46d07a 100644 --- a/libavfilter/vf_amf_common.c +++ b/libavfilter/vf_amf_common.c @@ -52,6 +52,7 @@ int amf_filter_init(AVFilterContext *avctx) return AVERROR(EINVAL); } } + ctx->format_opt = ctx->format; return 0; } @@ -289,6 +290,15 @@ int amf_copy_surface(AVFilterContext *avctx, const AVFrame *frame, return 0; } +enum AVPixelFormat amf_inlink_sw_format(AVFilterLink *inlink) +{ + FilterLink *inl = ff_filter_link(inlink); + + if (inl->hw_frames_ctx) + return ((AVHWFramesContext*)inl->hw_frames_ctx->data)->sw_format; + return inlink->format; +} + int amf_init_filter_config(AVFilterLink *outlink, enum AVPixelFormat *in_format) { int err; diff --git a/libavfilter/vf_amf_common.h b/libavfilter/vf_amf_common.h index 7879403d06..0d323dd480 100644 --- a/libavfilter/vf_amf_common.h +++ b/libavfilter/vf_amf_common.h @@ -31,6 +31,7 @@ typedef struct AMFFilterContext { int width, height; enum AVPixelFormat format; + enum AVPixelFormat format_opt; int scale_type; int in_color_range; int in_primaries; @@ -71,6 +72,7 @@ typedef struct AMFFilterContext { int amf_filter_init(AVFilterContext *avctx); void amf_filter_uninit(AVFilterContext *avctx); +enum AVPixelFormat amf_inlink_sw_format(AVFilterLink *inlink); int amf_init_filter_config(AVFilterLink *outlink, enum AVPixelFormat *in_format); int amf_copy_surface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface* surface); void amf_free_amfsurface(void *opaque, uint8_t *data); diff --git a/libavfilter/vf_frc_amf.c b/libavfilter/vf_frc_amf.c index 5b93f65ad1..1a84a2ee4f 100644 --- a/libavfilter/vf_frc_amf.c +++ b/libavfilter/vf_frc_amf.c @@ -110,6 +110,8 @@ static int amf_frc_filter_config_output(AVFilterLink *outlink) AMF_RESULT res; enum AVPixelFormat in_format; + amf_ctx->format = amf_inlink_sw_format(inlink); + err = amf_init_filter_config(outlink, &in_format); if (err < 0) return err; diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c index d9ed1dcd8a..fcaf2c5d2a 100644 --- a/libavfilter/vf_sr_amf.c +++ b/libavfilter/vf_sr_amf.c @@ -53,15 +53,6 @@ #endif -static enum AVPixelFormat amf_inlink_sw_format(AVFilterLink *inlink) -{ - FilterLink *inl = ff_filter_link(inlink); - - if (inl->hw_frames_ctx) - return ((AVHWFramesContext*)inl->hw_frames_ctx->data)->sw_format; - return inlink->format; -} - static int amf_hq_scaler_needs_packed_rgb(int algorithm) { return algorithm == AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 || @@ -140,21 +131,22 @@ static int amf_filter_config_output(AVFilterLink *outlink) needs_conversion = amf_hq_scaler_needs_packed_rgb(ctx->algorithm) && (in_sw_format == AV_PIX_FMT_NV12 || in_sw_format == AV_PIX_FMT_P010); - if (amf_hq_scaler_needs_packed_rgb(ctx->algorithm)) { - if (!needs_conversion) { - if (ctx->format != AV_PIX_FMT_NONE && ctx->format != in_sw_format) { - av_log(avctx, AV_LOG_ERROR, "The HQ scaler does not convert formats, format must be same or %s.\n", - av_get_pix_fmt_name(in_sw_format)); - return AVERROR(EINVAL); - } - ctx->format = in_sw_format; - } else if (ctx->format == AV_PIX_FMT_NONE) { + ctx->format = ctx->format_opt; + + if (needs_conversion) { + if (ctx->format == AV_PIX_FMT_NONE) ctx->format = in_sw_format == AV_PIX_FMT_P010 ? AV_PIX_FMT_X2BGR10 : AV_PIX_FMT_RGBA; - } else if (!amf_is_packed_rgb(ctx->format)) { + else if (!amf_is_packed_rgb(ctx->format)) { av_log(avctx, AV_LOG_ERROR, "This algorithm only outputs packed RGB, format=%s is not supported.\n", av_get_pix_fmt_name(ctx->format)); return AVERROR(EINVAL); } + } else if (ctx->format != AV_PIX_FMT_NONE && ctx->format != in_sw_format) { + av_log(avctx, AV_LOG_ERROR, "The HQ scaler does not convert formats, format must be same or %s.\n", + av_get_pix_fmt_name(in_sw_format)); + return AVERROR(EINVAL); + } else { + ctx->format = in_sw_format; } err = amf_init_filter_config(outlink, &in_format); diff --git a/libavfilter/vf_vqe_amf.c b/libavfilter/vf_vqe_amf.c index d53703ee58..901cfbdb5e 100644 --- a/libavfilter/vf_vqe_amf.c +++ b/libavfilter/vf_vqe_amf.c @@ -102,6 +102,8 @@ static int amf_vqe_filter_config_output(AVFilterLink *outlink) AMF_RESULT res; enum AVPixelFormat in_format; + amf_ctx->format = amf_inlink_sw_format(inlink); + err = amf_init_filter_config(outlink, &in_format); if (err < 0) return err; -- 2.52.0 >From 628a94509e2f7c36d7c1bc463f7de9ddb7547309 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 30 Aug 2026 15:53:03 +0200 Subject: [PATCH 10/14] avfilter/vf_amf_common: drain the component and tolerate AMF_REPEAT amf_filter_filter_frame() read the component output exactly once and treated anything but AMF_OK as fatal, so a component that had no output ready yet, which AMD's Video Converter programming guide documents as AMF_REPEAT, produced an error instead of simply consuming the frame. Poll until the component stops returning data, and accept AMF_REPEAT. A component that produces one output per input, which is what the scaler, the enhancer and the converter do today, behaves as before: one iteration with data, one without. Timestamps now come from the output surface, as vf_frc_amf.c already does, so a second output of the same submission does not inherit the first one's. Returning AVERROR(EAGAIN) from a filter_frame callback was wrong and is dropped. filter_frame_to_filter() latches a negative return into the link's status_out, which stalls a pull driven graph and stops EOF from crossing the filter; a frame that yields no output is simply consumed. AMF_INPUT_FULL is deliberately still fatal. It means the submission was not accepted, so tolerating it would silently drop the frame; handling it properly needs the retry loop libavcodec/amfenc.c has, which none of these components exercise. The input frame is kept until the loop ends, since its properties are copied onto every output, and freed once at the end. Tested on Windows 11, RX 9070 XT, driver 32.0.31041.1004: identical output and timestamps for every sr_amf algorithm, vpp_amf, vqe_amf and frc_amf, with software, AMF and D3D11VA input. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_amf_common.c | 118 ++++++++++++++++++++---------------- 1 file changed, 65 insertions(+), 53 deletions(-) diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c index 98ce46d07a..6edbb15ce7 100644 --- a/libavfilter/vf_amf_common.c +++ b/libavfilter/vf_amf_common.c @@ -98,6 +98,7 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) enum AVColorRange out_color_range; AVFrame *out = NULL; + int got_frame = 0; int ret = 0; if (!ctx->component) @@ -116,7 +117,11 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) surface_in->pVtbl->Release(surface_in); AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter SubmitInput() failed with error %d\n", res); res = ctx->pre_converter->pVtbl->QueryOutput(ctx->pre_converter, &data_conv); - AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK && data_conv, AVERROR_UNKNOWN, "Converter QueryOutput() failed with error %d\n", res); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK || res == AMF_REPEAT, AVERROR_UNKNOWN, "Converter QueryOutput() failed with error %d\n", res); + if (!data_conv) { + ret = 0; + goto fail; + } res = data_conv->pVtbl->QueryInterface(data_conv, &guid, (void**)&surface_conv); data_conv->pVtbl->Release(data_conv); AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter QueryInterface(IID_AMFSurface) failed with error %d\n", res); @@ -126,71 +131,78 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) res = ctx->component->pVtbl->SubmitInput(ctx->component, (AMFData*)surface_in); surface_in->pVtbl->Release(surface_in); // release surface after use AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); - res = ctx->component->pVtbl->QueryOutput(ctx->component, &data_out); - AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryOutput() failed with error %d\n", res); - if (data_out) { + while (1) { AMFGuid guid = IID_AMFSurface(); + + res = ctx->component->pVtbl->QueryOutput(ctx->component, &data_out); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK || res == AMF_REPEAT, AVERROR_UNKNOWN, "QueryOutput() failed with error %d\n", res); + if (!data_out) + break; + res = data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface_out); // query for buffer interface data_out->pVtbl->Release(data_out); + data_out = NULL; AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryInterface(IID_AMFSurface) failed with error %d\n", res); - } else { - ret = AVERROR(EAGAIN); - goto fail; - } - out = amf_amfsurface_to_avframe(avctx, surface_out); - AMF_GOTO_FAIL_IF_FALSE(avctx, out != NULL, AVERROR(ENOMEM), "Failed to convert AMFSurface to AVFrame\n"); + out = amf_amfsurface_to_avframe(avctx, surface_out); + AMF_GOTO_FAIL_IF_FALSE(avctx, out != NULL, AVERROR(ENOMEM), "Failed to convert AMFSurface to AVFrame\n"); - ret = av_frame_copy_props(out, in); - av_frame_unref(in); + ret = av_frame_copy_props(out, in); + if (ret < 0) + goto fail; + out->pts = surface_out->pVtbl->GetPts(surface_out); - out_colorspace = AVCOL_SPC_UNSPECIFIED; + out_colorspace = AVCOL_SPC_UNSPECIFIED; - if (ctx->color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) { - switch(ctx->color_profile) { - case AMF_VIDEO_CONVERTER_COLOR_PROFILE_601: - out_colorspace = AVCOL_SPC_SMPTE170M; - break; - case AMF_VIDEO_CONVERTER_COLOR_PROFILE_709: - out_colorspace = AVCOL_SPC_BT709; - break; - case AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020: - out_colorspace = AVCOL_SPC_BT2020_NCL; - break; - case AMF_VIDEO_CONVERTER_COLOR_PROFILE_JPEG: - out_colorspace = AVCOL_SPC_RGB; - break; - default: - out_colorspace = AVCOL_SPC_UNSPECIFIED; - break; + if (ctx->color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) { + switch(ctx->color_profile) { + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_601: + out_colorspace = AVCOL_SPC_SMPTE170M; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_709: + out_colorspace = AVCOL_SPC_BT709; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020: + out_colorspace = AVCOL_SPC_BT2020_NCL; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_JPEG: + out_colorspace = AVCOL_SPC_RGB; + break; + default: + out_colorspace = AVCOL_SPC_UNSPECIFIED; + break; + } + out->colorspace = out_colorspace; } - out->colorspace = out_colorspace; + + out_color_range = AVCOL_RANGE_UNSPECIFIED; + if (ctx->out_color_range == AMF_COLOR_RANGE_FULL) + out_color_range = AVCOL_RANGE_JPEG; + else if (ctx->out_color_range == AMF_COLOR_RANGE_STUDIO) + out_color_range = AVCOL_RANGE_MPEG; + + if (ctx->out_color_range != AMF_COLOR_RANGE_UNDEFINED) + out->color_range = out_color_range; + + if (ctx->out_primaries != AMF_COLOR_PRIMARIES_UNDEFINED) + out->color_primaries = ctx->out_primaries; + + if (ctx->out_trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) + out->color_trc = ctx->out_trc; + + if (ctx->pre_converter) + out->colorspace = AVCOL_SPC_RGB; + + ret = ff_filter_frame(outlink, out); + out = NULL; + if (ret < 0) + goto fail; + got_frame = 1; } - out_color_range = AVCOL_RANGE_UNSPECIFIED; - if (ctx->out_color_range == AMF_COLOR_RANGE_FULL) - out_color_range = AVCOL_RANGE_JPEG; - else if (ctx->out_color_range == AMF_COLOR_RANGE_STUDIO) - out_color_range = AVCOL_RANGE_MPEG; - - if (ctx->out_color_range != AMF_COLOR_RANGE_UNDEFINED) - out->color_range = out_color_range; - - if (ctx->out_primaries != AMF_COLOR_PRIMARIES_UNDEFINED) - out->color_primaries = ctx->out_primaries; - - if (ctx->out_trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) - out->color_trc = ctx->out_trc; - - if (ctx->pre_converter) - out->colorspace = AVCOL_SPC_RGB; - - if (ret < 0) - goto fail; - av_frame_free(&in); - return ff_filter_frame(outlink, out); + return got_frame ? ret : 0; fail: av_frame_free(&in); av_frame_free(&out); -- 2.52.0 >From b31bbd0f4ec25960534440c59aa299fe34854946 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 30 Aug 2026 15:53:03 +0200 Subject: [PATCH 11/14] avfilter/vf_vqe_amf: only offer the formats the VQ enhancer accepts vqe_amf advertised BGRA, RGBA, RGBAF16 and X2BGR10 alongside NV12 and P010, but AMFVQEnhancer::Init() rejects all four with AMF_INVALID_ARG. Now that the input list is used for negotiation, a packed RGB source reaches the filter and fails at configuration time instead of being converted to a format the component takes. Drop the four from both lists. libavfilter then converts such a source to NV12 in software and the filter works, where before it aborted. A hardware frames context is not covered by format negotiation, so reject an unsupported sw_format explicitly rather than leaving the user with AMF_INVALID_ARG from Init(). The component's own capability query cannot be used to build this list. AMFVQEnhancer enumerates NV12, BGRA, ARGB, RGBA, P010, RGBA_F16 and R10G10B10A2 through GetCaps()/GetInputCaps(), each flagged native and stable across runs, but Init() accepts only NV12 and P010 and rejects the other five. AMFHQScaler and AMFVideoConverter do not have this problem, so the list cannot simply follow the query for every component. Reported as GPUOpen-LibrariesAndSDKs/AMF#610. Tested on Windows 11, RX 9070 XT, driver 32.0.31041.1004. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_vqe_amf.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/libavfilter/vf_vqe_amf.c b/libavfilter/vf_vqe_amf.c index 901cfbdb5e..d368770c89 100644 --- a/libavfilter/vf_vqe_amf.c +++ b/libavfilter/vf_vqe_amf.c @@ -23,6 +23,7 @@ #include "libavutil/opt.h" +#include "libavutil/pixdesc.h" #include "libavutil/hwcontext.h" #include "libavutil/hwcontext_amf.h" #include "libavutil/hwcontext_amf_internal.h" @@ -67,20 +68,12 @@ static int amf_filter_query_formats(AVFilterContext *avctx) AV_PIX_FMT_DXVA2_VLD, AV_PIX_FMT_NV12, AV_PIX_FMT_P010, - AV_PIX_FMT_BGRA, - AV_PIX_FMT_RGBA, - AV_PIX_FMT_RGBAF16, - AV_PIX_FMT_X2BGR10, AV_PIX_FMT_NONE, }; static const enum AVPixelFormat output_pix_fmts_default[] = { AV_PIX_FMT_AMF_SURFACE, AV_PIX_FMT_NV12, AV_PIX_FMT_P010, - AV_PIX_FMT_BGRA, - AV_PIX_FMT_RGBA, - AV_PIX_FMT_RGBAF16, - AV_PIX_FMT_X2BGR10, AV_PIX_FMT_NONE, }; output_pix_fmts = output_pix_fmts_default; @@ -108,6 +101,12 @@ static int amf_vqe_filter_config_output(AVFilterLink *outlink) if (err < 0) return err; + if (in_format != AV_PIX_FMT_NV12 && in_format != AV_PIX_FMT_P010) { + av_log(avctx, AV_LOG_ERROR, "The VQ enhancer only accepts nv12 and p010, got %s.\n", + av_get_pix_fmt_name(in_format)); + return AVERROR(EINVAL); + } + device_ctx = amf_ctx->amf_device_ctx; res = AMF_IFACE_CALL(device_ctx->factory, CreateComponent, device_ctx->context, AMFVQEnhancer, &amf_ctx->component); -- 2.52.0 >From 440ef2225289e5f32b7a07d7fd2660995f3d53d3 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 30 Aug 2026 15:53:03 +0200 Subject: [PATCH 12/14] avfilter/amf: log CreateComponent failures on the filter context These four call sites pass the filter's private context to av_log() while every other call around them passes the AVFilterContext. Both happen to start with an AVClass, so nothing misbehaves, but the private class has no parent set, so the message loses the "[Parsed_sr_amf_0 @ ...]" prefix that identifies which filter instance failed. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_sr_amf.c | 4 ++-- libavfilter/vf_vpp_amf.c | 2 +- libavfilter/vsrc_amf.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c index fcaf2c5d2a..cfde4886b1 100644 --- a/libavfilter/vf_sr_amf.c +++ b/libavfilter/vf_sr_amf.c @@ -157,7 +157,7 @@ static int amf_filter_config_output(AVFilterLink *outlink) AMFSize in_size = { inlink->w, inlink->h }; res = ctx->amf_device_ctx->factory->pVtbl->CreateComponent(ctx->amf_device_ctx->factory, ctx->amf_device_ctx->context, AMFVideoConverter, &ctx->pre_converter); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); AMF_ASSIGN_PROPERTY_INT64(res, ctx->pre_converter, AMF_VIDEO_CONVERTER_OUTPUT_FORMAT, (amf_int32)av_av_to_amf_format(ctx->format)); AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); @@ -182,7 +182,7 @@ static int amf_filter_config_output(AVFilterLink *outlink) } // FIXME: add checks whether we have HW context res = ctx->amf_device_ctx->factory->pVtbl->CreateComponent(ctx->amf_device_ctx->factory, ctx->amf_device_ctx->context, AMFHQScaler, &ctx->component); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFHQScaler, res); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFHQScaler, res); mem_type = av_amf_get_memory_type(ctx->amf_device_ctx); if (mem_type != AMF_MEMORY_UNKNOWN) diff --git a/libavfilter/vf_vpp_amf.c b/libavfilter/vf_vpp_amf.c index 43277224ad..1cd7e81128 100644 --- a/libavfilter/vf_vpp_amf.c +++ b/libavfilter/vf_vpp_amf.c @@ -97,7 +97,7 @@ static int amf_filter_config_output(AVFilterLink *outlink) // FIXME: add checks whether we have HW context hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; res = ctx->amf_device_ctx->factory->pVtbl->CreateComponent(ctx->amf_device_ctx->factory, ctx->amf_device_ctx->context, AMFVideoConverter, &ctx->component); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); mem_type = av_amf_get_memory_type(ctx->amf_device_ctx); if (mem_type != AMF_MEMORY_UNKNOWN) diff --git a/libavfilter/vsrc_amf.c b/libavfilter/vsrc_amf.c index 2a9811ae4a..e3fb8b9d38 100644 --- a/libavfilter/vsrc_amf.c +++ b/libavfilter/vsrc_amf.c @@ -159,7 +159,7 @@ static int amf_init_vsrc(AVFilterLink *outlink) amf_device_ctx->context, AMFDisplayCapture, &ctx->capture); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFDisplayCapture, res); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFDisplayCapture, res); AMF_ASSIGN_PROPERTY_INT64(res, ctx->capture, AMF_DISPLAYCAPTURE_MONITOR_INDEX, ctx->monitor_index); if (res != AMF_OK) { -- 2.52.0 >From 8445c9ac6cbb29b445f731935689361150422c41 Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 30 Aug 2026 16:04:10 +0200 Subject: [PATCH 13/14] avfilter/amf: copy D3D11 decoder textures the components can read The AMF filter components read their input with a shader and reject a texture created without D3D11_BIND_SHADER_RESOURCE, which is what a D3D11VA decoder pool provides. Every AMF filter therefore fails on ffmpeg -hwaccel d3d11va -hwaccel_output_format d3d11 \ -i in.mp4 -vf vpp_amf ... with AMF_DIRECTX_FAILED from the first QueryOutput(), while frc_amf and the AMF encoders consume the same surfaces. The only way out was the device option SHADER=1, which the usual command lines do not set. Copy the slice into an AMF allocated surface, which carries the flags the components need. CopySubresourceRegion() runs on the copy engine, so it can read a decoder texture that a shader cannot. AMFSurface::Duplicate(), suggested upstream for this, returns AMF_OK but yields a surface that fails in exactly the same way, for AMF_MEMORY_DX11 and AMF_MEMORY_HOST alike. Only a frame whose texture lacks the flag is copied, so a pool created with SHADER=1 keeps the zero copy path. Tested on Windows 11, RX 9070 XT, driver 32.0.31041.1004, with -hwaccel d3d11va -hwaccel_output_format d3d11: vpp_amf, vqe_amf and every sr_amf algorithm produce output matching the plain decode where they previously failed, and frc_amf is unchanged. Refs https://github.com/GPUOpen-LibrariesAndSDKs/AMF/issues/605 Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_amf_common.c | 65 +++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c index 6edbb15ce7..65e01d811b 100644 --- a/libavfilter/vf_amf_common.c +++ b/libavfilter/vf_amf_common.c @@ -37,6 +37,7 @@ #if CONFIG_D3D11VA #include <d3d11.h> +#include "libavutil/hwcontext_d3d11va.h" #endif int amf_filter_init(AVFilterContext *avctx) @@ -508,6 +509,58 @@ fail: return NULL; } +#if CONFIG_D3D11VA +/* The AMF filter components read their input with a shader, so they reject a + * texture created without D3D11_BIND_SHADER_RESOURCE, which is what a D3D11VA + * decoder pool gives us. Copy the slice into an AMF allocated surface, which + * carries the flags the components need. CopySubresourceRegion() uses the copy + * engine, so it can read the decoder texture that a shader cannot. */ +static int amf_copy_d3d11_texture(AVFilterContext *avctx, const AVFrame *frame, + int index, AMFSurface **ppSurface) +{ + AMFFilterContext *ctx = avctx->priv; + AVHWFramesContext *frames = (AVHWFramesContext*)frame->hw_frames_ctx->data; + AVD3D11VADeviceContext *hwctx = frames->device_ctx->hwctx; + ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; + AMFSurface *surface = NULL; + AMFPlane *plane; + D3D11_TEXTURE2D_DESC desc; + D3D11_BOX box; + AMF_RESULT res; + + res = ctx->amf_device_ctx->context->pVtbl->AllocSurface(ctx->amf_device_ctx->context, + AMF_MEMORY_DX11, av_av_to_amf_format(frames->sw_format), + frame->width, frame->height, &surface); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); + + plane = surface->pVtbl->GetPlaneAt(surface, 0); + if (!plane) { + surface->pVtbl->Release(surface); + return AVERROR(ENOMEM); + } + + // The decoder pool is allocated with aligned dimensions, so copy the coded + // area rather than the whole source subresource. D3D11 wants even bounds + // for a planar format, and the source is at least that large. + texture->lpVtbl->GetDesc(texture, &desc); + box.left = 0; + box.top = 0; + box.front = 0; + box.right = FFMIN(FFALIGN(frame->width, 2), desc.Width); + box.bottom = FFMIN(FFALIGN(frame->height, 2), desc.Height); + box.back = 1; + + hwctx->lock(hwctx->lock_ctx); + hwctx->device_context->lpVtbl->CopySubresourceRegion(hwctx->device_context, + (ID3D11Resource*)plane->pVtbl->GetNative(plane), 0, 0, 0, 0, + (ID3D11Resource*)texture, index, &box); + hwctx->unlock(hwctx->lock_ctx); + + *ppSurface = surface; + return 0; +} +#endif + int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface** ppSurface) { AMFVariantStruct var = { 0 }; @@ -516,6 +569,7 @@ int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFS AMFSurface *surface; AMF_RESULT res; int hw_surface = 0; + int ret; switch (frame->format) { #if CONFIG_D3D11VA @@ -524,6 +578,17 @@ int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFS static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture int index = (intptr_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use + D3D11_TEXTURE2D_DESC desc; + + texture->lpVtbl->GetDesc(texture, &desc); + if (!(desc.BindFlags & D3D11_BIND_SHADER_RESOURCE) && frame->hw_frames_ctx) { + ret = amf_copy_d3d11_texture(avctx, frame, index, &surface); + if (ret < 0) + return ret; + hw_surface = 1; + break; + } + texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); res = ctx->amf_device_ctx->context->pVtbl->CreateSurfaceFromDX11Native(ctx->amf_device_ctx->context, texture, &surface, NULL); // wrap to AMF surface -- 2.52.0 >From 04bf472cb7da95200fb1601756c1a6f3f2e36b0e Mon Sep 17 00:00:00 2001 From: Julius Bairaktaris <[email protected]> Date: Sun, 30 Aug 2026 16:53:59 +0200 Subject: [PATCH 14/14] avfilter/amf: flush the component at EOF with an activate callback The AMF filters ran off a filter_frame callback, which has no way to tell a component that no more input is coming. Anything a component still held when the input link ended was lost, and there was nowhere to call AMFComponent::Drain(). The scaler, the enhancer and the converter return one output per input today, so nothing is dropped in practice, but the call sequence was only correct as long as that stayed true. Convert the three filters that share amf_filter_filter_frame() to an activate callback: consume one input frame at a time, acknowledge the input status, and on EOF drain the pre-converter and the component before forwarding the status downstream. The output side of filter_frame moves into amf_deliver_output(), which activate reuses for the frames the drain produces, with the timestamp taken from the surface since there is no input frame to copy properties from. vf_frc_amf.c keeps its own filter_frame; it emits several frames per input and warrants the same treatment separately. Tested on Windows 11, RX 9070 XT, driver 32.0.31041.1004: sr_amf, vpp_amf and vqe_amf return 60 frames for a 60 frame clip, bit identical to master by framecrc, over AMF and D3D11VA surfaces, and a truncated read with -frames:v still terminates. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- libavfilter/vf_amf_common.c | 147 +++++++++++++++++++++++++----------- libavfilter/vf_amf_common.h | 5 ++ libavfilter/vf_sr_amf.c | 2 +- libavfilter/vf_vpp_amf.c | 2 +- libavfilter/vf_vqe_amf.c | 2 +- 5 files changed, 109 insertions(+), 49 deletions(-) diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c index 65e01d811b..a1b83a2d17 100644 --- a/libavfilter/vf_amf_common.c +++ b/libavfilter/vf_amf_common.c @@ -21,6 +21,7 @@ #include "libavutil/avassert.h" #include "avfilter.h" #include "avfilter_internal.h" +#include "filters.h" #include "formats.h" #include "libavutil/mem.h" #include "libavutil/imgutils.h" @@ -86,58 +87,24 @@ void amf_filter_uninit(AVFilterContext *avctx) av_buffer_unref(&ctx->hwframes_out_ref); } -int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) +static int amf_deliver_output(AVFilterContext *avctx, const AVFrame *in, int *got_frame) { - AVFilterContext *avctx = inlink->dst; AMFFilterContext *ctx = avctx->priv; - AVFilterLink *outlink = avctx->outputs[0]; - AMF_RESULT res; - AMFSurface *surface_in; + AVFilterLink *outlink = avctx->outputs[0]; AMFSurface *surface_out; AMFData *data_out = NULL; enum AVColorSpace out_colorspace; enum AVColorRange out_color_range; - AVFrame *out = NULL; - int got_frame = 0; + AMF_RESULT res; int ret = 0; - if (!ctx->component) - return AVERROR(EINVAL); - - ret = amf_avframe_to_amfsurface(avctx, in, &surface_in); - if (ret < 0) - goto fail; - - if (ctx->pre_converter) { - AMFGuid guid = IID_AMFSurface(); - AMFData *data_conv = NULL; - AMFSurface *surface_conv = NULL; - - res = ctx->pre_converter->pVtbl->SubmitInput(ctx->pre_converter, (AMFData*)surface_in); - surface_in->pVtbl->Release(surface_in); - AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter SubmitInput() failed with error %d\n", res); - res = ctx->pre_converter->pVtbl->QueryOutput(ctx->pre_converter, &data_conv); - AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK || res == AMF_REPEAT, AVERROR_UNKNOWN, "Converter QueryOutput() failed with error %d\n", res); - if (!data_conv) { - ret = 0; - goto fail; - } - res = data_conv->pVtbl->QueryInterface(data_conv, &guid, (void**)&surface_conv); - data_conv->pVtbl->Release(data_conv); - AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter QueryInterface(IID_AMFSurface) failed with error %d\n", res); - surface_in = surface_conv; - } - - res = ctx->component->pVtbl->SubmitInput(ctx->component, (AMFData*)surface_in); - surface_in->pVtbl->Release(surface_in); // release surface after use - AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); - while (1) { AMFGuid guid = IID_AMFSurface(); res = ctx->component->pVtbl->QueryOutput(ctx->component, &data_out); - AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK || res == AMF_REPEAT, AVERROR_UNKNOWN, "QueryOutput() failed with error %d\n", res); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK || res == AMF_REPEAT || res == AMF_EOF, + AVERROR_UNKNOWN, "QueryOutput() failed with error %d\n", res); if (!data_out) break; @@ -149,9 +116,11 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) out = amf_amfsurface_to_avframe(avctx, surface_out); AMF_GOTO_FAIL_IF_FALSE(avctx, out != NULL, AVERROR(ENOMEM), "Failed to convert AMFSurface to AVFrame\n"); - ret = av_frame_copy_props(out, in); - if (ret < 0) - goto fail; + if (in) { + ret = av_frame_copy_props(out, in); + if (ret < 0) + goto fail; + } out->pts = surface_out->pVtbl->GetPts(surface_out); out_colorspace = AVCOL_SPC_UNSPECIFIED; @@ -199,17 +168,103 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) out = NULL; if (ret < 0) goto fail; - got_frame = 1; + *got_frame = 1; } - av_frame_free(&in); - return got_frame ? ret : 0; + return 0; fail: - av_frame_free(&in); av_frame_free(&out); return ret; } +int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *avctx = inlink->dst; + AMFFilterContext *ctx = avctx->priv; + AMF_RESULT res; + AMFSurface *surface_in; + int got_frame = 0; + int ret = 0; + + if (!ctx->component) + return AVERROR(EINVAL); + + ret = amf_avframe_to_amfsurface(avctx, in, &surface_in); + if (ret < 0) + goto fail; + + if (ctx->pre_converter) { + AMFGuid guid = IID_AMFSurface(); + AMFData *data_conv = NULL; + AMFSurface *surface_conv = NULL; + + res = ctx->pre_converter->pVtbl->SubmitInput(ctx->pre_converter, (AMFData*)surface_in); + surface_in->pVtbl->Release(surface_in); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter SubmitInput() failed with error %d\n", res); + res = ctx->pre_converter->pVtbl->QueryOutput(ctx->pre_converter, &data_conv); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK || res == AMF_REPEAT, AVERROR_UNKNOWN, "Converter QueryOutput() failed with error %d\n", res); + if (!data_conv) { + ret = 0; + goto fail; + } + res = data_conv->pVtbl->QueryInterface(data_conv, &guid, (void**)&surface_conv); + data_conv->pVtbl->Release(data_conv); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter QueryInterface(IID_AMFSurface) failed with error %d\n", res); + surface_in = surface_conv; + } + + res = ctx->component->pVtbl->SubmitInput(ctx->component, (AMFData*)surface_in); + surface_in->pVtbl->Release(surface_in); // release surface after use + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); + + ret = amf_deliver_output(avctx, in, &got_frame); +fail: + av_frame_free(&in); + return ret; +} + +int amf_filter_activate(AVFilterContext *avctx) +{ + AMFFilterContext *ctx = avctx->priv; + AVFilterLink *inlink = avctx->inputs[0]; + AVFilterLink *outlink = avctx->outputs[0]; + AVFrame *in = NULL; + int got_frame = 0; + int ret; + + FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink); + + if (!ctx->eof) { + ret = ff_inlink_consume_frame(inlink, &in); + if (ret < 0) + return ret; + if (in) + return amf_filter_filter_frame(inlink, in); + + if (ff_inlink_acknowledge_status(inlink, &ctx->status, &ctx->status_pts)) + ctx->eof = 1; + } + + if (ctx->eof) { + // let the component hand back anything it is still holding + if (ctx->component && !ctx->drained) { + ctx->drained = 1; + if (ctx->pre_converter) + ctx->pre_converter->pVtbl->Drain(ctx->pre_converter); + ctx->component->pVtbl->Drain(ctx->component); + ret = amf_deliver_output(avctx, NULL, &got_frame); + if (ret < 0) + return ret; + } + ff_outlink_set_status(outlink, ctx->status, ctx->status_pts); + return 0; + } + + FF_FILTER_FORWARD_WANTED(outlink, inlink); + + return FFERROR_NOT_READY; +} + int amf_setup_input_output_formats(AVFilterContext *avctx, diff --git a/libavfilter/vf_amf_common.h b/libavfilter/vf_amf_common.h index 0d323dd480..aa62458014 100644 --- a/libavfilter/vf_amf_common.h +++ b/libavfilter/vf_amf_common.h @@ -32,6 +32,10 @@ typedef struct AMFFilterContext { int width, height; enum AVPixelFormat format; enum AVPixelFormat format_opt; + int eof; + int drained; + int status; + int64_t status_pts; int scale_type; int in_color_range; int in_primaries; @@ -80,5 +84,6 @@ AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface) int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface** ppSurface); int amf_setup_input_output_formats(AVFilterContext *avctx, const enum AVPixelFormat *input_pix_fmts, const enum AVPixelFormat *output_pix_fmts); int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in); +int amf_filter_activate(AVFilterContext *avctx); #endif /* AVFILTER_AMF_COMMON_H */ diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c index cfde4886b1..58ec4483a0 100644 --- a/libavfilter/vf_sr_amf.c +++ b/libavfilter/vf_sr_amf.c @@ -244,7 +244,6 @@ static const AVFilterPad amf_filter_inputs[] = { { .name = "default", .type = AVMEDIA_TYPE_VIDEO, - .filter_frame = amf_filter_filter_frame, } }; @@ -265,6 +264,7 @@ FFFilter ff_vf_sr_amf = { .init = amf_filter_init, .uninit = amf_filter_uninit, + .activate = amf_filter_activate, FILTER_INPUTS(amf_filter_inputs), FILTER_OUTPUTS(amf_filter_outputs), FILTER_QUERY_FUNC(&amf_filter_query_formats), diff --git a/libavfilter/vf_vpp_amf.c b/libavfilter/vf_vpp_amf.c index 1cd7e81128..8b7d2f202c 100644 --- a/libavfilter/vf_vpp_amf.c +++ b/libavfilter/vf_vpp_amf.c @@ -280,7 +280,6 @@ static const AVFilterPad amf_filter_inputs[] = { { .name = "default", .type = AVMEDIA_TYPE_VIDEO, - .filter_frame = amf_filter_filter_frame, } }; @@ -300,6 +299,7 @@ FFFilter ff_vf_vpp_amf = { .priv_size = sizeof(AMFFilterContext), .init = amf_filter_init, .uninit = amf_filter_uninit, + .activate = amf_filter_activate, FILTER_INPUTS(amf_filter_inputs), FILTER_OUTPUTS(amf_filter_outputs), FILTER_QUERY_FUNC(amf_filter_query_formats), diff --git a/libavfilter/vf_vqe_amf.c b/libavfilter/vf_vqe_amf.c index d368770c89..cd9b58f8f3 100644 --- a/libavfilter/vf_vqe_amf.c +++ b/libavfilter/vf_vqe_amf.c @@ -148,7 +148,6 @@ static const AVFilterPad amf_filter_inputs[] = { { .name = "default", .type = AVMEDIA_TYPE_VIDEO, - .filter_frame = amf_filter_filter_frame, } }; @@ -168,6 +167,7 @@ FFFilter ff_vf_vqe_amf = { .priv_size = sizeof(AMFVQEFilterContext), .init = amf_vqe_init, .uninit = amf_filter_uninit, + .activate = amf_filter_activate, FILTER_INPUTS(amf_filter_inputs), FILTER_OUTPUTS(amf_filter_outputs), FILTER_QUERY_FUNC(&amf_filter_query_formats), -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
