This is an automated email from the git hooks/post-receive script. Git pushed a commit to branch master in repository ffmpeg.
commit 7db554bacfe56e001885baf5873956a7bccae89b Author: Julius Bairaktaris <[email protected]> AuthorDate: Fri Jul 31 13:25:20 2026 +0200 Commit: ArazIusubov <[email protected]> CommitDate: Wed Sep 16 22:05:28 2026 +0000 avfilter/vf_sr_amf: convert YUV input on the GPU for sr1-1 and point sr1-1 and point need a packed RGB surface. Software input was handled by restricting the negotiated formats so libavfilter inserts a converter, but a hardware surface backed by NV12 or P010 could only be rejected, which made both algorithms unusable with any AMF or D3D11VA producer upstream - including every player that hands libavfilter its decoder frames. Chain an AMFVideoConverter ahead of the scaler in that case instead. The conversion stays on the GPU, so no download is introduced, and the component is driven from the shared filter_frame path through a new pre_converter field. The HQ scaler has no output format property and emits the format it was initialised with, so one packed RGB target drives the converter output format, AMFHQScaler::Init() and the output frames context alike. P010 converts to X2BGR10 and NV12 to RGBA by default; an explicit format= is kept when it names a packed RGB format and rejected otherwise. Where no conversion is needed the scaler cannot change the format at all, so an explicit format= differing from the input is rejected and the target is pinned to the input format, rather than letting the output link negotiate a frames context that disagrees with the surface the scaler produces. AMD's Video Converter programming guide has the caller poll for output and stop submitting while the component reports AMF_INPUT_FULL, so both calls are retried instead of turning a valid asynchronous state into a dropped frame or an error. The component runs synchronously on the tested device and the retry never triggers there. The converter is asked for full range output, and the converted frame is tagged AVCOL_SPC_RGB with AVCOL_RANGE_JPEG rather than inheriting the input's YUV matrix. The output frames context follows the converted format, so the filter emits packed RGB where it previously emitted NV12; with format=same this is only reachable for the two algorithms that cannot produce YUV anyway. Tested on Windows 11, RX 9070 XT, driver 32.0.31041.1004, with -hwaccel amf -hwaccel_output_format amf: point and sr1-1 measure YMIN=YAVG=YMAX=0 before this change and match the plain decode to within 0.6/255 after, while bilinear, bicubic and sr1-0 keep their NV12 output. point at 2x scores 36.5 dB PSNR against a software neighbour reference, the converted frame is tagged pc/gbr, P010 reaches X2BGR10, the accepted and rejected format= values behave as described, and 300 frames run through the converter for both algorithms. Assisted-by: Claude Opus 5 Signed-off-by: Julius Bairaktaris <[email protected]> --- doc/filters.texi | 20 ++++++++----- libavfilter/vf_amf_common.c | 35 ++++++++++++++++++++++ libavfilter/vf_amf_common.h | 1 + libavfilter/vf_sr_amf.c | 73 +++++++++++++++++++++++++++++++++++++-------- 4 files changed, 110 insertions(+), 19 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 3d63288be9..9c00ffeca8 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -23274,17 +23274,17 @@ This is a default value @item point Point AMF returns an unwritten surface for this algorithm when the format is -@code{nv12} or @code{p010}, so software input in those formats is converted to -a packed RGB format (@code{rgba}, @code{bgra}, @code{x2bgr10le} or -@code{rgbaf16le}) automatically and a hardware surface in them is rejected. +@code{nv12} or @code{p010}, so inputs in those formats are converted to a packed +RGB format automatically. This is a driver bug rather than a documented +restriction; the conversion can be dropped once a driver that scales +@code{nv12} and @code{p010} correctly with @code{point} is in wide use. @item sr1-1 Video SR1.1 This algorithm only supports packed RGB formats and requires a DirectX 11 or -DirectX 12 device, so it is available on Windows only. Software input in -another format such as @code{nv12} or @code{p010} is converted to a packed RGB -format (@code{rgba}, @code{bgra}, @code{x2bgr10le} or @code{rgbaf16le}) -automatically; a hardware surface in such a format is rejected. +DirectX 12 device, so it is available on Windows only. Inputs in other formats +such as @code{nv12} or @code{p010} are converted to a packed RGB format +automatically. Note that the VideoSR algorithms (@code{sr1-0} and @code{sr1-1}) do not preserve the alpha channel (it is left zeroed); append e.g. @code{format=rgb24} downstream if an opaque result is required. @@ -23298,6 +23298,12 @@ Control hq scaler sharpening. The value is a float in the range of [0.0, 2.0] Controls the output pixel format. By default, or if none is specified, the input pixel format is used. +The @code{point} and @code{sr1-1} algorithms emit the packed RGB format the +scaler receives, so for them this option must name one of @code{rgba}, +@code{bgra}, @code{x2bgr10le} or @code{rgbaf16le}. When the input is converted +automatically, @code{x2bgr10le} is selected for @code{p010} input and +@code{rgba} otherwise, and the result is tagged as full range RGB. + @item keep-ratio Force the scaler to keep the aspect ratio of the input image when the output size has a different aspect ratio. Default value is false. diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c index 40c6fbceea..f7cd680cbe 100644 --- a/libavfilter/vf_amf_common.c +++ b/libavfilter/vf_amf_common.c @@ -25,6 +25,7 @@ #include "libavutil/mem.h" #include "libavutil/imgutils.h" #include "libavutil/pixdesc.h" +#include "libavutil/time.h" #include "AMF/components/VideoDecoderUVD.h" #include "libavutil/hwcontext_amf.h" @@ -66,6 +67,12 @@ void amf_filter_uninit(AVFilterContext *avctx) ctx->component = NULL; } + if (ctx->pre_converter) { + ctx->pre_converter->pVtbl->Terminate(ctx->pre_converter); + ctx->pre_converter->pVtbl->Release(ctx->pre_converter); + ctx->pre_converter = NULL; + } + if (ctx->master_display) av_freep(&ctx->master_display); @@ -100,6 +107,32 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) if (ret < 0) goto fail; + if (ctx->pre_converter) { + AMFGuid guid = IID_AMFSurface(); + AMFData *data_conv = NULL; + AMFSurface *surface_conv = NULL; + + do { + res = ctx->pre_converter->pVtbl->SubmitInput(ctx->pre_converter, (AMFData*)surface_in); + if (res == AMF_INPUT_FULL) + av_usleep(100); + } while (res == AMF_INPUT_FULL); + surface_in->pVtbl->Release(surface_in); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter SubmitInput() failed with error %d\n", res); + + do { + res = ctx->pre_converter->pVtbl->QueryOutput(ctx->pre_converter, &data_conv); + if (res == AMF_REPEAT && !data_conv) + av_usleep(100); + } while (res == AMF_REPEAT && !data_conv); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK && data_conv, AVERROR_UNKNOWN, "Converter QueryOutput() failed with error %d\n", res); + + res = data_conv->pVtbl->QueryInterface(data_conv, &guid, (void**)&surface_conv); + data_conv->pVtbl->Release(data_conv); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "Converter QueryInterface(IID_AMFSurface) failed with error %d\n", res); + surface_in = surface_conv; + } + res = ctx->component->pVtbl->SubmitInput(ctx->component, (AMFData*)surface_in); surface_in->pVtbl->Release(surface_in); // release surface after use AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); @@ -158,6 +191,8 @@ int amf_filter_filter_frame(AVFilterLink *inlink, AVFrame *in) if (ctx->out_trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) out->color_trc = ctx->out_trc; + if (ctx->pre_converter) + out->colorspace = AVCOL_SPC_RGB; if (ret < 0) goto fail; diff --git a/libavfilter/vf_amf_common.h b/libavfilter/vf_amf_common.h index 0290a52c45..7879403d06 100644 --- a/libavfilter/vf_amf_common.h +++ b/libavfilter/vf_amf_common.h @@ -58,6 +58,7 @@ typedef struct AMFFilterContext { int reset_sar; AMFComponent *component; + AMFComponent *pre_converter; AVBufferRef *amf_device_ref; AVBufferRef *hwframes_in_ref; diff --git a/libavfilter/vf_sr_amf.c b/libavfilter/vf_sr_amf.c index 1758c94ea9..deb020809d 100644 --- a/libavfilter/vf_sr_amf.c +++ b/libavfilter/vf_sr_amf.c @@ -35,6 +35,7 @@ #include "libavutil/hwcontext_amf_internal.h" #include "AMF/components/HQScaler.h" +#include "AMF/components/VideoConverter.h" #include "AMF/components/ColorSpace.h" #include "vf_amf_common.h" @@ -52,12 +53,27 @@ #endif +static enum AVPixelFormat amf_inlink_sw_format(AVFilterLink *inlink) +{ + FilterLink *inl = ff_filter_link(inlink); + + if (inl->hw_frames_ctx) + return ((AVHWFramesContext*)inl->hw_frames_ctx->data)->sw_format; + return inlink->format; +} + static int amf_hq_scaler_needs_packed_rgb(int algorithm) { return algorithm == AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 || algorithm == AMF_HQ_SCALER_ALGORITHM_POINT; } +static int amf_is_packed_rgb(enum AVPixelFormat format) +{ + return format == AV_PIX_FMT_RGBA || format == AV_PIX_FMT_BGRA || + format == AV_PIX_FMT_X2BGR10 || format == AV_PIX_FMT_RGBAF16; +} + static int amf_filter_query_formats(AVFilterContext *avctx) { AMFFilterContext *ctx = avctx->priv; @@ -82,7 +98,7 @@ static int amf_filter_query_formats(AVFilterContext *avctx) AV_PIX_FMT_RGBAF16, AV_PIX_FMT_NONE, }; - // sr1-1 and point produce a blank surface on YUV input + // sr1-1 and point need packed RGB; YUV is converted on the GPU static const enum AVPixelFormat pix_fmts_packed_rgb[] = { AV_PIX_FMT_RGBA, AV_PIX_FMT_BGRA, @@ -113,21 +129,54 @@ static int amf_filter_config_output(AVFilterLink *outlink) AMF_RESULT res; enum AVPixelFormat in_format; enum AMF_MEMORY_TYPE mem_type = AMF_MEMORY_UNKNOWN; + enum AVPixelFormat in_sw_format; + int needs_conversion; + + in_sw_format = amf_inlink_sw_format(inlink); + needs_conversion = amf_hq_scaler_needs_packed_rgb(ctx->algorithm) && + (in_sw_format == AV_PIX_FMT_NV12 || in_sw_format == AV_PIX_FMT_P010); + + if (amf_hq_scaler_needs_packed_rgb(ctx->algorithm)) { + if (!needs_conversion) { + if (ctx->format != AV_PIX_FMT_NONE && ctx->format != in_sw_format) { + av_log(avctx, AV_LOG_ERROR, "The HQ scaler does not convert formats, format must be same or %s.\n", + av_get_pix_fmt_name(in_sw_format)); + return AVERROR(EINVAL); + } + ctx->format = in_sw_format; + } else if (ctx->format == AV_PIX_FMT_NONE) { + ctx->format = in_sw_format == AV_PIX_FMT_P010 ? AV_PIX_FMT_X2BGR10 : AV_PIX_FMT_RGBA; + } else if (!amf_is_packed_rgb(ctx->format)) { + av_log(avctx, AV_LOG_ERROR, "This algorithm only outputs packed RGB, format=%s is not supported.\n", + av_get_pix_fmt_name(ctx->format)); + return AVERROR(EINVAL); + } + } err = amf_init_filter_config(outlink, &in_format); if (err < 0) return err; - if (amf_hq_scaler_needs_packed_rgb(ctx->algorithm) && - (in_format == AV_PIX_FMT_NV12 || in_format == AV_PIX_FMT_P010)) { - av_log(avctx, AV_LOG_ERROR, - "%s requires a packed RGB format (rgba); %s is not supported. " - "Convert the input first (e.g. format=rgba) or select another " - "algorithm.\n", - ctx->algorithm == AMF_HQ_SCALER_ALGORITHM_POINT ? "point" - : "sr1-1 (VideoSR1.1)", - av_get_pix_fmt_name(in_format)); - return AVERROR(EINVAL); + if (needs_conversion) { + AMFSize in_size = { inlink->w, inlink->h }; + + res = ctx->amf_device_ctx->factory->pVtbl->CreateComponent(ctx->amf_device_ctx->factory, ctx->amf_device_ctx->context, AMFVideoConverter, &ctx->pre_converter); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->pre_converter, AMF_VIDEO_CONVERTER_OUTPUT_FORMAT, (amf_int32)av_av_to_amf_format(ctx->format)); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + AMF_ASSIGN_PROPERTY_SIZE(res, ctx->pre_converter, AMF_VIDEO_CONVERTER_OUTPUT_SIZE, in_size); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + AMF_ASSIGN_PROPERTY_INT64(res, ctx->pre_converter, AMF_VIDEO_CONVERTER_OUTPUT_COLOR_RANGE, AMF_COLOR_RANGE_FULL); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + res = ctx->pre_converter->pVtbl->Init(ctx->pre_converter, av_av_to_amf_format(in_format), inlink->w, inlink->h); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, + "AMFConverter-Init() failed with error %d, %s to %s is not supported by this device%s\n", + res, av_get_pix_fmt_name(in_format), av_get_pix_fmt_name(ctx->format), + ctx->format == AV_PIX_FMT_X2BGR10 ? ", try format=rgba" : ""); + + in_format = ctx->format; } // HQ scaler should be used for upscaling only @@ -161,7 +210,7 @@ static int amf_filter_config_output(AVFilterLink *outlink) ctx->in_primaries = AMF_COLOR_PRIMARIES_UNDEFINED; ctx->in_trc = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED; ctx->color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; - ctx->out_color_range = AMF_COLOR_RANGE_UNDEFINED; + ctx->out_color_range = needs_conversion ? AMF_COLOR_RANGE_FULL : AMF_COLOR_RANGE_UNDEFINED; ctx->out_primaries = AMF_COLOR_PRIMARIES_UNDEFINED; ctx->out_trc = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED; -- To stop receiving notification emails like this one, please contact [email protected]. _______________________________________________ ffmpeg-cvslog mailing list -- [email protected] To unsubscribe send an email to [email protected]
