PR #25421 opened by Marcos Ashton (MarcosAsh) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25421 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25421.patch
Three fixes to the Vulkan decoder's profile query. The main one makes AV1 decoding work on drivers without film grain support. filmGrainSupport is optional, but vulkan_decode asked for it unless film grain was exported. ANV on my Meteor Lake laptop doesn't support it, so by default every AV1 stream failed with `av1 profile "Main" not supported`, even without film grain. The query is now retried without film grain when the stream has none. The unused external_fg field is set from the session profile and decides whether the hardware applies grain. If film grain turns on later in a stream, there's a one-time warning. Streams that start with film grain still fail on these drivers, now with a warning that -export_side_data film_grain exports the grain instead of applying it. ANV reports the missing film grain support as OPERATION_NOT_SUPPORTED rather than CODEC_NOT_SUPPORTED, so the retry accepts both. The allow_profile_mismatch fallback tested avctx->flags instead of avctx->hwaccel_flags, so it was really controlled by -flags +mv4, and it missed CODEC_NOT_SUPPORTED. On ANV, an HEVC stream with an unsupported profile now falls back to Main with -hwaccel_flags allow_profile_mismatch and matches software, where before only -flags +mv4 did that. Both retries now run in one loop, the film grain one first since it keeps the profile. The last commit stops passing NULL to %s when a profile has no name; master already prints `hevc profile "(null)" not supported!` for some HEVC RExt streams. On ANV (Meteor Lake, Mesa 26.0.8, ANV_DEBUG=video-decode), allintra, sizedown, cdfupdate, mv, mfmv and svc-L1T2 from av1-test-vectors now match libdav1d. svc-L2T1 and svc-L2T2 still differ there but match on NVIDIA, so that looks like a separate ANV issue. sizeup needs #24980: without it the frames grow past the frames context and the download reads past the end, which crashes on ANV now that AV1 decodes there. With #24980 on top, sizeup matches libdav1d too, so #24980 should go in first. Both film_grain-50 streams match libdav1d without grain when film grain is exported. On an RTX 5070 Ti (NVIDIA 595.91.07), which supports film grain, all 131 hardware decode tests of my nightly GPU runner (H.264, HEVC, VP9 and AV1 conformance streams against software, on both NVDEC and Vulkan) give the same results as master. >From 8dd15dcfd10bccc0150b3a5309978144c539fe8c Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Sat, 24 Oct 2026 15:25:20 +0100 Subject: [PATCH 1/3] avcodec/vulkan_decode: fix the allow_profile_mismatch fallback It tested avctx->flags instead of avctx->hwaccel_flags and missed VK_ERROR_VIDEO_PROFILE_CODEC_NOT_SUPPORTED_KHR, so -hwaccel_flags allow_profile_mismatch had no effect. --- libavcodec/vulkan_decode.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c index 5cf338b0d35..ddaba355efa 100644 --- a/libavcodec/vulkan_decode.c +++ b/libavcodec/vulkan_decode.c @@ -893,8 +893,9 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ caps, dec_caps, cur_profile); - if (ret == VK_ERROR_VIDEO_PROFILE_OPERATION_NOT_SUPPORTED_KHR && - avctx->flags & AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH && + if ((ret == VK_ERROR_VIDEO_PROFILE_OPERATION_NOT_SUPPORTED_KHR || + ret == VK_ERROR_VIDEO_PROFILE_CODEC_NOT_SUPPORTED_KHR) && + avctx->hwaccel_flags & AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH && avctx->profile != base_profile) { av_log(avctx, AV_LOG_VERBOSE, "%s profile %s not supported, attempting " "again with profile %s\n", -- 2.54.0 >From 7f858f3c26854d98f610b47c294c05230bfded49 Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Fri, 16 Oct 2026 16:08:45 +0100 Subject: [PATCH 2/3] avcodec/vulkan_decode: retry the AV1 profile query without film grain filmGrainSupport was requested unless film grain was exported, so drivers without it, like ANV, rejected every AV1 stream by default. --- libavcodec/vulkan_av1.c | 8 ++++- libavcodec/vulkan_decode.c | 60 +++++++++++++++++++++++++------------- libavcodec/vulkan_decode.h | 3 +- 3 files changed, 48 insertions(+), 23 deletions(-) diff --git a/libavcodec/vulkan_av1.c b/libavcodec/vulkan_av1.c index 99bdac46e58..6705cac6a31 100644 --- a/libavcodec/vulkan_av1.c +++ b/libavcodec/vulkan_av1.c @@ -265,12 +265,18 @@ static int vk_av1_start_frame(AVCodecContext *avctx, const AV1RawFilmGrainParams *film_grain = &s->cur_frame.film_grain; const int apply_grain = !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) && - film_grain->apply_grain; + !dec->external_fg && film_grain->apply_grain; StdVideoAV1FrameRestorationType remap_lr_type[4] = { STD_VIDEO_AV1_FRAME_RESTORATION_TYPE_NONE, STD_VIDEO_AV1_FRAME_RESTORATION_TYPE_SWITCHABLE, STD_VIDEO_AV1_FRAME_RESTORATION_TYPE_WIENER, STD_VIDEO_AV1_FRAME_RESTORATION_TYPE_SGRPROJ }; + if (film_grain->apply_grain && dec->external_fg && + !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN)) + av_log_once(avctx, AV_LOG_WARNING, AV_LOG_DEBUG, &dec->fg_warned, + "Film grain is not applied, the decode session was created " + "without film grain support\n"); + /* Use the current frame_ids in ref[] to decide occupied frame_ids */ for (int i = 0; i < STD_VIDEO_AV1_NUM_REF_FRAMES; i++) { const AV1VulkanDecodePicture* rp = s->ref[i].hwaccel_picture_private; diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c index ddaba355efa..52cc9cecb49 100644 --- a/libavcodec/vulkan_decode.c +++ b/libavcodec/vulkan_decode.c @@ -24,6 +24,7 @@ #include "libavutil/avassert.h" #include "libavutil/mem.h" #include "libavutil/vulkan_loader.h" +#include "av1dec.h" #include "h264dec.h" #define DECODER_IS_SDR(codec_id) \ @@ -753,6 +754,7 @@ static VkResult vulkan_setup_profile(AVCodecContext *avctx, VkVideoDecodeVP9ProfileInfoKHR *vp9_profile = &prof->vp9_profile; #endif VkVideoDecodeAV1ProfileInfoKHR *av1_profile = &prof->av1_profile; + const FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(avctx->sw_pix_fmt); if (!desc) @@ -791,7 +793,8 @@ static VkResult vulkan_setup_profile(AVCodecContext *avctx, usage->pNext = av1_profile; av1_profile->sType = VK_STRUCTURE_TYPE_VIDEO_DECODE_AV1_PROFILE_INFO_KHR; av1_profile->stdProfile = cur_profile; - av1_profile->filmGrainSupport = !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN); + av1_profile->filmGrainSupport = !dec->external_fg && + !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN); } usage->sType = VK_STRUCTURE_TYPE_VIDEO_DECODE_USAGE_INFO_KHR; @@ -883,26 +886,7 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ avctx->codec_id == AV_CODEC_ID_AV1 ? STD_VIDEO_AV1_PROFILE_MAIN : 0; - ret = vulkan_setup_profile(avctx, prof, hwctx, vk, vk_desc, - &h264_caps, - &h265_caps, -#if CONFIG_VP9_VULKAN_HWACCEL - &vp9_caps, -#endif - &av1_caps, - caps, - dec_caps, - cur_profile); - if ((ret == VK_ERROR_VIDEO_PROFILE_OPERATION_NOT_SUPPORTED_KHR || - ret == VK_ERROR_VIDEO_PROFILE_CODEC_NOT_SUPPORTED_KHR) && - avctx->hwaccel_flags & AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH && - avctx->profile != base_profile) { - av_log(avctx, AV_LOG_VERBOSE, "%s profile %s not supported, attempting " - "again with profile %s\n", - avcodec_get_name(avctx->codec_id), - avcodec_profile_name(avctx->codec_id, cur_profile), - avcodec_profile_name(avctx->codec_id, base_profile)); - cur_profile = base_profile; + while (1) { ret = vulkan_setup_profile(avctx, prof, hwctx, vk, vk_desc, &h264_caps, &h265_caps, @@ -913,6 +897,30 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ caps, dec_caps, cur_profile); + if (ret != VK_ERROR_VIDEO_PROFILE_OPERATION_NOT_SUPPORTED_KHR && + ret != VK_ERROR_VIDEO_PROFILE_CODEC_NOT_SUPPORTED_KHR) + break; + + /* Film grain support is optional, try without it if the stream has none. + * ANV reports it as OPERATION_NOT_SUPPORTED, not CODEC_NOT_SUPPORTED. */ + if (avctx->codec_id == AV_CODEC_ID_AV1 && prof->av1_profile.filmGrainSupport && + !((const AV1DecContext *)avctx->priv_data)->raw_seq->film_grain_params_present) { + av_log(avctx, AV_LOG_VERBOSE, "%s profile %s not supported, " + "attempting again without film grain\n", + avcodec_get_name(avctx->codec_id), + avcodec_profile_name(avctx->codec_id, cur_profile)); + dec->external_fg = 1; + } else if (avctx->hwaccel_flags & AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH && + cur_profile != base_profile) { + av_log(avctx, AV_LOG_VERBOSE, "%s profile %s not supported, attempting " + "again with profile %s\n", + avcodec_get_name(avctx->codec_id), + avcodec_profile_name(avctx->codec_id, cur_profile), + avcodec_profile_name(avctx->codec_id, base_profile)); + cur_profile = base_profile; + } else { + break; + } } if (ret == VK_ERROR_VIDEO_PROFILE_OPERATION_NOT_SUPPORTED_KHR || @@ -921,6 +929,10 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ "%s profile \"%s\" not supported!\n", avcodec_get_name(avctx->codec_id), avcodec_profile_name(avctx->codec_id, cur_profile)); + if (avctx->codec_id == AV_CODEC_ID_AV1 && prof->av1_profile.filmGrainSupport) + av_log(avctx, AV_LOG_WARNING, "The stream uses film grain, which the " + "device may not support; -export_side_data film_grain exports " + "it instead of applying it\n"); return AVERROR(EINVAL); } else if (ret == VK_ERROR_VIDEO_PICTURE_LAYOUT_NOT_SUPPORTED_KHR) { if (avctx->codec_id == AV_CODEC_ID_H264) @@ -1298,6 +1310,12 @@ int ff_vk_decode_init(AVCodecContext *avctx) return AVERROR(EINVAL); } + if (avctx->codec_id == AV_CODEC_ID_AV1) { + const VkVideoDecodeAV1ProfileInfoKHR *av1_profile = + ff_vk_find_struct(profile->pNext, VK_STRUCTURE_TYPE_VIDEO_DECODE_AV1_PROFILE_INFO_KHR); + dec->external_fg = !av1_profile || !av1_profile->filmGrainSupport; + } + /* Create queue context */ vk_desc = get_codecdesc(avctx->codec_id); ctx->qf = ff_vk_qf_find(s, vk_desc->queue_flags, vk_desc->decode_op); diff --git a/libavcodec/vulkan_decode.h b/libavcodec/vulkan_decode.h index eb334ea1aef..aa2c7de429d 100644 --- a/libavcodec/vulkan_decode.h +++ b/libavcodec/vulkan_decode.h @@ -56,7 +56,8 @@ typedef struct FFVulkanDecodeContext { VkVideoSessionParametersKHR *session_params; int dedicated_dpb; /* Oddity #1 - separate DPB images */ - int external_fg; /* Oddity #2 - hardware can't apply film grain */ + int external_fg; /* Oddity #2 - the session can't apply film grain */ + int fg_warned; /* Workaround for NVIDIA drivers tested with CTS version 1.3.8 for AV1. * The tests were incorrect as the OrderHints were offset by 1. */ -- 2.54.0 >From 68a5fed566d0ea70efd74914232e3880cac10dd7 Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Sat, 24 Oct 2026 16:05:53 +0100 Subject: [PATCH 3/3] avcodec/vulkan_decode: do not pass NULL profile names to av_log avcodec_profile_name() returns NULL for profiles it has no name for. --- libavcodec/vulkan_decode.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c index 52cc9cecb49..34ca3c6399b 100644 --- a/libavcodec/vulkan_decode.c +++ b/libavcodec/vulkan_decode.c @@ -827,6 +827,11 @@ static VkResult vulkan_setup_profile(AVCodecContext *avctx, caps); } +static const char *profile_name(enum AVCodecID codec_id, int profile) +{ + return av_x_if_null(avcodec_profile_name(codec_id, profile), "unknown"); +} + static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ref, enum AVPixelFormat *pix_fmt, VkFormat *vk_fmt, FFVulkanDecodeProfileData *prof, @@ -908,15 +913,15 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ av_log(avctx, AV_LOG_VERBOSE, "%s profile %s not supported, " "attempting again without film grain\n", avcodec_get_name(avctx->codec_id), - avcodec_profile_name(avctx->codec_id, cur_profile)); + profile_name(avctx->codec_id, cur_profile)); dec->external_fg = 1; } else if (avctx->hwaccel_flags & AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH && cur_profile != base_profile) { av_log(avctx, AV_LOG_VERBOSE, "%s profile %s not supported, attempting " "again with profile %s\n", avcodec_get_name(avctx->codec_id), - avcodec_profile_name(avctx->codec_id, cur_profile), - avcodec_profile_name(avctx->codec_id, base_profile)); + profile_name(avctx->codec_id, cur_profile), + profile_name(avctx->codec_id, base_profile)); cur_profile = base_profile; } else { break; @@ -928,7 +933,7 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ av_log(avctx, AV_LOG_VERBOSE, "Unable to initialize video session: " "%s profile \"%s\" not supported!\n", avcodec_get_name(avctx->codec_id), - avcodec_profile_name(avctx->codec_id, cur_profile)); + profile_name(avctx->codec_id, cur_profile)); if (avctx->codec_id == AV_CODEC_ID_AV1 && prof->av1_profile.filmGrainSupport) av_log(avctx, AV_LOG_WARNING, "The stream uses film grain, which the " "device may not support; -export_side_data film_grain exports " @@ -965,7 +970,7 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ av_log(avctx, AV_LOG_VERBOSE, "Decoder capabilities for %s profile \"%s\":\n", avcodec_get_name(avctx->codec_id), - avcodec_profile_name(avctx->codec_id, cur_profile)); + profile_name(avctx->codec_id, cur_profile)); av_log(avctx, AV_LOG_VERBOSE, " Maximum level: %i (stream %i)\n", max_level, avctx->level); av_log(avctx, AV_LOG_VERBOSE, " Width: from %i to %i\n", -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
