PR #25647 opened by Forgejo_Fairy URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25647 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25647.patch
Chained `overlay_cuda` filters after `scale_cuda` fail when a writable copy inherits the hardware pool’s padded dimensions. For example, a 1920×1080 frame becomes 1920×1088 and the next overlay rejects it. The reported failure starts at f2bf7a2440d1; its immediate parent succeeds. Set the visible width and height returned by `ff_default_get_video_buffer2()` to the requested dimensions without changing the hardware allocation. Add an optional CUDA FATE test with synthetic 158×120 input and two overlays, covering padding in both dimensions. A separate commit removes the overlay’s redundant cached CUDA context and uses the retained device context during module unloading. This fixes the secondary cleanup errors when filtering fails before the first blend callback. Validation on Linux x86-64, RTX 5090, driver 595.99.02: - The original libx264/NVDEC/scale/two-overlay graph produces 90 frames at 1920×1080; 3840×2160 and 1918×1080 also pass. - Single-overlay, aligned-height, direct-upload and forced-scale-after-upload controls pass. - All 90 downloaded frame records match the working direct-upload control, including the unpatched control. - The new FATE test fails before the allocation fix and passes afterward; its reference matches the direct-upload graph with scaling omitted. - All 36 selected FATE tests pass: the new overlay test, existing CUDA scale tests, testsrc2-yuv420p and filter-formats. - Applying only the cleanup correction preserves the original frame-size rejection while eliminating both secondary CUDA context errors. Build instructions, exact commands, revision results and diagnostic logs are preserved on [fairy/issue25598-tooling](https://code.ffmpeg.org/Forgejo_Fairy/FFmpeg/src/branch/fairy/issue25598-tooling/tools/issue25598). Windows and non-CUDA backends were not runtime-tested. No new FATE sample is needed. Fixes #25598. From a73684f5087e542196b3818c42d5a83c2de559c4 Mon Sep 17 00:00:00 2001 From: Michael Niedermayer <[email protected]> Date: Sun, 1 Nov 2026 01:02:53 +0000 Subject: [PATCH 1/2] avfilter/video: preserve requested dimensions in hardware buffers Hardware frame pools may have padded allocation dimensions. Set the returned frame's visible width and height to the requested dimensions, as for software frames, without changing the hardware allocation. This prevents ff_inlink_make_frame_writable() from changing the visible size of a frame from a padded pool. In particular, chained overlay_cuda filters after scale_cuda reject the padded copy since f2bf7a2440d1. Add an optional CUDA FATE test using a synthetic input with both dimensions unaligned to 32. Its reference also matches the direct-upload control without scale_cuda. The test fails before this fix and passes after it. Fixes #25598. Assisted-by: Fairy --- libavfilter/video.c | 7 ++++++- tests/fate/filter-video.mak | 8 ++++++++ tests/ref/fate/filter-overlay_cuda-padded | 10 ++++++++++ 3 files changed, 24 insertions(+), 1 deletion(-) create mode 100644 tests/ref/fate/filter-overlay_cuda-padded diff --git a/libavfilter/video.c b/libavfilter/video.c index 3480c150918..c64097a04cb 100644 --- a/libavfilter/video.c +++ b/libavfilter/video.c @@ -69,8 +69,13 @@ AVFrame *ff_default_get_video_buffer2(AVFilterLink *link, int w, int h, int alig return NULL; ret = av_hwframe_get_buffer(li->l.hw_frames_ctx, frame, 0); - if (ret < 0) + if (ret < 0) { av_frame_free(&frame); + return NULL; + } + + frame->width = w; + frame->height = h; return frame; } diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 95254ff3fcf..73bd0b82e31 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -1107,6 +1107,14 @@ $(FATE_SCALE_CUDA-yes): $(FFMPEG) fate-filter-scale_cuda: $(FATE_SCALE_CUDA-yes) +# A writable copy must keep the visible size of a padded hardware allocation. +# Like scale_cuda above, this test requires a CUDA device and belongs to fate-hw. +FATE_OVERLAY_CUDA-$(call FILTERFRAMECRC, TESTSRC2 COLOR FORMAT HWUPLOAD SCALE_CUDA SPLIT OVERLAY_CUDA HWDOWNLOAD, FFMPEG) += fate-filter-overlay_cuda-padded +fate-filter-overlay_cuda-padded: CMD = framecrc -init_hw_device cuda=cuda -filter_hw_device cuda -lavfi "testsrc2=s=158x120,format=yuv420p,hwupload,scale_cuda=passthrough=0[base];[email protected]:s=40x20,format=yuva420p,hwupload,split=2[a][b];[base][a]overlay_cuda=10:10[tmp];[tmp][b]overlay_cuda=80:80,hwdownload,format=yuv420p" -frames:v 5 +$(FATE_OVERLAY_CUDA-yes): $(FFMPEG) +FATE_HW-yes += $(FATE_OVERLAY_CUDA-yes) +fate-filter-overlay_cuda: $(FATE_OVERLAY_CUDA-yes) + fate-vfilter: $(FATE_FILTER-yes) $(FATE_FILTER_SAMPLES-yes) $(FATE_FILTER_VSYNTH-yes) $(FATE_LIBAVFILTER-yes) fate-filter: fate-afilter fate-vfilter $(FATE_METADATA_FILTER-yes) $(FATE_FILTER_FFPROBE-yes) diff --git a/tests/ref/fate/filter-overlay_cuda-padded b/tests/ref/fate/filter-overlay_cuda-padded new file mode 100644 index 00000000000..c77e6e4f9c6 --- /dev/null +++ b/tests/ref/fate/filter-overlay_cuda-padded @@ -0,0 +1,10 @@ +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 158x120 +#sar 0: 1/1 +0, 0, 0, 1, 28440, 0x0c677ffd +0, 1, 1, 1, 28440, 0x64787885 +0, 2, 2, 1, 28440, 0xf9127fae +0, 3, 3, 1, 28440, 0x6aae73e8 +0, 4, 4, 1, 28440, 0x6b0277a4 -- 2.54.0 From 374830609012e6dfb05a1f908aaeb590efaac841 Mon Sep 17 00:00:00 2001 From: Michael Niedermayer <[email protected]> Date: Sun, 1 Nov 2026 01:03:31 +0000 Subject: [PATCH 2/2] avfilter/overlay_cuda: use the device context when unloading the module The cached cu_ctx field is assigned only when overlay_cuda_blend() runs. If filtering fails before the first frame reaches an overlay, uninit instead tries to push a null CUDA context and emits secondary context errors. The second overlay in #25598 exercises this path. Use the context from the retained hardware device directly, as already done during module loading, and remove the redundant cached field. Tested independently with the original allocation bug still present: the graph still reports the frame-size error, but no longer emits the cuCtxPushCurrent/cuCtxPopCurrent failures during cleanup. With the allocation fix applied, the original 90-frame graph and CUDA FATE pass. Assisted-by: Fairy --- libavfilter/vf_overlay_cuda.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/libavfilter/vf_overlay_cuda.c b/libavfilter/vf_overlay_cuda.c index 3d39458f3fd..7201ce2fc12 100644 --- a/libavfilter/vf_overlay_cuda.c +++ b/libavfilter/vf_overlay_cuda.c @@ -101,7 +101,6 @@ typedef struct OverlayCUDAContext { AVBufferRef *hw_device_ctx; AVCUDADeviceContext *hwctx; - CUcontext cu_ctx; CUmodule cu_module; CUfunction cu_func; CUstream cu_stream; @@ -236,8 +235,6 @@ static int overlay_cuda_blend(FFFrameSync *fs) AVFrame *input_main, *input_overlay; - ctx->cu_ctx = cuda_ctx; - // read main and overlay frames from inputs ret = ff_framesync_dualinput_get(fs, &input_main, &input_overlay); if (ret < 0) @@ -388,7 +385,7 @@ static av_cold void overlay_cuda_uninit(AVFilterContext *avctx) if (ctx->hwctx && ctx->cu_module) { CUcontext dummy; CudaFunctions *cu = ctx->hwctx->internal->cuda_dl; - CHECK_CU(cu->cuCtxPushCurrent(ctx->cu_ctx)); + CHECK_CU(cu->cuCtxPushCurrent(ctx->hwctx->cuda_ctx)); CHECK_CU(cu->cuModuleUnload(ctx->cu_module)); CHECK_CU(cu->cuCtxPopCurrent(&dummy)); } -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
