PR #24679 opened by ArgaAAL URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24679 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24679.patch
# Summary of changes Set the current length of the media buffer wrapping a D3D11 input surface before adding it to the sample. On the tested Qualcomm H.264 Media Foundation encoder, the unpatched D3D11 path fails in `ProcessInput` with `0x80070057` (`E_INVALIDARG`), while system-memory input works. The patch queries `IMFMediaBuffer_GetMaxLength` and passes that length to `IMFMediaBuffer_SetCurrentLength`. If either call fails, it releases both the buffer and sample and returns an FFmpeg error. The successful ownership flow and system-memory path are unchanged. This is one file, 21 added lines. This follows [Chromium's existing Qualcomm interoperability approach](https://chromium.googlesource.com/chromium/src/media/+/4d9f97945a34124faa4cd745dd6b1a7e015ef549/gpu/windows/media_foundation_video_encode_accelerator_win.cc#1147), not a claim to have invented that technique or Windows ARM64 encoding support. Related user reports include [Sunshine #4732](https://github.com/LizardByte/Sunshine/issues/4732). Buffer capacity is not asserted to equal valid image size for every format or vendor; the evidence here is the measured interoperability result and precedent. ## Tested source and environment The same one-file change is submitted on `9058b3622ebbcfb820712c7da97a434b3cd1da7c`. The runtime results below remain pinned to the tested source; they are not a new run on that newer parent. - Matched unpatched/patched source: FFmpeg `bf9cb9f396c3f2afa4a1538e822495a7dae43217`; only `libavcodec/mfenc.c` changes. - Device: Xiaomi Pad 5 / Nabu, Snapdragon 860 / Adreno 640. - Windows 11 ARM64 build 22631, graphics driver `27.20.2060.0`. - Encoder builds: native ARM64 Clang 22.1.8, `aarch64-w64-windows-gnu`. - Independent decoder/probe: native ARM64 full FFmpeg build `N-126390-g9fc8c785e2-20260902`, not the minimal encoder-only build. - Input: generated moving `testsrc2`, 2560x1600 at 60 fps, 120 frames, NV12 / H.264 High, GOP 60, requested CBR 20 Mbps, `display_remoting`. The following hashes identify the actual tested tools, not reproducible-build guarantees for another compiler/package installation: | Tool | SHA256 | | --- | --- | | Unpatched encoder | `EDB95AE3CC6B5E3341E0958B29AECE88793DB9000BCC56695FCF2A4A3D156E7E` | | Patched encoder | `2A94679A97AB9CBEDA67DCB1C6DEED42675399EFB410BD6E9B013347CA85B3B4` | | Independent CPU decoder | `1E35E8926145D94730BD206C17AD1977B0B2C0884FD245BBC1178361C9AC7852` | | Independent ffprobe | `EB1F87AE03DD42E8142907F4517AC1D2CBB7E187EA5DD43C903B916C82B39BE1` | ## Results The four-case 20 Mbps test was run twice with the same result. Every case selected `QCOM Hardware Encoder - H264`; no process timed out. | Encoder / input | Encode exit | Packets | Empty packets | CPU-decoded frames | Output bytes | | --- | ---: | ---: | ---: | ---: | ---: | | Unpatched / system memory | 0 | 120 | 0 | 120 | 4748674 | | Unpatched / D3D11 | -542398533 | 0 | 0 | Not decoded | 0 | | Patched / system memory | 0 | 120 | 0 | 120 | 4748674 | | Patched / D3D11 | 0 | 120 | 0 | 120 | 4748674 | The negative control reports `ProcessInput` error `0x80070057`. The three working paths produce byte-identical streams, SHA256 `B15018C86E73E289772D16FCFA046A85BD41A4D62CEB20451D1D10C9EA1B77E5`. Each independent probe reports H.264, 2560x1600, yuv420p and 120 decoded frames; each framehash file contains 120 distinct decoded-frame hashes. Output hashes are observations on this device/driver, not expected constants for every system. At requested **1 Mbps**, the same moving-input test fails the full-frame requirement: all three working paths emit 120 packets, but 59 are zero-byte and only 61 pictures decode. This also occurs with unpatched system-memory input; the patch does not introduce or fix that separate limitation. Packet counts alone are therefore not treated as proof of complete decoded output. Additional controls on the same binaries found that changing only the scenario to `camera_record` restored 120 frames at requested 1 Mbps. Raising mean and maximum to 20 Mbps with the original buffer also passed, as did 1366x768 at 20 Mbps. Maximum-only, buffer-only and paced-input 1 Mbps controls still failed the strict 120-frame requirement. These are diagnostic observations, not a recommendation to change Sunshine defaults. Requested driver settings do not establish sustained bitrate compliance or latency; those were not measured. Seven selected sample-free CPU FATE tests passed on the patched ARM64 build: `imgutils`, `pixfmt_best`, `avpacket`, `libavcodec-avcodec`, `side_data_array`, `sws-pixdesc-query`, and `sws-floatimg-cmp`. This was not the complete FATE suite. The build retained an existing `dxva2.c` warning. `tools/patcheck` advisories about the COM output variable, error macro and missing changelog entry were reviewed; no warning-free-build claim is made. ## Reproduction: matched encoder builds Use an otherwise clean checkout of the tested source above in an MSYS2 CLANGARM64 shell. The recorded compiler target was `aarch64-w64-windows-gnu`. Configure and build the baseline: ```sh ./configure \ --target-os=mingw32 --arch=aarch64 --cc=clang --cxx=clang++ \ --disable-autodetect --disable-everything --disable-debug --disable-doc \ --disable-ffplay --disable-ffprobe --disable-network --disable-iconv \ --disable-swresample --disable-pthreads --enable-w32threads \ --enable-static --disable-shared --enable-ffmpeg \ --enable-d3d11va --enable-mediafoundation \ --enable-decoder=wrapped_avframe --enable-encoder=h264_mf \ --enable-indev=lavfi --enable-muxer=null,h264 --enable-protocol=file \ --enable-filter=color,testsrc2,format,hwupload,scale make -j4 ffmpeg.exe ``` Save that executable separately, apply this PR's one-file patch, and rebuild with `make -j4 ffmpeg.exe` without changing the configuration or compiler. Keep both executables. The minimal build has neither an H.264 decoder nor ffprobe; use a separate full build for independent decoding/probing. ## Reproduction: four encode controls and independent decode Run only on an idle device, with no active streaming session. The PowerShell templates below preserve the recorded executable arguments with sanitized paths. The variable setup, redirection and inspection commands are readable equivalents of the recording harness, not a claim that this newly written block was itself the historical test script. They do not implement the harness's timeout guards; a stalled run must be stopped and reported as a timeout, never counted as a pass. Set `$Encoder` to the selected unpatched/patched executable. Set `$Decoder` and `$Probe` to the independent full build's `ffmpeg.exe` and `ffprobe.exe`. Set `$OutputDirectory` to a new, empty directory you own; the output names must not already exist. Record each tool's `-version` and SHA256. Run exactly these cases: | `$Name` | `$Encoder` | Command template | | --- | --- | --- | | `unpatched-ram` | Baseline executable | System memory | | `unpatched-d3d11` | Baseline executable | D3D11 | | `patched-ram` | Patched executable | System memory | | `patched-d3d11` | Patched executable | D3D11 | System memory: ```powershell & $Encoder -hide_banner -nostdin -nostats -loglevel verbose -n -f lavfi -i 'testsrc2=size=2560x1600:rate=60' -frames:v 120 -vf 'format=nv12' -c:v h264_mf -hw_encoding 1 -rate_control cbr -scenario display_remoting -b:v 20000000 -maxrate 20000000 -bufsize 20000000 -g 60 -profile:v 100 -pix_fmt nv12 -stats_enc_post "$OutputDirectory/$Name-packet-sizes.txt" -stats_enc_post_fmt '{n} {pts} {dts} {size}' -f h264 "$OutputDirectory/$Name.h264" 1> "$OutputDirectory/$Name-encode.stdout.txt" 2> "$OutputDirectory/$Name-encode.stderr.txt" $encodeExit = $LASTEXITCODE ``` D3D11 input: ```powershell & $Encoder -hide_banner -nostdin -nostats -loglevel verbose -n -init_hw_device d3d11va=dev -filter_hw_device dev -f lavfi -i 'testsrc2=size=2560x1600:rate=60' -frames:v 120 -vf 'format=nv12,hwupload' -c:v h264_mf -hw_encoding 1 -rate_control cbr -scenario display_remoting -b:v 20000000 -maxrate 20000000 -bufsize 20000000 -g 60 -profile:v 100 -pix_fmt d3d11 -stats_enc_post "$OutputDirectory/$Name-packet-sizes.txt" -stats_enc_post_fmt '{n} {pts} {dts} {size}' -f h264 "$OutputDirectory/$Name.h264" 1> "$OutputDirectory/$Name-encode.stdout.txt" 2> "$OutputDirectory/$Name-encode.stderr.txt" $encodeExit = $LASTEXITCODE ``` The unpatched D3D11 case must select the Qualcomm encoder and fail specifically in `ProcessInput` with `0x80070057`, producing zero packets/output bytes. A missing encoder, crash or timeout is not the expected negative control. For each of the other three cases, require encode exit zero and verify that the log uniquely selects `QCOM Hardware Encoder - H264`; `-hw_encoding 1` alone does not prove the vendor selected. Probe and CPU-decode each successful file: ```powershell & $Probe -v error -err_detect explode -threads 2 -select_streams v:0 -count_frames -show_entries 'stream=codec_name,width,height,pix_fmt,nb_read_frames' -of json "$OutputDirectory/$Name.h264" 1> "$OutputDirectory/$Name-probe.json" 2> "$OutputDirectory/$Name-probe.stderr.txt" $probeExit = $LASTEXITCODE & $Decoder -hide_banner -nostdin -v error -xerror -err_detect explode -hwaccel none -c:v h264 -threads 2 -i "$OutputDirectory/$Name.h264" -map 0:v:0 -an -fps_mode passthrough -f framehash -hash sha256 -n "$OutputDirectory/$Name.framehash" 1> "$OutputDirectory/$Name-decode.stdout.txt" 2> "$OutputDirectory/$Name-decode.stderr.txt" $decodeExit = $LASTEXITCODE ``` Require both exits zero and empty probe/decode error logs. In each probe JSON, check the codec, dimensions and 120 decoded frames. Count framehash data lines (excluding `#` comments/blank lines) and compare their final SHA256 fields; the recorded result has 120 frames and 120 distinct hashes. In each packet-size file, require 120 records, all sizes greater than zero, with the sum of the fourth field equal to that H.264 file's length. Compare all three working-path output hashes and the actual bytes: ```powershell Get-FileHash -Algorithm SHA256 -LiteralPath "$OutputDirectory/unpatched-ram.h264", "$OutputDirectory/patched-ram.h264", "$OutputDirectory/patched-d3d11.h264" & "$env:SystemRoot/System32/fc.exe" /b "$OutputDirectory/unpatched-ram.h264" "$OutputDirectory/patched-ram.h264" if ($LASTEXITCODE -ne 0) { throw 'RAM output differs.' } & "$env:SystemRoot/System32/fc.exe" /b "$OutputDirectory/unpatched-ram.h264" "$OutputDirectory/patched-d3d11.h264" if ($LASTEXITCODE -ne 0) { throw 'D3D11 output differs.' } ``` Use another fresh output directory to repeat at requested 1 Mbps by setting all three options `-b:v`, `-maxrate` and `-bufsize` to `1000000`. Retain the observed failed full-frame result rather than accepting 120 packets as success. Other hardware/drivers may behave differently. ## Scope and disclosure This supports a bounded H.264 NV12 interoperability correction on one device/driver. It does not establish HEVC, HDR/P010, other vendors/Snapdragon generations, visual fidelity, latency, real-time throughput, power savings, zero-copy operation or complete Sunshine streaming compatibility. A portable hardware regression test is not supplied; hosted CPU runners lack this driver/MFT. The proposed change, testing analysis and this submission text were developed with substantial AI assistance. The draft has not received human code review. Its `Signed-off-by` trailer was explicitly authorized by the contributor; it is not a `Reviewed-by` or `Tested-by` certification. <!-- If this PR requires new FATE test samples, attach them to the PR and list their target paths below (relative to the fate-suite root). Attached filenames must match the sample's filename: ```fate-samples # e.g. vorbis/new-sample.ogg ``` --> >From 0f6c9afb45fa1110ac312137fcb6806e3afe0225 Mon Sep 17 00:00:00 2001 From: ArgaAAL <[email protected]> Date: Thu, 24 Sep 2026 19:37:34 +0000 Subject: [PATCH] avcodec/mfenc: set the current length of DXGI input buffers Set the current length of the media buffer wrapping each D3D11 input surface before adding it to the sample. On the tested Qualcomm H.264 Media Foundation encoder, the existing path fails in ProcessInput with E_INVALIDARG, while the system-memory input path succeeds. Query the surface buffer's maximum length and use it as its current length, matching Chromium's approach for Qualcomm encoder interoperability: https://chromium.googlesource.com/chromium/src/media/+/4d9f97945a34124faa4cd745dd6b1a7e015ef549/gpu/windows/media_foundation_video_encode_accelerator_win.cc#1147 Release the buffer and sample if either operation fails. The successful ownership flow and the system-memory input path are unchanged. Matched H.264 NV12 builds on bf9cb9f396c3 were tested on a Snapdragon 860 Windows ARM64 device, using moving 2560x1600 input at 60 fps and requested CBR 20 Mbps with the Qualcomm hardware transform required. Unpatched D3D11 input failed with 0x80070057 and no output. Patched D3D11 and both system-memory controls produced byte-identical streams with 120 nonempty packets and 120 independently CPU-decoded frames. The four-case test was repeated with the same result. Seven selected sample-free FATE tests passed on the patched ARM64 build; this was not the full FATE suite. At requested 1 Mbps, all three working paths produced 59 empty packets and only 61 decoded frames, including unpatched system-memory input. That separate limitation is not fixed by this change. The proposed change and its test analysis were developed with substantial AI assistance. This draft has not received human code review. Signed-off-by: ArgaAAL <[email protected]> --- libavcodec/mfenc.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/libavcodec/mfenc.c b/libavcodec/mfenc.c index db3c1e10a3..27476bda6b 100644 --- a/libavcodec/mfenc.c +++ b/libavcodec/mfenc.c @@ -365,6 +365,7 @@ static int process_d3d11_frame(AVCodecContext *avctx, const AVFrame *frame, IMFS ID3D11Texture2D *d3d11_texture = NULL; IMFSample *sample = NULL; IMFMediaBuffer *buffer = NULL; + DWORD buffer_length; int subIdx = 0; HRESULT hr; @@ -399,6 +400,26 @@ static int process_d3d11_frame(AVCodecContext *avctx, const AVFrame *frame, IMFS return AVERROR_EXTERNAL; } + hr = IMFMediaBuffer_GetMaxLength(buffer, &buffer_length); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, + "Failed to get DXGI surface buffer length: %s\n", + ff_hr_str(hr)); + IMFMediaBuffer_Release(buffer); + IMFSample_Release(sample); + return AVERROR_EXTERNAL; + } + + hr = IMFMediaBuffer_SetCurrentLength(buffer, buffer_length); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, + "Failed to set DXGI surface buffer length: %s\n", + ff_hr_str(hr)); + IMFMediaBuffer_Release(buffer); + IMFSample_Release(sample); + return AVERROR_EXTERNAL; + } + hr = IMFSample_AddBuffer(sample, buffer); if (FAILED(hr)) { av_log(avctx, AV_LOG_ERROR, "Failed to add buffer to sample: %s\n", ff_hr_str(hr)); -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
