cuvidDecodePicture() only reports errors that prevent a picture from being submitted. Errors detected by the hardware afterwards, including concealed ones, went unreported, so callers could not distinguish a corrupted frame from a clean one the way they can with software decoding.
cuvidMapVideoFrame() blocks until decoding of the picture is complete, per the NVDEC Video Decoder API Programming Guide. Query cuvidGetDecodeStatus() after it returns and map cuvidDecodeStatus_Error to FF_DECODE_ERROR_INVALID_BITSTREAM and cuvidDecodeStatus_Error_Concealed to FF_DECODE_ERROR_CONCEALMENT_ACTIVE. On the tested driver the status is final for H.264 and HEVC. JPEG can still report InProgress after the map; those frames are left unflagged. CUarray output has no equivalent blocking call, so it is not covered. cuvidGetDecodeStatus() is documented for H.264, HEVC and JPEG only, but the tested driver also returns a final status for AV1, including film grain. For unsupported codecs or GPUs, cuvidGetDecodeStatus() returns CUDA_ERROR_NOT_SUPPORTED; the result is cached so that an unsupported decoder only pays for one failed call. Only errors for the output picture are reported: reference-picture errors are not propagated, and FF_DECODE_ERROR_DECODE_SLICES is not set. ffmpeg now logs "corrupt decoded frame" and -xerror fails on such frames, as with software decoding. Signed-off-by: Qiaobin Lin <[email protected]> --- Tested on RTX 4070 (driver 570.124.04). - 30 s clean 1920x1080 H.264 over RTSP/TCP: 894 frames in this run, no flags, 0 decode errors. - Corrupted H.264 and HEVC: NVDEC and the software decoder flag the same PTS with FF_DECODE_ERROR_CONCEALMENT_ACTIVE; software also sets FF_DECODE_ERROR_DECODE_SLICES on the multi-slice sample. For HEVC, software did not flag the corrupted frame. - Low-delay H.264/HEVC with one thread: no InProgress, corrupt frames still flagged. Same for H.264 with B-frames. - MPEG-1/2/4, VP8 and VP9: CUDA_ERROR_NOT_SUPPORTED, one log per decoder. VC-1 not tested (no local encoder or sample). - AV1: Success on clean, status 9 on corrupt. A film grain sample (aomenc --film-grain-test=1) has apply_grain=1 and idx != ref_idx; status 9 is still returned for cf->idx. - JPEG: InProgress on clean samples; a corrupt frame is flagged only when a final status is available. Query is best-effort, InProgress logged once; dpb_size=1 likely explains the lag. - UDP/RTP loss: two concealed frames flagged; AV_FRAME_FLAG_CORRUPT unset. - unsafe_output + MPEG-2: unsupported, logged once. A corrupt IDR slice header fails in cuvidDecodePicture and produces no frame. - -xerror: AVERROR_INVALIDDATA on corrupt, 0 on clean. cuvidGetDecodeStatus is already a required LOAD_SYMBOL in the ffnvcodec loader; the existing configure dependency covers it, so there is no new runtime dependency. The query is not gated on err_recognition: hardware and software decoding should set these flags under the same conditions. It is one driver call per output picture in the existing map context, with no extra synchronization. H.264/HEVC samples were generated with libx264/libx265; corruption was injected into mdat or by removing a slice NAL. The AV1 Y4M input was generated with ffmpeg. Reproduction: ffmpeg -rtsp_transport tcp -hwaccel cuda -i <rtsp-url> -t 30 -f null - ffmpeg -hwaccel cuda -i corrupt.mp4 -f null - ffmpeg -hwaccel cuda -flags +low_delay -threads 1 -i corrupt.h264 -f null - ffmpeg -xerror -hwaccel cuda -i corrupt.mp4 -f null - aomenc --film-grain-test=1 --passes=1 --cpu-used=8 --end-usage=q \ --cq-level=40 --limit=50 -o grain.ivf input.y4m Status 8 was not observed; the mapping follows the SDK enum. Legacy cuvid is not covered either. No FATE test is possible, as NVDEC requires NVIDIA hardware. libavcodec/nvdec.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c index 3f5c7ea122..26afd0dad5 100644 --- a/libavcodec/nvdec.c +++ b/libavcodec/nvdec.c @@ -46,6 +46,9 @@ #define cudaVideoSurfaceFormat_YUV444_16Bit 3 #endif +/* The CUresult enum in nv-codec-headers lacks this value. */ +#define NVDEC_CUDA_ERROR_NOT_SUPPORTED 801 + typedef struct NVDECDecoder { CUvideodecoder decoder; @@ -58,8 +61,16 @@ typedef struct NVDECDecoder { CudaFunctions *cudl; CuvidFunctions *cvdl; + /* Cleared if cuvidGetDecodeStatus reports the codec/GPU as + * unsupported. */ + int decode_status_supported; + + /* Log InProgress once; JPEG can keep reporting it. */ + int in_progress_logged; + int unsafe_output; int opaque_output; + #ifdef NVDEC_HAVE_OPAQUE_OUTPUT_SUPPORT atomic_int *surface_in_use; int num_surfaces; @@ -235,6 +246,8 @@ static int nvdec_decoder_create(NVDECDecoder **out, AVBufferRef *hw_device_ref, if (!decoder) return AVERROR(ENOMEM); + decoder->decode_status_supported = 1; + decoder->hw_device_ref = av_buffer_ref(hw_device_ref); if (!decoder->hw_device_ref) { ret = AVERROR(ENOMEM); @@ -794,6 +807,53 @@ static void nvdec_opaque_release_slot(void *opaque, uint8_t *data) } #endif +/* + * The decoder's CUDA context must be current and decoding must have + * completed. idx is the surface index for cuvidMapVideoFrame and the + * picture index for cuvidDecodePicture. + */ +static void nvdec_set_decode_error_flags(void *logctx, NVDECDecoder *decoder, + int idx, AVFrame *frame) +{ + CUVIDGETDECODESTATUS status = { 0 }; + CUresult res; + + if (!decoder->decode_status_supported) + return; + + res = decoder->cvdl->cuvidGetDecodeStatus(decoder->decoder, idx, &status); + if (res == NVDEC_CUDA_ERROR_NOT_SUPPORTED) { + decoder->decode_status_supported = 0; + av_log(logctx, AV_LOG_VERBOSE, + "Decode status reporting is not supported for this codec on " + "this GPU, decode errors will not be reported\n"); + return; + } + if (res != CUDA_SUCCESS) + return; + + switch (status.decodeStatus) { + case cuvidDecodeStatus_Error: + frame->decode_error_flags |= FF_DECODE_ERROR_INVALID_BITSTREAM; + break; + case cuvidDecodeStatus_Error_Concealed: + frame->decode_error_flags |= FF_DECODE_ERROR_CONCEALMENT_ACTIVE; + break; + case cuvidDecodeStatus_InProgress: + /* H.264/HEVC report a final status after the blocking map. JPEG + * can still report InProgress here, even for clean pictures. */ + if (!decoder->in_progress_logged) { + av_log(logctx, AV_LOG_DEBUG, + "Decode status still InProgress after a blocking map, " + "errors not reported for such frames\n"); + decoder->in_progress_logged = 1; + } + break; + default: + break; + } +} + static int nvdec_retrieve_data(void *logctx, AVFrame *frame) { FrameDecodeData *fdd = frame->private_ref; @@ -922,6 +982,10 @@ static int nvdec_retrieve_data(void *logctx, AVFrame *frame) if (ret < 0) goto finish; + /* cuvidMapVideoFrame() blocks until decoding completes (NVDEC API + * guide). */ + nvdec_set_decode_error_flags(logctx, decoder, cf->idx, frame); + ret = av_buffer_replace(&frame->hw_frames_ctx, decoder->real_hw_frames_ref); if (ret < 0) goto copy_fail; -- 2.25.1 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
