PR #24322 opened by Bradyok URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24322 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24322.patch
alloc_picture() takes two motion_val and two ref_index entries from the refstruct pools for every H264Picture regardless of the decode path. Pool entries are zeroed on allocation, so each picture carries about 1 MiB of resident memory at 1080p (2 x 523344 B motion_val + 2 x 32912 B ref_index) that nothing reads when a hwaccel does the decoding: - h264_mvpred.h, h264_direct.c and fill_filter_caches() are reached only from decode_slice() via ff_h264_execute_decode_slices(), which returns before doing any work when avctx->hwaccel is set; - error resilience runs from ff_er_frame_end(), whose er_supported() returns 0 under a hwaccel, and ff_er_frame_end() allocates its own tables when cur_pic.motion_val[0] is NULL anyway; - ff_print_debug_info2() is guarded on motion_val[0] being non-NULL; - ff_h264_set_erpic() and the H264Picture copy/unref helpers only copy the pointers, through av_refstruct_replace()/av_refstruct_unref(), which accept NULL. avctx->hwaccel is settled by get_pixel_format() in h264_init_ps() before h264_frame_start() allocates the picture, and a pixel format change reinitializes through ff_h264_flush_change() -> ff_h264_remove_all_refs(), so a hwaccel picture without tables can never become a reference for a software-decoded one. The condition is the same avctx->hwaccel check that alloc_picture() and h264_frame_start() already use. Decoding 32 concurrent 1080p H.264 streams through VideoToolbox, this saves about 160 MB of RSS. Signed-off-by: Braden O'Keefe <[email protected]> Tested: `make fate-h264` passes (222 tests) on macOS arm64 with the patch applied to a clean n9.0 tree; the change is also in daily use on a VideoToolbox-based multi-stream player (that is where the 160 MB figure was measured). The patch applies unchanged to master. >From 2322a65960a54f1b03f4941b5b5e45f76011ef4d Mon Sep 17 00:00:00 2001 From: Braden O'Keefe <[email protected]> Date: Sat, 29 Aug 2026 18:42:50 -0400 Subject: [PATCH] avcodec/h264_slice: do not allocate motion_val/ref_index tables under a hwaccel alloc_picture() takes two motion_val and two ref_index entries from the refstruct pools for every H264Picture regardless of the decode path. Pool entries are zeroed on allocation, so each picture carries about 1 MiB of resident memory at 1080p (2 x 523344 B motion_val + 2 x 32912 B ref_index) that nothing reads when a hwaccel does the decoding: - h264_mvpred.h, h264_direct.c and fill_filter_caches() are reached only from decode_slice() via ff_h264_execute_decode_slices(), which returns before doing any work when avctx->hwaccel is set; - error resilience runs from ff_er_frame_end(), whose er_supported() returns 0 under a hwaccel, and ff_er_frame_end() allocates its own tables when cur_pic.motion_val[0] is NULL anyway; - ff_print_debug_info2() is guarded on motion_val[0] being non-NULL; - ff_h264_set_erpic() and the H264Picture copy/unref helpers only copy the pointers, through av_refstruct_replace()/av_refstruct_unref(), which accept NULL. avctx->hwaccel is settled by get_pixel_format() in h264_init_ps() before h264_frame_start() allocates the picture, and a pixel format change reinitializes through ff_h264_flush_change() -> ff_h264_remove_all_refs(), so a hwaccel picture without tables can never become a reference for a software-decoded one. The condition is the same avctx->hwaccel check that alloc_picture() and h264_frame_start() already use. Decoding 32 concurrent 1080p H.264 streams through VideoToolbox, this saves about 160 MB of RSS. Signed-off-by: Braden O'Keefe <[email protected]> --- libavcodec/h264_slice.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index a13047ffe8..5713d76948 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -250,13 +250,25 @@ static int alloc_picture(H264Context *h, H264Picture *pic) pic->mb_type = pic->mb_type_base + 2 * h->mb_stride + 1; pic->qscale_table = pic->qscale_table_base + 2 * h->mb_stride + 1; - for (i = 0; i < 2; i++) { - pic->motion_val_base[i] = av_refstruct_pool_get(h->motion_val_pool); - pic->ref_index[i] = av_refstruct_pool_get(h->ref_index_pool); - if (!pic->motion_val_base[i] || !pic->ref_index[i]) - goto fail; + /* The motion-vector and reference-index tables are read only by the + * software macroblock path (h264_mvpred.h, h264_direct.c, + * fill_filter_caches()), which is never entered under a hwaccel + * (ff_h264_execute_decode_slices() returns before doing anything), and + * by error resilience, which er_supported() disables under a hwaccel and + * which allocates its own tables when these are NULL. Every other user + * (ff_print_debug_info2(), ff_h264_set_erpic(), the picture copies) is + * NULL-safe. The pool entries are zeroed on allocation, so skipping them + * for hwaccel pictures saves about 1 MiB of resident memory per 1080p + * picture. */ + if (!h->avctx->hwaccel) { + for (i = 0; i < 2; i++) { + pic->motion_val_base[i] = av_refstruct_pool_get(h->motion_val_pool); + pic->ref_index[i] = av_refstruct_pool_get(h->ref_index_pool); + if (!pic->motion_val_base[i] || !pic->ref_index[i]) + goto fail; - pic->motion_val[i] = pic->motion_val_base[i] + 4; + pic->motion_val[i] = pic->motion_val_base[i] + 4; + } } pic->pps = av_refstruct_ref_c(h->ps.pps); -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
