On Tue, Nov 14, 2023 at 7:23 PM Dmitry Rogozhkin <dmitry.v.rogozhkin-at-intel....@ffmpeg.org> wrote: > > Gurd against segfault running VLC decoding under msys2 [1]: > > Thread 33 received signal SIGSEGV, Segmentation fault. > [Switching to Thread 37728.0xadd0] > ff_hwaccel_frame_priv_alloc (avctx=0x6447b00, > hwaccel_picture_private=0x65dfd00) > at libavcodec/decode.c:1848 > 1848 frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data; > (gdb) bt > at libavcodec/decode.c:1848 > at libavcodec/h264_slice.c:208 > first_slice=1) at libavcodec/h264_slice.c:1599 > at libavcodec/h264_slice.c:2130 > at libavcodec/h264dec.c:652 > got_frame=0x646e4b0, avpkt=0x64522c0) at libavcodec/h264dec.c:1048 > > (gdb) p avctx > $1 = (AVCodecContext *) 0x6447b00 > (gdb) p avctx->hw_frames_ctx > $2 = (AVBufferRef *) 0x0 > > See[1]: https://github.com/msys2/MINGW-packages/pull/19050 > Fixes: be07145109 ("avcodec: add AVHWAccel.free_frame_priv callback") > CC: Lynne <d...@lynne.ee> > CC: Christoph Reiter <reiter.christ...@gmail.com> > Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozh...@intel.com> > --- > libavcodec/decode.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/libavcodec/decode.c b/libavcodec/decode.c > index ad39021..3caaeec 100644 > --- a/libavcodec/decode.c > +++ b/libavcodec/decode.c > @@ -1845,9 +1845,9 @@ int ff_hwaccel_frame_priv_alloc(AVCodecContext *avctx, > void **hwaccel_picture_pr > > av_assert0(!*hwaccel_picture_private); > > - frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data; > + frames_ctx = (AVHWFramesContext *) avctx->hw_frames_ctx ? > avctx->hw_frames_ctx->data : NULL; > *hwaccel_picture_private = > ff_refstruct_alloc_ext(hwaccel->frame_priv_data_size, 0, > - frames_ctx->device_ctx, > + frames_ctx ? > frames_ctx->device_ctx : NULL, > > hwaccel->free_frame_priv);
The free_frame_priv callback is not usable when no context is available. The code should be updated to check if the hwaccel has a free_frame_priv callback and then require a frames_ctx to be set and otherwise error out (instead of crashing), and if free_frame_priv is not set, then it can allocate a buffer without these requirements. - Hendrik _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".