#11137: ffmpeg h265 decoder seeks past max_ts passed into avformat_seek_file
-------------------------------------+-------------------------------------
Reporter: ahmadsharif | Type: defect
Status: new | Priority: normal
Component: avformat | Version: git-
| master
Keywords: libx265 | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Summary of the bug:
I am using FFMPEG as a library. I create a AVFormatContext from a video
file, and then seek to a timestamp by calling:
{{{
double timestamp;
timestamp = 0.5
avformat_seek_file(format_context, 0, INT64_MIN, timestamp, timestamp);
}}}
My understanding is that avformat_seek_file() will seek to a position
where it's an I-Frame with a pts that is <= 0.5.
That is indeed what I see for most videos.
However, for certain H265 videos, FFMPEG seeks to a position beyond the
max_ts timestamp passed in. To repro this behavior you can run these
commands to generate a test video with just frame numbers on it:
{{{
# Create a clean conda environment for testing purposes
conda create --name test
conda activate test
conda install -c conda-forge x265
# Install some build pre-requisites
conda install pkg-config
# Build ffmpeg from source with x265 enabled
git clone https://github.com/FFmpeg/FFmpeg.git
./configure --enable-nonfree --enable-gpl --prefix=$(readlink -f ../bin)
--enable-libx265 --enable-rpath
--extra-ldflags=-Wl,-rpath=$CONDA_PREFIX/lib --enable-filter=drawtext
--enable-libfontconfig --enable-libfreetype --enable-libharfbuzz
make -j install
# Now generate a video with just frame numbers in the text per frame:
ffmpeg -f lavfi -i color=size=128x128:duration=1:rate=10:color=blue -vf
"drawtext=fontsize=30:fontcolor=white:x=(w-text_w)/2:y=(h-text_h)/2:text='Frame
%{frame_num}'" -vcodec libx265 -pix_fmt yuv420p -g 2 -crf 10 test.mp4 -y
Note that this video has 10 frames. ffprobe shows the following:
ffprobe -v error -select_streams v:0 -show_entries
frame=pts,pts_time,duration,pkt_pts_time,pkt_duration,key_frame -of csv
test.mp4
frame,1,0,0.000000,1024,1024,
frame,0,1024,0.100000,1024,1024
frame,1,2048,0.200000,1024,1024
frame,0,3072,0.300000,1024,1024
frame,1,4096,0.400000,1024,1024
frame,0,5120,0.500000,1024,1024
frame,1,6144,0.600000,1024,1024
frame,0,7168,0.700000,1024,1024
frame,1,8192,0.800000,1024,1024
frame,0,9216,0.900000,1024,1024
}}}
Now, when I open this video using FFMPEG as a library, I get an
AVFormatContext. I want to decode the frame with pts=5120. So I call
avformat_seek_file with min_ts=-INT64_MAX, ts=5120 and max_ts=5120.
I expect that FFMPEG will seek to the frame with pts=4096 (because it is
the last keyframe before pts=5120) so I can then decode forward and
eventually get frame with pts=5120 with avcodec_receive_frame(), but it
seems like the first frame that I get from avcodec_receive_frame() is the
one with pts=6144.
In other words:
{{{
frame,1,4096,0.400000,1024,1024
frame,0,5120,0.500000,1024,1024 # I try to seek with max_ts=5120...
frame,1,6144,0.600000,1024,1024 # .. but FFMPEG seeks to here!
}}}
More context:
I am writing a library that wraps FFMPEG and returns frames at arbitrary
timestamps. The full source code of the library is here:
https://github.com/pytorch/torchcodec. The pull-request that reproduces
this exact scenario is here:
https://github.com/pytorch/torchcodec/pull/178.
It would be nice if FFMPEG always seeked to a frame with pts <= the
max_pts passed into avformat_seek_file. This normally does work with other
codecs. Am I calling the library wrong? Should I be calling
avformat_seek_file() with other flags? The documentation of
avformat_seek_file is here:
https://ffmpeg.org/doxygen/7.0/group__lavf__decoding.html#ga3b40fc8d2fda6992ae6ea2567d71ba30
Here is the seek call in my code:
https://github.com/pytorch/torchcodec/blob/dbfef1223522639d2b036a185b444eecf7748466/src/torchcodec/decoders/_core/VideoDecoder.cpp#L735
Note that I accidentally sent out this email before filing this bug. I
thought I would check on the mailing list but I think a tracking bug is
better for discussion as the mailing list has mostly patches.
https://ffmpeg.org/pipermail/ffmpeg-devel/2024-August/332241.html
--
Ticket URL: <https://trac.ffmpeg.org/ticket/11137>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker_______________________________________________
FFmpeg-trac mailing list
[email protected]
https://ffmpeg.org/mailman/listinfo/ffmpeg-trac
To unsubscribe, visit link above, or email
[email protected] with subject "unsubscribe".
