Re: [FFmpeg-devel] Experiences in using ffmpeg to transcode broadcast video
A follow-up (I'll write this up better somewhere too once I get a chance).
I've rewritten my transcoding pipeline, avoiding -copyts and it looks
pretty promising. I now:
1. cut the input mpegts into pieces that have a) the same aspect ratio and
b) monotonic timestamps (for both audio and video); I cut it using dd
(rather than ffmpeg), which can't go wrong :-)
2. sync audio and video, overlay subtitles and transcode the pieces
(skipping pieces that are too small to contain decodable audio and video)
into mp4, with a GOP length suitable for fragmenting
3. concatenate the transcoded mp4 pieces with the concat demuxer
4. fragment the concatenated mp4 into ismv
Now I'm using ffmpeg's reasonably robust handling of mpegts (given
monotonic timestamps) without needing -copyts, as the concat demuxer will
fix up the timestamps for the pieces so that the pieces can all start at 0
and I'm syncing up audio and video before concatenating.
There are still places that can be a bit flaky (what is 'too small'?,
handling those frames that straddle the discontinuity where dts is on one
side and pts on the other, maybe badly broken audio input can't be synced).
Mika
On 29 October 2014 13:49, Mika Raento wrote:
> Tickets for sub2video and async not working with non-monotonic input,
> using -copyts:
>
> https://trac.ffmpeg.org/ticket/4062
> https://trac.ffmpeg.org/ticket/4064
>
> Hm. Lots of my issues seem to relate to using -copyts. I had trouble
> keeping all video and all audio if there were periods where one or the
> other were missing (esp. in the beginning). Maybe I could make this simpler
> if I'd relax that requirement (I don't know how much happier my users are
> if they get audio but no video, rather than nothing).
>
> Mika
>
> On 29 October 2014 10:38, Mika Raento wrote:
>
>> segment muxer PIDs were fixed in
>> https://github.com/FFmpeg/FFmpeg/commit/502fc3b3d4b36015562d19d74f27d0a4ff835c4e
>> by me
>>
>> On 29 October 2014 10:17, Mika Raento wrote:
>>
>>> Hi
>>>
>>> Thanks for the feedback.
>>>
>>> I'll try to capture at least some of these in tickets, and reply on this
>>> thread as I go along.
>>>
>>> Async not working with damaged audio input is
>>> https://trac.ffmpeg.org/ticket/2693 (existing ticket)
>>>
>>> Sub2video not working with negative timestamps is
>>> https://trac.ffmpeg.org/ticket/4062 (new ticket)
>>>
>>> Mika
>>>
>>> On 27 October 2014 01:00, Michael Niedermayer wrote:
>>>
>>>> On Wed, Sep 24, 2014 at 10:43:40AM +0300, Mika Raento wrote:
>>>> > Dear all
>>>> >
>>>> > This mail is meant mainly as a note to other potential users, but
>>>> > possibly as input to development - time allowing I might be able to
>>>> pick
>>>> > up some of the pieces myself. It's also a thank-you for all the hard
>>>> > work in ffmpeg.
>>>> >
>>>> > I've successfully implemented a transcoding pipeline for producing
>>>> > multi-bitrate fragmented mp4 files from broadcast DVB input. More
>>>> > concretely, I'm taking in broadcast TS captures with either mpeg2
>>>> video
>>>> > and mp2 audio (SD, varying aspect ratio) or h264 video and aac audio,
>>>> > both potentially with dvbsub. From that I'm producing ISMV output with
>>>> > multiple bitrate h264 video at fixed 16:9 aspect ratio and multiple
>>>> > bitrate aac audio, with burned subtitles. The ISMV outputs are
>>>> > post-processed with tools/ismindex and with the hls muxer.
>>>> >
>>>> > There are number of limitations in ffmpeg that I've had to work
>>>> around:
>>>> >
>>>> > - I haven't gotten sub2video or async working without reasonably
>>>> > monotonous DTS. Broadcast TS streams can easily contain backward
>>>> jumps
>>>> > in time (e.g., a cheapo source that plays mp4 files and starts each
>>>> file
>>>> > at 0). The fix is to cut the TS into pieces at timestamp jump
>>>> locations
>>>> > and using '-itsoffset' to rewrite the timestamps and then
>>>> concatenate.
>>>> > I'm using the segment and concat muxers for that.
>>>> > - Sub2video doesn't work with negative timestamps, so I use
>>>> '-itsoffset'
>>>> > to get positive timestamps
>>>> > - For HD streams, I nee
Re: [FFmpeg-devel] [PATCH] mov.c: fix handling of seek return in read_mfra
On 15 November 2014 09:00, Mika Raento wrote: > On 14 November 2014 21:51, Michael Niedermayer wrote: > >> On Fri, Nov 14, 2014 at 07:12:34PM +0200, Mika Raento wrote: >> > this would cause mfra to be ignored in files larger than 2G >> > --- >> > libavformat/mov.c | 22 ++ >> > 1 file changed, 14 insertions(+), 8 deletions(-) >> >> why not just chage the type of ret to int64_t instead of adding a >> second return variable ? >> > > Done, with a comment on why that works. > > Thanks. > Ah, I forgot the second reason: with a separate seek_ret I can keep the return code and not overwrite it with the return value of seeking back. I prefer the first version of the patch. Mika > > >> >> [...] >> -- >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >> >> Dictatorship naturally arises out of democracy, and the most aggravated >> form of tyranny and slavery out of the most extreme liberty. -- Plato >> >> ___ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] mov.c: fix handling of seek return in read_mfra
On 14 November 2014 21:51, Michael Niedermayer wrote: > On Fri, Nov 14, 2014 at 07:12:34PM +0200, Mika Raento wrote: > > this would cause mfra to be ignored in files larger than 2G > > --- > > libavformat/mov.c | 22 ++ > > 1 file changed, 14 insertions(+), 8 deletions(-) > > why not just chage the type of ret to int64_t instead of adding a > second return variable ? > Done, with a comment on why that works. Thanks. > > [...] > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > Dictatorship naturally arises out of democracy, and the most aggravated > form of tyranny and slavery out of the most extreme liberty. -- Plato > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] mov.c: fix handling of seek return in read_mfra
this would cause mfra to be ignored in files larger than 2G
---
libavformat/mov.c | 20
1 file changed, 12 insertions(+), 8 deletions(-)
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 6ba7b96..506c1a6 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -3820,35 +3820,39 @@ static int mov_read_mfra(MOVContext *c, AVIOContext *f)
int64_t stream_size = avio_size(f);
int64_t original_pos = avio_tell(f);
int32_t mfra_size;
-int ret = -1;
-if ((ret = avio_seek(f, stream_size - 4, SEEK_SET)) < 0) goto fail;
+// we return 0 or AV_ERROR, both fit in 32bits, but we check intermediate
+// calls that have 64 bits
+int64_t ret = -1;
+if ((ret = avio_seek(f, stream_size - 4, SEEK_SET)) < 0) {
+goto fail;
+}
mfra_size = avio_rb32(f);
if (mfra_size < 0 || mfra_size > stream_size) {
av_log(c->fc, AV_LOG_DEBUG, "doesn't look like mfra (unreasonable
size)\n");
-ret = -1;
goto fail;
}
-if ((ret = avio_seek(f, -mfra_size, SEEK_CUR)) < 0) goto fail;
+if ((ret = avio_seek(f, -mfra_size, SEEK_CUR)) < 0) {
+goto fail;
+}
if (avio_rb32(f) != mfra_size) {
av_log(c->fc, AV_LOG_DEBUG, "doesn't look like mfra (size
mismatch)\n");
-ret = -1;
goto fail;
}
if (avio_rb32(f) != MKBETAG('m', 'f', 'r', 'a')) {
av_log(c->fc, AV_LOG_DEBUG, "doesn't look like mfra (tag mismatch)\n");
goto fail;
}
+ret = 0;
av_log(c->fc, AV_LOG_VERBOSE, "stream has mfra\n");
while (!read_tfra(c, f)) {
/* Empty */
}
fail:
ret = avio_seek(f, original_pos, SEEK_SET);
-if (ret < 0)
+if (ret < 0) {
av_log(c->fc, AV_LOG_ERROR,
"failed to seek back after looking for mfra\n");
-else
-ret = 0;
+}
return ret;
}
--
1.9.3 (Apple Git-50)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] mov.c: fix handling of seek return in read_mfra
this would cause mfra to be ignored in files larger than 2G
---
libavformat/mov.c | 22 ++
1 file changed, 14 insertions(+), 8 deletions(-)
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 6ba7b96..8bf16e7 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -3819,36 +3819,42 @@ static int mov_read_mfra(MOVContext *c, AVIOContext *f)
{
int64_t stream_size = avio_size(f);
int64_t original_pos = avio_tell(f);
+int64_t seek_ret;
int32_t mfra_size;
int ret = -1;
-if ((ret = avio_seek(f, stream_size - 4, SEEK_SET)) < 0) goto fail;
+if ((seek_ret = avio_seek(f, stream_size - 4, SEEK_SET)) < 0) {
+ret = seek_ret;
+goto fail;
+}
mfra_size = avio_rb32(f);
if (mfra_size < 0 || mfra_size > stream_size) {
av_log(c->fc, AV_LOG_DEBUG, "doesn't look like mfra (unreasonable
size)\n");
-ret = -1;
goto fail;
}
-if ((ret = avio_seek(f, -mfra_size, SEEK_CUR)) < 0) goto fail;
+if ((seek_ret = avio_seek(f, -mfra_size, SEEK_CUR)) < 0) {
+ret = seek_ret;
+goto fail;
+}
if (avio_rb32(f) != mfra_size) {
av_log(c->fc, AV_LOG_DEBUG, "doesn't look like mfra (size
mismatch)\n");
-ret = -1;
goto fail;
}
if (avio_rb32(f) != MKBETAG('m', 'f', 'r', 'a')) {
av_log(c->fc, AV_LOG_DEBUG, "doesn't look like mfra (tag mismatch)\n");
goto fail;
}
+ret = 0;
av_log(c->fc, AV_LOG_VERBOSE, "stream has mfra\n");
while (!read_tfra(c, f)) {
/* Empty */
}
fail:
-ret = avio_seek(f, original_pos, SEEK_SET);
-if (ret < 0)
+seek_ret = avio_seek(f, original_pos, SEEK_SET);
+if (seek_ret < 0) {
av_log(c->fc, AV_LOG_ERROR,
"failed to seek back after looking for mfra\n");
-else
-ret = 0;
+ret = seek_ret;
+}
return ret;
}
--
1.9.3 (Apple Git-50)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Experiences in using ffmpeg to transcode broadcast video
Created a feature request to automatically scale subtitles for HD video https://trac.ffmpeg.org/ticket/4066 On 29 October 2014 14:25, Mika Raento wrote: > Created a feature request to support columnboxing input that changes > aspect-ratio mid-stream. > > https://trac.ffmpeg.org/ticket/4065 > > On 29 October 2014 13:49, Mika Raento wrote: > >> Tickets for sub2video and async not working with non-monotonic input, >> using -copyts: >> >> https://trac.ffmpeg.org/ticket/4062 >> https://trac.ffmpeg.org/ticket/4064 >> >> Hm. Lots of my issues seem to relate to using -copyts. I had trouble >> keeping all video and all audio if there were periods where one or the >> other were missing (esp. in the beginning). Maybe I could make this simpler >> if I'd relax that requirement (I don't know how much happier my users are >> if they get audio but no video, rather than nothing). >> >> Mika >> >> On 29 October 2014 10:38, Mika Raento wrote: >> >>> segment muxer PIDs were fixed in >>> https://github.com/FFmpeg/FFmpeg/commit/502fc3b3d4b36015562d19d74f27d0a4ff835c4e >>> by me >>> >>> On 29 October 2014 10:17, Mika Raento wrote: >>> >>>> Hi >>>> >>>> Thanks for the feedback. >>>> >>>> I'll try to capture at least some of these in tickets, and reply on >>>> this thread as I go along. >>>> >>>> Async not working with damaged audio input is >>>> https://trac.ffmpeg.org/ticket/2693 (existing ticket) >>>> >>>> Sub2video not working with negative timestamps is >>>> https://trac.ffmpeg.org/ticket/4062 (new ticket) >>>> >>>> Mika >>>> >>>> On 27 October 2014 01:00, Michael Niedermayer wrote: >>>> >>>>> On Wed, Sep 24, 2014 at 10:43:40AM +0300, Mika Raento wrote: >>>>> > Dear all >>>>> > >>>>> > This mail is meant mainly as a note to other potential users, but >>>>> > possibly as input to development - time allowing I might be able to >>>>> pick >>>>> > up some of the pieces myself. It's also a thank-you for all the hard >>>>> > work in ffmpeg. >>>>> > >>>>> > I've successfully implemented a transcoding pipeline for producing >>>>> > multi-bitrate fragmented mp4 files from broadcast DVB input. More >>>>> > concretely, I'm taking in broadcast TS captures with either mpeg2 >>>>> video >>>>> > and mp2 audio (SD, varying aspect ratio) or h264 video and aac audio, >>>>> > both potentially with dvbsub. From that I'm producing ISMV output >>>>> with >>>>> > multiple bitrate h264 video at fixed 16:9 aspect ratio and multiple >>>>> > bitrate aac audio, with burned subtitles. The ISMV outputs are >>>>> > post-processed with tools/ismindex and with the hls muxer. >>>>> > >>>>> > There are number of limitations in ffmpeg that I've had to work >>>>> around: >>>>> > >>>>> > - I haven't gotten sub2video or async working without reasonably >>>>> > monotonous DTS. Broadcast TS streams can easily contain backward >>>>> jumps >>>>> > in time (e.g., a cheapo source that plays mp4 files and starts >>>>> each file >>>>> > at 0). The fix is to cut the TS into pieces at timestamp jump >>>>> locations >>>>> > and using '-itsoffset' to rewrite the timestamps and then >>>>> concatenate. >>>>> > I'm using the segment and concat muxers for that. >>>>> > - Sub2video doesn't work with negative timestamps, so I use >>>>> '-itsoffset' >>>>> > to get positive timestamps >>>>> > - For HD streams, I need to scale up the sub2video results from SD to >>>>> > HD. Sub2video doesn't handle the HD subtitle geometries. I'm not >>>>> > enough of an expert to know whether that's the issue, or whether >>>>> that's >>>>> > just the way it's supposed to work with SD subs (typical) with HD >>>>> video. >>>>> > - For columnboxing, I use the scale, pad and setdar video filters. >>>>> These >>>>> > work fine, but their parameters are only evaluated at start, so I >>>>> need >>>>> > to cut the video into pieces with a single aspect ratio first and >>>>> > concatenate later. >>>>> > - Audio sync (using aresample) gets confused if the input contains >>>>> > errors, so I need to first re-encode audio (with '-copyts') and >>>>> only >>>>> > after that synchronize. >>>>> > - The TS PIDs are not kept over the segment muxer, so I given them on >>>>> > the command line with '-streamid'. >>>>> >>>>> a bit late reply but >>>>> bug reports or feature requests for all these are welcome unless there >>>>> are already tickets for them assuming these issues still exist >>>>> >>>>> thx >>>>> >>>>> [...] >>>>> >>>>> -- >>>>> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >>>>> >>>>> Why not whip the teacher when the pupil misbehaves? -- Diogenes of >>>>> Sinope >>>>> >>>>> ___ >>>>> ffmpeg-devel mailing list >>>>> ffmpeg-devel@ffmpeg.org >>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>>>> >>>>> >>>> >>> >> > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Experiences in using ffmpeg to transcode broadcast video
Created a feature request to support columnboxing input that changes aspect-ratio mid-stream. https://trac.ffmpeg.org/ticket/4065 On 29 October 2014 13:49, Mika Raento wrote: > Tickets for sub2video and async not working with non-monotonic input, > using -copyts: > > https://trac.ffmpeg.org/ticket/4062 > https://trac.ffmpeg.org/ticket/4064 > > Hm. Lots of my issues seem to relate to using -copyts. I had trouble > keeping all video and all audio if there were periods where one or the > other were missing (esp. in the beginning). Maybe I could make this simpler > if I'd relax that requirement (I don't know how much happier my users are > if they get audio but no video, rather than nothing). > > Mika > > On 29 October 2014 10:38, Mika Raento wrote: > >> segment muxer PIDs were fixed in >> https://github.com/FFmpeg/FFmpeg/commit/502fc3b3d4b36015562d19d74f27d0a4ff835c4e >> by me >> >> On 29 October 2014 10:17, Mika Raento wrote: >> >>> Hi >>> >>> Thanks for the feedback. >>> >>> I'll try to capture at least some of these in tickets, and reply on this >>> thread as I go along. >>> >>> Async not working with damaged audio input is >>> https://trac.ffmpeg.org/ticket/2693 (existing ticket) >>> >>> Sub2video not working with negative timestamps is >>> https://trac.ffmpeg.org/ticket/4062 (new ticket) >>> >>> Mika >>> >>> On 27 October 2014 01:00, Michael Niedermayer wrote: >>> >>>> On Wed, Sep 24, 2014 at 10:43:40AM +0300, Mika Raento wrote: >>>> > Dear all >>>> > >>>> > This mail is meant mainly as a note to other potential users, but >>>> > possibly as input to development - time allowing I might be able to >>>> pick >>>> > up some of the pieces myself. It's also a thank-you for all the hard >>>> > work in ffmpeg. >>>> > >>>> > I've successfully implemented a transcoding pipeline for producing >>>> > multi-bitrate fragmented mp4 files from broadcast DVB input. More >>>> > concretely, I'm taking in broadcast TS captures with either mpeg2 >>>> video >>>> > and mp2 audio (SD, varying aspect ratio) or h264 video and aac audio, >>>> > both potentially with dvbsub. From that I'm producing ISMV output with >>>> > multiple bitrate h264 video at fixed 16:9 aspect ratio and multiple >>>> > bitrate aac audio, with burned subtitles. The ISMV outputs are >>>> > post-processed with tools/ismindex and with the hls muxer. >>>> > >>>> > There are number of limitations in ffmpeg that I've had to work >>>> around: >>>> > >>>> > - I haven't gotten sub2video or async working without reasonably >>>> > monotonous DTS. Broadcast TS streams can easily contain backward >>>> jumps >>>> > in time (e.g., a cheapo source that plays mp4 files and starts each >>>> file >>>> > at 0). The fix is to cut the TS into pieces at timestamp jump >>>> locations >>>> > and using '-itsoffset' to rewrite the timestamps and then >>>> concatenate. >>>> > I'm using the segment and concat muxers for that. >>>> > - Sub2video doesn't work with negative timestamps, so I use >>>> '-itsoffset' >>>> > to get positive timestamps >>>> > - For HD streams, I need to scale up the sub2video results from SD to >>>> > HD. Sub2video doesn't handle the HD subtitle geometries. I'm not >>>> > enough of an expert to know whether that's the issue, or whether >>>> that's >>>> > just the way it's supposed to work with SD subs (typical) with HD >>>> video. >>>> > - For columnboxing, I use the scale, pad and setdar video filters. >>>> These >>>> > work fine, but their parameters are only evaluated at start, so I >>>> need >>>> > to cut the video into pieces with a single aspect ratio first and >>>> > concatenate later. >>>> > - Audio sync (using aresample) gets confused if the input contains >>>> > errors, so I need to first re-encode audio (with '-copyts') and only >>>> > after that synchronize. >>>> > - The TS PIDs are not kept over the segment muxer, so I given them on >>>> > the command line with '-streamid'. >>>> >>>> a bit late reply but >>>> bug reports or feature requests for all these are welcome unless there >>>> are already tickets for them assuming these issues still exist >>>> >>>> thx >>>> >>>> [...] >>>> >>>> -- >>>> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >>>> >>>> Why not whip the teacher when the pupil misbehaves? -- Diogenes of >>>> Sinope >>>> >>>> ___ >>>> ffmpeg-devel mailing list >>>> ffmpeg-devel@ffmpeg.org >>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>>> >>>> >>> >> > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Experiences in using ffmpeg to transcode broadcast video
Tickets for sub2video and async not working with non-monotonic input, using -copyts: https://trac.ffmpeg.org/ticket/4062 https://trac.ffmpeg.org/ticket/4064 Hm. Lots of my issues seem to relate to using -copyts. I had trouble keeping all video and all audio if there were periods where one or the other were missing (esp. in the beginning). Maybe I could make this simpler if I'd relax that requirement (I don't know how much happier my users are if they get audio but no video, rather than nothing). Mika On 29 October 2014 10:38, Mika Raento wrote: > segment muxer PIDs were fixed in > https://github.com/FFmpeg/FFmpeg/commit/502fc3b3d4b36015562d19d74f27d0a4ff835c4e > by me > > On 29 October 2014 10:17, Mika Raento wrote: > >> Hi >> >> Thanks for the feedback. >> >> I'll try to capture at least some of these in tickets, and reply on this >> thread as I go along. >> >> Async not working with damaged audio input is >> https://trac.ffmpeg.org/ticket/2693 (existing ticket) >> >> Sub2video not working with negative timestamps is >> https://trac.ffmpeg.org/ticket/4062 (new ticket) >> >> Mika >> >> On 27 October 2014 01:00, Michael Niedermayer wrote: >> >>> On Wed, Sep 24, 2014 at 10:43:40AM +0300, Mika Raento wrote: >>> > Dear all >>> > >>> > This mail is meant mainly as a note to other potential users, but >>> > possibly as input to development - time allowing I might be able to >>> pick >>> > up some of the pieces myself. It's also a thank-you for all the hard >>> > work in ffmpeg. >>> > >>> > I've successfully implemented a transcoding pipeline for producing >>> > multi-bitrate fragmented mp4 files from broadcast DVB input. More >>> > concretely, I'm taking in broadcast TS captures with either mpeg2 video >>> > and mp2 audio (SD, varying aspect ratio) or h264 video and aac audio, >>> > both potentially with dvbsub. From that I'm producing ISMV output with >>> > multiple bitrate h264 video at fixed 16:9 aspect ratio and multiple >>> > bitrate aac audio, with burned subtitles. The ISMV outputs are >>> > post-processed with tools/ismindex and with the hls muxer. >>> > >>> > There are number of limitations in ffmpeg that I've had to work around: >>> > >>> > - I haven't gotten sub2video or async working without reasonably >>> > monotonous DTS. Broadcast TS streams can easily contain backward >>> jumps >>> > in time (e.g., a cheapo source that plays mp4 files and starts each >>> file >>> > at 0). The fix is to cut the TS into pieces at timestamp jump >>> locations >>> > and using '-itsoffset' to rewrite the timestamps and then >>> concatenate. >>> > I'm using the segment and concat muxers for that. >>> > - Sub2video doesn't work with negative timestamps, so I use >>> '-itsoffset' >>> > to get positive timestamps >>> > - For HD streams, I need to scale up the sub2video results from SD to >>> > HD. Sub2video doesn't handle the HD subtitle geometries. I'm not >>> > enough of an expert to know whether that's the issue, or whether >>> that's >>> > just the way it's supposed to work with SD subs (typical) with HD >>> video. >>> > - For columnboxing, I use the scale, pad and setdar video filters. >>> These >>> > work fine, but their parameters are only evaluated at start, so I >>> need >>> > to cut the video into pieces with a single aspect ratio first and >>> > concatenate later. >>> > - Audio sync (using aresample) gets confused if the input contains >>> > errors, so I need to first re-encode audio (with '-copyts') and only >>> > after that synchronize. >>> > - The TS PIDs are not kept over the segment muxer, so I given them on >>> > the command line with '-streamid'. >>> >>> a bit late reply but >>> bug reports or feature requests for all these are welcome unless there >>> are already tickets for them assuming these issues still exist >>> >>> thx >>> >>> [...] >>> >>> -- >>> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >>> >>> Why not whip the teacher when the pupil misbehaves? -- Diogenes of Sinope >>> >>> ___ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >>> >> > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Experiences in using ffmpeg to transcode broadcast video
segment muxer PIDs were fixed in https://github.com/FFmpeg/FFmpeg/commit/502fc3b3d4b36015562d19d74f27d0a4ff835c4e by me On 29 October 2014 10:17, Mika Raento wrote: > Hi > > Thanks for the feedback. > > I'll try to capture at least some of these in tickets, and reply on this > thread as I go along. > > Async not working with damaged audio input is > https://trac.ffmpeg.org/ticket/2693 (existing ticket) > > Sub2video not working with negative timestamps is > https://trac.ffmpeg.org/ticket/4062 (new ticket) > > Mika > > On 27 October 2014 01:00, Michael Niedermayer wrote: > >> On Wed, Sep 24, 2014 at 10:43:40AM +0300, Mika Raento wrote: >> > Dear all >> > >> > This mail is meant mainly as a note to other potential users, but >> > possibly as input to development - time allowing I might be able to pick >> > up some of the pieces myself. It's also a thank-you for all the hard >> > work in ffmpeg. >> > >> > I've successfully implemented a transcoding pipeline for producing >> > multi-bitrate fragmented mp4 files from broadcast DVB input. More >> > concretely, I'm taking in broadcast TS captures with either mpeg2 video >> > and mp2 audio (SD, varying aspect ratio) or h264 video and aac audio, >> > both potentially with dvbsub. From that I'm producing ISMV output with >> > multiple bitrate h264 video at fixed 16:9 aspect ratio and multiple >> > bitrate aac audio, with burned subtitles. The ISMV outputs are >> > post-processed with tools/ismindex and with the hls muxer. >> > >> > There are number of limitations in ffmpeg that I've had to work around: >> > >> > - I haven't gotten sub2video or async working without reasonably >> > monotonous DTS. Broadcast TS streams can easily contain backward jumps >> > in time (e.g., a cheapo source that plays mp4 files and starts each >> file >> > at 0). The fix is to cut the TS into pieces at timestamp jump >> locations >> > and using '-itsoffset' to rewrite the timestamps and then concatenate. >> > I'm using the segment and concat muxers for that. >> > - Sub2video doesn't work with negative timestamps, so I use '-itsoffset' >> > to get positive timestamps >> > - For HD streams, I need to scale up the sub2video results from SD to >> > HD. Sub2video doesn't handle the HD subtitle geometries. I'm not >> > enough of an expert to know whether that's the issue, or whether >> that's >> > just the way it's supposed to work with SD subs (typical) with HD >> video. >> > - For columnboxing, I use the scale, pad and setdar video filters. These >> > work fine, but their parameters are only evaluated at start, so I need >> > to cut the video into pieces with a single aspect ratio first and >> > concatenate later. >> > - Audio sync (using aresample) gets confused if the input contains >> > errors, so I need to first re-encode audio (with '-copyts') and only >> > after that synchronize. >> > - The TS PIDs are not kept over the segment muxer, so I given them on >> > the command line with '-streamid'. >> >> a bit late reply but >> bug reports or feature requests for all these are welcome unless there >> are already tickets for them assuming these issues still exist >> >> thx >> >> [...] >> >> -- >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >> >> Why not whip the teacher when the pupil misbehaves? -- Diogenes of Sinope >> >> ___ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Experiences in using ffmpeg to transcode broadcast video
Hi Thanks for the feedback. I'll try to capture at least some of these in tickets, and reply on this thread as I go along. Async not working with damaged audio input is https://trac.ffmpeg.org/ticket/2693 (existing ticket) Sub2video not working with negative timestamps is https://trac.ffmpeg.org/ticket/4062 (new ticket) Mika On 27 October 2014 01:00, Michael Niedermayer wrote: > On Wed, Sep 24, 2014 at 10:43:40AM +0300, Mika Raento wrote: > > Dear all > > > > This mail is meant mainly as a note to other potential users, but > > possibly as input to development - time allowing I might be able to pick > > up some of the pieces myself. It's also a thank-you for all the hard > > work in ffmpeg. > > > > I've successfully implemented a transcoding pipeline for producing > > multi-bitrate fragmented mp4 files from broadcast DVB input. More > > concretely, I'm taking in broadcast TS captures with either mpeg2 video > > and mp2 audio (SD, varying aspect ratio) or h264 video and aac audio, > > both potentially with dvbsub. From that I'm producing ISMV output with > > multiple bitrate h264 video at fixed 16:9 aspect ratio and multiple > > bitrate aac audio, with burned subtitles. The ISMV outputs are > > post-processed with tools/ismindex and with the hls muxer. > > > > There are number of limitations in ffmpeg that I've had to work around: > > > > - I haven't gotten sub2video or async working without reasonably > > monotonous DTS. Broadcast TS streams can easily contain backward jumps > > in time (e.g., a cheapo source that plays mp4 files and starts each > file > > at 0). The fix is to cut the TS into pieces at timestamp jump locations > > and using '-itsoffset' to rewrite the timestamps and then concatenate. > > I'm using the segment and concat muxers for that. > > - Sub2video doesn't work with negative timestamps, so I use '-itsoffset' > > to get positive timestamps > > - For HD streams, I need to scale up the sub2video results from SD to > > HD. Sub2video doesn't handle the HD subtitle geometries. I'm not > > enough of an expert to know whether that's the issue, or whether that's > > just the way it's supposed to work with SD subs (typical) with HD > video. > > - For columnboxing, I use the scale, pad and setdar video filters. These > > work fine, but their parameters are only evaluated at start, so I need > > to cut the video into pieces with a single aspect ratio first and > > concatenate later. > > - Audio sync (using aresample) gets confused if the input contains > > errors, so I need to first re-encode audio (with '-copyts') and only > > after that synchronize. > > - The TS PIDs are not kept over the segment muxer, so I given them on > > the command line with '-streamid'. > > a bit late reply but > bug reports or feature requests for all these are welcome unless there > are already tickets for them assuming these issues still exist > > thx > > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > Why not whip the teacher when the pupil misbehaves? -- Diogenes of Sinope > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] mov.c: reasonable bitrate for fragmented mp4
If using MFRA for timestamps, the stream may start from a large offset
and/or have gaps. With this change we calculate the bitrate based on
frames we've seen.
---
libavformat/mov.c | 11 +++
1 file changed, 11 insertions(+)
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 80549ec..1444bce 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -3943,6 +3943,17 @@ static int mov_read_header(AVFormatContext *s)
}
}
+if (mov->use_mfra_for > 0) {
+for (i = 0; i < s->nb_streams; i++) {
+AVStream *st = s->streams[i];
+MOVStreamContext *sc = st->priv_data;
+if (sc->duration_for_fps > 0) {
+st->codec->bit_rate = sc->data_size * 8 * sc->time_scale /
+sc->duration_for_fps;
+}
+}
+}
+
for (i = 0; i < mov->bitrates_count && i < s->nb_streams; i++) {
if (mov->bitrates[i]) {
s->streams[i]->codec->bit_rate = mov->bitrates[i];
--
1.9.3 (Apple Git-50)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] hlsenc.c, segment.c: propagate defaults to mpegts
This fixes the abnormally high ts overhead in the files produced by the
HLS and segments muxers. See https://trac.ffmpeg.org/ticket/2857 . For
example makes it much more likely that it can produces streams that fit
under the 64kb App store limit.
---
libavformat/hlsenc.c | 10 ++
libavformat/segment.c | 1 +
2 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index 8b180dc..0525e70 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -85,14 +85,16 @@ static int hls_mux_init(AVFormatContext *s)
{
HLSContext *hls = s->priv_data;
AVFormatContext *oc;
-int i;
+int i, ret;
-hls->avf = oc = avformat_alloc_context();
-if (!oc)
-return AVERROR(ENOMEM);
+ret = avformat_alloc_output_context2(&hls->avf, hls->oformat, NULL, NULL);
+if (ret < 0)
+return ret;
+oc = hls->avf;
oc->oformat= hls->oformat;
oc->interrupt_callback = s->interrupt_callback;
+oc->max_delay = s->max_delay;
av_dict_copy(&oc->metadata, s->metadata, 0);
for (i = 0; i < s->nb_streams; i++) {
diff --git a/libavformat/segment.c b/libavformat/segment.c
index 6183208..9db2607 100644
--- a/libavformat/segment.c
+++ b/libavformat/segment.c
@@ -143,6 +143,7 @@ static int segment_mux_init(AVFormatContext *s)
oc = seg->avf;
oc->interrupt_callback = s->interrupt_callback;
+oc->max_delay = s->max_delay;
av_dict_copy(&oc->metadata, s->metadata, 0);
for (i = 0; i < s->nb_streams; i++) {
--
1.9.3 (Apple Git-50)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/2] avformat/mov: auodetect "use_mfra_for"
On 14 October 2014 23:48, Michael Niedermayer wrote: > On Mon, Oct 13, 2014 at 01:45:18PM +0200, Michael Niedermayer wrote: >> Signed-off-by: Michael Niedermayer >> --- >> libavformat/isom.h |1 + >> libavformat/mov.c | 29 +++-- >> 2 files changed, 28 insertions(+), 2 deletions(-) > > mika, any comments ? > does this look ok to you? > i assume you know more about the software and problem than i do ... This does correctly select the right behaviour for my input files, and looks reasonably conservative. LGTM. > > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > No human being will ever know the Truth, for even if they happen to say it > by chance, they would not even known they had done so. -- Xenophanes ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/2] avformat/mov: auodetect "use_mfra_for"
On 14 October 2014 23:48, Michael Niedermayer wrote: > On Mon, Oct 13, 2014 at 01:45:18PM +0200, Michael Niedermayer wrote: >> Signed-off-by: Michael Niedermayer >> --- >> libavformat/isom.h |1 + >> libavformat/mov.c | 29 +++-- >> 2 files changed, 28 insertions(+), 2 deletions(-) > > mika, any comments ? > does this look ok to you? > i assume you know more about the software and problem than i do ... Thanks for the change. I'll get back to you by end-of-day today. > > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > No human being will ever know the Truth, for even if they happen to say it > by chance, they would not even known they had done so. -- Xenophanes ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] mov.c: allow reading fragment start dts/pts from fragmented mp4
On 11 October 2014 23:02, Michael Niedermayer wrote:
> On Sat, Oct 11, 2014 at 08:38:14PM +0300, Mika Raento wrote:
>> On 11 October 2014 20:20, Michael Niedermayer wrote:
>> > On Sat, Oct 11, 2014 at 06:43:48PM +0300, Mika Raento wrote:
>> >> This introduces a new option to the mov demuxer: -use_mfra_for
>> >> (pts|dts). When it's given and moofs and a MFRA are present, the MFRA's
>> >> TFRAs are read for fragment start times.
>> >>
>> >> Unfortunately some programs that produce fragmented mp4s use the TFRA
>> >> time field for dts and some for pts. There is no realistic way to detect
>> >> which is the case, hence the responsibility is punted onto the user.
>> >> This also means that no behavioural change is enabled by default - you
>> >> must pass either dts or pts for anything to happen.
>> >>
>> >> Without this change, timestamps for some discontinuous fragmented mp4 are
>> >> wrong, and cause audio/video desync and are not usable for generating
>> >> HLS.
>> >> ---
>> > [...]
>> >> @@ -3943,6 +4111,15 @@ static const AVOption options[] = {
>> >> 0, 1, AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM},
>> >> {"ignore_editlist", "", offsetof(MOVContext, ignore_editlist),
>> >> FF_OPT_TYPE_INT, {.i64 = 0},
>> >> 0, 1, AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM},
>> >> +{"use_mfra_for",
>> >> +"use mfra for fragment timestamps",
>> >> +offsetof(MOVContext, use_mfra_for), FF_OPT_TYPE_INT, {.i64 = 0},
>> >> +0, FF_MOV_FLAG_MFRA_PTS,
>> >> AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM,
>> >> +"use_mfra_for"},
>> >
>> > maybe a FF_MOV_FLAG_MFRA_AUTO which is set by default could be used to
>> > guess based on some encoder/muxer identification what to do.
>> >
>> > do the files which need this option have something that idenifies
>> > them? like in metadata, compatible brands or other?
>> > and can you share a file/testcase which needs this patch ?
>>
>> I can share a file privately (copyrighted). I do not have a reduced
>> test case. Let me know if you think that's helpful.
>
> dunno, might be usefull
>
>
>>
>> The files come from a commercial packager that is getting
>> discontinuous input from a commercial transcoder. My understanding is
>> that, the timestamp issue comes from the packager, rather than the
>> transcoder. The underlying software in the packager is Code-shop's
>> Unified Streaming, but it's been fed from proprietary software.
>>
>> This is what ffprobe says, it's not very specific:
>>
>> (multiprofile-streamers)mikie@universe:~/devel/multiprofile-streamers$
>> ffprobe -i ../multiprofile-fixer/hls-disco/50603088.ismv
>> ffprobe version N-66765-g67d95df Copyright (c) 2007-2014 the FFmpeg
>> developers
>> built on Oct 11 2014 18:42:41 with Apple LLVM version 6.0
>> (clang-600.0.51) (based on LLVM 3.5svn)
>> configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD
>> --enable-shared --enable-pthreads --enable-gpl --enable-version3
>> --enable-nonfree --enable-hardcoded-tables --enable-avresample
>> --enable-vda --cc=clang --host-cflags= --host-ldflags=
>> --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid
>> --enable-debug --disable-stripping --enable-libass
>> libavutil 54. 10.100 / 54. 10.100
>> libavcodec 56. 4.101 / 56. 4.101
>> libavformat56. 9.100 / 56. 9.100
>> libavdevice56. 1.100 / 56. 1.100
>> libavfilter 5. 1.103 / 5. 1.103
>> libavresample 2. 1. 0 / 2. 1. 0
>> libswscale 3. 1.100 / 3. 1.100
>> libswresample 1. 1.100 / 1. 1.100
>> libpostproc53. 1.100 / 53. 1.100
>> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from
>> '../multiprofile-fixer/hls-disco/50603088.ismv':
>> Metadata:
>> major_brand : isom
>> minor_version : 1
>> compatible_brands: isomiso2
>> Duration: 02:17:00.14, start: 0.00, bitrate: 4411 kb/s
>> Stream #0:0(mul): Audio: aac (mp4a / 0x6134706D), 48000 Hz,
>> stereo, fltp, 124 kb/s (default)
>> Metadata:
>> handler_name: Sound Handler
>> Stream #0:1(mul): Audio: aac (mp4a / 0x6134706D), 48000 Hz,
>> stereo, fltp, 29 kb/s (default)
>> Metadata:
>> handler_name: Sound Handler
Re: [FFmpeg-devel] [PATCH] mov.c: allow reading fragment start dts/pts from fragmented mp4
Oh, and the transcoder was giving discontinuous input because there
was packetloss in the original DVB stream it was getting.
On 11 October 2014 20:38, Mika Raento wrote:
> On 11 October 2014 20:20, Michael Niedermayer wrote:
>> On Sat, Oct 11, 2014 at 06:43:48PM +0300, Mika Raento wrote:
>>> This introduces a new option to the mov demuxer: -use_mfra_for
>>> (pts|dts). When it's given and moofs and a MFRA are present, the MFRA's
>>> TFRAs are read for fragment start times.
>>>
>>> Unfortunately some programs that produce fragmented mp4s use the TFRA
>>> time field for dts and some for pts. There is no realistic way to detect
>>> which is the case, hence the responsibility is punted onto the user.
>>> This also means that no behavioural change is enabled by default - you
>>> must pass either dts or pts for anything to happen.
>>>
>>> Without this change, timestamps for some discontinuous fragmented mp4 are
>>> wrong, and cause audio/video desync and are not usable for generating
>>> HLS.
>>> ---
>> [...]
>>> @@ -3943,6 +4111,15 @@ static const AVOption options[] = {
>>> 0, 1, AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM},
>>> {"ignore_editlist", "", offsetof(MOVContext, ignore_editlist),
>>> FF_OPT_TYPE_INT, {.i64 = 0},
>>> 0, 1, AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM},
>>> +{"use_mfra_for",
>>> +"use mfra for fragment timestamps",
>>> +offsetof(MOVContext, use_mfra_for), FF_OPT_TYPE_INT, {.i64 = 0},
>>> +0, FF_MOV_FLAG_MFRA_PTS,
>>> AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM,
>>> +"use_mfra_for"},
>>
>> maybe a FF_MOV_FLAG_MFRA_AUTO which is set by default could be used to
>> guess based on some encoder/muxer identification what to do.
>>
>> do the files which need this option have something that idenifies
>> them? like in metadata, compatible brands or other?
>> and can you share a file/testcase which needs this patch ?
>
> I can share a file privately (copyrighted). I do not have a reduced
> test case. Let me know if you think that's helpful.
>
> The files come from a commercial packager that is getting
> discontinuous input from a commercial transcoder. My understanding is
> that, the timestamp issue comes from the packager, rather than the
> transcoder. The underlying software in the packager is Code-shop's
> Unified Streaming, but it's been fed from proprietary software.
>
> This is what ffprobe says, it's not very specific:
>
> (multiprofile-streamers)mikie@universe:~/devel/multiprofile-streamers$
> ffprobe -i ../multiprofile-fixer/hls-disco/50603088.ismv
> ffprobe version N-66765-g67d95df Copyright (c) 2007-2014 the FFmpeg developers
> built on Oct 11 2014 18:42:41 with Apple LLVM version 6.0
> (clang-600.0.51) (based on LLVM 3.5svn)
> configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD
> --enable-shared --enable-pthreads --enable-gpl --enable-version3
> --enable-nonfree --enable-hardcoded-tables --enable-avresample
> --enable-vda --cc=clang --host-cflags= --host-ldflags=
> --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid
> --enable-debug --disable-stripping --enable-libass
> libavutil 54. 10.100 / 54. 10.100
> libavcodec 56. 4.101 / 56. 4.101
> libavformat56. 9.100 / 56. 9.100
> libavdevice56. 1.100 / 56. 1.100
> libavfilter 5. 1.103 / 5. 1.103
> libavresample 2. 1. 0 / 2. 1. 0
> libswscale 3. 1.100 / 3. 1.100
> libswresample 1. 1.100 / 1. 1.100
> libpostproc53. 1.100 / 53. 1.100
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from
> '../multiprofile-fixer/hls-disco/50603088.ismv':
> Metadata:
> major_brand : isom
> minor_version : 1
> compatible_brands: isomiso2
> Duration: 02:17:00.14, start: 0.00, bitrate: 4411 kb/s
> Stream #0:0(mul): Audio: aac (mp4a / 0x6134706D), 48000 Hz,
> stereo, fltp, 124 kb/s (default)
> Metadata:
> handler_name: Sound Handler
> Stream #0:1(mul): Audio: aac (mp4a / 0x6134706D), 48000 Hz,
> stereo, fltp, 29 kb/s (default)
> Metadata:
> handler_name: Sound Handler
> Stream #0:2(und): Video: h264 (Main) (avc1 / 0x31637661),
> yuv420p(tv, bt470bg), 480x270 [SAR 1:1 DAR 16:9], 597 kb/s, 25 fps, 25
> tbr, 1k tbn, 50 tbc (default)
> Metadata:
> handler_name: Video Handler
> encoder : AVC Coding
> Stream #0:3(und): Video: h264 (Main) (avc1 / 0x31637661),
> yuv420p(tv, bt
Re: [FFmpeg-devel] [PATCH] mov.c: allow reading fragment start dts/pts from fragmented mp4
On 11 October 2014 20:20, Michael Niedermayer wrote:
> On Sat, Oct 11, 2014 at 06:43:48PM +0300, Mika Raento wrote:
>> This introduces a new option to the mov demuxer: -use_mfra_for
>> (pts|dts). When it's given and moofs and a MFRA are present, the MFRA's
>> TFRAs are read for fragment start times.
>>
>> Unfortunately some programs that produce fragmented mp4s use the TFRA
>> time field for dts and some for pts. There is no realistic way to detect
>> which is the case, hence the responsibility is punted onto the user.
>> This also means that no behavioural change is enabled by default - you
>> must pass either dts or pts for anything to happen.
>>
>> Without this change, timestamps for some discontinuous fragmented mp4 are
>> wrong, and cause audio/video desync and are not usable for generating
>> HLS.
>> ---
> [...]
>> @@ -3943,6 +4111,15 @@ static const AVOption options[] = {
>> 0, 1, AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM},
>> {"ignore_editlist", "", offsetof(MOVContext, ignore_editlist),
>> FF_OPT_TYPE_INT, {.i64 = 0},
>> 0, 1, AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM},
>> +{"use_mfra_for",
>> +"use mfra for fragment timestamps",
>> +offsetof(MOVContext, use_mfra_for), FF_OPT_TYPE_INT, {.i64 = 0},
>> +0, FF_MOV_FLAG_MFRA_PTS,
>> AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM,
>> +"use_mfra_for"},
>
> maybe a FF_MOV_FLAG_MFRA_AUTO which is set by default could be used to
> guess based on some encoder/muxer identification what to do.
>
> do the files which need this option have something that idenifies
> them? like in metadata, compatible brands or other?
> and can you share a file/testcase which needs this patch ?
I can share a file privately (copyrighted). I do not have a reduced
test case. Let me know if you think that's helpful.
The files come from a commercial packager that is getting
discontinuous input from a commercial transcoder. My understanding is
that, the timestamp issue comes from the packager, rather than the
transcoder. The underlying software in the packager is Code-shop's
Unified Streaming, but it's been fed from proprietary software.
This is what ffprobe says, it's not very specific:
(multiprofile-streamers)mikie@universe:~/devel/multiprofile-streamers$
ffprobe -i ../multiprofile-fixer/hls-disco/50603088.ismv
ffprobe version N-66765-g67d95df Copyright (c) 2007-2014 the FFmpeg developers
built on Oct 11 2014 18:42:41 with Apple LLVM version 6.0
(clang-600.0.51) (based on LLVM 3.5svn)
configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD
--enable-shared --enable-pthreads --enable-gpl --enable-version3
--enable-nonfree --enable-hardcoded-tables --enable-avresample
--enable-vda --cc=clang --host-cflags= --host-ldflags=
--enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid
--enable-debug --disable-stripping --enable-libass
libavutil 54. 10.100 / 54. 10.100
libavcodec 56. 4.101 / 56. 4.101
libavformat56. 9.100 / 56. 9.100
libavdevice56. 1.100 / 56. 1.100
libavfilter 5. 1.103 / 5. 1.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.100 / 3. 1.100
libswresample 1. 1.100 / 1. 1.100
libpostproc53. 1.100 / 53. 1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from
'../multiprofile-fixer/hls-disco/50603088.ismv':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isomiso2
Duration: 02:17:00.14, start: 0.00, bitrate: 4411 kb/s
Stream #0:0(mul): Audio: aac (mp4a / 0x6134706D), 48000 Hz,
stereo, fltp, 124 kb/s (default)
Metadata:
handler_name: Sound Handler
Stream #0:1(mul): Audio: aac (mp4a / 0x6134706D), 48000 Hz,
stereo, fltp, 29 kb/s (default)
Metadata:
handler_name: Sound Handler
Stream #0:2(und): Video: h264 (Main) (avc1 / 0x31637661),
yuv420p(tv, bt470bg), 480x270 [SAR 1:1 DAR 16:9], 597 kb/s, 25 fps, 25
tbr, 1k tbn, 50 tbc (default)
Metadata:
handler_name: Video Handler
encoder : AVC Coding
Stream #0:3(und): Video: h264 (Main) (avc1 / 0x31637661),
yuv420p(tv, bt470bg), 640x360 [SAR 1:1 DAR 16:9], 1195 kb/s, 25 fps,
25 tbr, 1k tbn, 50 tbc (default)
Metadata:
handler_name: Video Handler
encoder : AVC Coding
Stream #0:4(und): Video: h264 (High) (avc1 / 0x31637661),
yuv420p(tv, bt470bg), 768x432 [SAR 1:1 DAR 16:9], 2487 kb/s, 25 fps,
25 tbr, 1k tbn, 50 tbc (default)
Metadata:
handler_name: Video Handler
encoder : AVC Coding
>
> Also ISO/IEC 14496-12:2008(E) seems to say this:
> This box provides only a hint as to where random access points are; the
> movie
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
Resubmission follows Carl's advice on braces. On 11 October 2014 16:49, Carl Eugen Hoyos wrote: > Mika Raento iki.fi> writes: > >> Somehow I'd gotten the impression that the opposite >> was preferred, and indeed mov.c tends not to have >> those braces. Has the convention changed? > > I don't think so, tools/patcheck should tell you more. > > Carl Eugen > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] mov.c: allow reading fragment start dts/pts from fragmented mp4
This introduces a new option to the mov demuxer: -use_mfra_for
(pts|dts). When it's given and moofs and a MFRA are present, the MFRA's
TFRAs are read for fragment start times.
Unfortunately some programs that produce fragmented mp4s use the TFRA
time field for dts and some for pts. There is no realistic way to detect
which is the case, hence the responsibility is punted onto the user.
This also means that no behavioural change is enabled by default - you
must pass either dts or pts for anything to happen.
Without this change, timestamps for some discontinuous fragmented mp4 are
wrong, and cause audio/video desync and are not usable for generating
HLS.
---
libavformat/isom.h | 20 ++
libavformat/mov.c | 177 +
2 files changed, 197 insertions(+)
diff --git a/libavformat/isom.h b/libavformat/isom.h
index 979e967..343cdd3 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -78,6 +78,7 @@ typedef struct MOVFragment {
unsigned duration;
unsigned size;
unsigned flags;
+int64_t time;
} MOVFragment;
typedef struct MOVTrackExt {
@@ -93,6 +94,18 @@ typedef struct MOVSbgp {
unsigned int index;
} MOVSbgp;
+typedef struct MOVFragmentIndexItem {
+int64_t moof_offset;
+int64_t time;
+} MOVFragmentIndexItem;
+
+typedef struct MOVFragmentIndex {
+unsigned track_id;
+unsigned item_count;
+unsigned current_item;
+MOVFragmentIndexItem *items;
+} MOVFragmentIndex;
+
typedef struct MOVStreamContext {
AVIOContext *pb;
int pb_is_copied;
@@ -171,6 +184,10 @@ typedef struct MOVContext {
int *bitrates; ///< bitrates read before streams creation
int bitrates_count;
int moov_retry;
+int use_mfra_for;
+int has_looked_for_mfra;
+MOVFragmentIndex** fragment_index_data;
+unsigned fragment_index_count;
} MOVContext;
int ff_mp4_read_descr_len(AVIOContext *pb);
@@ -237,4 +254,7 @@ enum AVCodecID ff_mov_get_lpcm_codec_id(int bps, int flags);
int ff_mov_read_stsd_entries(MOVContext *c, AVIOContext *pb, int entries);
void ff_mov_write_chan(AVIOContext *pb, int64_t channel_layout);
+#define FF_MOV_FLAG_MFRA_DTS 1
+#define FF_MOV_FLAG_MFRA_PTS 2
+
#endif /* AVFORMAT_ISOM_H */
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 4ff46dd..71c2e29 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -68,6 +68,7 @@ typedef struct MOVParseTableEntry {
} MOVParseTableEntry;
static int mov_read_default(MOVContext *c, AVIOContext *pb, MOVAtom atom);
+static int mov_read_mfra(MOVContext *c, AVIOContext *f);
static int mov_metadata_track_or_disc_number(MOVContext *c, AVIOContext *pb,
unsigned len, const char *key)
@@ -776,6 +777,21 @@ static int mov_read_moov(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
static int mov_read_moof(MOVContext *c, AVIOContext *pb, MOVAtom atom)
{
+if (!c->has_looked_for_mfra && c->use_mfra_for > 0) {
+c->has_looked_for_mfra = 1;
+if (pb->seekable) {
+av_log(c->fc, AV_LOG_VERBOSE, "stream has moof boxes, will look "
+"for a mfra\n");
+int ret;
+if ((ret = mov_read_mfra(c, pb)) < 0) {
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but failed to "
+"read the mfra (may be a live ismv)\n");
+}
+} else {
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but stream is not "
+"seekable, can not look for mfra\n");
+}
+}
c->fragment.moof_offset = c->fragment.implicit_offset = avio_tell(pb) - 8;
av_dlog(c->fc, "moof offset %"PRIx64"\n", c->fragment.moof_offset);
return mov_read_default(c, pb, atom);
@@ -2807,6 +2823,7 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
{
MOVFragment *frag = &c->fragment;
MOVTrackExt *trex = NULL;
+MOVFragmentIndex* index = NULL;
int flags, track_id, i;
avio_r8(pb); /* version */
@@ -2825,6 +2842,15 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
return AVERROR_INVALIDDATA;
}
+for (i = 0; i < c->fragment_index_count; i++) {
+MOVFragmentIndex* candidate = c->fragment_index_data[i];
+if (candidate->track_id == frag->track_id) {
+av_log(c->fc, AV_LOG_DEBUG,
+ "found fragment index for track %u\n", frag->track_id);
+index = candidate;
+break;
+}
+}
frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
avio_rb64(pb) : flags &
MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
@@ -2837,6 +2863,25 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
avio_rb32(pb) : trex->size;
frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
avi
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
On 11 October 2014 16:41, Carl Eugen Hoyos wrote:
> Mika Raento iki.fi> writes:
>
>> +if (ret < 0)
>> +av_log(c->fc, AV_LOG_ERROR,
>> + "failed to seek back after looking for mfra\n");
>> +else
>> +ret = 0;
>
> If you resubmit please consider making this:
> if (ret < 0) {
> avl_log();
> } else {
> ret = 0;
> }
>
> This makes future patches much easier to read and costs only
> a line.
I can definitely do that. Somehow I'd gotten the impression that the
opposite was preferred, and indeed mov.c tends not to have those
braces. Has the convention changed?
>
> Thank you, Carl Eugen
Mika
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] mov.c: read fragment start pts from fragmented mp4
If present, an MFRA box and its TFRAs are read for fragment start times.
Without this change, timestamps for discontinuous fragmented mp4 are
wrong, and cause audio/video desync and are not usable for generating
HLS.
---
libavformat/isom.h | 16 ++
libavformat/mov.c | 158 +
2 files changed, 174 insertions(+)
diff --git a/libavformat/isom.h b/libavformat/isom.h
index 979e967..52bd9cb4e 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -78,6 +78,7 @@ typedef struct MOVFragment {
unsigned duration;
unsigned size;
unsigned flags;
+int64_t time;
} MOVFragment;
typedef struct MOVTrackExt {
@@ -93,6 +94,18 @@ typedef struct MOVSbgp {
unsigned int index;
} MOVSbgp;
+typedef struct MOVFragmentIndexItem {
+int64_t moof_offset;
+int64_t time;
+} MOVFragmentIndexItem;
+
+typedef struct MOVFragmentIndex {
+unsigned track_id;
+unsigned item_count;
+unsigned current_item;
+MOVFragmentIndexItem *items;
+} MOVFragmentIndex;
+
typedef struct MOVStreamContext {
AVIOContext *pb;
int pb_is_copied;
@@ -171,6 +184,9 @@ typedef struct MOVContext {
int *bitrates; ///< bitrates read before streams creation
int bitrates_count;
int moov_retry;
+int has_looked_for_mfra;
+MOVFragmentIndex** fragment_index_data;
+unsigned fragment_index_count;
} MOVContext;
int ff_mp4_read_descr_len(AVIOContext *pb);
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 4ff46dd..6e8f05b 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -68,6 +68,7 @@ typedef struct MOVParseTableEntry {
} MOVParseTableEntry;
static int mov_read_default(MOVContext *c, AVIOContext *pb, MOVAtom atom);
+static int mov_read_mfra(MOVContext *c, AVIOContext *f);
static int mov_metadata_track_or_disc_number(MOVContext *c, AVIOContext *pb,
unsigned len, const char *key)
@@ -776,6 +777,19 @@ static int mov_read_moov(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
static int mov_read_moof(MOVContext *c, AVIOContext *pb, MOVAtom atom)
{
+if (!c->has_looked_for_mfra) {
+c->has_looked_for_mfra = 1;
+if (pb->seekable) {
+av_log(c->fc, AV_LOG_VERBOSE, "stream has moof boxes, will look "
+"for a mfra\n");
+int ret;
+if ((ret = mov_read_mfra(c, pb)) < 0)
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but failed to "
+"read the mfra (may be a live ismv)\n");
+} else
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but stream is not "
+"seekable, can not look for mfra\n");
+}
c->fragment.moof_offset = c->fragment.implicit_offset = avio_tell(pb) - 8;
av_dlog(c->fc, "moof offset %"PRIx64"\n", c->fragment.moof_offset);
return mov_read_default(c, pb, atom);
@@ -2807,6 +2821,7 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
{
MOVFragment *frag = &c->fragment;
MOVTrackExt *trex = NULL;
+MOVFragmentIndex* index = NULL;
int flags, track_id, i;
avio_r8(pb); /* version */
@@ -2825,6 +2840,15 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
return AVERROR_INVALIDDATA;
}
+for (i = 0; i < c->fragment_index_count; i++) {
+MOVFragmentIndex* candidate = c->fragment_index_data[i];
+if (candidate->track_id == frag->track_id) {
+av_log(c->fc, AV_LOG_DEBUG,
+ "found fragment index for track %u\n", frag->track_id);
+index = candidate;
+break;
+}
+}
frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
avio_rb64(pb) : flags &
MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
@@ -2837,6 +2861,25 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
avio_rb32(pb) : trex->size;
frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
avio_rb32(pb) : trex->flags;
+frag->time = AV_NOPTS_VALUE;
+if (index) {
+int i, found = 0;
+for (i = index->current_item; i < index->item_count; i++) {
+if (frag->implicit_offset == index->items[i].moof_offset) {
+av_log(c->fc, AV_LOG_DEBUG, "found fragment index entry "
+"for track %u and moof_offset %"PRId64"\n",
+frag->track_id, index->items[i].moof_offset);
+frag->time = index->items[i].time;
+index->current_item = i + 1;
+found = 1;
+}
+}
+if (!found) {
+av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index "
+ "but it doesn't have an (in-order) entry for moof_offset "
+ "%"PRId64"\n", frag->tra
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
Hm. After fixing the obvious problems with matching moofs with tfra entries I noticed that the timestamps aren't right even after that. AFAICT, ffmpeg's fragmentation code puts dts in the tfra time. In movenc.c: mov->tracks[i].frag_start += mov->tracks[i].start_dts + mov->tracks[i].track_duration - mov->tracks[i].cluster[0].dts; Whereas l-smash's code definitely tries to put a presentation timestamp there. I wonder if we should add some flags after all - I don't think it'll be easy to automatically detect what the creating program has been doing. I'll submit the version with fixed matching shortly. Mika On 11 October 2014 15:27, Mika Raento wrote: > Interesting. The input.mov created with ./ffmpeg -i > matrixbench_mpeg2.mpg -t 1 -frag_duration 200k input.mov does not > have tfra entries for all the moofs for the audio track. The single > tfra it has is not for the first moof, but the last one. > > I'll improve the matching of the moofs and tfra entries. > > Thanks again for your help. > > Mika > > On 11 October 2014 14:21, Michael Niedermayer wrote: >> On Sat, Oct 11, 2014 at 07:25:52AM +0300, Mika Raento wrote: >>> If present, an MFRA box and its TFRAs are read for fragment start times. >>> >>> Without this change, timestamps for discontinuous fragmented mp4 are >>> wrong, and cause audio/video desync and are not usable for generating >>> HLS. >>> --- >>> libavformat/isom.h | 15 ++ >>> libavformat/mov.c | 146 >>> + >>> 2 files changed, 161 insertions(+) >> >> this appears to breaks timestamps for some files, i didnt investigate >> why >> >> ./ffmpeg -i matrixbench_mpeg2.mpg -t 1 -frag_duration 200k input.mov >> ./ffprobe input.mov -show_packets -print_format compact | grep audio >> >> >> packet|codec_type=audio|stream_index=1|pts=0|pts_time=0.00|dts=0|dts_time=0.00|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=172|pos=17590|flags=K >> >> packet|codec_type=audio|stream_index=1|pts=3840|pts_time=0.08|dts=3840|dts_time=0.08|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=315|pos=19048|flags=K >> >> packet|codec_type=audio|stream_index=1|pts=4864|pts_time=0.101333|dts=4864|dts_time=0.101333|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=297|pos=19363|flags=K >> >> packet|codec_type=audio|stream_index=1|pts=5888|pts_time=0.122667|dts=5888|dts_time=0.122667|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=9|pos=22964|flags=K >> >> packet|codec_type=audio|stream_index=1|pts=6912|pts_time=0.144000|dts=6912|dts_time=0.144000|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=216|pos=22973|flags=K >> >> packet|codec_type=audio|stream_index=1|pts=7936|pts_time=0.165333|dts=7936|dts_time=0.165333|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=327|pos=23521|flags=K >> >> packet|codec_type=audio|stream_index=1|pts=8960|pts_time=0.186667|dts=8960|dts_time=0.186667|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=327|pos=23848|flags=K >> -packet|codec_type=audio|stream_index=1|pts=9984|pts_time=0.208000|dts=9984|dts_time=0.208000|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=330|pos=35835|flags=K >> -packet|codec_type=audio|stream_index=1|pts=11008|pts_time=0.229333|dts=11008|dts_time=0.229333|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=317|pos=36165|flags=K >> -packet|codec_type=audio|stream_index=1|pts=12032|pts_time=0.250667|dts=12032|dts_time=0.250667|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=326|pos=36482|flags=K >> +packet|codec_type=audio|stream_index=1|pts=48896|pts_time=1.018667|dts=48896|dts_time=1.018667|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=330|pos=35835|flags=K >> +packet|codec_type=audio|stream_index=1|pts=49920|pts_time=1.04|dts=49920|dts_time=1.04|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=317|pos=36165|flags=K >> ... >> >> [...] >> -- >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >>
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
Interesting. The input.mov created with ./ffmpeg -i matrixbench_mpeg2.mpg -t 1 -frag_duration 200k input.mov does not have tfra entries for all the moofs for the audio track. The single tfra it has is not for the first moof, but the last one. I'll improve the matching of the moofs and tfra entries. Thanks again for your help. Mika On 11 October 2014 14:21, Michael Niedermayer wrote: > On Sat, Oct 11, 2014 at 07:25:52AM +0300, Mika Raento wrote: >> If present, an MFRA box and its TFRAs are read for fragment start times. >> >> Without this change, timestamps for discontinuous fragmented mp4 are >> wrong, and cause audio/video desync and are not usable for generating >> HLS. >> --- >> libavformat/isom.h | 15 ++ >> libavformat/mov.c | 146 >> + >> 2 files changed, 161 insertions(+) > > this appears to breaks timestamps for some files, i didnt investigate > why > > ./ffmpeg -i matrixbench_mpeg2.mpg -t 1 -frag_duration 200k input.mov > ./ffprobe input.mov -show_packets -print_format compact | grep audio > > > packet|codec_type=audio|stream_index=1|pts=0|pts_time=0.00|dts=0|dts_time=0.00|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=172|pos=17590|flags=K > > packet|codec_type=audio|stream_index=1|pts=3840|pts_time=0.08|dts=3840|dts_time=0.08|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=315|pos=19048|flags=K > > packet|codec_type=audio|stream_index=1|pts=4864|pts_time=0.101333|dts=4864|dts_time=0.101333|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=297|pos=19363|flags=K > > packet|codec_type=audio|stream_index=1|pts=5888|pts_time=0.122667|dts=5888|dts_time=0.122667|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=9|pos=22964|flags=K > > packet|codec_type=audio|stream_index=1|pts=6912|pts_time=0.144000|dts=6912|dts_time=0.144000|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=216|pos=22973|flags=K > > packet|codec_type=audio|stream_index=1|pts=7936|pts_time=0.165333|dts=7936|dts_time=0.165333|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=327|pos=23521|flags=K > > packet|codec_type=audio|stream_index=1|pts=8960|pts_time=0.186667|dts=8960|dts_time=0.186667|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=327|pos=23848|flags=K > -packet|codec_type=audio|stream_index=1|pts=9984|pts_time=0.208000|dts=9984|dts_time=0.208000|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=330|pos=35835|flags=K > -packet|codec_type=audio|stream_index=1|pts=11008|pts_time=0.229333|dts=11008|dts_time=0.229333|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=317|pos=36165|flags=K > -packet|codec_type=audio|stream_index=1|pts=12032|pts_time=0.250667|dts=12032|dts_time=0.250667|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=326|pos=36482|flags=K > +packet|codec_type=audio|stream_index=1|pts=48896|pts_time=1.018667|dts=48896|dts_time=1.018667|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=330|pos=35835|flags=K > +packet|codec_type=audio|stream_index=1|pts=49920|pts_time=1.04|dts=49920|dts_time=1.04|duration=1024|duration_time=0.021333|convergence_duration=N/A|convergence_duration_time=N/A|size=317|pos=36165|flags=K > ... > > [...] > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > During times of universal deceit, telling the truth becomes a > revolutionary act. -- George Orwell > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
If present, an MFRA box and its TFRAs are read for fragment start times.
Without this change, timestamps for discontinuous fragmented mp4 are
wrong, and cause audio/video desync and are not usable for generating
HLS.
---
libavformat/isom.h | 15 ++
libavformat/mov.c | 146 +
2 files changed, 161 insertions(+)
diff --git a/libavformat/isom.h b/libavformat/isom.h
index 979e967..80800f7 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -78,6 +78,7 @@ typedef struct MOVFragment {
unsigned duration;
unsigned size;
unsigned flags;
+int64_t time;
} MOVFragment;
typedef struct MOVTrackExt {
@@ -93,6 +94,17 @@ typedef struct MOVSbgp {
unsigned int index;
} MOVSbgp;
+typedef struct MOVFragmentIndexItem {
+int64_t time;
+} MOVFragmentIndexItem;
+
+typedef struct MOVFragmentIndex {
+unsigned track_id;
+unsigned item_count;
+unsigned current_item;
+MOVFragmentIndexItem *items;
+} MOVFragmentIndex;
+
typedef struct MOVStreamContext {
AVIOContext *pb;
int pb_is_copied;
@@ -171,6 +183,9 @@ typedef struct MOVContext {
int *bitrates; ///< bitrates read before streams creation
int bitrates_count;
int moov_retry;
+int has_looked_for_mfra;
+MOVFragmentIndex** fragment_index_data;
+unsigned fragment_index_count;
} MOVContext;
int ff_mp4_read_descr_len(AVIOContext *pb);
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 4ff46dd..c9da196 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -68,6 +68,7 @@ typedef struct MOVParseTableEntry {
} MOVParseTableEntry;
static int mov_read_default(MOVContext *c, AVIOContext *pb, MOVAtom atom);
+static int mov_read_mfra(MOVContext *c, AVIOContext *f);
static int mov_metadata_track_or_disc_number(MOVContext *c, AVIOContext *pb,
unsigned len, const char *key)
@@ -776,6 +777,19 @@ static int mov_read_moov(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
static int mov_read_moof(MOVContext *c, AVIOContext *pb, MOVAtom atom)
{
+if (!c->has_looked_for_mfra) {
+c->has_looked_for_mfra = 1;
+if (pb->seekable) {
+av_log(c->fc, AV_LOG_VERBOSE, "stream has moof boxes, will look "
+"for a mfra\n");
+int ret;
+if ((ret = mov_read_mfra(c, pb)) < 0)
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but failed to "
+"read the mfra (may be a live ismv)\n");
+} else
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but stream is not "
+"seekable, can not look for mfra\n");
+}
c->fragment.moof_offset = c->fragment.implicit_offset = avio_tell(pb) - 8;
av_dlog(c->fc, "moof offset %"PRIx64"\n", c->fragment.moof_offset);
return mov_read_default(c, pb, atom);
@@ -2807,6 +2821,7 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
{
MOVFragment *frag = &c->fragment;
MOVTrackExt *trex = NULL;
+MOVFragmentIndex* index = NULL;
int flags, track_id, i;
avio_r8(pb); /* version */
@@ -2825,6 +2840,15 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
return AVERROR_INVALIDDATA;
}
+for (i = 0; i < c->fragment_index_count; i++) {
+MOVFragmentIndex* candidate = c->fragment_index_data[i];
+if (candidate->track_id == frag->track_id) {
+av_log(c->fc, AV_LOG_DEBUG,
+ "found fragment index for track %u\n", frag->track_id);
+index = candidate;
+break;
+}
+}
frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
avio_rb64(pb) : flags &
MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
@@ -2837,6 +2861,18 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
avio_rb32(pb) : trex->size;
frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
avio_rb32(pb) : trex->flags;
+frag->time = AV_NOPTS_VALUE;
+if (index) {
+if (index->current_item == index->item_count) {
+av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index "
+ "but it doesn't have entries for all tfhds at tfhd "
+ "%u\n", frag->track_id, index->current_item);
+} else if (index->current_item < index->item_count) {
+frag->time =
+index->items[index->current_item].time;
+index->current_item++;
+}
+}
av_dlog(c->fc, "frag flags 0x%x\n", frag->flags);
return 0;
}
@@ -2945,6 +2981,18 @@ static int mov_read_trun(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
sc->ctts_data[sc->ctts_count].duration = (flags & MOV_TRUN_SAMPLE_CTS)
?
avio_rb32(pb
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
On 10 October 2014 23:08, Mika Raento wrote: > Firstly, thank you for the detailed explanation. > > Secondly, how should we proceed? > > I am not confident I'm able to implement that correctly, especially > with no test coverage. > > My current implementation improves discontinuous fragmented mp4s > significantly (from unusable to close-to-perfect) while slightly > worsening the timestamps for non-discontinuous fragmented mp4s. I > definitely need it for our streams, and I think it would help other > people in the same situation. I am quite willing to spend time on > this, but I fear that I just don't have enough known inputs and > outputs to verify my implementation. > > Normally fragments are supposed to start on key frames, which should > have pts close to dts, but there are no guarantees. > > Some alternatives: > > 1. I can leave my implementation behind a flag. That's not very > friendly to others, but breaks no existing usage. > > 2. We can merge my code as-is, and hope somebody more knowledgeable > can fix it up later. > > 3. I can try to implement the algorithm described. This is the one I picked. I'm submitting a version that produces identical timestamps as master for Michael's test case and fixes my discontinuous ismvs' timestamps. Mika > > 4. Somebody helps me with either implementation or by providing test cases. > > Opinions? > > Mika > > > On 10 October 2014 20:11, Yusuke Nakamura > wrote: >> 2014-10-10 13:38 GMT+09:00 Mika Raento : >> >>> On 9 October 2014 23:37, Yusuke Nakamura >>> wrote: >>> > 2014-10-10 4:49 GMT+09:00 Michael Niedermayer : >>> > >>> >> On Thu, Oct 09, 2014 at 09:44:43PM +0200, Michael Niedermayer wrote: >>> >> > On Thu, Oct 09, 2014 at 06:57:59PM +0300, Mika Raento wrote: >>> >> > > If present, an MFRA box and its TFRAs are read for fragment start >>> >> times. >>> >> > > >>> >> > > Without this change, timestamps for discontinuous fragmented mp4 are >>> >> > > wrong, and cause audio/video desync and are not usable for >>> generating >>> >> > > HLS. >>> >> > > --- >>> >> > > libavformat/isom.h | 15 ++ >>> >> > > libavformat/mov.c | 140 >>> >> + >>> >> > > 2 files changed, 155 insertions(+) >>> >> > >>> >> > this seems to break some files >>> >> > >>> >> > for example a file generated with the following 2 commands: >>> >> > ffmpeg -i matrixbench_mpeg2.mpg -t 10 in.mp4 >>> >> > l-smash/cli/remuxer -i in.mp4 --fragment 1 -o test.mp4 >>> >> > >>> >> > ive not investigated why this doesnt work >>> >> >>> >> maybe above was unclear, so to clarify before someone is confused >>> >> test.mp4 from above plays with ffplay before te patch but not really >>> >> aferwards. The 2 commads are just to create such file >>> >> >>> >> [...] >>> >> >>> >> -- >>> >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >>> >> >>> >> Good people do not need laws to tell them to act responsibly, while bad >>> >> people will find a way around the laws. -- Plato >>> >> >>> >> ___ >>> >> ffmpeg-devel mailing list >>> >> ffmpeg-devel@ffmpeg.org >>> >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >> >>> >> >>> > The 'time' field in the tfra box is defined in presentation timeline, not >>> > composition or decode timeline. >>> > Therefore, generally, the value of 'time' can't be used for DTS directly >>> as >>> > long as following 14496-12. >>> > Maybe some derivatives of ISO Base Media file format define differently, >>> > but the spec of ISO Base Media file format defines 'time' as the >>> > presentation time of the sync sample. >>> > Presentation times are composition times after the application of any >>> edit >>> > list for the track. >>> > >>> > I have also some samples which use the 'time' as DTS of sync sample. >>> > Historically, the term 'presentation time' was n
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
Firstly, thank you for the detailed explanation. Secondly, how should we proceed? I am not confident I'm able to implement that correctly, especially with no test coverage. My current implementation improves discontinuous fragmented mp4s significantly (from unusable to close-to-perfect) while slightly worsening the timestamps for non-discontinuous fragmented mp4s. I definitely need it for our streams, and I think it would help other people in the same situation. I am quite willing to spend time on this, but I fear that I just don't have enough known inputs and outputs to verify my implementation. Normally fragments are supposed to start on key frames, which should have pts close to dts, but there are no guarantees. Some alternatives: 1. I can leave my implementation behind a flag. That's not very friendly to others, but breaks no existing usage. 2. We can merge my code as-is, and hope somebody more knowledgeable can fix it up later. 3. I can try to implement the algorithm described. 4. Somebody helps me with either implementation or by providing test cases. Opinions? Mika On 10 October 2014 20:11, Yusuke Nakamura wrote: > 2014-10-10 13:38 GMT+09:00 Mika Raento : > >> On 9 October 2014 23:37, Yusuke Nakamura >> wrote: >> > 2014-10-10 4:49 GMT+09:00 Michael Niedermayer : >> > >> >> On Thu, Oct 09, 2014 at 09:44:43PM +0200, Michael Niedermayer wrote: >> >> > On Thu, Oct 09, 2014 at 06:57:59PM +0300, Mika Raento wrote: >> >> > > If present, an MFRA box and its TFRAs are read for fragment start >> >> times. >> >> > > >> >> > > Without this change, timestamps for discontinuous fragmented mp4 are >> >> > > wrong, and cause audio/video desync and are not usable for >> generating >> >> > > HLS. >> >> > > --- >> >> > > libavformat/isom.h | 15 ++ >> >> > > libavformat/mov.c | 140 >> >> + >> >> > > 2 files changed, 155 insertions(+) >> >> > >> >> > this seems to break some files >> >> > >> >> > for example a file generated with the following 2 commands: >> >> > ffmpeg -i matrixbench_mpeg2.mpg -t 10 in.mp4 >> >> > l-smash/cli/remuxer -i in.mp4 --fragment 1 -o test.mp4 >> >> > >> >> > ive not investigated why this doesnt work >> >> >> >> maybe above was unclear, so to clarify before someone is confused >> >> test.mp4 from above plays with ffplay before te patch but not really >> >> aferwards. The 2 commads are just to create such file >> >> >> >> [...] >> >> >> >> -- >> >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >> >> >> >> Good people do not need laws to tell them to act responsibly, while bad >> >> people will find a way around the laws. -- Plato >> >> >> >> ___ >> >> ffmpeg-devel mailing list >> >> ffmpeg-devel@ffmpeg.org >> >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> >> >> >> > The 'time' field in the tfra box is defined in presentation timeline, not >> > composition or decode timeline. >> > Therefore, generally, the value of 'time' can't be used for DTS directly >> as >> > long as following 14496-12. >> > Maybe some derivatives of ISO Base Media file format define differently, >> > but the spec of ISO Base Media file format defines 'time' as the >> > presentation time of the sync sample. >> > Presentation times are composition times after the application of any >> edit >> > list for the track. >> > >> > I have also some samples which use the 'time' as DTS of sync sample. >> > Historically, the term 'presentation time' was not defined clearly before >> > 14496-12:2012, this fact possibly may have brought about such >> inconsistency. >> >> Hm. So my changes aren't correct if there is an edit list? Because >> AFAICT without edit lists mov.c sets pkt->pts = pkt->dts. >> > > Wrong. PTS == DTS is nothing to do with edit list. Generally, CTS != DTS > occurs only when frame reordering exists. > Even if there is no edit list for a track, there is implicit edit for that > track, and in this case PTS == (CTS + alpha)*mvhd.timescale/mdhd.timescale, > where the constant alpha depends on the implementation. &
[FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
If present, an MFRA box and its TFRAs are read for fragment start times.
Without this change, timestamps for discontinuous fragmented mp4 are
wrong, and cause audio/video desync and are not usable for generating
HLS.
---
libavformat/isom.h | 15 ++
libavformat/mov.c | 140 +
2 files changed, 155 insertions(+)
diff --git a/libavformat/isom.h b/libavformat/isom.h
index 979e967..80800f7 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -78,6 +78,7 @@ typedef struct MOVFragment {
unsigned duration;
unsigned size;
unsigned flags;
+int64_t time;
} MOVFragment;
typedef struct MOVTrackExt {
@@ -93,6 +94,17 @@ typedef struct MOVSbgp {
unsigned int index;
} MOVSbgp;
+typedef struct MOVFragmentIndexItem {
+int64_t time;
+} MOVFragmentIndexItem;
+
+typedef struct MOVFragmentIndex {
+unsigned track_id;
+unsigned item_count;
+unsigned current_item;
+MOVFragmentIndexItem *items;
+} MOVFragmentIndex;
+
typedef struct MOVStreamContext {
AVIOContext *pb;
int pb_is_copied;
@@ -171,6 +183,9 @@ typedef struct MOVContext {
int *bitrates; ///< bitrates read before streams creation
int bitrates_count;
int moov_retry;
+int has_looked_for_mfra;
+MOVFragmentIndex** fragment_index_data;
+unsigned fragment_index_count;
} MOVContext;
int ff_mp4_read_descr_len(AVIOContext *pb);
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 4ff46dd..69fa5bd 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -68,6 +68,7 @@ typedef struct MOVParseTableEntry {
} MOVParseTableEntry;
static int mov_read_default(MOVContext *c, AVIOContext *pb, MOVAtom atom);
+static int mov_read_mfra(MOVContext *c, AVIOContext *f);
static int mov_metadata_track_or_disc_number(MOVContext *c, AVIOContext *pb,
unsigned len, const char *key)
@@ -776,6 +777,19 @@ static int mov_read_moov(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
static int mov_read_moof(MOVContext *c, AVIOContext *pb, MOVAtom atom)
{
+if (!c->has_looked_for_mfra) {
+c->has_looked_for_mfra = 1;
+if (pb->seekable) {
+av_log(c->fc, AV_LOG_VERBOSE, "stream has moof boxes, will look "
+"for a mfra\n");
+int ret;
+if ((ret = mov_read_mfra(c, pb)) < 0)
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but failed to "
+"read the mfra (may be a live ismv)\n");
+} else
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but stream is not "
+"seekable, can not look for mfra\n");
+}
c->fragment.moof_offset = c->fragment.implicit_offset = avio_tell(pb) - 8;
av_dlog(c->fc, "moof offset %"PRIx64"\n", c->fragment.moof_offset);
return mov_read_default(c, pb, atom);
@@ -2807,6 +2821,7 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
{
MOVFragment *frag = &c->fragment;
MOVTrackExt *trex = NULL;
+MOVFragmentIndex* index = NULL;
int flags, track_id, i;
avio_r8(pb); /* version */
@@ -2825,6 +2840,15 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
return AVERROR_INVALIDDATA;
}
+for (i = 0; i < c->fragment_index_count; i++) {
+MOVFragmentIndex* candidate = c->fragment_index_data[i];
+if (candidate->track_id == frag->track_id) {
+av_log(c->fc, AV_LOG_DEBUG,
+ "found fragment index for track %u\n", frag->track_id);
+index = candidate;
+break;
+}
+}
frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
avio_rb64(pb) : flags &
MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
@@ -2837,6 +2861,18 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
avio_rb32(pb) : trex->size;
frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
avio_rb32(pb) : trex->flags;
+frag->time = AV_NOPTS_VALUE;
+if (index) {
+if (index->current_item == index->item_count) {
+av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index "
+ "but it doesn't have entries for all tfhds at tfhd "
+ "%u\n", frag->track_id, index->current_item);
+} else if (index->current_item < index->item_count) {
+frag->time =
+index->items[index->current_item].time;
+index->current_item++;
+}
+}
av_dlog(c->fc, "frag flags 0x%x\n", frag->flags);
return 0;
}
@@ -2929,6 +2965,12 @@ static int mov_read_trun(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
if (flags & MOV_TRUN_DATA_OFFSET)data_offset=
avio_rb32(pb);
if (flags & MOV_TRUN_FIRST_SAMPLE_FLAGS) first_sample_flag
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
Ah. My approach to matching truns to fragment times was way too naive. Rewritten to look up the time by sequence number and to handle multiple truns inside a single traf. Resubmitting. Mika On 9 October 2014 22:44, Michael Niedermayer wrote: > On Thu, Oct 09, 2014 at 06:57:59PM +0300, Mika Raento wrote: >> If present, an MFRA box and its TFRAs are read for fragment start times. >> >> Without this change, timestamps for discontinuous fragmented mp4 are >> wrong, and cause audio/video desync and are not usable for generating >> HLS. >> --- >> libavformat/isom.h | 15 ++ >> libavformat/mov.c | 140 >> + >> 2 files changed, 155 insertions(+) > > this seems to break some files > > for example a file generated with the following 2 commands: > ffmpeg -i matrixbench_mpeg2.mpg -t 10 in.mp4 > l-smash/cli/remuxer -i in.mp4 --fragment 1 -o test.mp4 > > ive not investigated why this doesnt work > > [...] > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > I know you won't believe me, but the highest form of Human Excellence is > to question oneself and others. -- Socrates > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
On 9 October 2014 23:37, Yusuke Nakamura wrote: > 2014-10-10 4:49 GMT+09:00 Michael Niedermayer : > >> On Thu, Oct 09, 2014 at 09:44:43PM +0200, Michael Niedermayer wrote: >> > On Thu, Oct 09, 2014 at 06:57:59PM +0300, Mika Raento wrote: >> > > If present, an MFRA box and its TFRAs are read for fragment start >> times. >> > > >> > > Without this change, timestamps for discontinuous fragmented mp4 are >> > > wrong, and cause audio/video desync and are not usable for generating >> > > HLS. >> > > --- >> > > libavformat/isom.h | 15 ++ >> > > libavformat/mov.c | 140 >> + >> > > 2 files changed, 155 insertions(+) >> > >> > this seems to break some files >> > >> > for example a file generated with the following 2 commands: >> > ffmpeg -i matrixbench_mpeg2.mpg -t 10 in.mp4 >> > l-smash/cli/remuxer -i in.mp4 --fragment 1 -o test.mp4 >> > >> > ive not investigated why this doesnt work >> >> maybe above was unclear, so to clarify before someone is confused >> test.mp4 from above plays with ffplay before te patch but not really >> aferwards. The 2 commads are just to create such file >> >> [...] >> >> -- >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >> >> Good people do not need laws to tell them to act responsibly, while bad >> people will find a way around the laws. -- Plato >> >> ___ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> > The 'time' field in the tfra box is defined in presentation timeline, not > composition or decode timeline. > Therefore, generally, the value of 'time' can't be used for DTS directly as > long as following 14496-12. > Maybe some derivatives of ISO Base Media file format define differently, > but the spec of ISO Base Media file format defines 'time' as the > presentation time of the sync sample. > Presentation times are composition times after the application of any edit > list for the track. > > I have also some samples which use the 'time' as DTS of sync sample. > Historically, the term 'presentation time' was not defined clearly before > 14496-12:2012, this fact possibly may have brought about such inconsistency. Hm. So my changes aren't correct if there is an edit list? Because AFAICT without edit lists mov.c sets pkt->pts = pkt->dts. Would you mind explaining how edit lists and fragment times are supposed to work together? Mika > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
Right. Will definitely take a look. I wouldn't mind help turning that into a fate testcase. I've been struggling with writing fate tests. At least for somebody like me, having more coverage in the fate tests would help enormously, as I definitely do not have the required expertise to know what may break with my changes (and I'm thankful for the reviews that try to fix that problem). Mika On 9 October 2014 22:49, Michael Niedermayer wrote: > On Thu, Oct 09, 2014 at 09:44:43PM +0200, Michael Niedermayer wrote: >> On Thu, Oct 09, 2014 at 06:57:59PM +0300, Mika Raento wrote: >> > If present, an MFRA box and its TFRAs are read for fragment start times. >> > >> > Without this change, timestamps for discontinuous fragmented mp4 are >> > wrong, and cause audio/video desync and are not usable for generating >> > HLS. >> > --- >> > libavformat/isom.h | 15 ++ >> > libavformat/mov.c | 140 >> > + >> > 2 files changed, 155 insertions(+) >> >> this seems to break some files >> >> for example a file generated with the following 2 commands: >> ffmpeg -i matrixbench_mpeg2.mpg -t 10 in.mp4 >> l-smash/cli/remuxer -i in.mp4 --fragment 1 -o test.mp4 >> >> ive not investigated why this doesnt work > > maybe above was unclear, so to clarify before someone is confused > test.mp4 from above plays with ffplay before te patch but not really > aferwards. The 2 commads are just to create such file > > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > Good people do not need laws to tell them to act responsibly, while bad > people will find a way around the laws. -- Plato > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
If present, an MFRA box and its TFRAs are read for fragment start times.
Without this change, timestamps for discontinuous fragmented mp4 are
wrong, and cause audio/video desync and are not usable for generating
HLS.
---
libavformat/isom.h | 15 ++
libavformat/mov.c | 140 +
2 files changed, 155 insertions(+)
diff --git a/libavformat/isom.h b/libavformat/isom.h
index 979e967..f368275 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -78,6 +78,7 @@ typedef struct MOVFragment {
unsigned duration;
unsigned size;
unsigned flags;
+int64_t time;
} MOVFragment;
typedef struct MOVTrackExt {
@@ -93,6 +94,17 @@ typedef struct MOVSbgp {
unsigned int index;
} MOVSbgp;
+typedef struct MOVFragmentIndexItem {
+int64_t time;
+} MOVFragmentIndexItem;
+
+typedef struct MOVFragmentIndex {
+unsigned track_id;
+unsigned current_item_index;
+unsigned item_count;
+MOVFragmentIndexItem *items;
+} MOVFragmentIndex;
+
typedef struct MOVStreamContext {
AVIOContext *pb;
int pb_is_copied;
@@ -171,6 +183,9 @@ typedef struct MOVContext {
int *bitrates; ///< bitrates read before streams creation
int bitrates_count;
int moov_retry;
+int has_looked_for_mfra;
+MOVFragmentIndex** fragment_index_data;
+unsigned fragment_index_count;
} MOVContext;
int ff_mp4_read_descr_len(AVIOContext *pb);
diff --git a/libavformat/mov.c b/libavformat/mov.c
index fdd0671..e1fa546 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -68,6 +68,7 @@ typedef struct MOVParseTableEntry {
} MOVParseTableEntry;
static int mov_read_default(MOVContext *c, AVIOContext *pb, MOVAtom atom);
+static int mov_read_mfra(MOVContext *c, AVIOContext *f);
static int mov_metadata_track_or_disc_number(MOVContext *c, AVIOContext *pb,
unsigned len, const char *key)
@@ -776,6 +777,19 @@ static int mov_read_moov(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
static int mov_read_moof(MOVContext *c, AVIOContext *pb, MOVAtom atom)
{
+if (!c->has_looked_for_mfra) {
+c->has_looked_for_mfra = 1;
+if (pb->seekable) {
+av_log(c->fc, AV_LOG_VERBOSE, "stream has moof boxes, will look "
+"for a mfra\n");
+int ret;
+if ((ret = mov_read_mfra(c, pb)) < 0)
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but failed to "
+"read the mfra (may be a live ismv)\n");
+} else
+av_log(c->fc, AV_LOG_VERBOSE, "found a moof box but stream is not "
+"seekable, can not look for mfra\n");
+}
c->fragment.moof_offset = c->fragment.implicit_offset = avio_tell(pb) - 8;
av_dlog(c->fc, "moof offset %"PRIx64"\n", c->fragment.moof_offset);
return mov_read_default(c, pb, atom);
@@ -2738,6 +2752,7 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
{
MOVFragment *frag = &c->fragment;
MOVTrackExt *trex = NULL;
+MOVFragmentIndex* index = NULL;
int flags, track_id, i;
avio_r8(pb); /* version */
@@ -2756,6 +2771,15 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
return AVERROR_INVALIDDATA;
}
+for (i = 0; i < c->fragment_index_count; i++) {
+MOVFragmentIndex* candidate = c->fragment_index_data[i];
+if (candidate->track_id == frag->track_id) {
+av_log(c->fc, AV_LOG_DEBUG,
+ "found fragment index for track %u\n", frag->track_id);
+index = candidate;
+break;
+}
+}
frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
avio_rb64(pb) : flags &
MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
@@ -2768,6 +2792,20 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
avio_rb32(pb) : trex->size;
frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
avio_rb32(pb) : trex->flags;
+frag->time = AV_NOPTS_VALUE;
+if (index) {
+// TODO: should check moof index from mfhd, rather than just
+// relying on this code seeing the moofs in the same order as they
+// are in the mfra, and only once each.
+if (index->current_item_index == index->item_count) {
+av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index "
+ "but it doesn't have entries for all moofs, at moof "
+ "%u\n", frag->track_id, index->current_item_index);
+} else if (index->current_item_index < index->item_count) {
+frag->time = index->items[index->current_item_index].time;
+}
+index->current_item_index++;
+}
av_dlog(c->fc, "frag flags 0x%x\n", frag->flags);
return 0;
}
@@ -2860,6 +2898,10 @@ static
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
On 9 October 2014 18:30, Michael Niedermayer wrote:
> On Thu, Oct 09, 2014 at 08:52:29AM +0300, Mika Raento wrote:
>> On 8 October 2014 16:03, Michael Niedermayer wrote:
> [...]
>> >> +if (avio_rb32(f) != mfra_size) {
>> >> +av_log(s, AV_LOG_DEBUG, "doesn't look like mfra (size)\n");
>> >> +return -1;
>> >> +}
>> >> +if (avio_rb32(f) != MKBETAG('m', 'f', 'r', 'a')) {
>> >> +av_log(s, AV_LOG_DEBUG, "doesn't look like mfra (tag)\n");
>> >> +return -1;
>> >> +}
>> >> +av_log(s, AV_LOG_VERBOSE, "stream has mfra\n");
>> >> +while (!read_tfra(s)) {
>> >> +/* Empty */
>> >> +}
>> >> +return 0;
>> >> +}
>> >> +
>> >> static int mov_read_header(AVFormatContext *s)
>> >> {
>> >> MOVContext *mov = s->priv_data;
>> >
>> >> @@ -3565,6 +3678,9 @@ static int mov_read_header(AVFormatContext *s)
>> >> else
>> >> atom.size = INT64_MAX;
>> >>
>> >> +if (pb->seekable) {
>> >> +mov_read_mfra(s);
>> >> +}
>> >
>> > mov_read_mfra() requires multiple seeks, these are quite expensive if
>> > the file is not local but accessed over the network.
>> > Is it possible to detect (most) files which do not have mfra atoms
>> > reliably without having to try to read the mfra atom so as to avoid
>> > doing these potentially expensive seeks ?
>>
>> That's an excellent question. Here are some possibilities:
>> - we could only do it for filenames ending in .ismv (nothing forces
>> fragmented mp4s to be called .ismv)
>> - we could have a flag to look or not look for the mfra, and maybe
>> default it to true if the filename ends in .ismv (+ for performance, -
>> for usability)
>> - we could trigger the code only once we see a moof (I'd need to
>> restructure the code, reading mfra as the first thing looked like the
>> minimal change)
>
> i think it has to be automatic to be usefull, users cant be expected
> to set all kinds of flags on a per file basis, iam not sure using
> the file extension would be reliable enough so as not to break any
> files. So if its possible then using something from the files content
> would be best moof or otherwise.
> The goal here is to avoid the extra latency the back and forth seeking
> needs so a solution would make only sense if it itself doesnt require
> extra seeking outside the IO buffer
Right. Thanks for the input. I'll rewrite this to look for the mfra
when hitting a moof box.
That will still cause seeks, but there's no way to avoid them if we do
want to read the mfra - and without the mfra the dts are wrong.
Mika
>
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Everything should be made as simple as possible, but not simpler.
> -- Albert Einstein
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
fate passes with these changes, no new tests added. - mika
On 9 October 2014 08:53, Mika Raento wrote:
> If present, an MFRA box and its TFRAs are read for fragment start times.
>
> Without this change, timestamps for discontinuous fragmented mp4 are
> wrong, and cause audio/video desync and are not usable for generating
> HLS.
> ---
> libavformat/isom.h | 14 ++
> libavformat/mov.c | 131
> +
> 2 files changed, 145 insertions(+)
>
> diff --git a/libavformat/isom.h b/libavformat/isom.h
> index 979e967..2b49b55 100644
> --- a/libavformat/isom.h
> +++ b/libavformat/isom.h
> @@ -78,6 +78,7 @@ typedef struct MOVFragment {
> unsigned duration;
> unsigned size;
> unsigned flags;
> +int64_t time;
> } MOVFragment;
>
> typedef struct MOVTrackExt {
> @@ -93,6 +94,17 @@ typedef struct MOVSbgp {
> unsigned int index;
> } MOVSbgp;
>
> +typedef struct MOVFragmentIndexItem {
> +int64_t time;
> +} MOVFragmentIndexItem;
> +
> +typedef struct MOVFragmentIndex {
> +unsigned track_id;
> +unsigned current_item_index;
> +unsigned item_count;
> +MOVFragmentIndexItem *items;
> +} MOVFragmentIndex;
> +
> typedef struct MOVStreamContext {
> AVIOContext *pb;
> int pb_is_copied;
> @@ -171,6 +183,8 @@ typedef struct MOVContext {
> int *bitrates; ///< bitrates read before streams creation
> int bitrates_count;
> int moov_retry;
> +MOVFragmentIndex** fragment_index_data;
> +unsigned fragment_index_count;
> } MOVContext;
>
> int ff_mp4_read_descr_len(AVIOContext *pb);
> diff --git a/libavformat/mov.c b/libavformat/mov.c
> index fdd0671..2b14c48 100644
> --- a/libavformat/mov.c
> +++ b/libavformat/mov.c
> @@ -2738,6 +2738,7 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext
> *pb, MOVAtom atom)
> {
> MOVFragment *frag = &c->fragment;
> MOVTrackExt *trex = NULL;
> +MOVFragmentIndex* index = NULL;
> int flags, track_id, i;
>
> avio_r8(pb); /* version */
> @@ -2756,6 +2757,15 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext
> *pb, MOVAtom atom)
> av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
> return AVERROR_INVALIDDATA;
> }
> +for (i = 0; i < c->fragment_index_count; i++) {
> +MOVFragmentIndex* candidate = c->fragment_index_data[i];
> +if (candidate->track_id == frag->track_id) {
> +av_log(c->fc, AV_LOG_DEBUG,
> + "found fragment index for track %u\n", frag->track_id);
> +index = candidate;
> +break;
> +}
> +}
>
> frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
> avio_rb64(pb) : flags &
> MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
> @@ -2768,6 +2778,20 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext
> *pb, MOVAtom atom)
> avio_rb32(pb) : trex->size;
> frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
> avio_rb32(pb) : trex->flags;
> +frag->time = AV_NOPTS_VALUE;
> +if (index) {
> +// TODO: should check moof index from mfhd, rather than just
> +// relying on this code seeing the moofs in the same order as they
> +// are in the mfra, and only once each.
> +if (index->current_item_index == index->item_count) {
> +av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index "
> + "but it doesn't have entries for all moofs, at moof "
> + "%u\n", frag->track_id, index->current_item_index);
> +} else if (index->current_item_index < index->item_count) {
> +frag->time = index->items[index->current_item_index].time;
> +}
> +index->current_item_index++;
> +}
> av_dlog(c->fc, "frag flags 0x%x\n", frag->flags);
> return 0;
> }
> @@ -2860,6 +2884,10 @@ static int mov_read_trun(MOVContext *c, AVIOContext
> *pb, MOVAtom atom)
> if (flags & MOV_TRUN_DATA_OFFSET)data_offset=
> avio_rb32(pb);
> if (flags & MOV_TRUN_FIRST_SAMPLE_FLAGS) first_sample_flags =
> avio_rb32(pb);
> dts= sc->track_end - sc->time_offset;
> +if (frag->time != AV_NOPTS_VALUE) {
> +av_log(c->fc, AV_LOG_DEBUG, "found frag time %"PRId64"\n",
> frag->time);
> +dts = frag->time;
> +}
> offset = frag->base_data_of
[FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
If present, an MFRA box and its TFRAs are read for fragment start times.
Without this change, timestamps for discontinuous fragmented mp4 are
wrong, and cause audio/video desync and are not usable for generating
HLS.
---
libavformat/isom.h | 14 ++
libavformat/mov.c | 131 +
2 files changed, 145 insertions(+)
diff --git a/libavformat/isom.h b/libavformat/isom.h
index 979e967..2b49b55 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -78,6 +78,7 @@ typedef struct MOVFragment {
unsigned duration;
unsigned size;
unsigned flags;
+int64_t time;
} MOVFragment;
typedef struct MOVTrackExt {
@@ -93,6 +94,17 @@ typedef struct MOVSbgp {
unsigned int index;
} MOVSbgp;
+typedef struct MOVFragmentIndexItem {
+int64_t time;
+} MOVFragmentIndexItem;
+
+typedef struct MOVFragmentIndex {
+unsigned track_id;
+unsigned current_item_index;
+unsigned item_count;
+MOVFragmentIndexItem *items;
+} MOVFragmentIndex;
+
typedef struct MOVStreamContext {
AVIOContext *pb;
int pb_is_copied;
@@ -171,6 +183,8 @@ typedef struct MOVContext {
int *bitrates; ///< bitrates read before streams creation
int bitrates_count;
int moov_retry;
+MOVFragmentIndex** fragment_index_data;
+unsigned fragment_index_count;
} MOVContext;
int ff_mp4_read_descr_len(AVIOContext *pb);
diff --git a/libavformat/mov.c b/libavformat/mov.c
index fdd0671..2b14c48 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -2738,6 +2738,7 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
{
MOVFragment *frag = &c->fragment;
MOVTrackExt *trex = NULL;
+MOVFragmentIndex* index = NULL;
int flags, track_id, i;
avio_r8(pb); /* version */
@@ -2756,6 +2757,15 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
return AVERROR_INVALIDDATA;
}
+for (i = 0; i < c->fragment_index_count; i++) {
+MOVFragmentIndex* candidate = c->fragment_index_data[i];
+if (candidate->track_id == frag->track_id) {
+av_log(c->fc, AV_LOG_DEBUG,
+ "found fragment index for track %u\n", frag->track_id);
+index = candidate;
+break;
+}
+}
frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
avio_rb64(pb) : flags &
MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
@@ -2768,6 +2778,20 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
avio_rb32(pb) : trex->size;
frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
avio_rb32(pb) : trex->flags;
+frag->time = AV_NOPTS_VALUE;
+if (index) {
+// TODO: should check moof index from mfhd, rather than just
+// relying on this code seeing the moofs in the same order as they
+// are in the mfra, and only once each.
+if (index->current_item_index == index->item_count) {
+av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index "
+ "but it doesn't have entries for all moofs, at moof "
+ "%u\n", frag->track_id, index->current_item_index);
+} else if (index->current_item_index < index->item_count) {
+frag->time = index->items[index->current_item_index].time;
+}
+index->current_item_index++;
+}
av_dlog(c->fc, "frag flags 0x%x\n", frag->flags);
return 0;
}
@@ -2860,6 +2884,10 @@ static int mov_read_trun(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
if (flags & MOV_TRUN_DATA_OFFSET)data_offset=
avio_rb32(pb);
if (flags & MOV_TRUN_FIRST_SAMPLE_FLAGS) first_sample_flags =
avio_rb32(pb);
dts= sc->track_end - sc->time_offset;
+if (frag->time != AV_NOPTS_VALUE) {
+av_log(c->fc, AV_LOG_DEBUG, "found frag time %"PRId64"\n", frag->time);
+dts = frag->time;
+}
offset = frag->base_data_offset + data_offset;
distance = 0;
av_dlog(c->fc, "first sample flags 0x%x\n", first_sample_flags);
@@ -3513,6 +3541,13 @@ static int mov_read_close(AVFormatContext *s)
av_freep(&mov->trex_data);
av_freep(&mov->bitrates);
+for (i = 0; i < mov->fragment_index_count; i++) {
+MOVFragmentIndex* index = mov->fragment_index_data[i];
+av_freep(&index->items);
+av_freep(&mov->fragment_index_data[i]);
+}
+av_freep(&mov->fragment_index_data);
+
return 0;
}
@@ -3550,6 +3585,99 @@ static void export_orphan_timecode(AVFormatContext *s)
}
}
+static int read_tfra(AVFormatContext *s)
+{
+MOVContext *mov = s->priv_data;
+AVIOContext *f = s->pb;
+MOVFragmentIndex* index = NULL;
+int version, fieldlength, i, j, err;
+int64_t pos = avio_tell(f);
+uint32_t size = avio_rb32(f);
+if (avio_rb32(f) != MKBETAG('t',
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
On 8 October 2014 16:15, Michael Niedermayer wrote:
> On Wed, Oct 08, 2014 at 03:03:50PM +0200, Michael Niedermayer wrote:
> [...]
>> > @@ -3565,6 +3678,9 @@ static int mov_read_header(AVFormatContext *s)
>> > else
>> > atom.size = INT64_MAX;
>> >
>> > +if (pb->seekable) {
>> > +mov_read_mfra(s);
>> > +}
>>
>> mov_read_mfra() requires multiple seeks, these are quite expensive if
>> the file is not local but accessed over the network.
>> Is it possible to detect (most) files which do not have mfra atoms
>> reliably without having to try to read the mfra atom so as to avoid
>> doing these potentially expensive seeks ?
>
> also mov_read_mfra() is missing a seek back to 0 / the previous
> position
Done.
>
> [...]
>
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> The real ebay dictionary, page 2
> "100% positive feedback" - "All either got their money back or didnt complain"
> "Best seller ever, very honest" - "Seller refunded buyer after failed scam"
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
On 8 October 2014 16:03, Michael Niedermayer wrote:
> Hi
>
> On Wed, Oct 08, 2014 at 03:05:07PM +0300, Mika Raento wrote:
>> If present, an MFRA box and its TFRAs are read for fragment start times.
>>
>> Without this change, timestamps for discontinuous fragmented mp4 are
>> wrong, and cause audio/video desync and are not usable for generating
>> HLS.
>> ---
>> libavformat/isom.h | 14 +++
>> libavformat/mov.c | 116
>> +
>> 2 files changed, 130 insertions(+)
>>
>> diff --git a/libavformat/isom.h b/libavformat/isom.h
>> index 979e967..2b49b55 100644
>> --- a/libavformat/isom.h
>> +++ b/libavformat/isom.h
>> @@ -78,6 +78,7 @@ typedef struct MOVFragment {
>> unsigned duration;
>> unsigned size;
>> unsigned flags;
>> +int64_t time;
>> } MOVFragment;
>>
>> typedef struct MOVTrackExt {
>> @@ -93,6 +94,17 @@ typedef struct MOVSbgp {
>> unsigned int index;
>> } MOVSbgp;
>>
>> +typedef struct MOVFragmentIndexItem {
>> +int64_t time;
>> +} MOVFragmentIndexItem;
>> +
>> +typedef struct MOVFragmentIndex {
>> +unsigned track_id;
>> +unsigned current_item_index;
>> +unsigned item_count;
>> +MOVFragmentIndexItem *items;
>> +} MOVFragmentIndex;
>> +
>> typedef struct MOVStreamContext {
>> AVIOContext *pb;
>> int pb_is_copied;
>> @@ -171,6 +183,8 @@ typedef struct MOVContext {
>> int *bitrates; ///< bitrates read before streams creation
>> int bitrates_count;
>> int moov_retry;
>> +MOVFragmentIndex** fragment_index_data;
>> +unsigned fragment_index_count;
>> } MOVContext;
>>
>> int ff_mp4_read_descr_len(AVIOContext *pb);
>
>> diff --git a/libavformat/mov.c b/libavformat/mov.c
>> index fdd0671..bf92e60 100644
>> --- a/libavformat/mov.c
>> +++ b/libavformat/mov.c
>> @@ -2756,6 +2756,16 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext
>> *pb, MOVAtom atom)
>> av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
>> return AVERROR_INVALIDDATA;
>> }
>> +MOVFragmentIndex* index = NULL;
>
> mixing declaration and statement causes problems for some comilers
Done.
>
>
>> +for (i = 0; i < c->fragment_index_count; i++) {
>> +MOVFragmentIndex* candidate = c->fragment_index_data[i];
>> +if (candidate->track_id == frag->track_id) {
>> +av_log(c->fc, AV_LOG_VERBOSE,
>> + "found fragment index for track %u\n", frag->track_id);
>> +index = candidate;
>> +break;
>> +}
>> +}
>>
>> frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
>> avio_rb64(pb) : flags &
>> MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
>> @@ -2768,6 +2778,20 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext
>> *pb, MOVAtom atom)
>> avio_rb32(pb) : trex->size;
>> frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
>> avio_rb32(pb) : trex->flags;
>> +frag->time = AV_NOPTS_VALUE;
>> +if (index) {
>> +// TODO: should check moof index from mfhd, rather than just
>> +// relying on this code seeing the moofs in the same order as they
>> +// are in the mfra, and only once each.
>> +if (index->current_item_index == index->item_count) {
>> +av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index "
>> + "but it doesn't have entries for all moofs, at moof "
>> + "%u\n", frag->track_id, index->current_item_index);
>> +} else if (index->current_item_index < index->item_count) {
>> +frag->time = index->items[index->current_item_index].time;
>> +}
>> +index->current_item_index++;
>> +}
>> av_dlog(c->fc, "frag flags 0x%x\n", frag->flags);
>> return 0;
>> }
>> @@ -2860,6 +2884,10 @@ static int mov_read_trun(MOVContext *c, AVIOContext
>> *pb, MOVAtom atom)
>> if (flags & MOV_TRUN_DATA_OFFSET)data_offset=
>> avio_rb32(pb);
>> if (flags & MOV_TRUN_FIRST_SAMPLE_FLAGS) first_sample_flags =
>> avio_rb32(pb);
>> dts= sc->track_end - sc->time_o
Re: [FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
Thanks a lot for the detailed comments, will address and resubmit. - mika
On 8 October 2014 16:03, Michael Niedermayer wrote:
> Hi
>
> On Wed, Oct 08, 2014 at 03:05:07PM +0300, Mika Raento wrote:
>> If present, an MFRA box and its TFRAs are read for fragment start times.
>>
>> Without this change, timestamps for discontinuous fragmented mp4 are
>> wrong, and cause audio/video desync and are not usable for generating
>> HLS.
>> ---
>> libavformat/isom.h | 14 +++
>> libavformat/mov.c | 116
>> +
>> 2 files changed, 130 insertions(+)
>>
>> diff --git a/libavformat/isom.h b/libavformat/isom.h
>> index 979e967..2b49b55 100644
>> --- a/libavformat/isom.h
>> +++ b/libavformat/isom.h
>> @@ -78,6 +78,7 @@ typedef struct MOVFragment {
>> unsigned duration;
>> unsigned size;
>> unsigned flags;
>> +int64_t time;
>> } MOVFragment;
>>
>> typedef struct MOVTrackExt {
>> @@ -93,6 +94,17 @@ typedef struct MOVSbgp {
>> unsigned int index;
>> } MOVSbgp;
>>
>> +typedef struct MOVFragmentIndexItem {
>> +int64_t time;
>> +} MOVFragmentIndexItem;
>> +
>> +typedef struct MOVFragmentIndex {
>> +unsigned track_id;
>> +unsigned current_item_index;
>> +unsigned item_count;
>> +MOVFragmentIndexItem *items;
>> +} MOVFragmentIndex;
>> +
>> typedef struct MOVStreamContext {
>> AVIOContext *pb;
>> int pb_is_copied;
>> @@ -171,6 +183,8 @@ typedef struct MOVContext {
>> int *bitrates; ///< bitrates read before streams creation
>> int bitrates_count;
>> int moov_retry;
>> +MOVFragmentIndex** fragment_index_data;
>> +unsigned fragment_index_count;
>> } MOVContext;
>>
>> int ff_mp4_read_descr_len(AVIOContext *pb);
>
>> diff --git a/libavformat/mov.c b/libavformat/mov.c
>> index fdd0671..bf92e60 100644
>> --- a/libavformat/mov.c
>> +++ b/libavformat/mov.c
>> @@ -2756,6 +2756,16 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext
>> *pb, MOVAtom atom)
>> av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
>> return AVERROR_INVALIDDATA;
>> }
>> +MOVFragmentIndex* index = NULL;
>
> mixing declaration and statement causes problems for some comilers
>
>
>> +for (i = 0; i < c->fragment_index_count; i++) {
>> +MOVFragmentIndex* candidate = c->fragment_index_data[i];
>> +if (candidate->track_id == frag->track_id) {
>> +av_log(c->fc, AV_LOG_VERBOSE,
>> + "found fragment index for track %u\n", frag->track_id);
>> +index = candidate;
>> +break;
>> +}
>> +}
>>
>> frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
>> avio_rb64(pb) : flags &
>> MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
>> @@ -2768,6 +2778,20 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext
>> *pb, MOVAtom atom)
>> avio_rb32(pb) : trex->size;
>> frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
>> avio_rb32(pb) : trex->flags;
>> +frag->time = AV_NOPTS_VALUE;
>> +if (index) {
>> +// TODO: should check moof index from mfhd, rather than just
>> +// relying on this code seeing the moofs in the same order as they
>> +// are in the mfra, and only once each.
>> +if (index->current_item_index == index->item_count) {
>> +av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index "
>> + "but it doesn't have entries for all moofs, at moof "
>> + "%u\n", frag->track_id, index->current_item_index);
>> +} else if (index->current_item_index < index->item_count) {
>> +frag->time = index->items[index->current_item_index].time;
>> +}
>> +index->current_item_index++;
>> +}
>> av_dlog(c->fc, "frag flags 0x%x\n", frag->flags);
>> return 0;
>> }
>> @@ -2860,6 +2884,10 @@ static int mov_read_trun(MOVContext *c, AVIOContext
>> *pb, MOVAtom atom)
>> if (flags & MOV_TRUN_DATA_OFFSET)data_offset=
>> avio_rb32(pb);
>> if (flags & MOV_TRUN_FIRST_SAMPLE_FLAGS) first_sample_flags =
>>
[FFmpeg-devel] [PATCH] mov.c: read fragment start dts from fragmented mp4
If present, an MFRA box and its TFRAs are read for fragment start times.
Without this change, timestamps for discontinuous fragmented mp4 are
wrong, and cause audio/video desync and are not usable for generating
HLS.
---
libavformat/isom.h | 14 +++
libavformat/mov.c | 116 +
2 files changed, 130 insertions(+)
diff --git a/libavformat/isom.h b/libavformat/isom.h
index 979e967..2b49b55 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -78,6 +78,7 @@ typedef struct MOVFragment {
unsigned duration;
unsigned size;
unsigned flags;
+int64_t time;
} MOVFragment;
typedef struct MOVTrackExt {
@@ -93,6 +94,17 @@ typedef struct MOVSbgp {
unsigned int index;
} MOVSbgp;
+typedef struct MOVFragmentIndexItem {
+int64_t time;
+} MOVFragmentIndexItem;
+
+typedef struct MOVFragmentIndex {
+unsigned track_id;
+unsigned current_item_index;
+unsigned item_count;
+MOVFragmentIndexItem *items;
+} MOVFragmentIndex;
+
typedef struct MOVStreamContext {
AVIOContext *pb;
int pb_is_copied;
@@ -171,6 +183,8 @@ typedef struct MOVContext {
int *bitrates; ///< bitrates read before streams creation
int bitrates_count;
int moov_retry;
+MOVFragmentIndex** fragment_index_data;
+unsigned fragment_index_count;
} MOVContext;
int ff_mp4_read_descr_len(AVIOContext *pb);
diff --git a/libavformat/mov.c b/libavformat/mov.c
index fdd0671..bf92e60 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -2756,6 +2756,16 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n");
return AVERROR_INVALIDDATA;
}
+MOVFragmentIndex* index = NULL;
+for (i = 0; i < c->fragment_index_count; i++) {
+MOVFragmentIndex* candidate = c->fragment_index_data[i];
+if (candidate->track_id == frag->track_id) {
+av_log(c->fc, AV_LOG_VERBOSE,
+ "found fragment index for track %u\n", frag->track_id);
+index = candidate;
+break;
+}
+}
frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ?
avio_rb64(pb) : flags &
MOV_TFHD_DEFAULT_BASE_IS_MOOF ?
@@ -2768,6 +2778,20 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
avio_rb32(pb) : trex->size;
frag->flags= flags & MOV_TFHD_DEFAULT_FLAGS ?
avio_rb32(pb) : trex->flags;
+frag->time = AV_NOPTS_VALUE;
+if (index) {
+// TODO: should check moof index from mfhd, rather than just
+// relying on this code seeing the moofs in the same order as they
+// are in the mfra, and only once each.
+if (index->current_item_index == index->item_count) {
+av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index "
+ "but it doesn't have entries for all moofs, at moof "
+ "%u\n", frag->track_id, index->current_item_index);
+} else if (index->current_item_index < index->item_count) {
+frag->time = index->items[index->current_item_index].time;
+}
+index->current_item_index++;
+}
av_dlog(c->fc, "frag flags 0x%x\n", frag->flags);
return 0;
}
@@ -2860,6 +2884,10 @@ static int mov_read_trun(MOVContext *c, AVIOContext *pb,
MOVAtom atom)
if (flags & MOV_TRUN_DATA_OFFSET)data_offset=
avio_rb32(pb);
if (flags & MOV_TRUN_FIRST_SAMPLE_FLAGS) first_sample_flags =
avio_rb32(pb);
dts= sc->track_end - sc->time_offset;
+if (frag->time != AV_NOPTS_VALUE) {
+av_log(c->fc, AV_LOG_DEBUG, "found frag time %"PRId64"\n", frag->time);
+dts = frag->time;
+}
offset = frag->base_data_offset + data_offset;
distance = 0;
av_dlog(c->fc, "first sample flags 0x%x\n", first_sample_flags);
@@ -3513,6 +3541,13 @@ static int mov_read_close(AVFormatContext *s)
av_freep(&mov->trex_data);
av_freep(&mov->bitrates);
+for (i = 0; i < mov->fragment_index_count; i++) {
+MOVFragmentIndex* index = mov->fragment_index_data[i];
+av_freep(&index->items);
+av_freep(&mov->fragment_index_data[i]);
+}
+av_freep(&mov->fragment_index_data);
+
return 0;
}
@@ -3550,6 +3585,84 @@ static void export_orphan_timecode(AVFormatContext *s)
}
}
+static int read_tfra(AVFormatContext *s)
+{
+MOVContext *mov = s->priv_data;
+AVIOContext *f = s->pb;
+MOVFragmentIndex* index = NULL;
+int version, fieldlength, i, j, err;
+int64_t pos = avio_tell(f);
+uint32_t size = avio_rb32(f);
+if (avio_rb32(f) != MKBETAG('t', 'f', 'r', 'a')) {
+return -1;
+}
+av_log(s, AV_LOG_VERBOSE, "found tfra\n");
+index = av_mallocz(sizeof(MOVFragmentIndex));
+if (!index)
+return AVERROR(ENOMEM);
+mov->fragment_index_count++;
Re: [FFmpeg-devel] Not-very-good timestamps with fragmented mp4 input
I will send a patch that reads the MFRA. It's more a RFC than a patch, I for example have not yet run the regression suite with it. Mika On 8 October 2014 08:54, Mika Raento wrote: > Hi > > I have a number of ISMV files that have discontinuities (missing > frames between fragments). Using these as input to ffmpeg doesn't work > very well - the discontinuities are not reflected in the dts or pts, > e.g., using ffprobe -show_packets. Hence there are audio/video sync > problems and with large gaps I'm not able to create working HLS > output. > > Ffmpeg does not read the MFRA (fragmentation index), which would > contain the fragment start times. > > I don't know enough to say if the timestamp information might be > duplicated in the mdat. > > Would the solution be to read the MFRA in mov.c? > > (there is also an alternate MS-specific uuid atom that contains the > duration and time data for each moof, but my files don't have those) > > Or am I missing something? > > Mika ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] ismindex: handle discontinuous streams better
Reads the fragment duration from the trun sample data, rather than
assuming that there are no gaps. Creates much better playlists for our
inputs.
---
tools/ismindex.c | 95 ++--
1 file changed, 85 insertions(+), 10 deletions(-)
diff --git a/tools/ismindex.c b/tools/ismindex.c
index 734419b..b9bcede 100644
--- a/tools/ismindex.c
+++ b/tools/ismindex.c
@@ -50,6 +50,7 @@
#include "cmdutils.h"
#include "libavformat/avformat.h"
+#include "libavformat/isom.h"
#include "libavformat/os_support.h"
#include "libavutil/intreadwrite.h"
#include "libavutil/mathematics.h"
@@ -226,6 +227,78 @@ fail:
return ret;
}
+static int64_t read_trun_duration(AVIOContext *in, int64_t end)
+{
+int64_t ret = 0;
+int64_t pos;
+int flags, i;
+int entries;
+avio_r8(in); /* version */
+flags = avio_rb24(in);
+if (! (flags & MOV_TRUN_SAMPLE_DURATION)) {
+fprintf(stderr, "no sample duration in trun flags");
+return -1;
+}
+entries = avio_rb32(in);
+
+if (flags & MOV_TRUN_DATA_OFFSET) avio_rb32(in);
+if (flags & MOV_TRUN_FIRST_SAMPLE_FLAGS) avio_rb32(in);
+
+pos = avio_tell(in);
+for (i = 0; i < entries && pos < end; i++) {
+int sample_duration = 0;
+if (flags & MOV_TRUN_SAMPLE_DURATION) sample_duration = avio_rb32(in);
+if (flags & MOV_TRUN_SAMPLE_SIZE) avio_rb32(in);
+if (flags & MOV_TRUN_SAMPLE_FLAGS) avio_rb32(in);
+if (flags & MOV_TRUN_SAMPLE_CTS) avio_rb32(in);
+if (sample_duration < 0) {
+fprintf(stderr, "negative sample duration %d\n", sample_duration);
+return -1;
+}
+ret += sample_duration;
+pos = avio_tell(in);
+}
+
+return ret;
+}
+
+static int64_t read_moof_duration(AVIOContext *in, int64_t offset)
+{
+int64_t ret = -1;
+int32_t moof_size, size, tag;
+int64_t pos = 0;
+
+avio_seek(in, offset, SEEK_SET);
+moof_size = avio_rb32(in);
+tag = avio_rb32(in);
+if (expect_tag(tag, MKBETAG('m', 'o', 'o', 'f')) != 0) goto fail;
+while (pos < offset + moof_size) {
+pos = avio_tell(in);
+size = avio_rb32(in);
+tag = avio_rb32(in);
+if (tag == MKBETAG('t', 'r', 'a', 'f')) {
+int64_t traf_pos = pos;
+int64_t traf_size = size;
+while (pos < traf_pos + traf_size) {
+pos = avio_tell(in);
+size = avio_rb32(in);
+tag = avio_rb32(in);
+if (tag == MKBETAG('t', 'r', 'u', 'n')) {
+return read_trun_duration(in, pos + size);
+}
+avio_seek(in, pos + size, SEEK_SET);
+}
+fprintf(stderr, "couldn't find trun");
+goto fail;
+}
+avio_seek(in, pos + size, SEEK_SET);
+}
+fprintf(stderr, "couldn't find traf");
+
+fail:
+return ret;
+}
+
static int read_tfra(struct Tracks *tracks, int start_index, AVIOContext *f)
{
int ret = AVERROR_EOF, track_id;
@@ -255,12 +328,7 @@ static int read_tfra(struct Tracks *tracks, int
start_index, AVIOContext *f)
goto fail;
}
// The duration here is always the difference between consecutive
-// start times and doesn't even try to read the actual duration of the
-// media fragments. This is what other smooth streaming tools tend to
-// do too, but cannot express missing fragments, and the start times
-// may not match the stream metadata we get from libavformat. Correct
-// calculation would require parsing the tfxd atom (if present, it's
-// not mandatory) or parsing the full moof atoms separately.
+// start times.
for (i = 0; i < track->chunks; i++) {
if (version == 1) {
track->offsets[i].time = avio_rb64(f);
@@ -283,6 +351,17 @@ static int read_tfra(struct Tracks *tracks, int
start_index, AVIOContext *f)
track->offsets[track->chunks - 1].duration = track->offsets[0].time +
track->duration -
track->offsets[track->chunks - 1].time;
+}
+// Now try and read the actual durations from the trun sample data.
+for (i = 0; i < track->chunks; i++) {
+int64_t duration = read_moof_duration(f, track->offsets[i].offset);
+if (duration > 0 && abs(duration - track->offsets[i].duration) > 3) {
+// 3 allows for integer duration to drift a few units,
+// e.g., for 1/3 durations
+track->offsets[i].duration = duration;
+}
+}
+if (track->chunks > 0) {
if (track->offsets[track->chunks - 1].duration <= 0) {
fprintf(stderr, "Calculated last chunk duration for track %d "
"was non-positive (%"PRId64"), probably due to missing "
@@ -577,10 +656,6 @@ static void print_track_chunks(FILE *out, struct Tracks
*tracks, int main,
Re: [FFmpeg-devel] [PATCH] ismindex: handle discontinuous streams better
On 8 October 2014 08:12, Mika Raento wrote:
> Reads the fragment duration from the trun sample data, rather than
> assuming that there are no gaps. Creates much better playlists for our
> inputs.
> ---
> tools/ismindex.c | 96
> +---
> 1 file changed, 85 insertions(+), 11 deletions(-)
>
> diff --git a/tools/ismindex.c b/tools/ismindex.c
> index 734419b..dcbc115 100644
> --- a/tools/ismindex.c
> +++ b/tools/ismindex.c
> @@ -50,6 +50,7 @@
> #include "cmdutils.h"
>
> #include "libavformat/avformat.h"
> +#include "libavformat/isom.h"
> #include "libavformat/os_support.h"
> #include "libavutil/intreadwrite.h"
> #include "libavutil/mathematics.h"
> @@ -226,6 +227,78 @@ fail:
> return ret;
> }
>
> +static int64_t read_trun_duration(AVIOContext *in, int64_t end)
> +{
> +int64_t ret = 0;
> +int64_t pos;
> +int flags, i;
> +int entries;
> +avio_r8(in); /* version */
> +flags = avio_rb24(in);
> +if (! (flags & MOV_TRUN_SAMPLE_DURATION)) {
> +fprintf(stderr, "no sample duration in trun flags");
> +return -1;
> +}
> +entries = avio_rb32(in);
> +
> +if (flags & MOV_TRUN_DATA_OFFSET) avio_rb32(in);
> +if (flags & MOV_TRUN_FIRST_SAMPLE_FLAGS) avio_rb32(in);
> +
> +pos = avio_tell(in);
> +for (i = 0; i < entries && pos < end; i++) {
> +int sample_duration = 0;
> +if (flags & MOV_TRUN_SAMPLE_DURATION) sample_duration =
> avio_rb32(in);
> +if (flags & MOV_TRUN_SAMPLE_SIZE) avio_rb32(in);
> +if (flags & MOV_TRUN_SAMPLE_FLAGS) avio_rb32(in);
> +if (flags & MOV_TRUN_SAMPLE_CTS) avio_rb32(in);
> +if (sample_duration < 0) {
> +fprintf(stderr, "negative sample duration %d\n",
> sample_duration);
> +return -1;
> +}
> +ret += sample_duration;
> +pos = avio_tell(in);
> +}
> +
> +return ret;
> +}
> +
> +static int64_t read_moof_duration(AVIOContext *in, int64_t offset)
> +{
> +int64_t ret = -1;
> +int32_t moof_size, size, tag;
> +int64_t pos = 0;
> +
> +avio_seek(in, offset, SEEK_SET);
> +moof_size = avio_rb32(in);
> +tag = avio_rb32(in);
> +if (expect_tag(tag, MKBETAG('m', 'o', 'o', 'f')) != 0) goto fail;
> +while (pos < offset + moof_size) {
> +pos = avio_tell(in);
> +size = avio_rb32(in);
> +tag = avio_rb32(in);
> +if (tag == MKBETAG('t', 'r', 'a', 'f')) {
> +int64_t traf_pos = pos;
> +int64_t traf_size = size;
> +while (pos < traf_pos + traf_size) {
> +pos = avio_tell(in);
> +size = avio_rb32(in);
> +tag = avio_rb32(in);
> +if (tag == MKBETAG('t', 'r', 'u', 'n')) {
> +return read_trun_duration(in, pos + size);
> +}
> +avio_seek(in, pos + size, SEEK_SET);
> +}
> +fprintf(stderr, "couldn't find trun");
> +goto fail;
> +}
> +avio_seek(in, pos + size, SEEK_SET);
> +}
> +fprintf(stderr, "couldn't find traf");
> +
> +fail:
> +return ret;
> +}
> +
> static int read_tfra(struct Tracks *tracks, int start_index, AVIOContext *f)
> {
> int ret = AVERROR_EOF, track_id;
> @@ -255,12 +328,7 @@ static int read_tfra(struct Tracks *tracks, int
> start_index, AVIOContext *f)
> goto fail;
> }
> // The duration here is always the difference between consecutive
> -// start times and doesn't even try to read the actual duration of the
> -// media fragments. This is what other smooth streaming tools tend to
> -// do too, but cannot express missing fragments, and the start times
> -// may not match the stream metadata we get from libavformat. Correct
> -// calculation would require parsing the tfxd atom (if present, it's
> -// not mandatory) or parsing the full moof atoms separately.
> +// start times.
> for (i = 0; i < track->chunks; i++) {
> if (version == 1) {
> track->offsets[i].time = avio_rb64(f);
> @@ -283,6 +351,15 @@ static int read_tfra(struct Tracks *tracks, int
> start_index, AVIOContext *f)
> track->offsets[track->chunks - 1].duration = track->offsets[0].ti
Re: [FFmpeg-devel] [PATCH] ismindex: handle discontinuous streams better
For better or worse, ismindex doesn't use av_log anywhere, I'm following the existing style in the file. Changing the whole file should be a separate patch? Mika On 8 October 2014 09:33, Carl Eugen Hoyos wrote: > Mika Raento iki.fi> writes: > >> +fprintf(stderr, "negative sample duration %d\n", >> sample_duration); > > fprintf (& friends) are not allowed, please use > av_log(). > > Note to others: Why did we remove the defines? > > Carl Eugen > > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] Not-very-good timestamps with fragmented mp4 input
Hi I have a number of ISMV files that have discontinuities (missing frames between fragments). Using these as input to ffmpeg doesn't work very well - the discontinuities are not reflected in the dts or pts, e.g., using ffprobe -show_packets. Hence there are audio/video sync problems and with large gaps I'm not able to create working HLS output. Ffmpeg does not read the MFRA (fragmentation index), which would contain the fragment start times. I don't know enough to say if the timestamp information might be duplicated in the mdat. Would the solution be to read the MFRA in mov.c? (there is also an alternate MS-specific uuid atom that contains the duration and time data for each moof, but my files don't have those) Or am I missing something? Mika ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] ismindex: handle discontinuous streams better
Reads the fragment duration from the trun sample data, rather than
assuming that there are no gaps. Creates much better playlists for our
inputs.
---
tools/ismindex.c | 96 +---
1 file changed, 85 insertions(+), 11 deletions(-)
diff --git a/tools/ismindex.c b/tools/ismindex.c
index 734419b..dcbc115 100644
--- a/tools/ismindex.c
+++ b/tools/ismindex.c
@@ -50,6 +50,7 @@
#include "cmdutils.h"
#include "libavformat/avformat.h"
+#include "libavformat/isom.h"
#include "libavformat/os_support.h"
#include "libavutil/intreadwrite.h"
#include "libavutil/mathematics.h"
@@ -226,6 +227,78 @@ fail:
return ret;
}
+static int64_t read_trun_duration(AVIOContext *in, int64_t end)
+{
+int64_t ret = 0;
+int64_t pos;
+int flags, i;
+int entries;
+avio_r8(in); /* version */
+flags = avio_rb24(in);
+if (! (flags & MOV_TRUN_SAMPLE_DURATION)) {
+fprintf(stderr, "no sample duration in trun flags");
+return -1;
+}
+entries = avio_rb32(in);
+
+if (flags & MOV_TRUN_DATA_OFFSET) avio_rb32(in);
+if (flags & MOV_TRUN_FIRST_SAMPLE_FLAGS) avio_rb32(in);
+
+pos = avio_tell(in);
+for (i = 0; i < entries && pos < end; i++) {
+int sample_duration = 0;
+if (flags & MOV_TRUN_SAMPLE_DURATION) sample_duration = avio_rb32(in);
+if (flags & MOV_TRUN_SAMPLE_SIZE) avio_rb32(in);
+if (flags & MOV_TRUN_SAMPLE_FLAGS) avio_rb32(in);
+if (flags & MOV_TRUN_SAMPLE_CTS) avio_rb32(in);
+if (sample_duration < 0) {
+fprintf(stderr, "negative sample duration %d\n", sample_duration);
+return -1;
+}
+ret += sample_duration;
+pos = avio_tell(in);
+}
+
+return ret;
+}
+
+static int64_t read_moof_duration(AVIOContext *in, int64_t offset)
+{
+int64_t ret = -1;
+int32_t moof_size, size, tag;
+int64_t pos = 0;
+
+avio_seek(in, offset, SEEK_SET);
+moof_size = avio_rb32(in);
+tag = avio_rb32(in);
+if (expect_tag(tag, MKBETAG('m', 'o', 'o', 'f')) != 0) goto fail;
+while (pos < offset + moof_size) {
+pos = avio_tell(in);
+size = avio_rb32(in);
+tag = avio_rb32(in);
+if (tag == MKBETAG('t', 'r', 'a', 'f')) {
+int64_t traf_pos = pos;
+int64_t traf_size = size;
+while (pos < traf_pos + traf_size) {
+pos = avio_tell(in);
+size = avio_rb32(in);
+tag = avio_rb32(in);
+if (tag == MKBETAG('t', 'r', 'u', 'n')) {
+return read_trun_duration(in, pos + size);
+}
+avio_seek(in, pos + size, SEEK_SET);
+}
+fprintf(stderr, "couldn't find trun");
+goto fail;
+}
+avio_seek(in, pos + size, SEEK_SET);
+}
+fprintf(stderr, "couldn't find traf");
+
+fail:
+return ret;
+}
+
static int read_tfra(struct Tracks *tracks, int start_index, AVIOContext *f)
{
int ret = AVERROR_EOF, track_id;
@@ -255,12 +328,7 @@ static int read_tfra(struct Tracks *tracks, int
start_index, AVIOContext *f)
goto fail;
}
// The duration here is always the difference between consecutive
-// start times and doesn't even try to read the actual duration of the
-// media fragments. This is what other smooth streaming tools tend to
-// do too, but cannot express missing fragments, and the start times
-// may not match the stream metadata we get from libavformat. Correct
-// calculation would require parsing the tfxd atom (if present, it's
-// not mandatory) or parsing the full moof atoms separately.
+// start times.
for (i = 0; i < track->chunks; i++) {
if (version == 1) {
track->offsets[i].time = avio_rb64(f);
@@ -283,6 +351,15 @@ static int read_tfra(struct Tracks *tracks, int
start_index, AVIOContext *f)
track->offsets[track->chunks - 1].duration = track->offsets[0].time +
track->duration -
track->offsets[track->chunks - 1].time;
+}
+// Now try and read the actual durations from the trun sample data.
+for (i = 0; i < track->chunks; i++) {
+int64_t duration = read_moof_duration(f, track->offsets[i].offset);
+if (duration > 0) {
+track->offsets[i].duration = duration;
+}
+}
+if (track->chunks > 0) {
if (track->offsets[track->chunks - 1].duration <= 0) {
fprintf(stderr, "Calculated last chunk duration for track %d "
"was non-positive (%"PRId64"), probably due to missing "
@@ -576,11 +653,8 @@ static void print_track_chunks(FILE *out, struct Tracks
*tracks, int main,
}
fprintf(out, "\t\toffsets[i].duration);
-if (pos != track->offsets[i].time) {
-// With the current logic for calculatio
[FFmpeg-devel] Experiences in using ffmpeg to transcode broadcast video
Dear all This mail is meant mainly as a note to other potential users, but possibly as input to development - time allowing I might be able to pick up some of the pieces myself. It's also a thank-you for all the hard work in ffmpeg. I've successfully implemented a transcoding pipeline for producing multi-bitrate fragmented mp4 files from broadcast DVB input. More concretely, I'm taking in broadcast TS captures with either mpeg2 video and mp2 audio (SD, varying aspect ratio) or h264 video and aac audio, both potentially with dvbsub. From that I'm producing ISMV output with multiple bitrate h264 video at fixed 16:9 aspect ratio and multiple bitrate aac audio, with burned subtitles. The ISMV outputs are post-processed with tools/ismindex and with the hls muxer. There are number of limitations in ffmpeg that I've had to work around: - I haven't gotten sub2video or async working without reasonably monotonous DTS. Broadcast TS streams can easily contain backward jumps in time (e.g., a cheapo source that plays mp4 files and starts each file at 0). The fix is to cut the TS into pieces at timestamp jump locations and using '-itsoffset' to rewrite the timestamps and then concatenate. I'm using the segment and concat muxers for that. - Sub2video doesn't work with negative timestamps, so I use '-itsoffset' to get positive timestamps - For HD streams, I need to scale up the sub2video results from SD to HD. Sub2video doesn't handle the HD subtitle geometries. I'm not enough of an expert to know whether that's the issue, or whether that's just the way it's supposed to work with SD subs (typical) with HD video. - For columnboxing, I use the scale, pad and setdar video filters. These work fine, but their parameters are only evaluated at start, so I need to cut the video into pieces with a single aspect ratio first and concatenate later. - Audio sync (using aresample) gets confused if the input contains errors, so I need to first re-encode audio (with '-copyts') and only after that synchronize. - The TS PIDs are not kept over the segment muxer, so I given them on the command line with '-streamid'. I think all of the above would make great out-of-the-box ffmpeg features :-) If somebody is interested I can provide private copies of my inputs for playing with, since they are copyrighted material I can't easily distribute them. I've automated the above steps with a bunch of python, using ffprobe to parse the video stream. The result manages to correctly transcode about 98% of the TS inputs that our commercial transcoding and packaging pipe fails to handle (it would probably do even better on the rest). Thanks for all the features of ffmpeg that did work out-of-the-box :-) BR, Mika Raento ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] hlsenc: single_file, support HLS ver 4 byteranges
This adds a new option -hls_flags single_file that creates one .ts file
for HLS and adds byteranges to the .m3u8 file, instead of creating one
.ts file for each segment.
This is helpful at least for storing large number of videos, as the
number of files per video is drastically reduced and copying and storing
those files takes less requests and inodes.
This is based on work by Nicolas Martyanoff, discussed on ffmpeg-devel
in July 2014. That patch seems abandoned by the author, and contained
unrelated changes. This patch tries to add the minimum amount of code to
support the byterange playlists.
---
doc/muxers.texi | 23 +++
libavformat/hlsenc.c | 64 ++--
2 files changed, 71 insertions(+), 16 deletions(-)
diff --git a/doc/muxers.texi b/doc/muxers.texi
index 57e81f4..40ae857 100644
--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -194,15 +194,19 @@ can not be smaller than one centi second.
Apple HTTP Live Streaming muxer that segments MPEG-TS according to
the HTTP Live Streaming (HLS) specification.
-It creates a playlist file and numbered segment files. The output
-filename specifies the playlist filename; the segment filenames
-receive the same basename as the playlist, a sequential number and
-a .ts extension.
+It creates a playlist file, and one or more segment files. The output filename
+specifies the playlist filename.
+
+By default, the muxer creates a file for each segment produced. These files
+have the same name as the playlist, followed by a sequential number and a
+.ts extension.
For example, to convert an input file with @command{ffmpeg}:
@example
ffmpeg -i in.nut out.m3u8
@end example
+This example will produce the playlist, @file{out.m3u8}, and segment files:
+@file{out0.ts}, @file{out1.ts}, @file{out2.ts}, etc.
See also the @ref{segment} muxer, which provides a more generic and
flexible implementation of a segmenter, and can be used to perform HLS
@@ -241,6 +245,17 @@ Note that the playlist sequence number must be unique for
each segment
and it is not to be confused with the segment filename sequence number
which can be cyclic, for example if the @option{wrap} option is
specified.
+
+@item hls_flags single_file
+If this flag is set, the muxer will store all segments in a single MPEG-TS
+file, and will use byte ranges in the playlist. HLS playlists generated with
+this way will have the version number 4.
+For example:
+@example
+ffmpeg -i in.nut -hls_flags single_file out.m3u8
+@end example
+Will produce the playlist, @file{out.m3u8}, and a single segment file,
+@file{out.ts}.
@end table
@anchor{ico}
diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index 11f1e5b..c03737a 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -34,10 +34,17 @@
typedef struct HLSSegment {
char filename[1024];
double duration; /* in seconds */
+int64_t pos;
+int64_t size;
struct HLSSegment *next;
} HLSSegment;
+typedef enum HLSFlags {
+// Generate a single media file and use byte ranges in the playlist.
+HLS_SINGLE_FILE = (1 << 0),
+} HLSFlags;
+
typedef struct HLSContext {
const AVClass *class; // Class for private options.
unsigned number;
@@ -50,12 +57,15 @@ typedef struct HLSContext {
float time;// Set by a private option.
int max_nb_segments; // Set by a private option.
int wrap; // Set by a private option.
+uint32_t flags;// enum HLSFlags
int64_t recording_time;
int has_video;
int64_t start_pts;
int64_t end_pts;
double duration; // last segment duration computed so far, in seconds
+int64_t start_pos;// last segment starting position
+int64_t size; // last segment size
int nb_entries;
HLSSegment *segments;
@@ -88,12 +98,14 @@ static int hls_mux_init(AVFormatContext *s)
avcodec_copy_context(st->codec, s->streams[i]->codec);
st->sample_aspect_ratio = s->streams[i]->sample_aspect_ratio;
}
+hls->start_pos = 0;
return 0;
}
/* Create a new segment and append it to the segment list */
-static int hls_append_segment(HLSContext *hls, double duration)
+static int hls_append_segment(HLSContext *hls, double duration, int64_t pos,
+ int64_t size)
{
HLSSegment *en = av_malloc(sizeof(*en));
@@ -103,6 +115,8 @@ static int hls_append_segment(HLSContext *hls, double
duration)
av_strlcpy(en->filename, av_basename(hls->avf->filename),
sizeof(en->filename));
en->duration = duration;
+en->pos = pos;
+en->size = size;
en->next = NULL;
if (!hls->segments)
@@ -142,6 +156,7 @@ static int hls_window(AVFormatContext *s, int last)
int target_duration = 0;
int ret = 0;
int64_t sequence = FFMAX(hls->start_sequence, hls->sequence -
hls->nb_entries);
+int version = hls->flags & HLS_SINGLE_FILE ? 4 : 3;
if ((ret = avio_open
[FFmpeg-devel] [PATCH] hlsenc: single_file, support HLS ver 4 byteranges
This adds a new option -hls_flags single_file that creates one .ts file
for HLS and adds byteranges to the .m3u8 file, instead of creating one
.ts file for each segment.
This is helpful at least for storing large number of videos, as the
number of files per video is drastically reduced and copying and storing
those files takes less requests and inodes.
This is based on work by Nicolas Martyanoff, discussed on ffmpeg-devel
in July 2014. That patch seems abandoned by the author, and contained
unrelated changes. This patch tries to add the minimum amount of code to
support the byterange playlists.
---
doc/muxers.texi | 23 +++
libavformat/hlsenc.c | 63 ++--
2 files changed, 70 insertions(+), 16 deletions(-)
diff --git a/doc/muxers.texi b/doc/muxers.texi
index 57e81f4..40ae857 100644
--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -194,15 +194,19 @@ can not be smaller than one centi second.
Apple HTTP Live Streaming muxer that segments MPEG-TS according to
the HTTP Live Streaming (HLS) specification.
-It creates a playlist file and numbered segment files. The output
-filename specifies the playlist filename; the segment filenames
-receive the same basename as the playlist, a sequential number and
-a .ts extension.
+It creates a playlist file, and one or more segment files. The output filename
+specifies the playlist filename.
+
+By default, the muxer creates a file for each segment produced. These files
+have the same name as the playlist, followed by a sequential number and a
+.ts extension.
For example, to convert an input file with @command{ffmpeg}:
@example
ffmpeg -i in.nut out.m3u8
@end example
+This example will produce the playlist, @file{out.m3u8}, and segment files:
+@file{out0.ts}, @file{out1.ts}, @file{out2.ts}, etc.
See also the @ref{segment} muxer, which provides a more generic and
flexible implementation of a segmenter, and can be used to perform HLS
@@ -241,6 +245,17 @@ Note that the playlist sequence number must be unique for
each segment
and it is not to be confused with the segment filename sequence number
which can be cyclic, for example if the @option{wrap} option is
specified.
+
+@item hls_flags single_file
+If this flag is set, the muxer will store all segments in a single MPEG-TS
+file, and will use byte ranges in the playlist. HLS playlists generated with
+this way will have the version number 4.
+For example:
+@example
+ffmpeg -i in.nut -hls_flags single_file out.m3u8
+@end example
+Will produce the playlist, @file{out.m3u8}, and a single segment file,
+@file{out.ts}.
@end table
@anchor{ico}
diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index 11f1e5b..c3fb341 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -34,10 +34,17 @@
typedef struct HLSSegment {
char filename[1024];
double duration; /* in seconds */
+int64_t pos;
+int64_t size;
struct HLSSegment *next;
} HLSSegment;
+typedef enum HLSFlags {
+// Generate a single media file and use byte ranges in the playlist.
+HLS_SINGLE_FILE = (1 << 0),
+} HLSFlags;
+
typedef struct HLSContext {
const AVClass *class; // Class for private options.
unsigned number;
@@ -50,12 +57,15 @@ typedef struct HLSContext {
float time;// Set by a private option.
int max_nb_segments; // Set by a private option.
int wrap; // Set by a private option.
+uint32_t flags;// enum HLSFlags
int64_t recording_time;
int has_video;
int64_t start_pts;
int64_t end_pts;
double duration; // last segment duration computed so far, in seconds
+int64_t start_pos;// last segment starting position
+int64_t size; // last segment size
int nb_entries;
HLSSegment *segments;
@@ -88,12 +98,14 @@ static int hls_mux_init(AVFormatContext *s)
avcodec_copy_context(st->codec, s->streams[i]->codec);
st->sample_aspect_ratio = s->streams[i]->sample_aspect_ratio;
}
+hls->start_pos = 0;
return 0;
}
/* Create a new segment and append it to the segment list */
-static int hls_append_segment(HLSContext *hls, double duration)
+static int hls_append_segment(HLSContext *hls, double duration, int64_t pos,
+ int64_t size)
{
HLSSegment *en = av_malloc(sizeof(*en));
@@ -103,6 +115,8 @@ static int hls_append_segment(HLSContext *hls, double
duration)
av_strlcpy(en->filename, av_basename(hls->avf->filename),
sizeof(en->filename));
en->duration = duration;
+en->pos = pos;
+en->size = size;
en->next = NULL;
if (!hls->segments)
@@ -142,6 +156,7 @@ static int hls_window(AVFormatContext *s, int last)
int target_duration = 0;
int ret = 0;
int64_t sequence = FFMAX(hls->start_sequence, hls->sequence -
hls->nb_entries);
+int version = hls->flags & HLS_SINGLE_FILE ? 4 : 3;
if ((ret = avio_open
[FFmpeg-devel] [PATCH] hlsenc: single_file, support HLS ver 4 byteranges
This adds a new option -hls_flags single_file that creates one .ts file
for HLS and adds byteranges to the .m3u8 file, instead of creating one
.ts file for each segment.
This is helpful at least for storing large number of videos, as the
number of files per video is drastically reduced and copying and storing
those files takes less requests and inodes.
This is based on work by Nicolas Martyanoff, discussed on ffmpeg-devel
in July 2014. That patch seems abandoned by the author, and contained
unrelated changes. This patch tries to add the minimum amount of code to
support the byterange playlists.
---
doc/muxers.texi | 23 +++
libavformat/hlsenc.c | 65 ++--
2 files changed, 72 insertions(+), 16 deletions(-)
diff --git a/doc/muxers.texi b/doc/muxers.texi
index 57e81f4..40ae857 100644
--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -194,15 +194,19 @@ can not be smaller than one centi second.
Apple HTTP Live Streaming muxer that segments MPEG-TS according to
the HTTP Live Streaming (HLS) specification.
-It creates a playlist file and numbered segment files. The output
-filename specifies the playlist filename; the segment filenames
-receive the same basename as the playlist, a sequential number and
-a .ts extension.
+It creates a playlist file, and one or more segment files. The output filename
+specifies the playlist filename.
+
+By default, the muxer creates a file for each segment produced. These files
+have the same name as the playlist, followed by a sequential number and a
+.ts extension.
For example, to convert an input file with @command{ffmpeg}:
@example
ffmpeg -i in.nut out.m3u8
@end example
+This example will produce the playlist, @file{out.m3u8}, and segment files:
+@file{out0.ts}, @file{out1.ts}, @file{out2.ts}, etc.
See also the @ref{segment} muxer, which provides a more generic and
flexible implementation of a segmenter, and can be used to perform HLS
@@ -241,6 +245,17 @@ Note that the playlist sequence number must be unique for
each segment
and it is not to be confused with the segment filename sequence number
which can be cyclic, for example if the @option{wrap} option is
specified.
+
+@item hls_flags single_file
+If this flag is set, the muxer will store all segments in a single MPEG-TS
+file, and will use byte ranges in the playlist. HLS playlists generated with
+this way will have the version number 4.
+For example:
+@example
+ffmpeg -i in.nut -hls_flags single_file out.m3u8
+@end example
+Will produce the playlist, @file{out.m3u8}, and a single segment file,
+@file{out.ts}.
@end table
@anchor{ico}
diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index 11f1e5b..9fcb999 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -34,10 +34,17 @@
typedef struct HLSSegment {
char filename[1024];
double duration; /* in seconds */
+int64_t pos;
+int64_t size;
struct HLSSegment *next;
} HLSSegment;
+typedef enum HLSFlags {
+// Generate a single media file and use byte ranges in the playlist.
+HLS_SINGLE_FILE = (1 << 0),
+} HLSFlags;
+
typedef struct HLSContext {
const AVClass *class; // Class for private options.
unsigned number;
@@ -50,12 +57,15 @@ typedef struct HLSContext {
float time;// Set by a private option.
int max_nb_segments; // Set by a private option.
int wrap; // Set by a private option.
+uint32_t flags;// enum HLSFlags
int64_t recording_time;
int has_video;
int64_t start_pts;
int64_t end_pts;
double duration; // last segment duration computed so far, in seconds
+int64_t start_pos;// last segment starting position
+int64_t size; // last segment size
int nb_entries;
HLSSegment *segments;
@@ -88,12 +98,14 @@ static int hls_mux_init(AVFormatContext *s)
avcodec_copy_context(st->codec, s->streams[i]->codec);
st->sample_aspect_ratio = s->streams[i]->sample_aspect_ratio;
}
+hls->start_pos = 0;
return 0;
}
/* Create a new segment and append it to the segment list */
-static int hls_append_segment(HLSContext *hls, double duration)
+static int hls_append_segment(HLSContext *hls, double duration, int64_t pos,
+ int64_t size)
{
HLSSegment *en = av_malloc(sizeof(*en));
@@ -103,6 +115,8 @@ static int hls_append_segment(HLSContext *hls, double
duration)
av_strlcpy(en->filename, av_basename(hls->avf->filename),
sizeof(en->filename));
en->duration = duration;
+en->pos = pos;
+en->size = size;
en->next = NULL;
if (!hls->segments)
@@ -142,6 +156,7 @@ static int hls_window(AVFormatContext *s, int last)
int target_duration = 0;
int ret = 0;
int64_t sequence = FFMAX(hls->start_sequence, hls->sequence -
hls->nb_entries);
+int version = hls->flags & HLS_SINGLE_FILE ? 4 : 3;
if ((ret = avio_open
[FFmpeg-devel] [PATCH] hlsenc: single_file, support HLS ver 4 byteranges
This adds a new option -hls_flags single_file that creates one .ts file
for HLS and adds byteranges to the .m3u8 file, instead of creating one
.ts file for each segment.
This is helpful at least for storing large number of videos, as the
number of files per video is drastically reduced and copying and storing
those files takes less requests and inodes.
This is based on work by Nicolas Martyanoff, discussed on ffmpeg-devel
in July 2014. That patch seems abandoned by the author, and contained
unrelated changes. This patch tries to add the minimum amount of code to
support the byterange playlists.
---
doc/muxers.texi | 23 +++
libavformat/hlsenc.c | 64 ++--
2 files changed, 71 insertions(+), 16 deletions(-)
diff --git a/doc/muxers.texi b/doc/muxers.texi
index 57e81f4..40ae857 100644
--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -194,15 +194,19 @@ can not be smaller than one centi second.
Apple HTTP Live Streaming muxer that segments MPEG-TS according to
the HTTP Live Streaming (HLS) specification.
-It creates a playlist file and numbered segment files. The output
-filename specifies the playlist filename; the segment filenames
-receive the same basename as the playlist, a sequential number and
-a .ts extension.
+It creates a playlist file, and one or more segment files. The output filename
+specifies the playlist filename.
+
+By default, the muxer creates a file for each segment produced. These files
+have the same name as the playlist, followed by a sequential number and a
+.ts extension.
For example, to convert an input file with @command{ffmpeg}:
@example
ffmpeg -i in.nut out.m3u8
@end example
+This example will produce the playlist, @file{out.m3u8}, and segment files:
+@file{out0.ts}, @file{out1.ts}, @file{out2.ts}, etc.
See also the @ref{segment} muxer, which provides a more generic and
flexible implementation of a segmenter, and can be used to perform HLS
@@ -241,6 +245,17 @@ Note that the playlist sequence number must be unique for
each segment
and it is not to be confused with the segment filename sequence number
which can be cyclic, for example if the @option{wrap} option is
specified.
+
+@item hls_flags single_file
+If this flag is set, the muxer will store all segments in a single MPEG-TS
+file, and will use byte ranges in the playlist. HLS playlists generated with
+this way will have the version number 4.
+For example:
+@example
+ffmpeg -i in.nut -hls_flags single_file out.m3u8
+@end example
+Will produce the playlist, @file{out.m3u8}, and a single segment file,
+@file{out.ts}.
@end table
@anchor{ico}
diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index 11f1e5b..b5b41ae 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -34,10 +34,17 @@
typedef struct HLSSegment {
char filename[1024];
double duration; /* in seconds */
+int64_t pos;
+int64_t size;
struct HLSSegment *next;
} HLSSegment;
+typedef enum HLSFlags {
+// Generate a single media file and use byte ranges in the playlist.
+HLS_SINGLE_FILE = (1 << 0),
+} HLSFlags;
+
typedef struct HLSContext {
const AVClass *class; // Class for private options.
unsigned number;
@@ -50,12 +57,15 @@ typedef struct HLSContext {
float time;// Set by a private option.
int max_nb_segments; // Set by a private option.
int wrap; // Set by a private option.
+uint32_t flags;// enum HLSFlags
int64_t recording_time;
int has_video;
int64_t start_pts;
int64_t end_pts;
double duration; // last segment duration computed so far, in seconds
+int64_t start_pos;// last segment starting position
+int64_t size; // last segment size
int nb_entries;
HLSSegment *segments;
@@ -88,12 +98,14 @@ static int hls_mux_init(AVFormatContext *s)
avcodec_copy_context(st->codec, s->streams[i]->codec);
st->sample_aspect_ratio = s->streams[i]->sample_aspect_ratio;
}
+hls->start_pos = 0;
return 0;
}
/* Create a new segment and append it to the segment list */
-static int hls_append_segment(HLSContext *hls, double duration)
+static int hls_append_segment(HLSContext *hls, double duration, int64_t pos,
+ int64_t size)
{
HLSSegment *en = av_malloc(sizeof(*en));
@@ -103,6 +115,8 @@ static int hls_append_segment(HLSContext *hls, double
duration)
av_strlcpy(en->filename, av_basename(hls->avf->filename),
sizeof(en->filename));
en->duration = duration;
+en->pos = pos;
+en->size = size;
en->next = NULL;
if (!hls->segments)
@@ -142,6 +156,7 @@ static int hls_window(AVFormatContext *s, int last)
int target_duration = 0;
int ret = 0;
int64_t sequence = FFMAX(hls->start_sequence, hls->sequence -
hls->nb_entries);
+int version = hls->flags & HLS_SINGLE_FILE ? 4 : 3;
if ((ret = avio_open
[FFmpeg-devel] [PATCH] segment: don't access outside seg->frames array
Fixes wrong number of segments output and undefined memory access.
---
libavformat/segment.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavformat/segment.c b/libavformat/segment.c
index ce784da..1cb6454 100644
--- a/libavformat/segment.c
+++ b/libavformat/segment.c
@@ -684,7 +684,7 @@ static int seg_write_packet(AVFormatContext *s, AVPacket
*pkt)
end_pts = seg->segment_count < seg->nb_times ?
seg->times[seg->segment_count] : INT64_MAX;
} else if (seg->frames) {
-start_frame = seg->segment_count <= seg->nb_frames ?
+start_frame = seg->segment_count < seg->nb_frames ?
seg->frames[seg->segment_count] : INT_MAX;
} else {
if (seg->use_clocktime) {
--
1.8.5.2 (Apple Git-48)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] segment: fix remuxing and copy metadata
To get mpegts metadata copied when segmenting.
---
libavformat/segment.c | 18 --
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git a/libavformat/segment.c b/libavformat/segment.c
index 1cb6454..0be77da 100644
--- a/libavformat/segment.c
+++ b/libavformat/segment.c
@@ -139,6 +139,15 @@ static int segment_mux_init(AVFormatContext *s)
return AVERROR(ENOMEM);
oc->oformat= seg->oformat;
+if (oc->oformat->priv_data_size > 0) {
+oc->priv_data = av_mallocz(oc->oformat->priv_data_size);
+if (!oc->priv_data)
+return AVERROR(ENOMEM);
+if (oc->oformat->priv_class) {
+*(const AVClass **) oc->priv_data = oc->oformat->priv_class;
+av_opt_set_defaults(oc->priv_data);
+}
+}
oc->interrupt_callback = s->interrupt_callback;
av_dict_copy(&oc->metadata, s->metadata, 0);
@@ -159,6 +168,9 @@ static int segment_mux_init(AVFormatContext *s)
ocodec->codec_tag = 0;
}
st->sample_aspect_ratio = s->streams[i]->sample_aspect_ratio;
+if (s->streams[i]->metadata) {
+av_dict_copy(&st->metadata, s->streams[i]->metadata, 0);
+}
}
return 0;
@@ -220,8 +232,10 @@ static int segment_start(AVFormatContext *s, int
write_header)
return err;
}
-if (oc->oformat->priv_class && oc->priv_data)
-av_opt_set(oc->priv_data, "resend_headers", "1", 0); /* mpegts
specific */
+if (oc->oformat->priv_class && oc->priv_data) {
+av_log(s, AV_LOG_VERBOSE, "setting mpegts resend_headers\n");
+av_opt_set(oc->priv_data, "mpegts_flags", "resend_headers", 0);
+}
if (write_header) {
if ((err = avformat_write_header(oc, NULL)) < 0)
--
1.8.5.2 (Apple Git-48)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
