> On Jun 23, 2018, at 12:07 AM, Ronak <ronak2121-at-yahoo....@ffmpeg.org> wrote: > > > >> On Jun 22, 2018, at 7:59 PM, Ronak Patel <ronak2...@yahoo.com> wrote: >> >> >>> On Jun 22, 2018, at 5:03 PM, Carl Zwanzig <c...@tuunq.com> wrote: >>> >>>> On 6/22/2018 1:37 PM, Ronak wrote: >>>> We have audio files that are more than 100 hours long, and we need them >>>> to be fragmented quickly. It totally seems like there's an I/O problem >>>> here because fragmentation is faster the longer we set our fragment size >>>> to. >>> Sounds like it's time to break out iostat or sar (or even dtrace) and do >>> some snooping. >>> >>> Are you reading and writing to different drives? (The usual metric for a >>> spinning drive is 150 IOPS for random access, RAID doesn't necessarily >>> speed that up.) >>> >> >> Nope it’s the same disk that we’re reading and writing to. I’m really >> curious why we can’t just buffer fragments in ram and write them out in >> larger bursts. Why not make this an option we can set? >> >> This would simulate the performance for a 10 sec fragment size. >> >> I’d imagine you have the same reading buffer regardless of the fragment size. >> >> I’m thinking of running this on even more powerful machines to see if I can >> get the times down. But is it possible to get an option to tweak the output >> buffer size? >> > > The problem is probably because of this: > > [dash @ 0x7f92cd005600] Opening 'media_0.m3u8.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.m4s' for reading > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.mpd.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'media_0.m3u8.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.m4s' for reading > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.mpd.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'media_0.m3u8.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.m4s' for reading > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.mpd.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'media_0.m3u8.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.m4s' for reading > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.mpd.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'media_0.m3u8.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.m4s' for reading > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.mpd.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'media_0.m3u8.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.m4s' for reading > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.mpd.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'media_0.m3u8.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.m4s' for reading > [dash @ 0x7f92cd005600] Opening 'test_60_hr_10sec.mpd.tmp' for writing > [dash @ 0x7f92cd005600] Opening 'media_0.m3u8.tmp' for writing > > Ffmpeg constantly makes this tmp files, and seems to be opening and closing > the file handles constantly. > Is there a way to just keep the main files open and just keep appending to > it? Why make temp files at all? >
I just tested Apple's segmentation tools: mediafilesegmenter, and I can confirm that a 26 hour file took only: real 1m35.576s user 1m32.617s sys 0m29.705s time to fragment on a 2015 Macbook Pro 2.2 GHz, 16 GB of RAM. The big differences I see are: 1. Apple does not create temp files nor open & close file handles repeatedly. 2. Apple does not write out sidx boxes, only moof for these fMP4 files. It seems like sidx boxes are not required when the content only contains audio. Is this true? Can ffmpeg be configured to skip generating them? 3. Apple's fragmentation is consistent. Every single fragment has exactly the same duration. 4. Apple sets the defaultSampleDuration in the moof atoms to 1024 for HE-AACv2 streams, the same as LC-AAC at the same sampling rate. I would like to see how I can fix these 4 flaws in ffmpeg Can anyone please give me pointers on which source files to look at and pointers on how to make these code changes? I'd like to submit patches for these issues soon. Thanks, Ronak >>> Later, >>> >>> z! >>> _______________________________________________ >>> ffmpeg-user mailing list >>> ffmpeg-user@ffmpeg.org <mailto:ffmpeg-user@ffmpeg.org> >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >>> <http://ffmpeg.org/mailman/listinfo/ffmpeg-user> >>> >>> To unsubscribe, visit link above, or email >>> ffmpeg-user-requ...@ffmpeg.org <mailto:ffmpeg-user-requ...@ffmpeg.org> with >>> subject "unsubscribe". > > _______________________________________________ > ffmpeg-user mailing list > ffmpeg-user@ffmpeg.org <mailto:ffmpeg-user@ffmpeg.org> > http://ffmpeg.org/mailman/listinfo/ffmpeg-user > <http://ffmpeg.org/mailman/listinfo/ffmpeg-user> > > To unsubscribe, visit link above, or email > ffmpeg-user-requ...@ffmpeg.org <mailto:ffmpeg-user-requ...@ffmpeg.org> with > subject "unsubscribe". _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".
