> On Oct 31, 2020, at 5:15 PM, Marton Balint <c...@passwd.hu> wrote: > On Sat, 31 Oct 2020, Dave Rice wrote: >>> On Oct 31, 2020, at 3:47 PM, Marton Balint <c...@passwd.hu> wrote: >>> On Sat, 31 Oct 2020, Dave Rice wrote: >>>> Hi Marton, >>>>> On Oct 31, 2020, at 12:56 PM, Marton Balint <c...@passwd.hu> wrote: >>>>> Fixes out of sync timestamps in ticket #8762. >>>> Although Michael’s recent patch does address the issue documented in 8762, >>>> I haven’t found this patch to fix the issue. I tried with -c:a copy and >>>> with -c:a pcm_s16le with some sample files that exhibit this issue but >>>> each output was out of sync. I put an output at >>>> https://gist.github.com/dericed/659bd843bd38b6f24a60198b5e345795. That >>>> output notes that 3597 packages of video are read and 3586 packets of >>>> audio. In the resulting file, at the end of the timeline the audio is 9 >>>> frames out of sync and my output video stream is 00:02:00.020 and output >>>> audio stream is 00:01:59.653. >>>> Beyond copying or encoding the audio, are there other options I should use >>>> to test this? >>> Well, it depends on what you want. After this patch you should get a file >>> which has audio packets synced to video, but the audio stream is sparse, >>> not every video packet has a corresponding audio packet. (It looks like our >>> MOV muxer does not support muxing of sparse audio therefore does not >>> produce proper timestamps. But MKV does, please try that.) >>> You can also make ffmpeg generate the missing audio based on packet >>> timestamps. Swresample has an async=1 option, so something like this should >>> get you synced audio with continous audio packets: >>> ffmpeg -y -i 1670520000_12.dv -c:v copy \ >>> -af aresample=async=1:min_hard_comp=0.01 -c:a pcm_s16le 1670520000_12.mov >> >> Thank you for this. With the patch and async, the result is synced and the >> resulting audio was the same as Michael’s patch. >> >> Could you explain why you used min_hard_comp here? IIUC min_hard_comp is a >> set a threshold between the strategies of trim/fill or stretch/squeeze to >> align the audio to time; however, the async documentation says "Setting this >> to 1 will enable filling and trimming, larger values represent the maximum >> amount in samples that the data may be stretched or squeezed” so I thought >> that async=1 would not permit stretch/squeeze anyway. > > It is documented poorly, but if you check the source code you will see that > async=1 implicitly sets min_comp to 0.001 enabling trimming/dropping. > min_hard_comp decides the threshold when silence injection actually happens, > and the default for that is 0.1, which is more than a frame, therefore not > acceptable if we want to maintain <1 frame accuracy. Or at least that is how > I think it should work.
Thanks for the explanation. I’ve tested this patch with larger sample sets and really appreciate that the audio timestamps are now properly ordered. For instance with https://archive.org/download/dvr-007/DVR_007.dv, before the patch the timestamps for audio around 20 - 30 seconds get severely jumbled which makes it difficult to use astats metadata. While the documentation could be better, as I’ve seen many forums entries of ffmpeg users struggling to keep dv from tape transfers in sync, I don’t think that’s blocking. I’d really like to see this merged. Although this patch resolves the issue, I think Michael’s patch is interesting too as it would allow access to the audio dif blocks of dv frames with no audio source pack metadata. I’d have to test more to find an instance where the 'dvaudio_concealment pass’ option of Michael’s patch provides a different result than the added silence from async. Thanks so much. Dave > Regards, > Marton > >>>>> Signed-off-by: Marton Balint <c...@passwd.hu> >>>>> --- >>>>> libavformat/dv.c | 16 ++-------------- >>>>> tests/ref/seek/lavf-dv | 18 +++++++++--------- >>>>> 2 files changed, 11 insertions(+), 23 deletions(-) >>>>> diff --git a/libavformat/dv.c b/libavformat/dv.c >>>>> index 3e0d12c0e3..26a78139f5 100644 >>>>> --- a/libavformat/dv.c >>>>> +++ b/libavformat/dv.c >>>>> @@ -49,7 +49,6 @@ struct DVDemuxContext { >>>>> uint8_t audio_buf[4][8192]; >>>>> int ach; >>>>> int frames; >>>>> - uint64_t abytes; >>>>> }; >>>>> static inline uint16_t dv_audio_12to16(uint16_t sample) >>>>> @@ -258,7 +257,7 @@ static int dv_extract_audio_info(DVDemuxContext *c, >>>>> const uint8_t *frame) >>>>> c->ast[i] = avformat_new_stream(c->fctx, NULL); >>>>> if (!c->ast[i]) >>>>> break; >>>>> - avpriv_set_pts_info(c->ast[i], 64, 1, 30000); >>>>> + avpriv_set_pts_info(c->ast[i], 64, c->sys->time_base.num, >>>>> c->sys->time_base.den); >>>>> c->ast[i]->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; >>>>> c->ast[i]->codecpar->codec_id = AV_CODEC_ID_PCM_S16LE; >>>>> @@ -387,8 +386,7 @@ int avpriv_dv_produce_packet(DVDemuxContext *c, >>>>> AVPacket *pkt, >>>>> for (i = 0; i < c->ach; i++) { >>>>> c->audio_pkt[i].pos = pos; >>>>> c->audio_pkt[i].size = size; >>>>> - c->audio_pkt[i].pts = c->abytes * 30000 * 8 / >>>>> - c->ast[i]->codecpar->bit_rate; >>>>> + c->audio_pkt[i].pts = (c->sys->height == 720) ? (c->frames & >>>>> ~1) : c->frames; >>>>> ppcm[i] = c->audio_buf[i]; >>>>> } >>>>> if (c->ach) >>>>> @@ -401,10 +399,7 @@ int avpriv_dv_produce_packet(DVDemuxContext *c, >>>>> AVPacket *pkt, >>>>> c->audio_pkt[2].size = c->audio_pkt[3].size = 0; >>>>> } else { >>>>> c->audio_pkt[0].size = c->audio_pkt[1].size = 0; >>>>> - c->abytes += size; >>>>> } >>>>> - } else { >>>>> - c->abytes += size; >>>>> } >>>>> >>>>> /* Now it's time to return video packet */ >>>>> @@ -444,13 +439,6 @@ static int64_t dv_frame_offset(AVFormatContext *s, >>>>> DVDemuxContext *c, >>>>> void ff_dv_offset_reset(DVDemuxContext *c, int64_t frame_offset) >>>>> { >>>>> c->frames = frame_offset; >>>>> - if (c->ach) { >>>>> - if (c->sys) { >>>>> - c->abytes = av_rescale_q(c->frames, c->sys->time_base, >>>>> - (AVRational) { 8, >>>>> c->ast[0]->codecpar->bit_rate }); >>>>> - } else >>>>> - av_log(c->fctx, AV_LOG_ERROR, "cannot adjust audio bytes\n"); >>>>> - } >>>>> c->audio_pkt[0].size = c->audio_pkt[1].size = 0; >>>>> c->audio_pkt[2].size = c->audio_pkt[3].size = 0; >>>>> } >>>>> diff --git a/tests/ref/seek/lavf-dv b/tests/ref/seek/lavf-dv >>>>> index 0000ff5abe..f63e4460be 100644 >>>>> --- a/tests/ref/seek/lavf-dv >>>>> +++ b/tests/ref/seek/lavf-dv >>>>> @@ -7,9 +7,9 @@ ret: 0 st: 0 flags:0 ts: 0.800000 >>>>> ret: 0 st: 0 flags:1 dts: 0.800000 pts: 0.800000 pos:2880000 >>>>> size:144000 >>>>> ret: 0 st: 0 flags:1 ts:-0.320000 >>>>> ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 >>>>> size:144000 >>>>> -ret: 0 st: 1 flags:0 ts: 2.576667 >>>>> +ret: 0 st: 1 flags:0 ts: 2.560000 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> -ret: 0 st: 1 flags:1 ts: 1.470833 >>>>> +ret: 0 st: 1 flags:1 ts: 1.480000 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> ret: 0 st:-1 flags:0 ts: 0.365002 >>>>> ret: 0 st: 0 flags:1 dts: 0.360000 pts: 0.360000 pos:1296000 >>>>> size:144000 >>>>> @@ -19,9 +19,9 @@ ret: 0 st: 0 flags:0 ts: 2.160000 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> ret: 0 st: 0 flags:1 ts: 1.040000 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> -ret: 0 st: 1 flags:0 ts:-0.058333 >>>>> +ret: 0 st: 1 flags:0 ts:-0.040000 >>>>> ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 >>>>> size:144000 >>>>> -ret: 0 st: 1 flags:1 ts: 2.835833 >>>>> +ret: 0 st: 1 flags:1 ts: 2.840000 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> ret: 0 st:-1 flags:0 ts: 1.730004 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> @@ -31,10 +31,10 @@ ret: 0 st: 0 flags:0 ts:-0.480000 >>>>> ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 >>>>> size:144000 >>>>> ret: 0 st: 0 flags:1 ts: 2.400000 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> -ret: 0 st: 1 flags:0 ts: 1.306667 >>>>> -ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> -ret: 0 st: 1 flags:1 ts: 0.200833 >>>>> +ret: 0 st: 1 flags:0 ts: 1.320000 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> +ret: 0 st: 1 flags:1 ts: 0.200000 >>>>> +ret: 0 st: 0 flags:1 dts: 0.200000 pts: 0.200000 pos: 720000 >>>>> size:144000 >>>>> ret: 0 st:-1 flags:0 ts:-0.904994 >>>>> ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 >>>>> size:144000 >>>>> ret: 0 st:-1 flags:1 ts: 1.989173 >>>>> @@ -43,9 +43,9 @@ ret: 0 st: 0 flags:0 ts: 0.880000 >>>>> ret: 0 st: 0 flags:1 dts: 0.880000 pts: 0.880000 pos:3168000 >>>>> size:144000 >>>>> ret: 0 st: 0 flags:1 ts:-0.240000 >>>>> ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 >>>>> size:144000 >>>>> -ret: 0 st: 1 flags:0 ts: 2.671667 >>>>> +ret: 0 st: 1 flags:0 ts: 2.680000 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> -ret: 0 st: 1 flags:1 ts: 1.565833 >>>>> +ret: 0 st: 1 flags:1 ts: 1.560000 >>>>> ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 >>>>> size:144000 >>>>> ret: 0 st:-1 flags:0 ts: 0.460008 >>>>> ret: 0 st: 0 flags:1 dts: 0.480000 pts: 0.480000 pos:1728000 >>>>> size:144000 >>>>> — 2.26.2 >>>> Best Regards, >>>> Dave Rice >>>> _______________________________________________ >>>> ffmpeg-devel mailing list >>>> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org> >>>> <mailto:ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>> >>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>>> <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel><https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>>> <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>> >>>> To unsubscribe, visit link above, or email >>>> ffmpeg-devel-requ...@ffmpeg.org <mailto:ffmpeg-devel-requ...@ffmpeg.org> >>>> <mailto:ffmpeg-devel-requ...@ffmpeg.org >>>> <mailto:ffmpeg-devel-requ...@ffmpeg.org>> with subject "unsubscribe". >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org> >>> <mailto:ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>> >>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel><https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>> >>> To unsubscribe, visit link above, or email >>> ffmpeg-devel-requ...@ffmpeg.org <mailto:ffmpeg-devel-requ...@ffmpeg.org> >>> <mailto:ffmpeg-devel-requ...@ffmpeg.org >>> <mailto:ffmpeg-devel-requ...@ffmpeg.org>> with subject "unsubscribe". >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org> >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
