Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 31-08-2019 08:11 PM, Marton Balint wrote:



On Sat, 31 Aug 2019, Gyan wrote:




On 31-08-2019 04:14 AM, Marton Balint wrote:



On Fri, 30 Aug 2019, Gyan wrote:




On 22-08-2019 01:10 AM, Marton Balint wrote:



On Tue, 20 Aug 2019, Devin Heitmueller wrote:


A couple of follow-up Qs:

Is auto-detection available for all Decklink devices?


No, but AFAIK it is for all devices which support SDI. Generally 
it's

the older analog capture devices which don't support it.

For those for which it is available, are there any edge cases in 
which

it sets inaccurate mode?


I don't trust the existing detection code enough to use it in
production.  It often fails to detect and thus ffmpeg will exit at
startup.


Can this be fixed by simply increasing the timeout from 1 sec to 2 
seconds?



Also, there are cases where it will misdetect 1080i59 as
1080p30 depending on the card.  It's been on my TODO list for a 
while

to make that code more robust (I believe I know what most of the
issues are), but it hasn't been critical for any of my use cases.


Hmm, interesting... When you say the issue is card-dependant, does 
this mean card _model_ dependant or that the issue can affect one 
card and not the other with of the same model/fw?


If an user can't or won't use format auto detection, then there is 
no way to reliably obtain the correct timecode without this patch 
or some similar hack.


Maybe the offset-related ops could be guarded by a check for auto 
detection, even though the patch won't hurt even if it was used in 
that case.


I am still not liking this too much because we are guessing some 
timecode which we never actually saw, and the guessing is based on 
the number of the VideoFrameArrived callbacks. Yes, usually it is 
called once for every frame, but some frames might not contain 
audio, also there are some examples in the SDK docs where only an 
audio frame (and no video frame) is available in the callback.


Yes, that's why there is a new counter, incremented only upon getting 
a video frame and reset until the demuxer successfully queues the 
packet of the first video frame. I just noticed an issue with it, but 
I can change that.



So it is up to the receiving application to do something to keep 
audio and video in sync, it either drops/duplicates video or audio. 
If it drops/dups video then our guessed start timecode can be wrong 
even if we were only counting the video frames.


By receiving app, do you mean the Decklink driver   or 
decklink_dec/ffmpeg.c?


ffmpeg.c




So I think that if a workaround/hack is needed, we should do that 
differently. One idea is to introduce an option which makes the code 
drop video till the first valid video frame, or the first valid 
timecode. But if autodetection works (or it can be fixed), then I 
just don't see too much use even for this.


As Ilinca's tests showed, without auto format detection, the stored 
timecode is *wrong*. That's a bug so some change is *required*.


The bug is that decklink_dec sets a timecode to AVStream metadata 
which is not the timecode of the first frame. As far as I recall when 
accepting the decklink timecode support patch, I pointed out that it 
is not very useful because it might not be the TC of the first frame.


I am fine with a patch which only sets AVStream timecode if it is 
indeed the first frame. I am fine with a patch which drops starter 
frames without timecode when some option is set. I am not fine with a 
patch which does calculations in deckink_dec to guess a starter timecode.


Ok. Will get back.

Thanks,
Gyan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Marton Balint]

On Sat, 31 Aug 2019, Gyan wrote:




On 31-08-2019 04:14 AM, Marton Balint wrote:



On Fri, 30 Aug 2019, Gyan wrote:




On 22-08-2019 01:10 AM, Marton Balint wrote:



On Tue, 20 Aug 2019, Devin Heitmueller wrote:


A couple of follow-up Qs:

Is auto-detection available for all Decklink devices?


No, but AFAIK it is for all devices which support SDI. Generally it's
the older analog capture devices which don't support it.

For those for which it is available, are there any edge cases in 
which

it sets inaccurate mode?


I don't trust the existing detection code enough to use it in
production.  It often fails to detect and thus ffmpeg will exit at
startup.


Can this be fixed by simply increasing the timeout from 1 sec to 2 
seconds?



Also, there are cases where it will misdetect 1080i59 as
1080p30 depending on the card.  It's been on my TODO list for a while
to make that code more robust (I believe I know what most of the
issues are), but it hasn't been critical for any of my use cases.


Hmm, interesting... When you say the issue is card-dependant, does 
this mean card _model_ dependant or that the issue can affect one 
card and not the other with of the same model/fw?


If an user can't or won't use format auto detection, then there is no 
way to reliably obtain the correct timecode without this patch or 
some similar hack.


Maybe the offset-related ops could be guarded by a check for auto 
detection, even though the patch won't hurt even if it was used in 
that case.


I am still not liking this too much because we are guessing some 
timecode which we never actually saw, and the guessing is based on the 
number of the VideoFrameArrived callbacks. Yes, usually it is called 
once for every frame, but some frames might not contain audio, also 
there are some examples in the SDK docs where only an audio frame (and 
no video frame) is available in the callback.


Yes, that's why there is a new counter, incremented only upon getting a 
video frame and reset until the demuxer successfully queues the packet 
of the first video frame. I just noticed an issue with it, but I can 
change that.



So it is up to the receiving application to do something to keep audio 
and video in sync, it either drops/duplicates video or audio. If it 
drops/dups video then our guessed start timecode can be wrong even if 
we were only counting the video frames.


By receiving app, do you mean the Decklink driver   or 
decklink_dec/ffmpeg.c?


ffmpeg.c




So I think that if a workaround/hack is needed, we should do that 
differently. One idea is to introduce an option which makes the code 
drop video till the first valid video frame, or the first valid 
timecode. But if autodetection works (or it can be fixed), then I just 
don't see too much use even for this.


As Ilinca's tests showed, without auto format detection, the stored 
timecode is *wrong*. That's a bug so some change is *required*.


The bug is that decklink_dec sets a timecode to AVStream metadata which is 
not the timecode of the first frame. As far as I recall when 
accepting the decklink timecode support patch, I pointed out that it is 
not very useful because it might not be the TC of the first frame.


I am fine with a patch which only sets AVStream timecode if it is indeed 
the first frame. I am fine with a patch which drops starter frames without 
timecode when some option is set. I am not fine with a patch which does 
calculations in deckink_dec to guess a starter timecode.


Regards,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 31-08-2019 04:14 AM, Marton Balint wrote:



On Fri, 30 Aug 2019, Gyan wrote:




On 22-08-2019 01:10 AM, Marton Balint wrote:



On Tue, 20 Aug 2019, Devin Heitmueller wrote:


A couple of follow-up Qs:

Is auto-detection available for all Decklink devices?


No, but AFAIK it is for all devices which support SDI. Generally it's
the older analog capture devices which don't support it.

For those for which it is available, are there any edge cases in 
which

it sets inaccurate mode?


I don't trust the existing detection code enough to use it in
production.  It often fails to detect and thus ffmpeg will exit at
startup.


Can this be fixed by simply increasing the timeout from 1 sec to 2 
seconds?



Also, there are cases where it will misdetect 1080i59 as
1080p30 depending on the card.  It's been on my TODO list for a while
to make that code more robust (I believe I know what most of the
issues are), but it hasn't been critical for any of my use cases.


Hmm, interesting... When you say the issue is card-dependant, does 
this mean card _model_ dependant or that the issue can affect one 
card and not the other with of the same model/fw?


If an user can't or won't use format auto detection, then there is no 
way to reliably obtain the correct timecode without this patch or 
some similar hack.


Maybe the offset-related ops could be guarded by a check for auto 
detection, even though the patch won't hurt even if it was used in 
that case.


I am still not liking this too much because we are guessing some 
timecode which we never actually saw, and the guessing is based on the 
number of the VideoFrameArrived callbacks. Yes, usually it is called 
once for every frame, but some frames might not contain audio, also 
there are some examples in the SDK docs where only an audio frame (and 
no video frame) is available in the callback.


Yes, that's why there is a new counter, incremented only upon getting a 
video frame and reset until the demuxer successfully queues the packet 
of the first video frame. I just noticed an issue with it, but I can 
change that.



So it is up to the receiving application to do something to keep audio 
and video in sync, it either drops/duplicates video or audio. If it 
drops/dups video then our guessed start timecode can be wrong even if 
we were only counting the video frames.


By receiving app, do you mean the Decklink driver   or 
decklink_dec/ffmpeg.c?



So I think that if a workaround/hack is needed, we should do that 
differently. One idea is to introduce an option which makes the code 
drop video till the first valid video frame, or the first valid 
timecode. But if autodetection works (or it can be fixed), then I just 
don't see too much use even for this.


As Ilinca's tests showed, without auto format detection, the stored 
timecode is *wrong*. That's a bug so some change is *required*.


Gyan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Marton Balint]

On Fri, 30 Aug 2019, Gyan wrote:




On 22-08-2019 01:10 AM, Marton Balint wrote:



On Tue, 20 Aug 2019, Devin Heitmueller wrote:


A couple of follow-up Qs:

Is auto-detection available for all Decklink devices?


No, but AFAIK it is for all devices which support SDI. Generally it's
the older analog capture devices which don't support it.


For those for which it is available, are there any edge cases in which
it sets inaccurate mode?


I don't trust the existing detection code enough to use it in
production.  It often fails to detect and thus ffmpeg will exit at
startup.


Can this be fixed by simply increasing the timeout from 1 sec to 2 
seconds?



Also, there are cases where it will misdetect 1080i59 as
1080p30 depending on the card.  It's been on my TODO list for a while
to make that code more robust (I believe I know what most of the
issues are), but it hasn't been critical for any of my use cases.


Hmm, interesting... When you say the issue is card-dependant, does 
this mean card _model_ dependant or that the issue can affect one card 
and not the other with of the same model/fw?


If an user can't or won't use format auto detection, then there is no 
way to reliably obtain the correct timecode without this patch or some 
similar hack.


Maybe the offset-related ops could be guarded by a check for auto 
detection, even though the patch won't hurt even if it was used in that 
case.


I am still not liking this too much because we are guessing some timecode 
which we never actually saw, and the guessing is based on the number of 
the VideoFrameArrived callbacks. Yes, usually it is called once for every 
frame, but some frames might not contain audio, also there are some 
examples in the SDK docs where only an audio frame (and no video frame) is 
available in the callback.


So it is up to the receiving application to do something to keep audio 
and video in sync, it either drops/duplicates video or audio. If it 
drops/dups video then our guessed start timecode can be wrong even if we 
were only counting the video frames.


So I think that if a workaround/hack is needed, we should do that 
differently. One idea is to introduce an option which makes the 
code drop video till the first valid video frame, or the first valid 
timecode. But if autodetection works (or it can be fixed), then I just 
don't see too much use even for this.


Regards,
Marton

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 22-08-2019 01:10 AM, Marton Balint wrote:



On Tue, 20 Aug 2019, Devin Heitmueller wrote:


A couple of follow-up Qs:

Is auto-detection available for all Decklink devices?


No, but AFAIK it is for all devices which support SDI. Generally it's
the older analog capture devices which don't support it.


For those for which it is available, are there any edge cases in which
it sets inaccurate mode?


I don't trust the existing detection code enough to use it in
production.  It often fails to detect and thus ffmpeg will exit at
startup.


Can this be fixed by simply increasing the timeout from 1 sec to 2 
seconds?



Also, there are cases where it will misdetect 1080i59 as
1080p30 depending on the card.  It's been on my TODO list for a while
to make that code more robust (I believe I know what most of the
issues are), but it hasn't been critical for any of my use cases.


Hmm, interesting... When you say the issue is card-dependant, does 
this mean card _model_ dependant or that the issue can affect one card 
and not the other with of the same model/fw?


If an user can't or won't use format auto detection, then there is no 
way to reliably obtain the correct timecode without this patch or some 
similar hack.


Maybe the offset-related ops could be guarded by a check for auto 
detection, even though the patch won't hurt even if it was used in that 
case.


Gyan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Marton Balint]

On Tue, 20 Aug 2019, Devin Heitmueller wrote:


A couple of follow-up Qs:

Is auto-detection available for all Decklink devices?


No, but AFAIK it is for all devices which support SDI.  Generally it's
the older analog capture devices which don't support it.


For those for which it is available, are there any edge cases in which
it sets inaccurate mode?


I don't trust the existing detection code enough to use it in
production.  It often fails to detect and thus ffmpeg will exit at
startup.


Can this be fixed by simply increasing the timeout from 1 sec to 2 
seconds?



Also, there are cases where it will misdetect 1080i59 as
1080p30 depending on the card.  It's been on my TODO list for a while
to make that code more robust (I believe I know what most of the
issues are), but it hasn't been critical for any of my use cases.


Hmm, interesting... When you say the issue is card-dependant, does this 
mean card _model_ dependant or that the issue can affect one card and not 
the other with of the same model/fw?


Thanks,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Devin Heitmueller]

> A couple of follow-up Qs:
>
> Is auto-detection available for all Decklink devices?

No, but AFAIK it is for all devices which support SDI.  Generally it's
the older analog capture devices which don't support it.

> For those for which it is available, are there any edge cases in which
> it sets inaccurate mode?

I don't trust the existing detection code enough to use it in
production.  It often fails to detect and thus ffmpeg will exit at
startup.  Also, there are cases where it will misdetect 1080i59 as
1080p30 depending on the card.  It's been on my TODO list for a while
to make that code more robust (I believe I know what most of the
issues are), but it hasn't been critical for any of my use cases.

Devin

-- 
Devin J. Heitmueller - Kernel Labs
http://www.kernellabs.com
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 19-08-2019 07:15 PM, Ilinca Tudose wrote:

Hi Marton,

I want to confirm that we get correctly synced TCs when leaving out the
format_code parameter and using an ffmpeg build from head (not with Gyan's
patch). However, I was under the impression that we need the format_code so
that we can capture different content types, in the original format. For
example multiple frame rates or interlaced content. How would we set this
up if we are not using format_code? Is this identified automatically?


A couple of follow-up Qs:

Is auto-detection available for all Decklink devices?

For those for which it is available, are there any edge cases in which 
it sets inaccurate mode?


Gyan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Ilinca Tudose]

Hi Marton,

I want to confirm that we get correctly synced TCs when leaving out the
format_code parameter and using an ffmpeg build from head (not with Gyan's
patch). However, I was under the impression that we need the format_code so
that we can capture different content types, in the original format. For
example multiple frame rates or interlaced content. How would we set this
up if we are not using format_code? Is this identified automatically?

Thanks,
ilinca

On Sat, Aug 17, 2019 at 6:06 PM Gyan  wrote:

>
>
> On 17-08-2019 09:21 PM, Marton Balint wrote:
> >
> >
> > On Sat, 17 Aug 2019, Gyan wrote:
> >
> >>
> >>
> >> On 17-08-2019 08:42 PM, Marton Balint wrote:
> >>>
> >>>
> >>> On Wed, 14 Aug 2019, Marton Balint wrote:
> >>>
> 
> 
>  On Wed, 14 Aug 2019, Gyan wrote:
> 
> >
> >
> > On 09-08-2019 04:59 PM, Ilinca Tudose wrote:
> >> Hi Marton,
> >>
> >> The issue with the out of sync TC was reproducible on all tapes
> >> and decks
> >> that we tested. I don't have the exact number now, but a few
> >> dozens, less
> >> than 100. They all had between 7 and 17 frames out of sync. We
> >> were not
> >> able to obtain anything more in sync than 7 frames.
> >>
> >> The TC sync was tested by setting up the deck to "burn" the TC
> >> with the
> >> image while capturing the content with TC through ffmpeg. We then
> >> play the
> >> file in a player that supports timecodes and compare the
> >> burned-in TC with
> >> the one captured in the metadata.
> >>
> >> We used Decklink quad 2 and several Sony decks: J30, J3, JH3. We
> >> tested on
> >> multiple decks from each model and confirmed the issue was
> >> present + that
> >> Gyan's patch seemed to fix it. We have used several types of
> >> Betacam tapes
> >> and HDCAM tapes. I can not comment on whether this is the best
> >> solution,
> >> but can confirm it works.
> >>
> >> Let me know if you need more info.
> >>
> >> Thanks,
> >> ilinca
> >
> > Ping.
> 
>  Sorry, I need a bit more time to investigate.
> >>>
> >>> OK, I did a couple of tests myself for HD and SD pal signals, and it
> >>> seems to me if signal autodetection is used, so no format_code is
> >>> specified then timecode is readily available in the first frame. I
> >>> was using BlackMagic SDK 11.1 for testing by the way. Can you
> >>> confirm the same behaviour with your setup?
> >>
> >> So, were you able to reproduce the issue _with_  format_code specified?
> >
> > I was able to reproduce the part where after starting the capture the
> > decklink device records empty frames (with flag
> > bmdFrameHasNoInputSource set) for a couple of frames.
> >
> > I was not able to reproduce the part where the timecode was earlier or
> > later available then the first frame of the useful video signal.
>
> Ok, thanks. I'll get back when I have more info.
>
> Gyan
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".



-- 

---
Ilinca Tudose | Technical Solutions Consultant, Digitization Operations |
go/digitops 
Google Germany GmbH
Erika-Mann-Str. 33
80636 München

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado Diese E-Mail ist
vertraulich. Wenn Sie nicht der richtige Adressat sind, leiten Sie diese
bitte nicht weiter, informieren Sie den Absender und löschen Sie die E-Mail
und alle Anhänge. Vielen Dank. This e-mail is confidential. If you are not
the right addressee please do not forward it, please inform the sender, and
please erase this e-mail including any attachments. Thanks. Der Inhalt
dieser E-Mail spiegelt den derzeitigen Stand der Verhandlungen wider und
dient als Basis für weitere Gespräche. Er soll keine rechtlich verbindliche
Verpflichtung begründen. Eine solche Verpflichtung wird allein durch einen
zwischen allen beteiligten Parteien abgeschlossenen, schriftlichen Vertrag
geschaffen. The above terms reflect a potential business arrangement, are
provided solely as a basis for further discussion, and are not intended to
be and do not constitute a legally binding obligation. No legally binding
obligations will be created, implied, or inferred until an agreement in
final form is executed in writing by all parties involved.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 17-08-2019 09:21 PM, Marton Balint wrote:



On Sat, 17 Aug 2019, Gyan wrote:




On 17-08-2019 08:42 PM, Marton Balint wrote:



On Wed, 14 Aug 2019, Marton Balint wrote:




On Wed, 14 Aug 2019, Gyan wrote:




On 09-08-2019 04:59 PM, Ilinca Tudose wrote:

Hi Marton,

The issue with the out of sync TC was reproducible on all tapes 
and decks
that we tested. I don't have the exact number now, but a few 
dozens, less
than 100. They all had between 7 and 17 frames out of sync. We 
were not

able to obtain anything more in sync than 7 frames.

The TC sync was tested by setting up the deck to "burn" the TC 
with the
image while capturing the content with TC through ffmpeg. We then 
play the
file in a player that supports timecodes and compare the 
burned-in TC with

the one captured in the metadata.

We used Decklink quad 2 and several Sony decks: J30, J3, JH3. We 
tested on
multiple decks from each model and confirmed the issue was 
present + that
Gyan's patch seemed to fix it. We have used several types of 
Betacam tapes
and HDCAM tapes. I can not comment on whether this is the best 
solution,

but can confirm it works.

Let me know if you need more info.

Thanks,
ilinca


Ping.


Sorry, I need a bit more time to investigate.


OK, I did a couple of tests myself for HD and SD pal signals, and it 
seems to me if signal autodetection is used, so no format_code is 
specified then timecode is readily available in the first frame. I 
was using BlackMagic SDK 11.1 for testing by the way. Can you 
confirm the same behaviour with your setup?


So, were you able to reproduce the issue _with_  format_code specified?


I was able to reproduce the part where after starting the capture the 
decklink device records empty frames (with flag 
bmdFrameHasNoInputSource set) for a couple of frames.


I was not able to reproduce the part where the timecode was earlier or 
later available then the first frame of the useful video signal.


Ok, thanks. I'll get back when I have more info.

Gyan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Marton Balint]

On Sat, 17 Aug 2019, Gyan wrote:




On 17-08-2019 08:42 PM, Marton Balint wrote:



On Wed, 14 Aug 2019, Marton Balint wrote:




On Wed, 14 Aug 2019, Gyan wrote:




On 09-08-2019 04:59 PM, Ilinca Tudose wrote:

Hi Marton,

The issue with the out of sync TC was reproducible on all tapes and 
decks
that we tested. I don't have the exact number now, but a few 
dozens, less
than 100. They all had between 7 and 17 frames out of sync. We were 
not

able to obtain anything more in sync than 7 frames.

The TC sync was tested by setting up the deck to "burn" the TC with 
the
image while capturing the content with TC through ffmpeg. We then 
play the
file in a player that supports timecodes and compare the burned-in 
TC with

the one captured in the metadata.

We used Decklink quad 2 and several Sony decks: J30, J3, JH3. We 
tested on
multiple decks from each model and confirmed the issue was present 
+ that
Gyan's patch seemed to fix it. We have used several types of 
Betacam tapes
and HDCAM tapes. I can not comment on whether this is the best 
solution,

but can confirm it works.

Let me know if you need more info.

Thanks,
ilinca


Ping.


Sorry, I need a bit more time to investigate.


OK, I did a couple of tests myself for HD and SD pal signals, and it 
seems to me if signal autodetection is used, so no format_code is 
specified then timecode is readily available in the first frame. I was 
using BlackMagic SDK 11.1 for testing by the way. Can you confirm the 
same behaviour with your setup?


So, were you able to reproduce the issue _with_  format_code specified?


I was able to reproduce the part where after starting the capture the 
decklink device records empty frames (with flag bmdFrameHasNoInputSource 
set) for a couple of frames.


I was not able to reproduce the part where the timecode was earlier or 
later available then the first frame of the useful video signal.


Regards,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 17-08-2019 08:42 PM, Marton Balint wrote:



On Wed, 14 Aug 2019, Marton Balint wrote:




On Wed, 14 Aug 2019, Gyan wrote:




On 09-08-2019 04:59 PM, Ilinca Tudose wrote:

Hi Marton,

The issue with the out of sync TC was reproducible on all tapes and 
decks
that we tested. I don't have the exact number now, but a few 
dozens, less
than 100. They all had between 7 and 17 frames out of sync. We were 
not

able to obtain anything more in sync than 7 frames.

The TC sync was tested by setting up the deck to "burn" the TC with 
the
image while capturing the content with TC through ffmpeg. We then 
play the
file in a player that supports timecodes and compare the burned-in 
TC with

the one captured in the metadata.

We used Decklink quad 2 and several Sony decks: J30, J3, JH3. We 
tested on
multiple decks from each model and confirmed the issue was present 
+ that
Gyan's patch seemed to fix it. We have used several types of 
Betacam tapes
and HDCAM tapes. I can not comment on whether this is the best 
solution,

but can confirm it works.

Let me know if you need more info.

Thanks,
ilinca


Ping.


Sorry, I need a bit more time to investigate.


OK, I did a couple of tests myself for HD and SD pal signals, and it 
seems to me if signal autodetection is used, so no format_code is 
specified then timecode is readily available in the first frame. I was 
using BlackMagic SDK 11.1 for testing by the way. Can you confirm the 
same behaviour with your setup?


So, were you able to reproduce the issue _with_  format_code specified?

Gyan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Marton Balint]

On Wed, 14 Aug 2019, Marton Balint wrote:




On Wed, 14 Aug 2019, Gyan wrote:




On 09-08-2019 04:59 PM, Ilinca Tudose wrote:

Hi Marton,

The issue with the out of sync TC was reproducible on all tapes and decks
that we tested. I don't have the exact number now, but a few dozens, less
than 100. They all had between 7 and 17 frames out of sync. We were not
able to obtain anything more in sync than 7 frames.

The TC sync was tested by setting up the deck to "burn" the TC with the
image while capturing the content with TC through ffmpeg. We then play the
file in a player that supports timecodes and compare the burned-in TC with
the one captured in the metadata.

We used Decklink quad 2 and several Sony decks: J30, J3, JH3. We tested on
multiple decks from each model and confirmed the issue was present + that
Gyan's patch seemed to fix it. We have used several types of Betacam tapes
and HDCAM tapes. I can not comment on whether this is the best solution,
but can confirm it works.

Let me know if you need more info.

Thanks,
ilinca


Ping.


Sorry, I need a bit more time to investigate.


OK, I did a couple of tests myself for HD and SD pal signals, and it seems 
to me if signal autodetection is used, so no format_code is specified 
then timecode is readily available in the first frame. I was using 
BlackMagic SDK 11.1 for testing by the way. Can you confirm the same 
behaviour with your setup?


Because that would make the patch uneeded I believe.

Thanks,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Marton Balint]

On Wed, 14 Aug 2019, Gyan wrote:




On 09-08-2019 04:59 PM, Ilinca Tudose wrote:

Hi Marton,

The issue with the out of sync TC was reproducible on all tapes and decks
that we tested. I don't have the exact number now, but a few dozens, less
than 100. They all had between 7 and 17 frames out of sync. We were not
able to obtain anything more in sync than 7 frames.

The TC sync was tested by setting up the deck to "burn" the TC with the
image while capturing the content with TC through ffmpeg. We then play the
file in a player that supports timecodes and compare the burned-in TC with
the one captured in the metadata.

We used Decklink quad 2 and several Sony decks: J30, J3, JH3. We tested on
multiple decks from each model and confirmed the issue was present + that
Gyan's patch seemed to fix it. We have used several types of Betacam tapes
and HDCAM tapes. I can not comment on whether this is the best solution,
but can confirm it works.

Let me know if you need more info.

Thanks,
ilinca


Ping.


Sorry, I need a bit more time to investigate.

Thanks,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 09-08-2019 04:59 PM, Ilinca Tudose wrote:

Hi Marton,

The issue with the out of sync TC was reproducible on all tapes and decks
that we tested. I don't have the exact number now, but a few dozens, less
than 100. They all had between 7 and 17 frames out of sync. We were not
able to obtain anything more in sync than 7 frames.

The TC sync was tested by setting up the deck to "burn" the TC with the
image while capturing the content with TC through ffmpeg. We then play the
file in a player that supports timecodes and compare the burned-in TC with
the one captured in the metadata.

We used Decklink quad 2 and several Sony decks: J30, J3, JH3. We tested on
multiple decks from each model and confirmed the issue was present + that
Gyan's patch seemed to fix it. We have used several types of Betacam tapes
and HDCAM tapes. I can not comment on whether this is the best solution,
but can confirm it works.

Let me know if you need more info.

Thanks,
ilinca


Ping.

Gyan




On Fri, Aug 9, 2019 at 1:08 PM Gyan  wrote:


On 28-07-2019 01:37 AM, Marton Balint wrote:


On Fri, 26 Jul 2019, Gyan wrote:


Patch supported by and tested at Google.

Seems like a huge hack for a decklink issue... ;)

I wonder if there are better approaches to work around this. Like
dropping all frames until we have a valid signal.

Let's say the scenario is like this,

Frame #TC Status
  1   no   dropped
  2   no   demuxed
  3   no   demuxed
  4   no   dropped
  5  04:11:21:04   dropped
  6  04:11:21:05   demuxed


We don't want to drop frames before 6; they still should be digitized
(sometimes the TC appears after ~15 frames). But the stored TC needs
to be 04:11:21:04 + (2 - 5) =  04:11:21:01


How is this repoducible? do you have an exact model / signal type /
TC source requirement?

(Anchor for reply)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Ilinca Tudose]

Hi Marton,

The issue with the out of sync TC was reproducible on all tapes and decks
that we tested. I don't have the exact number now, but a few dozens, less
than 100. They all had between 7 and 17 frames out of sync. We were not
able to obtain anything more in sync than 7 frames.

The TC sync was tested by setting up the deck to "burn" the TC with the
image while capturing the content with TC through ffmpeg. We then play the
file in a player that supports timecodes and compare the burned-in TC with
the one captured in the metadata.

We used Decklink quad 2 and several Sony decks: J30, J3, JH3. We tested on
multiple decks from each model and confirmed the issue was present + that
Gyan's patch seemed to fix it. We have used several types of Betacam tapes
and HDCAM tapes. I can not comment on whether this is the best solution,
but can confirm it works.

Let me know if you need more info.

Thanks,
ilinca

On Fri, Aug 9, 2019 at 1:08 PM Gyan  wrote:

>
> > On 28-07-2019 01:37 AM, Marton Balint wrote:
> >>
> >>
> >> On Fri, 26 Jul 2019, Gyan wrote:
> >>
> >>>
> >>> Patch supported by and tested at Google.
> >>
> >> Seems like a huge hack for a decklink issue... ;)
> >>
> >> I wonder if there are better approaches to work around this. Like
> >> dropping all frames until we have a valid signal.
> >
> > Let's say the scenario is like this,
> >
> > Frame #TC Status
> >  1   no   dropped
> >  2   no   demuxed
> >  3   no   demuxed
> >  4   no   dropped
> >  5  04:11:21:04   dropped
> >  6  04:11:21:05   demuxed
> >
> >
> > We don't want to drop frames before 6; they still should be digitized
> > (sometimes the TC appears after ~15 frames). But the stored TC needs
> > to be 04:11:21:04 + (2 - 5) =  04:11:21:01
> >
> >> How is this repoducible? do you have an exact model / signal type /
> >> TC source requirement?
>
> (Anchor for reply)
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 28-07-2019 01:37 AM, Marton Balint wrote:



On Fri, 26 Jul 2019, Gyan wrote:



Patch supported by and tested at Google.


Seems like a huge hack for a decklink issue... ;)

I wonder if there are better approaches to work around this. Like 
dropping all frames until we have a valid signal.


Let's say the scenario is like this,

Frame #    TC     Status
 1       no       dropped
 2       no       demuxed
 3       no       demuxed
 4       no       dropped
 5  04:11:21:04   dropped
 6  04:11:21:05   demuxed


We don't want to drop frames before 6; they still should be digitized 
(sometimes the TC appears after ~15 frames). But the stored TC needs 
to be 04:11:21:04 + (2 - 5) =  04:11:21:01


How is this repoducible? do you have an exact model / signal type / 
TC source requirement?


(Anchor for reply)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 28-07-2019 01:37 AM, Marton Balint wrote:



On Fri, 26 Jul 2019, Gyan wrote:



Patch supported by and tested at Google.


Seems like a huge hack for a decklink issue... ;)

I wonder if there are better approaches to work around this. Like 
dropping all frames until we have a valid signal.


Let's say the scenario is like this,

Frame #    TC     Status
 1       no       dropped
 2       no       demuxed
 3       no       demuxed
 4       no       dropped
 5  04:11:21:04   dropped
 6  04:11:21:05   demuxed


We don't want to drop frames before 6; they still should be digitized 
(sometimes the TC appears after ~15 frames). But the stored TC needs to 
be 04:11:21:04 + (2 - 5) =  04:11:21:01


How is this repoducible? do you have an exact model / signal type / TC 
source requirement?


Usually RP188 but tested over various configurations afaik, but my 
contact at Google has the details. I'll ask them to post.


Gyan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Marton Balint]

On Fri, 26 Jul 2019, Gyan wrote:



Patch supported by and tested at Google.


Seems like a huge hack for a decklink issue... ;)

I wonder if there are better approaches to work around this. Like dropping 
all frames until we have a valid signal.


How is this repoducible? do you have an exact model / signal type / TC 
source requirement?


Thanks,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

Patch supported by and tested at Google.

Gyan
From e51bd8201ddf618dce33ada70a9bc6ce2f33b07b Mon Sep 17 00:00:00 2001
From: Gyan Doshi 
Date: Mon, 1 Jul 2019 23:43:44 +0530
Subject: [PATCH] avdevice/decklink: adjust for timecode lag

The decklink demuxer may not start receiving timecode with
the first frame, but only some frames later.

The demuxer, at present, naively stores the first received timecode.
This patch monitors the lag in video frames, and adjusts the timecode
before saving it in stream metadata.
---
 libavdevice/decklink_common.h |  4 +++
 libavdevice/decklink_dec.cpp  | 53 +--
 2 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h
index 921818ba41..f149e7ff66 100644
--- a/libavdevice/decklink_common.h
+++ b/libavdevice/decklink_common.h
@@ -149,6 +149,10 @@ struct decklink_ctx {
 
 int channels;
 int audio_depth;
+
+/* Fields for timecode correction */
+unsigned long vidframeCount;//  frameCount tracks audio-only 
packets as well
+unsigned long firstframeIndex;  //  value of frameCount for first 
successfully demuxed video frame
 };
 
 typedef enum { DIRECTION_IN, DIRECTION_OUT} decklink_direction_t;
diff --git a/libavdevice/decklink_dec.cpp b/libavdevice/decklink_dec.cpp
index 4da9122bff..d0ebd0ac0a 100644
--- a/libavdevice/decklink_dec.cpp
+++ b/libavdevice/decklink_dec.cpp
@@ -41,6 +41,7 @@ extern "C" {
 #include "libavutil/imgutils.h"
 #include "libavutil/intreadwrite.h"
 #include "libavutil/time.h"
+#include "libavutil/timecode.h"
 #include "libavutil/mathematics.h"
 #include "libavutil/reverse.h"
 #include "avdevice.h"
@@ -736,6 +737,8 @@ HRESULT decklink_input_callback::VideoInputFrameArrived(
 videoFrame->GetStreamTime(, ,
   ctx->video_st->time_base.den);
 
+ctx->vidframeCount++;
+
 if (videoFrame->GetFlags() & bmdFrameHasNoInputSource) {
 if (ctx->draw_bars && videoFrame->GetPixelFormat() == 
bmdFormat8BitYUV) {
 unsigned bars[8] = {
@@ -765,6 +768,8 @@ HRESULT decklink_input_callback::VideoInputFrameArrived(
 
 // Handle Timecode (if requested)
 if (ctx->tc_format) {
+if (!ctx->firstframeIndex)
+ctx->firstframeIndex = ctx->vidframeCount;
 IDeckLinkTimecode *timecode;
 if (videoFrame->GetTimecode(ctx->tc_format, ) == 
S_OK) {
 const char *tc = NULL;
@@ -778,7 +783,9 @@ HRESULT decklink_input_callback::VideoInputFrameArrived(
 AVDictionary* metadata_dict = NULL;
 int metadata_len;
 uint8_t* packed_metadata;
+unsigned long offset = ctx->firstframeIndex - 
ctx->vidframeCount;  // represents offset of first returned frame relative to 
first frame with timecode
 if (av_dict_set(_dict, "timecode", tc, 
AV_DICT_DONT_STRDUP_VAL) >= 0) {
+av_dict_set_int(_dict, "tc_offset", 
offset, 0);
 packed_metadata = 
av_packet_pack_dictionary(metadata_dict, _len);
 av_dict_free(_dict);
 if (packed_metadata) {
@@ -880,6 +887,8 @@ HRESULT decklink_input_callback::VideoInputFrameArrived(
 videoFrame->AddRef();
 
 if (avpacket_queue_put(>queue, ) < 0) {
+if (ctx->firstframeIndex == ctx->vidframeCount)
+ctx->firstframeIndex = 0;
 ++ctx->dropped;
 }
 }
@@ -1266,10 +1275,48 @@ int ff_decklink_read_packet(AVFormatContext *avctx, 
AVPacket *pkt)
 if (ctx->tc_format && !(av_dict_get(ctx->video_st->metadata, "timecode", 
NULL, 0))) {
 int size;
 const uint8_t *side_metadata = av_packet_get_side_data(pkt, 
AV_PKT_DATA_STRINGS_METADATA, );
+
 if (side_metadata) {
-   if (av_packet_unpack_dictionary(side_metadata, size, 
>video_st->metadata) < 0)
-   av_log(avctx, AV_LOG_ERROR, "Unable to set timecode\n");
-}
+char offset_tc[AV_TIMECODE_STR_SIZE];
+int ret;
+AVTimecode tc;
+AVRational vid_rate = ctx->video_st->r_frame_rate;
+int64_t offset = 0;
+AVDictionary *first_tc = NULL;
+AVDictionaryEntry *tcstr = NULL;
+AVDictionaryEntry *offsetstr = NULL;
+
+if (av_packet_unpack_dictionary(side_metadata, size, _tc) < 
0 ||
+!(tcstr = av_dict_get(first_tc, "timecode", NULL, 0)) ) {
+av_log(avctx, AV_LOG_ERROR, "Unable to find timecode\n");
+return 0;
+}
+
+if (tcstr)
+av_log(avctx, AV_LOG_DEBUG, "Serial frame timecode is %s.\n", 
tcstr->value);
+
+if (av_timecode_init_from_string(, vid_rate, tcstr->value, 
avctx) < 0) {
+