On Mon, 24 Apr 2023, Devin Heitmueller wrote:
Hello Marton,
Thanks for reviewing. Comments inline:
On Sun, Apr 23, 2023 at 2:43 PM Marton Balint <c...@passwd.hu> wrote:
In general, queueing packets in specific components should be avoided if
possible. Muxed packets are normally ordered by DTS and stream id, generic
code ensures that. If you want something other than that, then I think
the perferred way of doing it is by providing a custom interleave
function. (e.g. to ensure you get data packets before video even if data
stream has a higher stream ID.)
To be clear, using a queue was not first choice. It's the result of
trying different approaches, and I'm open to constructive suggestions
on alternatives.
While what you're are saying is correct "in general", there are some
really important reasons why it doesn't work in this case. Permit me
to explain...
By default, the behavior of the mux interleaver is to wait until there
is at least one packet available for each stream before writing to the
output module (in this case decklink). However data formats such as
SMPTE ST2038 are considered to be "sparse" as there isn't necessarily
a continuous stream of packets like with video and audio (there may be
many seconds between packets, or no packets at all). As a result you
can't wait for a packet to be available on all streams since on some
streams it will simply wait continuously until hitting the
max_interleave_delta, at which point it will burst out everything in
the queue. This would cause stalls and/or stuttering playback on the
decklink output.
To accommodate these sparse streams we added code to mux.c to not wait
for 2038 packets. A side-effect of that though is that packets will
be sent through as soon as they hit the mux, which in most cases will
be significantly ahead of the video (potentially hundreds of
milliseconds). This can easily be seen experimentally by adding an
av_log() line to ff_decklink_write_packet(), which will show in many
cases the PTS values of the data frames being sent 20+ frames before
the corresponding video.
Okay, I realized there is one thing here I don't understand. What if we
interleave data packets the same way as others, but we don't wait for them
in order to start flushing packet queues?
So I wonder, if you removed the AV_CODEC_ID_SMPTE_2038 exception
from init_muxer when calculating si->nb_interleaved_streams but keep the
exception in ff_interleave_packet_per_dts, and set
max_interleave_delta to 1, would that work?
Regards,
Marton
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
