I'm trying to build a stereo camera streamer that uses two individual
cameras, each producing a h264 rtp stream using libav.
A client application then receives the two streams, and synchronously
displays the two images.
It is crucial that the frames get captured at the same time (no problem
there), and that the client has a way of associating the two frames with
each other.
I was able to get encoding / muxing /demuxing/ decoding working, but
the PTS on the client side don't agree with the sender's.
More details:
-----------------
The application encodes with a monotonically increasing frame number
(works just fine), and sends it as well.
(I cut out error checking etc to make it more readable)
frame.pts = some_increasing_number;
av_init_packet(&packet);
packet.data = NULL;
packet.size = 0;
packet.stream_index = 0;
int ret = avcodec_encode_video2(codecContext, outPacket,
&frame, &got_output);
ffio_open_dyn_packet_buf(&m_ctx->pb, m_maxPacketSize);
av_interleaved_write_frame(m_ctx, &packet);
avio_close_dyn_buf(m_ctx->pb, data);
After that the data is written to the network.
When received on the client:
rc = avcodec_decode_video2(codecContext, &frame, &frameFinished,
&packet);
the frame.pts is always NO_PTS, and the packet.pts is not identical to
the one on the sent packet. By observation and by looking at rtpdec.c it
seems that the pts is reset to start at 1 when the first frame comes in.
At that point, the server may be at pts = 10000, but the client starts
at pts = 1. That rules out using the packet pts as an indicator of when
actually the frame was generated on the camera server.
The question:
-----------------
How can I recover the original pts of the frame, given that frame pts is
not set, and packet pts is lost in transit?
(I looked at http://dranger.com/ffmpeg/tutorial05.html, but that assumes
packet pts is correct)
There are also rtcp packets being sent, and some fields in the
(private!) RTPDemuxContext like rtcp_ts_offset that could potentially be
useful to extract the original pts. How is that supposed to work?
Any suggestions?
Thanks!
Bernd
_______________________________________________
libav-devel mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-devel
