>Thanks again. Just to make sure I'm clear can we run through an example?
>So I have file with audio and video. >The audio stream has a start time of 56994 >The video stream has a start time of 64194 >Both have a time base of 1/90000 >The first audio packet I get has a pts of 56994 >The first video packet I get has a pts of 64194 >The next video packets have a pts of: (coming out in this order) >56994 >60594 >74994 <- need packet re-order here, if all these are video? >67794 <- need packet re-order here, if all these are video? >71394 >I get a frame on the 2nd call to avcodec_decode_video2() (i.e on the packet >with a pts of 56994). >So given this, should I be playing the first set of decoded audio at the >same time that I show the first decoded frame? or is there some delay I need >to add to one of the output streams. The fact that my lowest pts for video >is 56994, but the start_time is 64194 is a little confusing to me, just I'm >just trying to understand the offsets here. Your video decoder pulls data from the video queue. You have the correlate the timestamp from the video packet to the audio. In your case, video with pts 64194 needs to be played when audio around that is played. I don't know what the timestamp is after audio 56994, but your video prob needs to be played between audio pts 56994 and the next one (say audio pts 8000). Now, audio usually comes in a AVPacket that when decompressed is one second or so long. So, you need to interpolate your timestamp across the audio buffer (use channel count, frequency, etc), ffplay does it with is->audio_clock += (double)data_size / (double)(n * dec->sample_rate); in audio_decode_frame(). For video we need to build a "wait/sleep" until we around pts 64194 (the pts of the video). Code that does that can be found in ffplay, in the function compute_frame_delay(). Referring to that, since you might be able to step through it, and see what it does. Seeing your video queue, I see timestamps that are non-lineair? In a nutshell, Add audio and video to separate queues. Poll both queues, each in a separate thread (with enough locking to protect the queues of course). Start playing the audio (by feeding the packets you stored in the audio queue to the decoder, and from there your WAVE/PCM player). When you feed to audio to the PCM player, keep its timestamp for the packet, call that the clock. Remember that one AVPacket can contain several audio buffers when decoded, so you may need to interpolate your timestamps over the size of the buffer. Sleep/delay your video thread until the timestamp of the audio packet that is playing matches your video (bigger then). Display your video frame. Keep feeding packets to both queues (and threads) until stop. _______________________________________________ libav-user mailing list [email protected] https://lists.mplayerhq.hu/mailman/listinfo/libav-user
