Hi, Sven, On Wed, Sep 15, 2010 at 7:11 PM, Sven Wasmer <[email protected]> wrote:
> Hi Alex, > > thank you for your help! YES, network streamed h264 decoding is a big > secret on the net, but h264 is far more complex than h263. > > Well, I think I can't use "av_open_input_file", because my server and > client use live555 for the rtsp/rtp handling. Thus I get (not > fragmented) nalus by each RTP-Frame and get them into a buffer. > The current procedere is that I get a frame (first SPS, second PPS) > and hand them over to the decoder (error: no frame). > all the other frames are handled the same way. > > As I said in my previous post, SPS and PPS must have an IDR slice followed them, if not you get errors. Another point is, that I think that "ffplay" works the same way you > described in your code, but ffplay does also not play the streamed > h264 correctly for me (only grey difference pictures are played, might > be missing the IDR frame?). It can on the other hand play the "file" > very well, thus I think, the decoder is not initialized correctly. > > FFplay can work with streaming. Do you check this ? http://www.wu.ece.ufl.edu/projects/wirelessVideo/project/realTimeCoding/ When you pass an IDR slice to avcodec_decode_video, FFmpeg will decode that frame, and store it for reference. FFMpeg will not output this IDR until you feed another frame to avcodec_decode_video. Hope it helps. Huy. Maybe I need to backbuffer some of these RTP-frames and let > av_read_frame work through that package? That would make only sense, > if av_read_frame would write some data to the AVFormatContext, right? > > I will try it and come back later. > > cheers > Sven > > 2010/9/15 Alex Grosu <[email protected]>: > > Hi Sven > > > > AFAIK, there isn't any other possibility when decoding raw NALs is > wanted. > > The correct manner to handle this kind of issue is very vague discussed > over > > the net. At least, I couldn't find proper information for this. > > Anyway, you said you want to decode streams which are received through > rtsp > > protocol. Why don't you use: > > > > if(av_open_input_file(&pFormatCtx, "rtsp_address", NULL, 0, NULL)!=0) > > return -1; // Couldn't open file > > videoStream=-1; > > audioStream=-1; > > for(i=0; i<pFormatCtx->nb_streams; i++) { > > if(pFormatCtx->streams[i]->codec->codec_type==CODEC_TYPE_VIDEO && > > videoStream < 0) { > > videoStream=i; > > } > > if(pFormatCtx->streams[i]->codec->codec_type==CODEC_TYPE_AUDIO && > > audioStream < 0) { > > audioStream=i; > > } > > } > > vCodecCtxp = FormatCtx->streams[videoStream]->codec; > > aCodecCtx = pFormatCtx->streams[audioStream]->codec; > > > > After this, use av_read_frame: > > > > while(av_read_frame(pFormatCtx, &packet)>=0) { > > // Is this a packet from the video stream? > > if(packet.stream_index==videoStream) { > > // Decode video frame > > avcodec_decode_video(pCodecCtx, pFrame, &frameFinished, > > packet.data, packet.size); > > if(frameFinished) { > > //do something with decoded data > > } > > } > > } > > > > The board I am using sends the streams in two ways: > > 1. using rtsp protocol (for testing purpose) > > 2. sending raw NALs in order when they appear (which I have to use) > > Using the above code, I can properly decode the rtsp protocol. > > Hope this helps > > Regards, Alex > > > > > > Sven Wasmer wrote: > >> > >> Hey, > >> > >> I got the same. I want to stream raw h264 streams via RTSP/RTP. In > >> another post "Re: [libav-user] Decode h264/rtp video -> problem with > >> SPS/PPS and extradata" I mentioned already my problem. > >> > >> Did you get any further? I tried to concatenate multiple received > >> RTP-Frames, always prepending 0x00, 0x00,0x00,0x01. But I still get > >> "-1" after avcodec_video_decode. > >> > >> Is there no other way to initialize the decoder correctly for h264 > >> using SDP (SPS,PPS)? > >> > >> Cheers and thx for your help! > >> Sven > >> > >> 2010/9/14 Alex Grosu <[email protected]>: > >> > >>> > >>> Hello Huy > >>> > >>> I will check your information. Great advices, thanks a lot > >>> Regards, > >>> Alex > >>> > >>> Nhat Huy wrote: > >>> > >>>> > >>>> On Fri, Sep 10, 2010 at 5:38 PM, Alex Grosu <[email protected]> wrote: > >>>> > >>>> > >>>> > >>>>> > >>>>> Hello > >>>>> > >>>>> The information you passed will help me later, thanks. In this > moment, > >>>>> my > >>>>> application communicates with an embedded product which sends only > one > >>>>> frame > >>>>> per NAL in this moment. Later, it will send inceed multiple slices, > and > >>>>> we > >>>>> said that we will see what we are going to do then. Is there any > >>>>> possibility > >>>>> to find which slice is the start of a frame sequence and which slice > is > >>>>> the > >>>>> end of it? In you case for example, is there a possibility to find > the > >>>>> first > >>>>> 5 from I frame and the last 5 from I frame (or, the first 1 and last > 1 > >>>>> from > >>>>> B frame / P frame) ? > >>>>> Regards, > >>>>> Alex > >>>>> > >>>>> > >>>>> Nhat Huy wrote: > >>>>> > >>>>> > >>>>> > >>>>>> > >>>>>> On Thu, Sep 9, 2010 at 9:00 PM, Alex Grosu <[email protected]> wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> Hello Huy > >>>>>>> > >>>>>>> Thanks a lot for the answer. I found this solution 2 days ago, but > >>>>>>> first > >>>>>>> I > >>>>>>> thought it was a dirty one. I wanted to check this before posting > >>>>>>> here > >>>>>>> again, but you took it ahead. Your pointing shows me that in fact > >>>>>>> this > >>>>>>> is > >>>>>>> not a dirty solution, so thanks a lot for support. > >>>>>>> So, what I had to do (as Huy suggested), was to couple 7 8 and 5 > NAL > >>>>>>> types > >>>>>>> into only one buffer and to present it to libavcodec (in > >>>>>>> avcodec_decode_video2 function). > >>>>>>> Thus, using the [ and ] characters to delimit buffer boundaries, I > >>>>>>> was > >>>>>>> coupling following before send them to libavcodec: > >>>>>>> [7 8 5] [1] [1] [1] [1] [7 8 5] [1] [1] [1] and so on. > >>>>>>> I don't receive errors anymore. Everything works now > >>>>>>> > >>>>>>> Thank you > >>>>>>> > >>>>>>> > >>>>>>> Nhat Huy wrote: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> > >>>>>>>> On Thu, Sep 2, 2010 at 3:58 PM, Alex Grosu <[email protected]> wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> > >>>>>>>>> Hello > >>>>>>>>> > >>>>>>>>> I am currently decoding h264 streams (with libavcodec) and I am > >>>>>>>>> stuck > >>>>>>>>> with > >>>>>>>>> the logic for SPS (sequence parameter set) and PPS (picture > >>>>>>>>> parameter > >>>>>>>>> set). > >>>>>>>>> The board from which I am receiving the packages is sending a NAL > >>>>>>>>> unit > >>>>>>>>> in > >>>>>>>>> each packet. The sequence for NAL types is: > >>>>>>>>> 7(SPS) 8(PPS) 5 1 1 1 1 7 8 5 1 1 1... > >>>>>>>>> I am putting in front of each NAL the start sequence 0x000001 and > >>>>>>>>> sending > >>>>>>>>> this packet to avcodec_decode_video2. When 7 and 8 types are > >>>>>>>>> coming, > >>>>>>>>> I > >>>>>>>>> am > >>>>>>>>> doing the same logic, but avcodec_decode_video2 is returning -1. > >>>>>>>>> After > >>>>>>>>> this, > >>>>>>>>> all the received NALs are decoded and the images are displayed. > >>>>>>>>> Every > >>>>>>>>> time > >>>>>>>>> when SPS and PPS are coming, avcodec_decode_video2 returns -1 > >>>>>>>>> (fails). > >>>>>>>>> I > >>>>>>>>> searched all over the net , and I still can't understand how to > >>>>>>>>> fill > >>>>>>>>> up > >>>>>>>>> the > >>>>>>>>> extradata and extradata_size from AVCodecContext used. All I > found > >>>>>>>>> is > >>>>>>>>> this > >>>>>>>>> link: > >>>>>>>>> > http://www.mail-archive.com/[email protected]/msg04939.html. > >>>>>>>>> As I saw from here: > >>>>>>>>> "To decode H.264 stream you need to have SPS and PPS NAL units > also > >>>>>>>>> .". > >>>>>>>>> Ok, > >>>>>>>>> I have them, but frankly I don't know how to use them. At first, > I > >>>>>>>>> thought > >>>>>>>>> that avcodec_decode_video2 will "automatically" use them. But > >>>>>>>>> returning > >>>>>>>>> -1, > >>>>>>>>> I don't see how. Also, If I am discarding SPS and PPS from > sending > >>>>>>>>> them > >>>>>>>>> to > >>>>>>>>> libavcodec, nothing is decoded anymore. > >>>>>>>>> Can you please give me a hint? > >>>>>>>>> Thank you a lot ! > >>>>>>>>> _______________________________________________ > >>>>>>>>> libav-user mailing list > >>>>>>>>> [email protected] > >>>>>>>>> https://lists.mplayerhq.hu/mailman/listinfo/libav-user > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> I think you should read H.264 bit stream structure and use these > >>>>>>>> tools > >>>>>>>> to > >>>>>>>> analyze a NAL unit of H.264 bitstream. > >>>>>>>> http://www.codecian.com/ > >>>>>>>> http://tsviatko.jongov.com/index_prj_h264videoesviewer.htm > >>>>>>>> > >>>>>>>> When FFmpeg decode a NAL unit, if the nal_unit_type equal 7,8 it > >>>>>>>> will > >>>>>>>> continue decode to find the IDR nal_unit. In your case, I think > that > >>>>>>>> FFmpeg > >>>>>>>> found SPS, PPS but it can not find IDR thus it returns -1. > >>>>>>>> > >>>>>>>> Hope it helps. > >>>>>>>> > >>>>>>>> Huy. > >>>>>>>> _______________________________________________ > >>>>>>>> libav-user mailing list > >>>>>>>> [email protected] > >>>>>>>> https://lists.mplayerhq.hu/mailman/listinfo/libav-user > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> libav-user mailing list > >>>>>>> [email protected] > >>>>>>> https://lists.mplayerhq.hu/mailman/listinfo/libav-user > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> If you coupled H.264 slice as [7 8 5] [1] [1] [1] [1] [7 8 5] [1] > [1] > >>>>>> [1] > >>>>>> and so on, it can work well only in single slice/ frame case. > >>>>>> If you use multiple slice , it goes wrong. > >>>>>> > >>>>>> For ex. my sequence has 5 frame I B B B P, and number of slices is 4 > >>>>>> per > >>>>>> frame. The NAL_unit_type of bit stream is: > >>>>>> [7 8 5 5 5 5][1 1 1 1][1 1 1 1][1 1 1 1][1 1 1 1] > >>>>>> I use [ and ] to mark the frame boundary. > >>>>>> > >>>>>> When you feed avcodec_decode_video2, the input buffer must contains > >>>>>> whole > >>>>>> frame data. > >>>>>> Thus, the problem that you have to calculate exactly the input > buffer > >>>>>> length, and feed it enough data plus some padding if necessary > before > >>>>>> pass > >>>>>> it to avcodec_decode_video2. > >>>>>> > >>>>>> Huy. > >>>>>> _______________________________________________ > >>>>>> libav-user mailing list > >>>>>> [email protected] > >>>>>> https://lists.mplayerhq.hu/mailman/listinfo/libav-user > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> libav-user mailing list > >>>>> [email protected] > >>>>> https://lists.mplayerhq.hu/mailman/listinfo/libav-user > >>>>> > >>>>> > >>>>> > >>>> > >>>> Hi, > >>>> > >>>> There is a syntax element named first_mb_in_slice in slice header. It > >>>> describes the position of the first macroblock in that slice. It > equals > >>>> 0 > >>>> to > >>>> indicate a new frame. > >>>> > >>>> I am not sure that this information can help or not. > >>>> > >>>> I just know that when I call av_read_frame, I can receive a buffer > >>>> contained > >>>> whole frame data. But I do not know how can it do this ? > >>>> > >>>> Huy. > >>>> _______________________________________________ > >>>> libav-user mailing list > >>>> [email protected] > >>>> https://lists.mplayerhq.hu/mailman/listinfo/libav-user > >>>> > >>>> > >>>> > >>> > >>> _______________________________________________ > >>> libav-user mailing list > >>> [email protected] > >>> https://lists.mplayerhq.hu/mailman/listinfo/libav-user > >>> > >>> > >> > >> _______________________________________________ > >> libav-user mailing list > >> [email protected] > >> https://lists.mplayerhq.hu/mailman/listinfo/libav-user > >> > > > > _______________________________________________ > > libav-user mailing list > > [email protected] > > https://lists.mplayerhq.hu/mailman/listinfo/libav-user > > > _______________________________________________ > libav-user mailing list > [email protected] > https://lists.mplayerhq.hu/mailman/listinfo/libav-user > _______________________________________________ libav-user mailing list [email protected] https://lists.mplayerhq.hu/mailman/listinfo/libav-user
