Hello Ben, I have a similar problem: I use livemedia/live555 for the RTSP/RTP Part. A working server implementation (also live555) was given to me by my faculty. I have to implement a client for receiving and decoding the h264 stream. it's supposed to decode multi view codec (3d, stereo etc.) later. But first I need it working for plain single-view.
For the client part I use the latest omnimeeting implementation (which in huge parts relies on livemedia). So far, I did many changes for the h264 as it supported only h263/h263+. client infrastructure: - mac osx 10.6 - ffmpeg 0.6 (compiled and linked 32bit) - omnimeeting 0.4 -> http://sourceforge.net/projects/omnimeeting/files/omnimeeting/0.4/omnimeeting-linux-0.4-src.tar.gz/download - live555 v2009.09.28 ( compiled 32 bit ) (32 bit was needed, as live555 in that version does not support 64 bit properly. I will post my compile options later in a separate posting as it seems of some interest to other developers.) Video-info: "foreman_qcif.yuv" -> http://trace.eas.asu.edu/yuv/foreman/foreman_qcif.7z Encoder: "JM 17.2, h264/AVC Reference Software" -> http://iphome.hhi.de/suehring/tml/download - using hierarchical b-frames with full hierarchie - 5 reference pictures - 25 Frames per second - 300 frames Some info, what my client receives: - SDP - SPS/PPS, base64 decode works fine - all following RTP-Frames - My decoded pictures look like h264 difference-pictures for motion-estimation - after a look on my codec_context I see, that most stuff is still initialized with default parameters (except: codec_id, and gop_size, which i did manually) - I did prepend the delimiters (00000001) to each received frame, before sending it to the decoder. - also I tried a dirty solution to collect multiple frames an sending them together to the decoder (nalus 7,8,5) ... did not work. - nalus are not fragmented some test results with other clients: - ffplay rtsp://... does not work properly. pictures are playing at correct speed, but are broken or like differencepictures h264 @ 0x1058e00]error while decoding MB 9 3, bytestream (-18) [h264 @ 0x1058e00]concealing 99 DC, 99 AC, 99 MV errors [rtsp @ 0x1805e00]Unhandled type (29) (See RFC for implementation details Last message repeated 1 times [rtsp @ 0x1805e00]Estimating duration from bitrate, this may be inaccurate Input #0, rtsp, from 'rtsp://localhost:8554/foreman-long-gop8': Metadata: title : Session streamed by "Multi-View Streaming Server" comment : foreman-long-gop8 Duration: N/A, start: 0.083500, bitrate: N/A Stream #0.0: Video: h264, yuv420p, 176x144, 20 tbr, 90k tbn, 180k tbc [h264 @ 0x1058e00]error while decoding MB 9 3, bytestream (-18) [h264 @ 0x1058e00]concealing 99 DC, 99 AC, 99 MV errors [h264 @ 0x1058e00]error while decoding MB 5 8, bytestream (-10) f=0/0 [h264 @ 0x1058e00]concealing 55 DC, 55 AC, 55 MV errors [h264 @ 0x1058e00]error while decoding MB 7 8, bytestream (-26) [h264 @ 0x1058e00]concealing 53 DC, 53 AC, 53 MV errors ... [h264 @ 0x1058e00]non-existing PPS 3 referenced [h264 @ 0x1058e00]decode_slice_header error [h264 @ 0x1058e00]mmco: unref short failure ... [h264 @ 0x1058e00]concealing 51 DC, 51 AC, 51 MV errors [h264 @ 0x1058e00]Missing reference picture Last message repeated 3 times [h264 @ 0x1058e00]mmco: unref short failure [h264 @ 0x1058e00]Missing reference picture Last message repeated 2 times [h264 @ 0x1058e00]mmco: unref short failure [h264 @ 0x1058e00]Missing reference picture Last message repeated 1 times [h264 @ 0x1058e00]mmco: unref short failure [h264 @ 0x1058e00]Missing reference picture - ffplay local file works perfectly (with -autoexit set, as h264 has no "end-frame" and thus will not stop playing after the last frame) - vlc 1.1. does not work at all. pictures are jumping forward and backward when playing the local file. - vlc 1.1. per RTSP shows broken pictures (difference pics) jumoing forward and backward. My Opinion: It seems to me, that the decoder doesn't get initialized correctly. I thought it would be enough to send SPS and PPS to it and then automagically it will be done, just like when you start playing the same file from console in ffplay, which works fine. How can I use my session description protocol (SDP) and/or my sequence parameter set and picture parameter set (SPS, PPS) to completely initiliaze my codec_context, so that it can decode h264 correctly? Any suggetions or comments will help me! kR Sven Wasmer 2010/8/19 bben <[email protected]>: > > Hello, > > I have several problem when I want to decode a H264 video send by > > RTP. > > I'm using libavcodec to decode video but I build my own SDP > > parser. > > If you have time, can you tell me where is the problem in my > > method. > > I create a structure to store SDP data : > > > typedef struct{ > int packetization_mode; > uint8_t profile_idc; > uint8_t profile_iop; > uint8_t level_idc; > int extradata_size; > uint8_t* extradata; > int width; > int height; > } H264CONTEXT; > > H264CONTEXT h264context; > > > I parse the fmtp line of SDP as the function of ffmpeg > > sdp_parse_fmtp_config_h264 > > > Then I found : > > level_idc= 41 (base 10) > profile_idc= 66 (base 10) > extradata_size=30 > I store 00 00 01 67 42 00 1e e2 90 16 02 4d 81 > > 27 05 01 05 e1 e2 44 54 00 00 01 68 ce 3c 80 00 00 in extradata > > > * I initialize CodecContext > > > codecContext->level = h264context.level_idc; > codecContext->profile = h264context.profile_idc; > > > > codecContext->extradata_size= > > h264context.extradata_size; > codecContext->extradata =h264context.extradata; > > codecContext->hurry_up = 1; > codecContext->debug = 1; > codecContext->stream_codec_tag = 462h; > > //0x31637661; > > codecContext->flags |= CODEC_FLAG_4MV; > codecContext->flags |= CODEC_FLAG_PART; > codecContext->flags |=CODEC_FLAG_TRUNCATED; > codecContext->flags2 |= CODEC_FLAG2_CHUNKS; > > > codecContext->error_recognition = 1; > > > codecContext->error_concealment = > > FF_EC_GUESS_MVS; > codecContext->skip_top = 0; > codecContext->skip_bottom = 0; > codecContext->skip_loop_filter = AVDISCARD_NONE; > codecContext->skip_idct = > > AVDISCARD_NONE; > codecContext->skip_frame = AVDISCARD_NONE > codecContext->workaround_bugs = 1; > > codecContext->codec_type = CODEC_TYPE_VIDEO; > codecContext->codec_id = CODEC_ID_H264; > > codecContext->pix_fmt = PIX_FMT_YUV420P; > codecContext->sample_fmt = SAMPLE_FMT_S16; > > > codecContext->codec_tag = > > avcodec_pix_fmt_to_codec_tag (codecContext->pix_fmt ) ; > codecContext->width = 768; > codecContext->height = 576; > > I try decode IDR frame. > > > > My IDR frame (nal_type = 5) are fragmented : fragment_type=28 > > > I find in a forum this method : > > > > If the NAL is 28 (1C) then it means that following payload > > represents one H264 IDR (I-Frame) > fragment and that I need to collect all of them to > > reconstruct H264 IDR (I-Frame). > > Fragment that has START BIT = 1: > > First byte: [ 3 NAL UNIT BITS | 5 FRAGMENT TYPE BITS] > Second byte: [ START BIT | RESERVED BIT | END BIT | 5 NAL UNIT > > BITS] > Other bytes: [... IDR FRAGMENT DATA...] > > Other fragments: > > First byte: [ 3 NAL UNIT BITS | 5 FRAGMENT TYPE BITS] > Other bytes: [... IDR FRAGMENT DATA...] > > To reconstruct IDR I collect this info: > > int fragment_type = Data[0] & 0x1F; > int nal_type = Data[1] & 0x1F; > int start_bit = Data[1] & 0x80; > int end_bit = Data[1] & 0x40; > > If fragment_type == 28 then payload following it is one > > fragment of IDR. Next check is start_bit set, > if it is, then that fragment is the first one in a sequence. I > > use it to reconstruct IDR's NAL byte by taking > the first 3 bits from first payload byte (3 NAL UNIT BITS) and > > combine them with last 5 bits from > > second payload byte (5 NAL UNIT BITS) so I would get a byte > > like this > > [3 NAL UNIT BITS | 5 NAL UNIT BITS]. Then I write that NAL > > byte first > into a clear buffer with all other following bytes from that > > fragment. Remember to skip first byte > > in a sequence since it is not a part of IDR, but only > > identifies the fragment. > > > If start_bit and end_bit are 0 then I just write the payload > > (skipping first > payload byte that identifies the fragment) to the buffer. > > > If start_bit is 0 and end_bit is 1, that means that it is the > > last fragment, > and I just write its payload (skipping the first byte that > > identifies the fragment) > to the buffer, and now I have my IDR reconstructed. > > > > Then I send my IDR to decoder but I some error occurs : > > pps_id (32) out of range > > illegal POC type 5 > > > it think is due SPS/PPS setting but I don't understant why > > > > Thx > > > > > > _______________________________________________ > libav-user mailing list > [email protected] > https://lists.mplayerhq.hu/mailman/listinfo/libav-user > _______________________________________________ libav-user mailing list [email protected] https://lists.mplayerhq.hu/mailman/listinfo/libav-user
