Re: [FFmpeg-user] Low latency H.264 streaming with ffmpeg

[rameshperumal]

Greger Burman wrote
> Observation:
> x264 codec + mpegts format compared to mjpeg codec + mjpeg format adds
> about 300 ms (10 frames) latency.

Hi,

I just started to transcode the 1-minute input video(mjpeg) from the USB
camera with the below command.

time ~/bin/ffmpeg -benchmark -threads 4 -t 00:01:00.00 -f video4linux2
-input_format mjpeg -framerate 30 -i /dev/video0 -an -c:v libx264 -preset
ultrafast -tune zerolatency -thread_type 1 -r:v 30 -f null - 

Output:
ffmpeg version 3.3 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 6.2.0 (Ubuntu 6.2.0-5ubuntu12) 20161005
  configuration: --prefix=/usr --pkg-config-flags=--static
--extra-cflags=-I/usr/include --extra-ldflags=-L/usr/lib --bindir=/root/bin
--enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype
--enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264
--enable-libx265 --enable-nonfree --enable-vaapi --enable-libopus
--enable-libvpx --enable-hwaccel=h263_vaapi --enable-hwaccel=h264_vaapi
--enable-hwaccel=mpeg2_vaapi --enable-hwaccel=mpeg4_vaapi
--enable-hwaccel=vc1_vaapi --enable-hwaccel=vp8_vaapi
--enable-hwaccel=vp9_vaapi --enable-hwaccel=wmv3_vaapi
--enable-decoder='vp8,vp9'
  libavutil  55. 58.100 / 55. 58.100
  libavcodec 57. 89.100 / 57. 89.100
  libavformat57. 71.100 / 57. 71.100
  libavdevice57.  6.100 / 57.  6.100
  libavfilter 6. 82.100 /  6. 82.100
  libswscale  4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc54.  5.100 / 54.  5.100
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 5864.558330, bitrate: N/A
Stream #0:0: Video: mjpeg, yuvj422p(pc, bt470bg/unknown/unknown),
640x480, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
No pixel format specified, yuvj422p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 0x555b636ea220] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x555b636ea220] profile High 4:2:2, level 3.0, 4:2:2 8-bit
Output #0, null, to 'pipe:':
  Metadata:
encoder : Lavf57.71.100
Stream #0:0: Video: h264 (libx264), yuvj422p(pc), 640x480, q=-1--1, 30
fps, 30 tbn, 30 tbc
Metadata:
  encoder : Lavc57.89.100 libx264
Side data:
  cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame= 1800 fps= 30 q=-1.0 Lsize=N/A time=00:01:00.00 bitrate=N/A speed=  
1x
video:8981kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: unknown
bench: utime=5.746s
bench: maxrss=57952kB
[libx264 @ 0x555b636ea220] frame I:8 Avg QP:19.12  size: 62426
[libx264 @ 0x555b636ea220] frame P:1792  Avg QP:22.09  size:  4853
[libx264 @ 0x555b636ea220] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x555b636ea220] mb P  I16..4:  1.5%  0.0%  0.0%  P16..4: 58.3% 
0.0%  0.0%  0.0%  0.0%skip:40.2%
[libx264 @ 0x555b636ea220] coded y,uvDC,uvAC intra: 29.4% 66.3% 13.1% inter:
14.5% 42.2% 8.9%
[libx264 @ 0x555b636ea220] i16 v,h,dc,p: 32% 38% 18% 12%
[libx264 @ 0x555b636ea220] i8c dc,h,v,p: 36% 23% 31% 10%
[libx264 @ 0x555b636ea220] kb/s:1226.19

real1m1.359s
user0m5.772s
sys 0m0.115s


May I know the values (bench: utime=5.746s or user0m5.772s or  sys
0m0.115s) I need to consider to measure the end-to-end transcoding latency?
I am interested in measuring the combined encode and decode latency per
video frame. Please suggest me.



--
Sent from: http://www.ffmpeg-archive.org/
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Low latency H.264 streaming with ffmpeg

[Greger Burman]

Ok, so the ffmpeg based player is buffering frames somewhere even though
I'm telling it not to. This happens with h264 video but not with mjpeg.
That suggests to me that the latency is caused by either the h264 decoder
or the mpegts demuxer. I have also tried m4v instead of mpegts, with about
the same result.

Anyone who knows enough about the h264 decoding process to confirm/deny?
Is there a way to affect this behaviour without diving deep into the code?

cheers
//Greger

>
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

Re: [FFmpeg-user] Low latency H.264 streaming with ffmpeg

[Roger Pack]

On 1/13/16, Greger Burman  wrote:
> Ok, so the ffmpeg based player is buffering frames somewhere even though
> I'm telling it not to. This happens with h264 video but not with mjpeg.
> That suggests to me that the latency is caused by either the h264 decoder
> or the mpegts demuxer. I have also tried m4v instead of mpegts, with about
> the same result.
>
> Anyone who knows enough about the h264 decoding process to confirm/deny?
> Is there a way to affect this behaviour without diving deep into the code?

I don't know much about the h264 decoder.  If you put I-frames in more
often it might help.  I've heard rumors of this flag
 -fflags nobuffer
but really no little of any internal workings.
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

Re: [FFmpeg-user] Low latency H.264 streaming with ffmpeg

[Roger Pack]

On 1/12/16, Kenneth Fields  wrote:
> yes, we’re having the same issue now.
> but -benchmark also really degrades the stream.
> Is there any options possible for -benchmark (moderate, strict)?

what type of degredation? are you using udp? are you including audio?
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

Re: [FFmpeg-user] Low latency H.264 streaming with ffmpeg

[Roger Pack]

On 1/12/16, Greger Burman  wrote:
> I would like to hear from anyone who has experience with low latency video
> streaming and ffmpeg.
>
> In my application I take video from a PCIe capture device and stream live
> over network. The end to end latency must be below 200ms otherwise it is
> not usable. Let us assume that the network latency is near zero. This
> question is about encoder+decoder+buffering latency.
>
> Very low latency can be achieved when using the x264 encoder by using
> settings like tune zerolatency. From my own testing with x264 (without
> ffmpeg) I can confirm that this works. I have measured approx 90 ms (3
> frames) of latency added by encoder+decoder combined. Awesome.
>
> Sadly I have been unable to get anywhere near those numbers when using
> ffmpeg and libx264. As player I use either ffplay or a dot-net player built
> on the ffmpeg libraries, with about equal results. The lowest end to end
> latency I have seen is 400-500ms. That includes capture latency also.

mplayer -benchmark has been the only client I've found with reasonable
latency, though I'm sure there are others/more.
I've gotten pretty low latency before with ultrafast+zerolatency

Basically my advice is "be wary of ffplay (and vlc)" I'm not sure if
its latency is low enough...

https://trac.ffmpeg.org/wiki/StreamingGuide
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

Re: [FFmpeg-user] Low latency H.264 streaming with ffmpeg

[Kenneth Fields]

yes, we’re having the same issue now.
but -benchmark also really degrades the stream.
Is there any options possible for -benchmark (moderate, strict)?

Ken



> mplayer -benchmark has been the only client I've found with reasonable
> latency, though I'm sure there are others/more.
> I've gotten pretty low latency before with ultrafast+zerolatency
> 
> Basically my advice is "be wary of ffplay (and vlc)" I'm not sure if
> its latency is low enough...


> On Jan 13, 2016, at 2:30 AM, Roger Pack  wrote:
> 
> On 1/12/16, Greger Burman > wrote:
>> I would like to hear from anyone who has experience with low latency video
>> streaming and ffmpeg.
>> 
>> In my application I take video from a PCIe capture device and stream live
>> over network. The end to end latency must be below 200ms otherwise it is
>> not usable. Let us assume that the network latency is near zero. This
>> question is about encoder+decoder+buffering latency.
>> 
>> Very low latency can be achieved when using the x264 encoder by using
>> settings like tune zerolatency. From my own testing with x264 (without
>> ffmpeg) I can confirm that this works. I have measured approx 90 ms (3
>> frames) of latency added by encoder+decoder combined. Awesome.
>> 
>> Sadly I have been unable to get anywhere near those numbers when using
>> ffmpeg and libx264. As player I use either ffplay or a dot-net player built
>> on the ffmpeg libraries, with about equal results. The lowest end to end
>> latency I have seen is 400-500ms. That includes capture latency also.
> 
> 
> https://trac.ffmpeg.org/wiki/StreamingGuide 
> 
> ___
> ffmpeg-user mailing list
> ffmpeg-user@ffmpeg.org 
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user 
> 
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user