I would like to hear from anyone who has experience with low latency video streaming and ffmpeg.
In my application I take video from a PCIe capture device and stream live over network. The end to end latency must be below 200ms otherwise it is not usable. Let us assume that the network latency is near zero. This question is about encoder+decoder+buffering latency. Very low latency can be achieved when using the x264 encoder by using settings like tune zerolatency. From my own testing with x264 (without ffmpeg) I can confirm that this works. I have measured approx 90 ms (3 frames) of latency added by encoder+decoder combined. Awesome. Sadly I have been unable to get anywhere near those numbers when using ffmpeg and libx264. As player I use either ffplay or a dot-net player built on the ffmpeg libraries, with about equal results. The lowest end to end latency I have seen is 400-500ms. That includes capture latency also. Yes, there are many unknowns here. To help pin point the problem I used mjpeg streaming for comparison. Same everything except I change ffmpeg vcodec to mjpeg and format to mjpeg. I change nothing with ffplay. With mjpeg I now get approx 150ms end to end latency. Observation: x264 codec + mpegts format compared to mjpeg codec + mjpeg format adds about 300 ms (10 frames) latency. I have tried every ffplay-, ffmpeg- and libx264 setting that I believe relevant. Most have not made any noticable difference in latency. I will not list a lot of settings here. Here are the basic settings and commands that I have used. I can post more details on request. x264-preset: vcodec=libx264 thread_type=slice slices=1 # x264 profile=baseline level=32 preset=superfast tune=zerolatency intra-refresh=1 crf=15 x264-params=vbv-maxrate=5000:vbv-bufsize=1:slice-max-size=1500:keyint=60 $ ffmpeg -r 30 -f dshow -i video="devicename" -pix_fmt yuv420p -an -vpre x264-preset -f mpegts udp://127.0.0.1:8888 $ ffplay -fflags nobuffer -flags low_delay -framedrop -strict experimental udp://127.0.0.1:8888 Everything is run on Windows 7 64-bit, with 32-bit executables. ffmpeg version N-76539-g480bad7 Copyright (c) 2000-2015 the FFmpeg developers built with gcc 5.2.0 (GCC) configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib libavutil 55. 5.100 / 55. 5.100 libavcodec 57. 15.100 / 57. 15.100 libavformat 57. 14.100 / 57. 14.100 libavdevice 57. 0.100 / 57. 0.100 libavfilter 6. 15.100 / 6. 15.100 libswscale 4. 0.100 / 4. 0.100 libswresample 2. 0.101 / 2. 0.101 libpostproc 54. 0.100 / 54. 0.100 cheers -- Greger Burman _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-user