Re: [FFmpeg-user] Stereoscopic hwaccel

[JackDesBwa]

2019-10-03 09:56, JackDesBwa:
> 2019-10-03 01:23, Mark Thompson :
> > On 17/09/2019 17:33, JackDesBwa wrote:
> > > Hi,
> > >
> > > I do stereoscopic (3d) photography for a few years and I just start to
> > > experiment on stereoscopic videos.
> > > The filter_complex argument of ffmpeg allows me to do the editing I have 
> > > in
> > > mind more precisely and less painfully than I was able to do with regular
> > > non-linear editors before.
> > > Now that the basic editing tools seem to be right for my video projects, I
> > > want to accelerate rendering with hardware, and here starts my question.
> > >
> > > *How to have frame-packing information in h264 streams when generated with
> > > hardware acceleration?*
> > >
> > > ...
> > > Accelerating with vaapi:
> > > ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi
> > > -hwaccel_output_format vaapi -i aligned/l_01.mp4 -i aligned/r_01.mp4
> > > -filter_complex '[0][1]framepack=sbs,format=nv12|vaapi,hwupload' -c:v
> > > h264_vaapi -qp 22 -profile:v high out_vaapi.mp4
> >
> > Try this with 
> > .
>
> Wow, thanks to have worked on it.
> Sure I can try the patch to confirm it works, but you might have to help me.
>
> 1. This patch seems to be in a series. Should I apply the whole series
> or only this patch would suffice?
> 2. On which base should I apply it/them? Only master seems to be
> recent, but it does not have your patches yet.
> 3. In this patch, you do not use av_frame_get_side_data(frame,
> AV_FRAME_DATA_STEREO3D) as in libx264.c codec, but I see you added a
> frame_packing. Does it mean I have to add an option in the command
> line?

Mark Thompson, you did not answered my questions.
As your code is still not on master, I cannot test to confirm if your
patch is working without your instructions.

JackDesBwa
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[JackDesBwa]

2019-10-03 01:23, Mark Thompson :
> On 17/09/2019 17:33, JackDesBwa wrote:
> > Hi,
> >
> > I do stereoscopic (3d) photography for a few years and I just start to
> > experiment on stereoscopic videos.
> > The filter_complex argument of ffmpeg allows me to do the editing I have in
> > mind more precisely and less painfully than I was able to do with regular
> > non-linear editors before.
> > Now that the basic editing tools seem to be right for my video projects, I
> > want to accelerate rendering with hardware, and here starts my question.
> >
> > *How to have frame-packing information in h264 streams when generated with
> > hardware acceleration?*
> >
> > ...
> > Accelerating with vaapi:
> > ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi
> > -hwaccel_output_format vaapi -i aligned/l_01.mp4 -i aligned/r_01.mp4
> > -filter_complex '[0][1]framepack=sbs,format=nv12|vaapi,hwupload' -c:v
> > h264_vaapi -qp 22 -profile:v high out_vaapi.mp4
>
> Try this with 
> .

Wow, thanks to have worked on it.
Sure I can try the patch to confirm it works, but you might have to help me.

1. This patch seems to be in a series. Should I apply the whole series
or only this patch would suffice?
2. On which base should I apply it/them? Only master seems to be
recent, but it does not have your patches yet.
3. In this patch, you do not use av_frame_get_side_data(frame,
AV_FRAME_DATA_STEREO3D) as in libx264.c codec, but I see you added a
frame_packing. Does it mean I have to add an option in the command
line?

JackDesBwa
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[Mark Thompson]

On 17/09/2019 17:33, JackDesBwa wrote:
> Hi,
> 
> I do stereoscopic (3d) photography for a few years and I just start to
> experiment on stereoscopic videos.
> The filter_complex argument of ffmpeg allows me to do the editing I have in
> mind more precisely and less painfully than I was able to do with regular
> non-linear editors before.
> Now that the basic editing tools seem to be right for my video projects, I
> want to accelerate rendering with hardware, and here starts my question.
> 
> *How to have frame-packing information in h264 streams when generated with
> hardware acceleration?*
> 
> ...
> Accelerating with vaapi:
> ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi
> -hwaccel_output_format vaapi -i aligned/l_01.mp4 -i aligned/r_01.mp4
> -filter_complex '[0][1]framepack=sbs,format=nv12|vaapi,hwupload' -c:v
> h264_vaapi -qp 22 -profile:v high out_vaapi.mp4

Try this with 
.

- Mark
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[JackDesBwa]

> How/where are you checking for the frame packing information?

On https://git.ffmpeg.org/gitweb/ffmpeg.git I found the code of the
filter in libavfilter/ and the code of the different encoders in
libavcodec/
For example the AV_STEREO3D_SIDEBYSIDE symbol is used in the filter
and in the libx264.c codec, but not in the other codecs I wanted to
use. I also checked other symbols but I do not remember which ones.

JackDesBwa
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[Carl Eugen Hoyos]

Am Fr., 27. Sept. 2019 um 19:23 Uhr schrieb JackDesBwa :

> > You would have to ask in #ffmpeg as such a filter would not
> > be part of libx264 (which by definition always re-encodes).
>
> Ok, I will go there when I have some time.

Sorry for my misleading comment:
I meant that people could have told you in #ffmpeg that such
a bitstream filter does not exist (yet).

Carl Eugen
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[Ted Park]

How/where are you checking for the frame packing information?
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[JackDesBwa]

> I wonder if VAProfileH264StereoHigh is really what you are looking
> for...

It is hard to find information about it, as most answers come from
a pasting of the results of vainfo command.
However, I could have determined (through driver code for example)
that it is related to Multiview Video Coding which sounds good for
stereoscopic videos I would like to create.

> You would have to ask in #ffmpeg as such a filter would not
> be part of libx264 (which by definition always re-encodes).

Ok, I will go there when I have some time.

> Note that I was not aware vaapi supports 3d information, did
> you find any additional information?

Nope. It is mainly based on this observation on the
VAProfileH264StereoHigh profile.
Thus I wondered if I could use it to accelerate encoding of my
stereoscopic videos (or with nvenc, whatever).


JackDesBwa
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[Carl Eugen Hoyos]

Am Fr., 27. Sept. 2019 um 18:06 Uhr schrieb JackDesBwa :
>
> I answer to myself.
>
> > (1) Which way could I be certain that it is not currently supported or
> > if there is some option to pass?
>
> By reading the source code, I found how it is set in libx264 codec,
> but there is nothing similar at all in nvenc nor in vaapi. In nvenc, I
> saw stereo fields in the configuration of the encoder but no glue to
> use them.

I wonder if VAProfileH264StereoHigh is really what you are looking
for...

> Although it is probably only a configuration bit or two to add in
> nvenc case, it seems a bit too complex for me to do it myself.
>
> > (2) Is there a way to add the frame-packing information without a full 
> > encoding?
>
> On the #x264 IRC channel

You would have to ask in #ffmpeg as such a filter would not
be part of libx264 (which by definition always re-encodes).

Note that I was not aware vaapi supports 3d information, did
you find any additional information?

Carl Eugen
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[JackDesBwa]

I answer to myself.

> (1) Which way could I be certain that it is not currently supported or
> if there is some option to pass?

By reading the source code, I found how it is set in libx264 codec,
but there is nothing similar at all in nvenc nor in vaapi. In nvenc, I
saw stereo fields in the configuration of the encoder but no glue to
use them.
Although it is probably only a configuration bit or two to add in
nvenc case, it seems a bit too complex for me to do it myself.

> (2) Is there a way to add the frame-packing information without a full 
> encoding?

On the #x264 IRC channel (freenode) someone answered that there is no
such feature in the x264 command line tool (the one created by the
same people as the libx264 library if I understand well)

Thus, at least for the moment, I will encode with nvenc to check my
filters and encode with libx264 for the final render.

JackDesBwa
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[JackDesBwa]

> So I feel like the encoding part is the biggest bottleneck and it wouldn’t 
> make much of a difference if the decoder and filter part are software. I 
> don’t think the filter supports hw formats anyway, and for the second test 
> only one of the decoders are vaapi. Setting the profile on the encoder might 
> be what gets rid of the stereo frame packing settings, though I don’t know if 
> the hw accelerated encoders configure those settings in the first place.

Thanks for your reply.
I also think that the encoding part is the main point to focus on, and
I am not familiar enough with this part of the cli to have caught that
I enabled hardware decoding as well (was based on an example as I was
not able to make it work by myself).
I tried to remove all custom options I could.

I tested with:
ffmpeg -i aligned/l_01.mp4 -i aligned/r_01.mp4 -filter_complex
[0][1]framepack=sbs,format=nv12|vaapi,hwupload -vaapi_device
/dev/dri/renderD128 -c:v h264_vaapi out_vaapi.mp4
It takes 6.9s wall time and produces a 40Mio output. (slower, bigger)
And there is no frame-packing information neither.

Similarly with nvenc:
ffmpeg -i aligned/l_01.mp4 -i aligned/r_01.mp4 -filter_complex
[0][1]framepack=sbs -c:v h264_nvenc out_nvenc.mp4
It takes 2.4s wall time and produces a 2.6Mio output. (faster, bigger,
still horrible compression)
And there is no frame-packing information neither.

It seems like they do not support the frame-packing although both
hardware is supposed to do (VAProfileH264StereoHigh listed by vainfo
for example)
(1) Which way could I be certain that it is not currently supported or
if there is some option to pass?
By the way, I use ffmpeg version 4.2.1 which seems to be the latest
release today.

Alternatively, I could create an accelerated regular video and then
add the frame-packing data in the video.
However, when I try this with -x264-params frame-packing=3, the video
is encoded again and I loose all the time I gained with the
acceleration.
(2) Is there a way to add the frame-packing information without a full encoding?

If my questions are too specific for this list, do not hesitate to
orient me toward better suited place.


JackDesBwa
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Stereoscopic hwaccel

[Ted Park]

> On Sep 17, 2019, at 12:33 PM, JackDesBwa  wrote:
> 
> Removing the full chain of filter_complex to keep the simplest example,
> computing using CPU would give:
> ffmpeg -i aligned/l_01.mp4 -i aligned/r_01.mp4 -filter_complex
> [0][1]framepack=sbs -c:v libx264 out_libx264.mp4
> It takes 22s wall time and produces a 18Mio output.
> 
> Accelerating with vaapi:
> ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi
> -hwaccel_output_format vaapi -i aligned/l_01.mp4 -i aligned/r_01.mp4
> -filter_complex '[0][1]framepack=sbs,format=nv12|vaapi,hwupload' -c:v
> h264_vaapi -qp 22 -profile:v high out_vaapi.mp4
> It takes 5.6s wall time and produces a 32Mio output.
> 
> Accelerating with nvenc:
> ffmpeg -i aligned/l_01.mp4 -i aligned/r_01.mp4 -filter_complex
> [0][1]framepack=sbs -c:v h264_nvenc -rc:v vbr_hq -qp:v 22 -profile:v high
> out_nvenc.mp4
> It takes 2.7s wall time and produces a 2.3Mio output.
> 
> The numbers are only for one short 240-frames
> 3604x1064-unfriendly-dimentions clip here.
> The nvenc version is fast and create small file, but has very bad quality.
> Perhaps I do not use the right parameters.
> The vaapi version is quite fast too, but create a bigger file. Perhaps I
> should tweak parameters here too.
> Advices on parameters are welcomed, but the more import thing is the
> following.
> 
> *The two accelerated versions do not add the SBS frame-packing information
> in the stream*. The documentation of framepack filter says that it would
> set proper metadata on supported codecs. Does it mean that I am stuck with
> CPU codecs to get the proper output stream with stereo format information
> in it?
> 
> JackDesBwa

So I feel like the encoding part is the biggest bottleneck and it wouldn’t make 
much of a difference if the decoder and filter part are software. I don’t think 
the filter supports hw formats anyway, and for the second test only one of the 
decoders are vaapi. Setting the profile on the encoder might be what gets rid 
of the stereo frame packing settings, though I don’t know if the hw accelerated 
encoders configure those settings in the first place.
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-user] Stereoscopic hwaccel

[JackDesBwa]

Hi,

I do stereoscopic (3d) photography for a few years and I just start to
experiment on stereoscopic videos.
The filter_complex argument of ffmpeg allows me to do the editing I have in
mind more precisely and less painfully than I was able to do with regular
non-linear editors before.
Now that the basic editing tools seem to be right for my video projects, I
want to accelerate rendering with hardware, and here starts my question.

*How to have frame-packing information in h264 streams when generated with
hardware acceleration?*

I have both a UHD Graphics 620 and GeForce GTX 1050 GPUs on my machine.
The first (or at least a graphical subpart integrated to my processor) is
accessible with VAAPI while the second can use NVENC. Perhaps there are
others ways I am not aware of.

Removing the full chain of filter_complex to keep the simplest example,
computing using CPU would give:
ffmpeg -i aligned/l_01.mp4 -i aligned/r_01.mp4 -filter_complex
[0][1]framepack=sbs -c:v libx264 out_libx264.mp4
It takes 22s wall time and produces a 18Mio output.

Accelerating with vaapi:
ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi
-hwaccel_output_format vaapi -i aligned/l_01.mp4 -i aligned/r_01.mp4
-filter_complex '[0][1]framepack=sbs,format=nv12|vaapi,hwupload' -c:v
h264_vaapi -qp 22 -profile:v high out_vaapi.mp4
It takes 5.6s wall time and produces a 32Mio output.

Accelerating with nvenc:
ffmpeg -i aligned/l_01.mp4 -i aligned/r_01.mp4 -filter_complex
[0][1]framepack=sbs -c:v h264_nvenc -rc:v vbr_hq -qp:v 22 -profile:v high
out_nvenc.mp4
It takes 2.7s wall time and produces a 2.3Mio output.

The numbers are only for one short 240-frames
3604x1064-unfriendly-dimentions clip here.
The nvenc version is fast and create small file, but has very bad quality.
Perhaps I do not use the right parameters.
The vaapi version is quite fast too, but create a bigger file. Perhaps I
should tweak parameters here too.
Advices on parameters are welcomed, but the more import thing is the
following.

*The two accelerated versions do not add the SBS frame-packing information
in the stream*. The documentation of framepack filter says that it would
set proper metadata on supported codecs. Does it mean that I am stuck with
CPU codecs to get the proper output stream with stereo format information
in it?

JackDesBwa
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".