[FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
This is the only format supported by MFC5 HW decoders (e.g. Samsung exynos
4412).
---
libavutil/pixdesc.c | 12
libavutil/pixfmt.h | 1 +
2 files changed, 13 insertions(+)
diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c
index 648d014..426dfd4 100644
--- a/libavutil/pixdesc.c
+++ b/libavutil/pixdesc.c
@@ -1900,6 +1900,18 @@ const AVPixFmtDescriptor
av_pix_fmt_descriptors[AV_PIX_FMT_NB] = {
.name = "vda",
.flags = AV_PIX_FMT_FLAG_HWACCEL,
},
+[AV_PIX_FMT_NV12T] = {
+.name = "nv12t",
+.nb_components = 3,
+.log2_chroma_w = 1,
+.log2_chroma_h = 1,
+.comp = {
+{ 0, 0, 1, 0, 7 },/* Y */
+{ 1, 1, 1, 0, 7 },/* U */
+{ 1, 1, 2, 0, 7 },/* V */
+},
+.flags = AV_PIX_FMT_FLAG_PLANAR,
+},
};
static const char *color_range_names[AVCOL_RANGE_NB] = {
diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h
index 756a1a7..c6709a1 100644
--- a/libavutil/pixfmt.h
+++ b/libavutil/pixfmt.h
@@ -289,6 +289,7 @@ enum AVPixelFormat {
#if !FF_API_XVMC
AV_PIX_FMT_XVMC,///< XVideo Motion Acceleration via common packet passing
#endif /* !FF_API_XVMC */
+AV_PIX_FMT_NV12T, ///< Same as NV12 except the coordinates differ:
Z-shape tiled 64x32 macroblocks. V4L2 specific format corresponding to
V4L2_PIX_FMT_NV12MT
AV_PIX_FMT_NB,///< number of pixel formats, DO NOT USE THIS if you
want to link with shared libav* because the number of formats might differ
between versions
--
2.1.3
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On 20 Nov, Alexis Ballier wrote : > This is the only format supported by MFC5 HW decoders (e.g. Samsung exynos > 4412). Why not convert it to a normal format? With my kindest regards, -- Jean-Baptiste Kempf http://www.jbkempf.com/ - +33 672 704 734 Sent from my Electronic Device ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Thu, 20 Nov 2014 17:51:54 +0100
Alexis Ballier wrote:
> This is the only format supported by MFC5 HW decoders (e.g. Samsung exynos
> 4412).
> ---
> libavutil/pixdesc.c | 12
> libavutil/pixfmt.h | 1 +
> 2 files changed, 13 insertions(+)
>
> diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c
> index 648d014..426dfd4 100644
> --- a/libavutil/pixdesc.c
> +++ b/libavutil/pixdesc.c
> @@ -1900,6 +1900,18 @@ const AVPixFmtDescriptor
> av_pix_fmt_descriptors[AV_PIX_FMT_NB] = {
> .name = "vda",
> .flags = AV_PIX_FMT_FLAG_HWACCEL,
> },
> +[AV_PIX_FMT_NV12T] = {
> +.name = "nv12t",
> +.nb_components = 3,
> +.log2_chroma_w = 1,
> +.log2_chroma_h = 1,
> +.comp = {
> +{ 0, 0, 1, 0, 7 },/* Y */
> +{ 1, 1, 1, 0, 7 },/* U */
> +{ 1, 1, 2, 0, 7 },/* V */
> +},
> +.flags = AV_PIX_FMT_FLAG_PLANAR,
> +},
> };
>
> static const char *color_range_names[AVCOL_RANGE_NB] = {
> diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h
> index 756a1a7..c6709a1 100644
> --- a/libavutil/pixfmt.h
> +++ b/libavutil/pixfmt.h
> @@ -289,6 +289,7 @@ enum AVPixelFormat {
> #if !FF_API_XVMC
> AV_PIX_FMT_XVMC,///< XVideo Motion Acceleration via common packet passing
> #endif /* !FF_API_XVMC */
> +AV_PIX_FMT_NV12T, ///< Same as NV12 except the coordinates differ:
> Z-shape tiled 64x32 macroblocks. V4L2 specific format corresponding to
> V4L2_PIX_FMT_NV12MT
>
> AV_PIX_FMT_NB,///< number of pixel formats, DO NOT USE THIS if
> you want to link with shared libav* because the number of formats might
> differ between versions
>
I think this should be rejected. It's far too special to be useful in
general, and it's not even pixel- or line-addressable (Z-shaped tiles,
seriously?). It's pretty much a raw codec, not a pixel format.
Also, doesn't libv4l2 handle converting this to something sane
transparently?
If this is needed for the m2m filter, then maybe it should be part of
the v4l2 libavdevice.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Thu, 20 Nov 2014 18:42:19 +0100 wm4 wrote: > I think this should be rejected. It's far too special to be useful in > general, and it's not even pixel- or line-addressable (Z-shaped tiles, > seriously?). It's pretty much a raw codec, not a pixel format. How do you put this in an AVFrame then ? > Also, doesn't libv4l2 handle converting this to something sane > transparently? "transparently" yes, but in sw. A <10W arm soc wouldn't like to "transparently" convert 1080p@25fps like that also, last time I checked, libv4l2 didnt support NV12MT > If this is needed for the m2m filter, then maybe it should be part of > the v4l2 libavdevice. I don't understand this. This is needed for HW decoding on MFCv5: it is the only format decoders can produce. To use it in your application, you send it to the m2m filter to get something sane. Alexis. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Thu, 20 Nov 2014 18:40:18 +0100 Jean-Baptiste Kempf wrote: > On 20 Nov, Alexis Ballier wrote : > > This is the only format supported by MFC5 HW decoders (e.g. Samsung > > exynos 4412). > > Why not convert it to a normal format? > That is exactly what the m2m filter is for: on 4412 you have MFC HW codecs and fimc (camera & m2m module); the fimc m2m module acts like a filter and accepts this format and outputs much saner ones. Alexis. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 08:45:47 +0100 Alexis Ballier wrote: > On Thu, 20 Nov 2014 18:42:19 +0100 > wm4 wrote: > > I think this should be rejected. It's far too special to be useful in > > general, and it's not even pixel- or line-addressable (Z-shaped tiles, > > seriously?). It's pretty much a raw codec, not a pixel format. > > How do you put this in an AVFrame then ? Preferably not at all. Is there a need to? Only the end result (nv12 I suppose) needs to be in an AVFrame. > > Also, doesn't libv4l2 handle converting this to something sane > > transparently? > > "transparently" yes, but in sw. A <10W arm soc wouldn't like to > "transparently" convert 1080p@25fps like that > > also, last time I checked, libv4l2 didnt support NV12MT > > > If this is needed for the m2m filter, then maybe it should be part of > > the v4l2 libavdevice. > > I don't understand this. > > This is needed for HW decoding on MFCv5: it is the only format decoders > can produce. To use it in your application, you send it to the m2m > filter to get something sane. Sorry, I didn't look too close at the other patches. So this is a decoder. Why can't the decoder take care of this conversion too? This macroblock format isn't really useful for anything other than the m2m filter, but breaks all AVFrame/pixfmt conventions. Additionally, it breaks libavfilter conventions: conversions between pixfmts are supposed to be handled by libswscale, not arbitrary filters. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 09:01:43 +0100 wm4 wrote: > On Fri, 21 Nov 2014 08:45:47 +0100 > Alexis Ballier wrote: > > > On Thu, 20 Nov 2014 18:42:19 +0100 > > wm4 wrote: > > > I think this should be rejected. It's far too special to be > > > useful in general, and it's not even pixel- or line-addressable > > > (Z-shaped tiles, seriously?). It's pretty much a raw codec, not a > > > pixel format. > > > > How do you put this in an AVFrame then ? > > Preferably not at all. Is there a need to? Only the end result (nv12 I > suppose) needs to be in an AVFrame. It can be done but is ugly IMHO: - I was under the impression that lavc decoder convention was to output non processed formats and let the application handle (or not) the conversions - The decoder code would look like: * probe and open decoder * feed it with some frames / extradata to get the format from the decoder * if i dont like the format then: * probe and open another random device that may or may not exist for format conversion * copy/paste the m2m filter code in the decoder to postprocess the frames - If you're thinking about manual conversion then it's a no go: this would break the zero copy chain I'm trying to obtain. > > > Also, doesn't libv4l2 handle converting this to something sane > > > transparently? > > > > "transparently" yes, but in sw. A <10W arm soc wouldn't like to > > "transparently" convert 1080p@25fps like that > > > > also, last time I checked, libv4l2 didnt support NV12MT > > > > > If this is needed for the m2m filter, then maybe it should be > > > part of the v4l2 libavdevice. > > > > I don't understand this. > > > > This is needed for HW decoding on MFCv5: it is the only format > > decoders can produce. To use it in your application, you send it to > > the m2m filter to get something sane. > > Sorry, I didn't look too close at the other patches. It'd be hard to guess this from the other patches anyway, v4l2 driver is the one that will tell the format, so you won't get much information without looking at kernel driver code :) > So this is a > decoder. Why can't the decoder take care of this conversion too? See above. > This > macroblock format isn't really useful for anything other than the m2m > filter, but breaks all AVFrame/pixfmt conventions. Additionally, it > breaks libavfilter conventions: conversions between pixfmts are > supposed to be handled by libswscale, not arbitrary filters. I didn't take the time to write swscale support for this because the format is a bit insane, the code would be ugly and it would be useless in practice. I probably missed something but I was under the impression that av_image_copy & friends should work if frames have the correct width & height alignment. The command: ./ffmpeg_g -y -c:v mpeg4_v4l2m2m -i bli.mkv -vf v4l2_m2m -c:v h264_v4l2m2m -pix_fmt nv12 -c:a ac3 bli.mp4 works like a charm here, no auto-inserted scaler, mpeg4_v4l2m2m produces nv12mt that is fed to the m2m filter that produces standard nv12 which is fed to the h264_v4l2m2m encoder Alexis. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On 21 Nov, Alexis Ballier wrote : > On Thu, 20 Nov 2014 18:40:18 +0100 > Jean-Baptiste Kempf wrote: > > > On 20 Nov, Alexis Ballier wrote : > > > This is the only format supported by MFC5 HW decoders (e.g. Samsung > > > exynos 4412). > > > > Why not convert it to a normal format? > > > > That is exactly what the m2m filter is for: on 4412 you have MFC HW > codecs and fimc (camera & m2m module); the fimc m2m module acts > like a filter and accepts this format and outputs much saner ones. So you have a format outputted that is of no use, except if you use the filter. And you have a filter that is of no use, except if you use this v4l2 output. In my opinion, you should just merge them, so you output something sane. With my kindest regards, -- Jean-Baptiste Kempf http://www.jbkempf.com/ - +33 672 704 734 Sent from my Electronic Device ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On 21 Nov, Alexis Ballier wrote : > > > How do you put this in an AVFrame then ? > > > > Preferably not at all. Is there a need to? Only the end result (nv12 I > > suppose) needs to be in an AVFrame. > > It can be done but is ugly IMHO: > > - I was under the impression that lavc decoder convention was to > output non processed formats and let the application handle (or not) > the conversions Sure, but this is when you can actually process them in applications, like planar audio. No a hardware-specific format that noone can play except with a special library. > > So this is a > > decoder. Why can't the decoder take care of this conversion too? > > See above. Sorry, but that's far from clear. > > This > > macroblock format isn't really useful for anything other than the m2m > > filter, but breaks all AVFrame/pixfmt conventions. Additionally, it > > breaks libavfilter conventions: conversions between pixfmts are > > supposed to be handled by libswscale, not arbitrary filters. > > I didn't take the time to write swscale support for this because the > format is a bit insane, the code would be ugly and it would be > useless in practice. If it is insane, keep it internal. > works like a charm here, no auto-inserted scaler, mpeg4_v4l2m2m > produces nv12mt that is fed to the m2m filter that produces standard > nv12 which is fed to the h264_v4l2m2m encoder Then, merge them together, so it only output standard nv12. With my kindest regards, -- Jean-Baptiste Kempf http://www.jbkempf.com/ - +33 672 704 734 Sent from my Electronic Device ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 09:58:40 +0100 Jean-Baptiste Kempf wrote: > On 21 Nov, Alexis Ballier wrote : > > On Thu, 20 Nov 2014 18:40:18 +0100 > > Jean-Baptiste Kempf wrote: > > > > > On 20 Nov, Alexis Ballier wrote : > > > > This is the only format supported by MFC5 HW decoders (e.g. > > > > Samsung exynos 4412). > > > > > > Why not convert it to a normal format? > > > > > > > That is exactly what the m2m filter is for: on 4412 you have MFC HW > > codecs and fimc (camera & m2m module); the fimc m2m module acts > > like a filter and accepts this format and outputs much saner ones. > > So you have a format outputted that is of no use, except if you use > the filter. On MFCv5 yes; I don't assume because I'm working on this that it is the only one :) MFCv6 and newer don't even understand nv12mt and can output standard nv12: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c#n35 there are other v4l2 mem2mem drivers than mfc ones, and I expect more to come in the future > And you have a filter that is of no use, except if you > use this v4l2 output. again, filter is not bound to samsung socs; even fimc is useful for other format conversions and even scaling: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/exynos4-is/fimc-core.c (the filter supports fmt conversion and scaling but see also 'missing features' in the filter patch i've submitted) I've been using it to convert yuv output of an usb cam to nv12 fed to the mfc hw enc so that I can stream H263 while leaving the cpu free for other things. Alexis. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On 21 Nov, Alexis Ballier wrote : > > So you have a format outputted that is of no use, except if you use > > the filter. > > On MFCv5 yes; I don't assume because I'm working on this that it is the > only one :) So, basically, noone can display it, reencode or do anything with it, without the filter. Therefore, you should not introduce a new fmt for this. > MFCv6 and newer don't even understand nv12mt and can output standard > nv12: > http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c#n35 This is yet another good reason to NOT introduce a new weird pix-fmt that will stay in FFmpeg for the next 10 years. > I've been using it to convert yuv output of an usb cam to nv12 fed to > the mfc hw enc so that I can stream H263 while leaving the cpu > free for other things. Then keep the weird format internal. With my kindest regards, -- Jean-Baptiste Kempf http://www.jbkempf.com/ - +33 672 704 734 Sent from my Electronic Device ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 10:02:36 +0100 Jean-Baptiste Kempf wrote: > On 21 Nov, Alexis Ballier wrote : > > > > How do you put this in an AVFrame then ? > > > > > > Preferably not at all. Is there a need to? Only the end result > > > (nv12 I suppose) needs to be in an AVFrame. > > > > It can be done but is ugly IMHO: > > > > - I was under the impression that lavc decoder convention was to > > output non processed formats and let the application handle (or > > not) the conversions > > Sure, but this is when you can actually process them in applications, > like planar audio. No a hardware-specific format that noone can play > except with a special library. I think it's safe to assume that if you have a SOC with a HW dec that outputs only this insane and uncommon format, the SOC also has a HW filter that can convert it. I've not played with it, but you can also have a look at s5p-tv driver that is a V4L2 outdev which accepts NV12MT. So you can even play it. > > > So this is a > > > decoder. Why can't the decoder take care of this conversion too? > > > > See above. > > Sorry, but that's far from clear. What is not ? The important part was the one you cut :) Alexis. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On 21 Nov, Alexis Ballier wrote : > I think it's safe to assume that if you have a SOC with a HW dec that > outputs only this insane and uncommon format, the SOC also has a HW > filter that can convert it. Then use this filter to output a standard format. > > Sorry, but that's far from clear. > > What is not ? The important part was the one you cut :) Introducing yet another PIX_FMT that has almost no meaning in the outside world, that is very linked to a specific hardware and that will be needed to be supported for the next 10 years is a very bad idea. With my kindest regards, -- Jean-Baptiste Kempf http://www.jbkempf.com/ - +33 672 704 734 Sent from my Electronic Device ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
Hi, - Mail original - [...] > I think it's safe to assume that if you have a SOC with a HW dec that > outputs only this insane and uncommon format, the SOC also has a HW > filter that can convert it. This is a good argument IMHO. On a side note, I don't think it would be that complicated to support both, through an option. -- Ben ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 10:15:29 +0100 Jean-Baptiste Kempf wrote: > On 21 Nov, Alexis Ballier wrote : > > > So you have a format outputted that is of no use, except if you > > > use the filter. > > > > On MFCv5 yes; I don't assume because I'm working on this that it is > > the only one :) > > So, basically, noone can display it, reencode or do anything with it, > without the filter. > Therefore, you should not introduce a new fmt for this. First, I fail to see how this differs from AV_PIX_FMT_VDPAU & friends. Second, I don't understand all the drama of using _one_ member of an enum to avoid cluttering the code. Let me re-explain my other mail: Current decoding code is: * open the decoder * feed it with some data * get the output format the decoder uses * advertise it in codec context * decoding loop is: get an avpacket, feed to decoder, obtain an avframe Keeping it internal would mean, for the sole decoder: * open the decoder * feed it with some data * get the output format * if i dont like the format then: * probe and open another random device that may or may not exist for format conversion * look for an output format i like * advertise the output format i will give to codec context * decoding loop is: * get an avpacket, feed it to decoder, obtain an avframe * if conversion is needed: * feed frame to filter, obtain the converted frame Now, if I want to use the fimc device to apply some effects I can't because the decoder is magically using it behind my back. If I want to use s5p-tv to display the decoded video over HDMI, then I have post-processed the frame with fimc for nothing since NV12MT is accepted. > > MFCv6 and newer don't even understand nv12mt and can output standard > > nv12: > > http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c#n35 > > This is yet another good reason to NOT introduce a new weird pix-fmt > that will stay in FFmpeg for the next 10 years. It will be in kernel headers and kernel ABI for like forever, so I don't understand why this is so much of a problem. Alexis. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 10:38:37 +0100 Jean-Baptiste Kempf wrote: > On 21 Nov, Alexis Ballier wrote : > > I think it's safe to assume that if you have a SOC with a HW dec > > that outputs only this insane and uncommon format, the SOC also has > > a HW filter that can convert it. > > Then use this filter to output a standard format. I think you should read again the part you cut and reply there, or my other reply in the thread. I explained why I don't like this. Alexis. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On 21 Nov, Alexis Ballier wrote : > On Fri, 21 Nov 2014 10:15:29 +0100 > Jean-Baptiste Kempf wrote: > > > On 21 Nov, Alexis Ballier wrote : > > > > So you have a format outputted that is of no use, except if you > > > > use the filter. > > > > > > On MFCv5 yes; I don't assume because I'm working on this that it is > > > the only one :) > > > > So, basically, noone can display it, reencode or do anything with it, > > without the filter. > > Therefore, you should not introduce a new fmt for this. > > First, I fail to see how this differs from AV_PIX_FMT_VDPAU & friends. Because it's called NV12T, not AV_PIX_FMT_V4L_FMT. And because it's not and HWAccel. > Second, I don't understand all the drama of using _one_ member of an > enum to avoid cluttering the code. Because you don't use libav* libraries directly. > Let me re-explain my other mail: > > Current decoding code is: > * open the decoder > * feed it with some data > * get the output format the decoder uses > * advertise it in codec context > * decoding loop is: get an avpacket, feed to decoder, obtain an avframe > > Keeping it internal would mean, for the sole decoder: > * open the decoder > * feed it with some data > * get the output format > * if i dont like the format then: > * probe and open another random device that may or may not > exist for format conversion > * look for an output format i like > * advertise the output format i will give to codec context > * decoding loop is: > * get an avpacket, feed it to decoder, obtain an avframe > * if conversion is needed: > * feed frame to filter, obtain the converted frame So what? Your 2 cases are completly unfair, and you don't give the same level of details. > Now, if I want to use the fimc device to apply some effects I can't > because the decoder is magically using it behind my back. How is that relevant? Just block it. And it seems really like a technical limitation of this chip. > If I want to use s5p-tv to display the decoded video over HDMI, then I > have post-processed the frame with fimc for nothing since NV12MT is > accepted. Then maybe, in those cases, FFmpeg is not the right tool, and you should do a HWAccel. > > > MFCv6 and newer don't even understand nv12mt and can output standard > > > nv12: > > > http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c#n35 > > > > This is yet another good reason to NOT introduce a new weird pix-fmt > > that will stay in FFmpeg for the next 10 years. > > It will be in kernel headers and kernel ABI for like forever, so I > don't understand why this is so much of a problem. The linux kernel is not the only one around here. With my kindest regards, -- Jean-Baptiste Kempf http://www.jbkempf.com/ - +33 672 704 734 Sent from my Electronic Device ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On 21 Nov, Alexis Ballier wrote : > On Fri, 21 Nov 2014 10:38:37 +0100 > Jean-Baptiste Kempf wrote: > > > On 21 Nov, Alexis Ballier wrote : > > > I think it's safe to assume that if you have a SOC with a HW dec > > > that outputs only this insane and uncommon format, the SOC also has > > > a HW filter that can convert it. > > > > Then use this filter to output a standard format. > > I think you should read again the part you cut and reply there, or my > other reply in the thread. I explained why I don't like this. I explained also why I don't like it. Which gives us what? With my kindest regards, -- Jean-Baptiste Kempf http://www.jbkempf.com/ - +33 672 704 734 Sent from my Electronic Device ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 10:43:15 +0100 Alexis Ballier wrote: > On Fri, 21 Nov 2014 10:15:29 +0100 > Jean-Baptiste Kempf wrote: > > > On 21 Nov, Alexis Ballier wrote : > > > > So you have a format outputted that is of no use, except if you > > > > use the filter. > > > > > > On MFCv5 yes; I don't assume because I'm working on this that it is > > > the only one :) > > > > So, basically, noone can display it, reencode or do anything with it, > > without the filter. > > Therefore, you should not introduce a new fmt for this. > > First, I fail to see how this differs from AV_PIX_FMT_VDPAU & friends. AV_PIX_FMT_VDPAU is marked as hwaccel format, and thus completely opaque. It can't be cropped, copied, etc. - just passed around. It also means the generic AVFrame code can't allocate frames of such a format. Personally I'd have much less of a problem with this if it were to be marked as opaque. > Second, I don't understand all the drama of using _one_ member of an > enum to avoid cluttering the code. Every addition is something the API user has to care about, since there is no automatic conversion between libavcodec and the API user. > Let me re-explain my other mail: > > Current decoding code is: > * open the decoder > * feed it with some data > * get the output format the decoder uses > * advertise it in codec context > * decoding loop is: get an avpacket, feed to decoder, obtain an avframe > > Keeping it internal would mean, for the sole decoder: > * open the decoder > * feed it with some data > * get the output format > * if i dont like the format then: > * probe and open another random device that may or may not > exist for format conversion > * look for an output format i like > * advertise the output format i will give to codec context > * decoding loop is: > * get an avpacket, feed it to decoder, obtain an avframe > * if conversion is needed: > * feed frame to filter, obtain the converted frame > > Now, if I want to use the fimc device to apply some effects I can't > because the decoder is magically using it behind my back. > > If I want to use s5p-tv to display the decoded video over HDMI, then I > have post-processed the frame with fimc for nothing since NV12MT is > accepted. It also means that _every_ software which wants to use this new decocer has to know how to insert the filter etc. - what's the point? If it's so special, it might as well be outside of libavcodec and libavfilter. But if it just works without requiring the API user to jump through hoops, I see at least some value in it. > > > MFCv6 and newer don't even understand nv12mt and can output standard > > > nv12: > > > http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c#n35 > > > > This is yet another good reason to NOT introduce a new weird pix-fmt > > that will stay in FFmpeg for the next 10 years. > > It will be in kernel headers and kernel ABI for like forever, so I > don't understand why this is so much of a problem. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 10:51:52 +0100 wm4 wrote: > On Fri, 21 Nov 2014 10:43:15 +0100 > Alexis Ballier wrote: > > > On Fri, 21 Nov 2014 10:15:29 +0100 > > Jean-Baptiste Kempf wrote: > > > > > On 21 Nov, Alexis Ballier wrote : > > > > > So you have a format outputted that is of no use, except if > > > > > you use the filter. > > > > > > > > On MFCv5 yes; I don't assume because I'm working on this that > > > > it is the only one :) > > > > > > So, basically, noone can display it, reencode or do anything with > > > it, without the filter. > > > Therefore, you should not introduce a new fmt for this. > > > > First, I fail to see how this differs from AV_PIX_FMT_VDPAU & > > friends. > > AV_PIX_FMT_VDPAU is marked as hwaccel format, and thus completely > opaque. It can't be cropped, copied, etc. - just passed around. It > also means the generic AVFrame code can't allocate frames of such a > format. > > Personally I'd have much less of a problem with this if it were to be > marked as opaque. I didn't know such "opaque" things existed; it probably makes more sense with it indeed. How should I do it ? add .flags = AV_PIX_FMT_FLAG_HWACCEL and be done ? I would like to keep 'av_pix_fmt_count_planes()' working for it at least [...] > It also means that _every_ software which wants to use this new > decocer has to know how to insert the filter etc. - what's the point? This could be "fixed" by adding support in swscale for it, but I see little to no point to it. I think you have to specifically ask for v4l2m2m codecs if you want to use them do hw (de/en)code; so why not do everything in hw and also setup the filter ? (note: I shall write some doc about all this once this has settled) > But if it just works without requiring the API user to jump through > hoops, I see at least some value in it. Would AV_PIX_FMT_FLAG_HWACCEL be enough ? Alexis. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 11:15:08 +0100 Alexis Ballier wrote: > On Fri, 21 Nov 2014 10:51:52 +0100 > wm4 wrote: > > > On Fri, 21 Nov 2014 10:43:15 +0100 > > Alexis Ballier wrote: > > > > > On Fri, 21 Nov 2014 10:15:29 +0100 > > > Jean-Baptiste Kempf wrote: > > > > > > > On 21 Nov, Alexis Ballier wrote : > > > > > > So you have a format outputted that is of no use, except if > > > > > > you use the filter. > > > > > > > > > > On MFCv5 yes; I don't assume because I'm working on this that > > > > > it is the only one :) > > > > > > > > So, basically, noone can display it, reencode or do anything with > > > > it, without the filter. > > > > Therefore, you should not introduce a new fmt for this. > > > > > > First, I fail to see how this differs from AV_PIX_FMT_VDPAU & > > > friends. > > > > AV_PIX_FMT_VDPAU is marked as hwaccel format, and thus completely > > opaque. It can't be cropped, copied, etc. - just passed around. It > > also means the generic AVFrame code can't allocate frames of such a > > format. > > > > Personally I'd have much less of a problem with this if it were to be > > marked as opaque. > > I didn't know such "opaque" things existed; it probably makes more > sense with it indeed. > > How should I do it ? add .flags = AV_PIX_FMT_FLAG_HWACCEL and be done ? > I would like to keep 'av_pix_fmt_count_planes()' working for it at least No, this would imply that the pointer to the opaque data is in AVFrame.data[3], and all other pointers are ignored. So you have only 1 pointer. AVFrame.linesize has no meaning either in this case. > [...] > > It also means that _every_ software which wants to use this new > > decocer has to know how to insert the filter etc. - what's the point? > > This could be "fixed" by adding support in swscale for it, but I see > little to no point to it. > > I think you have to specifically ask for v4l2m2m codecs if you want to > use them do hw (de/en)code; so why not do everything in hw and also > setup the filter ? Well, if the filter copies to system RAM anyway (if that chip distinguishes between GPU and system RAM at all), then why do you need such a filter as user-visible thing at all? Where exactly is the problem in putting this code into libavcodec? > (note: I shall write some doc about all this once this has settled) > > > But if it just works without requiring the API user to jump through > > hoops, I see at least some value in it. > > > Would AV_PIX_FMT_FLAG_HWACCEL be enough ? I can't speak for others whether this approach is acceptable. But I think this is all too messy, and the decoder should just output a useful format. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 10:50:42 +0100 Jean-Baptiste Kempf wrote: > > Now, if I want to use the fimc device to apply some effects I can't > > because the decoder is magically using it behind my back. > > How is that relevant? Just block it. Block what ? automatic conversion ? what fmt should i advertise if NV12T is the only supported output ? > > If I want to use s5p-tv to display the decoded video over HDMI, > > then I have post-processed the frame with fimc for nothing since > > NV12MT is accepted. > > Then maybe, in those cases, FFmpeg is not the right tool, and you > should do a HWAccel. Besides that this belongs more to the lavd outdev, you may or may not have the tv driver loaded & working. And this would be very specific to socs with m2m codecs, m2m filters and v4l2 outdev, while the current code tries to keep everything well separated: one device per "context" so that it can work with various setups. This is probably increasing work at application level though, but HWAccel always requires application specific support afaik. > > > > MFCv6 and newer don't even understand nv12mt and can output > > > > standard nv12: > > > > http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c#n35 > > > > > > This is yet another good reason to NOT introduce a new weird > > > pix-fmt that will stay in FFmpeg for the next 10 years. > > > > It will be in kernel headers and kernel ABI for like forever, so I > > don't understand why this is so much of a problem. > > The linux kernel is not the only one around here. you'll never get to see this format in an application outside of the linux kernel, and probably never outside of samsung socs :) Alexis. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 11:23:49 +0100 wm4 wrote: > > I didn't know such "opaque" things existed; it probably makes more > > sense with it indeed. > > > > How should I do it ? add .flags = AV_PIX_FMT_FLAG_HWACCEL and be > > done ? I would like to keep 'av_pix_fmt_count_planes()' working for > > it at least > > No, this would imply that the pointer to the opaque data is in > AVFrame.data[3], and all other pointers are ignored. So you have only > 1 pointer. AVFrame.linesize has no meaning either in this case. > > > [...] > > > It also means that _every_ software which wants to use this new > > > decocer has to know how to insert the filter etc. - what's the > > > point? > > > > This could be "fixed" by adding support in swscale for it, but I see > > little to no point to it. > > > > I think you have to specifically ask for v4l2m2m codecs if you want > > to use them do hw (de/en)code; so why not do everything in hw and > > also setup the filter ? > > Well, if the filter copies to system RAM anyway (if that chip > distinguishes between GPU and system RAM at all), Speaking only for that chip (this will likely differ for others), there is no such thing as "gpu ram", but there is cma allocated & reserved memory in system ram. Default settings ask the driver to mmap this memory and use this as a buffer; this indeed introduces some copies to feed the filter or codec with data: if we get random input buffers, we have to memcpy() them at the mmap adress given by the driver. mfc codec refuses to handle anything but mmaped io. You can feed the filter with userptr memory (basically a userspace pointer that the driver will translate himself) but this memory must be _physically_ contiguous, which I don't think we have any way to ensure at userspace level. The solution is to take the data from the decoder, feed the frame with zero copy (thanks to refcounted frames) to the filter: Since the buffers come from the decoder who has mmaped them in its cma-allocated memory it is guaranteed to be physically contiguous. > then why do you need > such a filter as user-visible thing at all? > Where exactly is the problem in putting this code into libavcodec? There is a limited # of hw filters; you may also want to scale the video to fit your display, which the filter can do together with the fmt conversion in one pass. I don't think up/down scaling the decoded video should be done in lavc. > > (note: I shall write some doc about all this once this has settled) > > > > > But if it just works without requiring the API user to jump > > > through hoops, I see at least some value in it. > > > > > > Would AV_PIX_FMT_FLAG_HWACCEL be enough ? > > I can't speak for others whether this approach is acceptable. I don't understand: > No, this would imply that the pointer to the opaque data is in > AVFrame.data[3], and all other pointers are ignored. So you have only > 1 pointer. AVFrame.linesize has no meaning either in this case. this won't work with NV12(T) since the format has 2 planes ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.
On Fri, 21 Nov 2014 11:52:01 +0100 Alexis Ballier wrote: > > No, this would imply that the pointer to the opaque data is in > > AVFrame.data[3], and all other pointers are ignored. So you have > > only 1 pointer. AVFrame.linesize has no meaning either in this case. > > this won't work with NV12(T) since the format has 2 planes tried that and it indeed completely breaks linesize computing which i need. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
