Re: How to express planar formats with mediabus format code?
Hi Jiaquan and Laurent, Apologies for my delayed reply. Su Jiaquan wrote: Hi Sakari, On Thu, Aug 22, 2013 at 7:29 PM, Laurent Pinchart wrote: Hi Jiaquan, On Wednesday 21 August 2013 18:14:50 Su Jiaquan wrote: On Tue, Aug 20, 2013 at 8:53 PM, Laurent Pinchart wrote: Hi Jiaquan, I'm not sure if that's needed here. Vendor-specific formats still need to be documented, so we could just create a custom YUV format for your case. Let's start with the beginning, could you describe what gets transmitted on the bus when that special format is selected ? For YUV420P format, the data format sent from IPC is similar to V4L2_MBUS_FMT_YUYV8_1_5X8, but the content for each line is different: For odd line, it's YYU YYU YYU... For even line, it's YYV YYV YYV... then DMA engine send them to RAM in planar format. For YUV420SP format, the data format sent from IPC is YYUV YYUV YYUV(maybe called V4L2_MBUS_FMT_YYUV8_2X8?), but DMA engine drop UV every other line, then send them to RAM as semi-planar. V4L2_MBUS_FMT_YYUV8_2X8 looks good to me. Well, the first data format is too odd, I don't have a clue how to call it, do you have suggestion? Maybe V4L2_MBUS_FMT_YU8_YV8_1_5X8 ? I've CC'ed Sakari Ailus, he's often pretty creative for these issues. -- Regards, Laurent Pinchart Does the format V4L2_MBUS_FMT_YU8_YV8_1_5X8 sounds good to you? Do you have better idea how we should describe this format? If bus (or DMA) transfers 8 bits at a time, then yes. Otherwise perhaps ...1X12. The documentation should be extended to cover different components on alternating lines; I don't think we've had such cases before. I think just a note telling to do exactly as above should suffice. -- Kind regards, Sakari Ailus sakari.ai...@iki.fi -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to express planar formats with mediabus format code?
Hi Sakari, On Thu, Aug 22, 2013 at 7:29 PM, Laurent Pinchart wrote: > Hi Jiaquan, > > On Wednesday 21 August 2013 18:14:50 Su Jiaquan wrote: >> On Tue, Aug 20, 2013 at 8:53 PM, Laurent Pinchart wrote: >> > Hi Jiaquan, >> > >> > I'm not sure if that's needed here. Vendor-specific formats still need to >> > be documented, so we could just create a custom YUV format for your case. >> > Let's start with the beginning, could you describe what gets transmitted >> > on the bus when that special format is selected ? >> >> For YUV420P format, the data format sent from IPC is similar to >> V4L2_MBUS_FMT_YUYV8_1_5X8, but the content for each line is different: >> For odd line, it's YYU YYU YYU... For even line, it's YYV YYV YYV... >> then DMA engine send them to RAM in planar format. >> >> For YUV420SP format, the data format sent from IPC is YYUV YYUV >> YYUV(maybe called V4L2_MBUS_FMT_YYUV8_2X8?), but DMA engine drop UV >> every other line, then send them to RAM as semi-planar. > > V4L2_MBUS_FMT_YYUV8_2X8 looks good to me. > >> Well, the first data format is too odd, I don't have a clue how to >> call it, do you have suggestion? > > Maybe V4L2_MBUS_FMT_YU8_YV8_1_5X8 ? I've CC'ed Sakari Ailus, he's often pretty > creative for these issues. > > -- > Regards, > > Laurent Pinchart > Does the format V4L2_MBUS_FMT_YU8_YV8_1_5X8 sounds good to you? Do you have better idea how we should describe this format? If there is no further concern, I'll prepare a patch Thanks -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to express planar formats with mediabus format code?
Hi Jiaquan, On Wednesday 21 August 2013 18:14:50 Su Jiaquan wrote: > On Tue, Aug 20, 2013 at 8:53 PM, Laurent Pinchart wrote: > > Hi Jiaquan, > > > > I'm not sure if that's needed here. Vendor-specific formats still need to > > be documented, so we could just create a custom YUV format for your case. > > Let's start with the beginning, could you describe what gets transmitted > > on the bus when that special format is selected ? > > For YUV420P format, the data format sent from IPC is similar to > V4L2_MBUS_FMT_YUYV8_1_5X8, but the content for each line is different: > For odd line, it's YYU YYU YYU... For even line, it's YYV YYV YYV... > then DMA engine send them to RAM in planar format. > > For YUV420SP format, the data format sent from IPC is YYUV YYUV > YYUV(maybe called V4L2_MBUS_FMT_YYUV8_2X8?), but DMA engine drop UV > every other line, then send them to RAM as semi-planar. V4L2_MBUS_FMT_YYUV8_2X8 looks good to me. > Well, the first data format is too odd, I don't have a clue how to > call it, do you have suggestion? Maybe V4L2_MBUS_FMT_YU8_YV8_1_5X8 ? I've CC'ed Sakari Ailus, he's often pretty creative for these issues. -- Regards, Laurent Pinchart -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to express planar formats with mediabus format code?
Hi Laurent, Thanks for the replay, I've removed earlier mail content and only keep you question: On Tue, Aug 20, 2013 at 8:53 PM, Laurent Pinchart wrote: > Hi Jiaquan, > > I'm not sure if that's needed here. Vendor-specific formats still need to be > documented, so we could just create a custom YUV format for your case. Let's > start with the beginning, could you describe what gets transmitted on the bus > when that special format is selected ? > > -- > Regards, > > Laurent Pinchart > For YUV420P format, the data format sent from IPC is similar to V4L2_MBUS_FMT_YUYV8_1_5X8, but the content for each line is different: For odd line, it's YYU YYU YYU... For even line, it's YYV YYV YYV... then DMA engine send them to RAM in planar format. For YUV420SP format, the data format sent from IPC is YYUV YYUV YYUV(maybe called V4L2_MBUS_FMT_YYUV8_2X8?), but DMA engine drop UV every other line, then send them to RAM as semi-planar. Well, the first data format is too odd, I don't have a clue how to call it, do you have suggestion? Thanks a lot! Jiaquan -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to express planar formats with mediabus format code?
Hi Jiaquan,
On Thursday 15 August 2013 16:27:43 Su Jiaquan wrote:
> On Sat, Aug 10, 2013 at 1:06 AM, Su Jiaquan wrote:
> > On Fri, Aug 9, 2013 at 5:12 AM, Laurent Pinchart wrote:
> >> On Tuesday 06 August 2013 17:18:14 Su Jiaquan wrote:
> >>> On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski wrote:
> >>> > On Sun, 4 Aug 2013, Su Jiaquan wrote:
> >>> >> Hi,
> >>> >>
> >>> >> I know the title looks crazy, but here is our problem:
> >>> >>
> >>> >> In our SoC based ISP, the hardware can be divide to several blocks.
> >>> >> Some blocks can do color space conversion(raw to YUV
> >>> >> interleave/planar), others can do the pixel re-
> >>> >> order(interleave/planar/semi-planar conversion, UV planar switch). We
> >>> >> use one subdev to describe each of them, then came the problem: How
> >>> >> can we express the planar formats with mediabus format code?
> >>> >
> >>> > Could you please explain more exactly what you mean? How are those
> >>> > your blocks connected? How do they exchange data? If they exchange
> >>> > data over a serial bus, then I don't think planar formats make sense,
> >>> > right? Or do your blocks really output planes one after another,
> >>> > reordering data internally? That would be odd... If OTOH your blocks
> >>> > output data to RAM, and the next block takes data from there, then you
> >>> > use V4L2_PIX_FMT_* formats to describe them and any further processing
> >>> > block should be a mem2mem device. Wouldn't this work?
> >>>
> >>> These two hardware blocks are both located inside of ISP, and is
> >>> connected by a hardware data bus.
> >>>
> >>> Actually, there are three blocks inside ISP: One is close to sensor, and
> >>> can do color space conversion(RGB->YUV), we call it IPC; The other two
> >>> are at back end, which are basically DMA Engine, and they are
> >>> identical. When data flow out of IPC, it can go into each one of these
> >>> DMA Engines and finally into RAM. Whether the DMA Engine is turned
> >>> on/off and the output format can be controlled independently. Since
> >>> they are DMA Engines, they have some basic pixel reordering
> >>> ability(i.e. interleave->planar/semi-planar).
> >>>
> >>> In our H/W design, when we want to get YUV semi-planar format, the IPC
> >>> output should be configured to interleave, and the DMA engine will do
> >>> the interleave->semi-planar job. If we want planar / interleave format,
> >>> the IPC will output planar format directly, DMA engine simply send the
> >>> data to RAM, and don't do any re-order. So in the planar output case,
> >>> media-bus formats can't express the format of the data between IPC and
> >>> DMA Engine, that's the problem we meet.
> >>
> >> If the format between the two subdevs is really planar, I don't see any
> >> problem defining a media bus pixel code for it. You will have to properly
> >> document the format of course.
> >>
> >> I'm a bit surprised that the IPC could output planar data. It would need
> >> to buffer a whole image to do so, do you need to give it a temporary
> >> system RAM buffer ?
> >>
> >>> We want to adopt a formal solution before we send our patch to the
> >>> community, that's where our headache comes.
> >
> > Thanks for the reply!
> >
> > Actually, we don't need to buffer the frame inside IPC, there are
> > three channels in the data bus. When transfering interleave format,
> > only one channel is used, for planar formats, three channels send one
> > planar each, and to difference address(Let me confirm this with our
> > H/W team and get back to you later). So the planars is not sent one
> > after an other, but in parallel.
> >
> > This may be a bit different from the planar formats as people think it
> > should be. Can we use planar format to describe it? Since this won't
> > cause any misunderstanding given it's used in this special case.
> > Please advice.
> >
>
> I have to say I'm sorry for the wrong information. I just double checked
> with hardware team, and turns out there is only one channel for the data bus
> between IPC and ispdma.
> If ispdma output planar format, the data format between IPC and ispdma
> should be configured to a special format, that is not the same with any know
> media-bus format.
> So I think what we need is to define vendor-specific media-bus code.
> Since others any want to do the same thing, shall we define a base address
> for the vendor specific formats? For example:
>
> enum v4l2_mbus_pixelcode {
> V4L2_MBUS_FMT_FIXED = 0x0001,
>
> ...
>
> /* JPEG compressed formats - next is 0x4002 */
> V4L2_MBUS_FMT_JPEG_1X8 = 0x4001,
> + V4L2_MBUS_FMT_PRIVATE_BASE = 0xF001,
> };
>
> If you are OK with this, I'll prepare a patch to add it
I'm not sure if that's needed here. Vendor-specific formats still need to be
documented, so we could just create a custom YUV format for your case. Let's
start with the beginning, could you describe what gets transmitted on the bus
when that special format is selected ?
--
Regards,
Laure
Re: How to express planar formats with mediabus format code?
Hi Laurent / Guennadi,
On Sat, Aug 10, 2013 at 1:06 AM, Su Jiaquan wrote:
> Hi Laurent / Guennadi,
>
> On Fri, Aug 9, 2013 at 5:12 AM, Laurent Pinchart
> wrote:
>> Hi,
>>
>> On Tuesday 06 August 2013 17:18:14 Su Jiaquan wrote:
>>> Hi Guennadi,
>>>
>>> Thanks for the reply! Please see my description inline.
>>>
>>> On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski wrote:
>>> > On Sun, 4 Aug 2013, Su Jiaquan wrote:
>>> >> Hi,
>>> >>
>>> >> I know the title looks crazy, but here is our problem:
>>> >>
>>> >> In our SoC based ISP, the hardware can be divide to several blocks.
>>> >> Some blocks can do color space conversion(raw to YUV interleave/planar),
>>> >> others can do the pixel re-order(interleave/planar/semi-planar
>>> >> conversion, UV planar switch). We use one subdev to describe each of
>>> >> them, then came the problem: How can we express the planar formats with
>>> >> mediabus format code?
>>> >
>>> > Could you please explain more exactly what you mean? How are those your
>>> > blocks connected? How do they exchange data? If they exchange data over a
>>> > serial bus, then I don't think planar formats make sense, right? Or do
>>> > your blocks really output planes one after another, reordering data
>>> > internally? That would be odd... If OTOH your blocks output data to RAM,
>>> > and the next block takes data from there, then you use V4L2_PIX_FMT_*
>>> > formats to describe them and any further processing block should be a
>>> > mem2mem device. Wouldn't this work?
>>>
>>> These two hardware blocks are both located inside of ISP, and is connected
>>> by a hardware data bus.
>>>
>>> Actually, there are three blocks inside ISP: One is close to sensor, and can
>>> do color space conversion(RGB->YUV), we call it IPC; The other two are at
>>> back end, which are basically DMA Engine, and they are identical. When data
>>> flow out of IPC, it can go into each one of these DMA Engines and finally
>>> into RAM. Whether the DMA Engine is turned on/off and the output format can
>>> be controlled independently. Since they are DMA Engines, they have some
>>> basic pixel reordering ability(i.e. interleave->planar/semi-planar).
>>>
>>> In our H/W design, when we want to get YUV semi-planar format, the IPC
>>> output should be configured to interleave, and the DMA engine will do the
>>> interleave->semi-planar job. If we want planar / interleave format, the IPC
>>> will output planar format directly, DMA engine simply send the data to RAM,
>>> and don't do any re-order. So in the planar output case, media-bus formats
>>> can't express the format of the data between IPC and DMA Engine, that's the
>>> problem we meet.
>>
>> If the format between the two subdevs is really planar, I don't see any
>> problem defining a media bus pixel code for it. You will have to properly
>> document the format of course.
>>
>> I'm a bit surprised that the IPC could output planar data. It would need to
>> buffer a whole image to do so, do you need to give it a temporary system RAM
>> buffer ?
>>
>>> We want to adopt a formal solution before we send our patch to the
>>> community, that's where our headache comes.
>>
>> --
>> Regards,
>>
>> Laurent Pinchart
>>
>
> Thanks for the reply!
>
> Actually, we don't need to buffer the frame inside IPC, there are
> three channels in the data bus. When transfering interleave format,
> only one channel is used, for planar formats, three channels send one
> planar each, and to difference address(Let me confirm this with our
> H/W team and get back to you later). So the planars is not sent one
> after an other, but in parallel.
>
> This may be a bit different from the planar formats as people think it
> should be. Can we use planar format to describe it? Since this won't
> cause any misunderstanding given it's used in this special case.
> Please advice.
>
> Thanks a lot!
>
> Jiaquan
I have to say I'm sorry for the wrong information. I just double
checked with hardware team, and turns out there is only one channel
for the data bus between IPC and ispdma.
If ispdma output planar format, the data format between IPC and ispdma
should be configured to a special format, that is not the same with
any know media-bus format.
So I think what we need is to define vendor-specific media-bus code.
Since others any want to do the same thing, shall we define a base
address for the vendor specific formats? For example:
enum v4l2_mbus_pixelcode {
V4L2_MBUS_FMT_FIXED = 0x0001,
...
/* JPEG compressed formats - next is 0x4002 */
V4L2_MBUS_FMT_JPEG_1X8 = 0x4001,
+ V4L2_MBUS_FMT_PRIVATE_BASE = 0xF001,
};
If you are OK with this, I'll prepare a patch to add it
Thanks!
Jiaquan
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to express planar formats with mediabus format code?
Hi Laurent / Guennadi, On Fri, Aug 9, 2013 at 5:12 AM, Laurent Pinchart wrote: > Hi, > > On Tuesday 06 August 2013 17:18:14 Su Jiaquan wrote: >> Hi Guennadi, >> >> Thanks for the reply! Please see my description inline. >> >> On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski wrote: >> > On Sun, 4 Aug 2013, Su Jiaquan wrote: >> >> Hi, >> >> >> >> I know the title looks crazy, but here is our problem: >> >> >> >> In our SoC based ISP, the hardware can be divide to several blocks. >> >> Some blocks can do color space conversion(raw to YUV interleave/planar), >> >> others can do the pixel re-order(interleave/planar/semi-planar >> >> conversion, UV planar switch). We use one subdev to describe each of >> >> them, then came the problem: How can we express the planar formats with >> >> mediabus format code? >> > >> > Could you please explain more exactly what you mean? How are those your >> > blocks connected? How do they exchange data? If they exchange data over a >> > serial bus, then I don't think planar formats make sense, right? Or do >> > your blocks really output planes one after another, reordering data >> > internally? That would be odd... If OTOH your blocks output data to RAM, >> > and the next block takes data from there, then you use V4L2_PIX_FMT_* >> > formats to describe them and any further processing block should be a >> > mem2mem device. Wouldn't this work? >> >> These two hardware blocks are both located inside of ISP, and is connected >> by a hardware data bus. >> >> Actually, there are three blocks inside ISP: One is close to sensor, and can >> do color space conversion(RGB->YUV), we call it IPC; The other two are at >> back end, which are basically DMA Engine, and they are identical. When data >> flow out of IPC, it can go into each one of these DMA Engines and finally >> into RAM. Whether the DMA Engine is turned on/off and the output format can >> be controlled independently. Since they are DMA Engines, they have some >> basic pixel reordering ability(i.e. interleave->planar/semi-planar). >> >> In our H/W design, when we want to get YUV semi-planar format, the IPC >> output should be configured to interleave, and the DMA engine will do the >> interleave->semi-planar job. If we want planar / interleave format, the IPC >> will output planar format directly, DMA engine simply send the data to RAM, >> and don't do any re-order. So in the planar output case, media-bus formats >> can't express the format of the data between IPC and DMA Engine, that's the >> problem we meet. > > If the format between the two subdevs is really planar, I don't see any > problem defining a media bus pixel code for it. You will have to properly > document the format of course. > > I'm a bit surprised that the IPC could output planar data. It would need to > buffer a whole image to do so, do you need to give it a temporary system RAM > buffer ? > >> We want to adopt a formal solution before we send our patch to the >> community, that's where our headache comes. > > -- > Regards, > > Laurent Pinchart > Thanks for the reply! Actually, we don't need to buffer the frame inside IPC, there are three channels in the data bus. When transfering interleave format, only one channel is used, for planar formats, three channels send one planar each, and to difference address(Let me confirm this with our H/W team and get back to you later). So the planars is not sent one after an other, but in parallel. This may be a bit different from the planar formats as people think it should be. Can we use planar format to describe it? Since this won't cause any misunderstanding given it's used in this special case. Please advice. Thanks a lot! Jiaquan -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to express planar formats with mediabus format code?
Hi, On Tuesday 06 August 2013 17:18:14 Su Jiaquan wrote: > Hi Guennadi, > > Thanks for the reply! Please see my description inline. > > On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski wrote: > > On Sun, 4 Aug 2013, Su Jiaquan wrote: > >> Hi, > >> > >> I know the title looks crazy, but here is our problem: > >> > >> In our SoC based ISP, the hardware can be divide to several blocks. > >> Some blocks can do color space conversion(raw to YUV interleave/planar), > >> others can do the pixel re-order(interleave/planar/semi-planar > >> conversion, UV planar switch). We use one subdev to describe each of > >> them, then came the problem: How can we express the planar formats with > >> mediabus format code? > > > > Could you please explain more exactly what you mean? How are those your > > blocks connected? How do they exchange data? If they exchange data over a > > serial bus, then I don't think planar formats make sense, right? Or do > > your blocks really output planes one after another, reordering data > > internally? That would be odd... If OTOH your blocks output data to RAM, > > and the next block takes data from there, then you use V4L2_PIX_FMT_* > > formats to describe them and any further processing block should be a > > mem2mem device. Wouldn't this work? > > These two hardware blocks are both located inside of ISP, and is connected > by a hardware data bus. > > Actually, there are three blocks inside ISP: One is close to sensor, and can > do color space conversion(RGB->YUV), we call it IPC; The other two are at > back end, which are basically DMA Engine, and they are identical. When data > flow out of IPC, it can go into each one of these DMA Engines and finally > into RAM. Whether the DMA Engine is turned on/off and the output format can > be controlled independently. Since they are DMA Engines, they have some > basic pixel reordering ability(i.e. interleave->planar/semi-planar). > > In our H/W design, when we want to get YUV semi-planar format, the IPC > output should be configured to interleave, and the DMA engine will do the > interleave->semi-planar job. If we want planar / interleave format, the IPC > will output planar format directly, DMA engine simply send the data to RAM, > and don't do any re-order. So in the planar output case, media-bus formats > can't express the format of the data between IPC and DMA Engine, that's the > problem we meet. If the format between the two subdevs is really planar, I don't see any problem defining a media bus pixel code for it. You will have to properly document the format of course. I'm a bit surprised that the IPC could output planar data. It would need to buffer a whole image to do so, do you need to give it a temporary system RAM buffer ? > We want to adopt a formal solution before we send our patch to the > community, that's where our headache comes. -- Regards, Laurent Pinchart -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to express planar formats with mediabus format code?
Hi Su Jiaquan, On Tue, 6 Aug 2013, Su Jiaquan wrote: > Hi Guennadi, > > Thanks for the reply! Please see my description inline. > > On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski > wrote: > > Hi Su Jiaquan > > > > On Sun, 4 Aug 2013, Su Jiaquan wrote: > > > >> Hi, > >> > >> I know the title looks crazy, but here is our problem: > >> > >> In our SoC based ISP, the hardware can be divide to several blocks. > >> Some blocks can do color space conversion(raw to YUV > >> interleave/planar), others can do the pixel > >> re-order(interleave/planar/semi-planar conversion, UV planar switch). > >> We use one subdev to describe each of them, then came the problem: How > >> can we express the planar formats with mediabus format code? > > > > Could you please explain more exactly what you mean? How are those your > > blocks connected? How do they exchange data? If they exchange data over a > > serial bus, then I don't think planar formats make sense, right? Or do > > your blocks really output planes one after another, reordering data > > internally? That would be odd... If OTOH your blocks output data to RAM, > > and the next block takes data from there, then you use V4L2_PIX_FMT_* > > formats to describe them and any further processing block should be a > > mem2mem device. Wouldn't this work? > > These two hardware blocks are both located inside of ISP, and is > connected by a hardware data bus. > > Actually, there are three blocks inside ISP: One is close to sensor, > and can do color space conversion(RGB->YUV), we call it IPC; The other > two are at back end, which are basically DMA Engine, and they are > identical. When data flow out of IPC, it can go into each one of these > DMA Engines and finally into RAM. Whether the DMA Engine is turned > on/off and the output format can be controlled independently. Since > they are DMA Engines, they have some basic pixel reordering > ability(i.e. interleave->planar/semi-planar). > > In our H/W design, when we want to get YUV semi-planar format, the IPC > output should be configured to interleave, and the DMA engine will do > the interleave->semi-planar job. If we want planar / interleave > format, the IPC will output planar format directly, DMA engine simply > send the data to RAM, and don't do any re-order. So in the planar > output case, media-bus formats can't express the format of the data > between IPC and DMA Engine, that's the problem we meet. Ok, so, do I understand you correctly, that in the case, where IPC outputs planar data you have: 1. your sensor is sending data to IPC Then one of the following happens 2a. IPC stores the complete frame first, and only when the frame is complete, it first outputs the Y plane amd then the UV plane. A slight optimisation of this would be 2b. it outputs Y components as pixels arrive and stores UV data internally, and at the end of the frame it sends out UV If this is indeed the case, well, then I'm on the same page with you - I don't know a standard solution for this, sorry. It seems to me, that you will indeed need a new mediabus pixel code for this - either a generic one or a vendor-specific one. Let's see what others say. Thanks Guennadi > We want to adopt a formal solution before we send our patch to the > community, that's where our headache comes. > > > > Thanks > > Guennadi > > > >> I understand at beginning, media-bus was designed to describe the data > >> link between camera sensor and camera controller, where sensor is > >> described in subdev. So interleave formats looks good enough at that > >> time. But now as Media-controller is introduced, subdev can describe a > >> much wider range of hardware, which is not limited to camera sensor. > >> So now planar formats are possible to be passed between subdevs. > >> > >> I think the problem we meet can be very common for SoC based ISP > >> solutions, what do you think about it? > >> > >> there are many possible solution for it: > >> > >> 1> change the definition of v4l2_subdev_format::format, use v4l2_format; > >> > >> 2> extend the mediabus format code, add planar format code; > >> > >> 3> use a extra bit to tell the meaning of v4l2_mbus_framefmt::code, is > >> it in mediabus-format or in fourcc > >> > >> Do you have any suggestions? > >> > >> Thanks a lot! > >> -- > >> To unsubscribe from this list: send the line "unsubscribe linux-media" in > >> the body of a message to majord...@vger.kernel.org > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> > > > > --- > > Guennadi Liakhovetski, Ph.D. > > Freelance Open-Source Software Developer > > http://www.open-technology.de/ > > Thanks! > > Jiaquan > --- Guennadi Liakhovetski, Ph.D. Freelance Open-Source Software Developer http://www.open-technology.de/ -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to express planar formats with mediabus format code?
Hi Guennadi, Thanks for the reply! Please see my description inline. On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski wrote: > Hi Su Jiaquan > > On Sun, 4 Aug 2013, Su Jiaquan wrote: > >> Hi, >> >> I know the title looks crazy, but here is our problem: >> >> In our SoC based ISP, the hardware can be divide to several blocks. >> Some blocks can do color space conversion(raw to YUV >> interleave/planar), others can do the pixel >> re-order(interleave/planar/semi-planar conversion, UV planar switch). >> We use one subdev to describe each of them, then came the problem: How >> can we express the planar formats with mediabus format code? > > Could you please explain more exactly what you mean? How are those your > blocks connected? How do they exchange data? If they exchange data over a > serial bus, then I don't think planar formats make sense, right? Or do > your blocks really output planes one after another, reordering data > internally? That would be odd... If OTOH your blocks output data to RAM, > and the next block takes data from there, then you use V4L2_PIX_FMT_* > formats to describe them and any further processing block should be a > mem2mem device. Wouldn't this work? These two hardware blocks are both located inside of ISP, and is connected by a hardware data bus. Actually, there are three blocks inside ISP: One is close to sensor, and can do color space conversion(RGB->YUV), we call it IPC; The other two are at back end, which are basically DMA Engine, and they are identical. When data flow out of IPC, it can go into each one of these DMA Engines and finally into RAM. Whether the DMA Engine is turned on/off and the output format can be controlled independently. Since they are DMA Engines, they have some basic pixel reordering ability(i.e. interleave->planar/semi-planar). In our H/W design, when we want to get YUV semi-planar format, the IPC output should be configured to interleave, and the DMA engine will do the interleave->semi-planar job. If we want planar / interleave format, the IPC will output planar format directly, DMA engine simply send the data to RAM, and don't do any re-order. So in the planar output case, media-bus formats can't express the format of the data between IPC and DMA Engine, that's the problem we meet. We want to adopt a formal solution before we send our patch to the community, that's where our headache comes. > > Thanks > Guennadi > >> I understand at beginning, media-bus was designed to describe the data >> link between camera sensor and camera controller, where sensor is >> described in subdev. So interleave formats looks good enough at that >> time. But now as Media-controller is introduced, subdev can describe a >> much wider range of hardware, which is not limited to camera sensor. >> So now planar formats are possible to be passed between subdevs. >> >> I think the problem we meet can be very common for SoC based ISP >> solutions, what do you think about it? >> >> there are many possible solution for it: >> >> 1> change the definition of v4l2_subdev_format::format, use v4l2_format; >> >> 2> extend the mediabus format code, add planar format code; >> >> 3> use a extra bit to tell the meaning of v4l2_mbus_framefmt::code, is >> it in mediabus-format or in fourcc >> >> Do you have any suggestions? >> >> Thanks a lot! >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-media" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > --- > Guennadi Liakhovetski, Ph.D. > Freelance Open-Source Software Developer > http://www.open-technology.de/ Thanks! Jiaquan -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to express planar formats with mediabus format code?
Hi Su Jiaquan On Sun, 4 Aug 2013, Su Jiaquan wrote: > Hi, > > I know the title looks crazy, but here is our problem: > > In our SoC based ISP, the hardware can be divide to several blocks. > Some blocks can do color space conversion(raw to YUV > interleave/planar), others can do the pixel > re-order(interleave/planar/semi-planar conversion, UV planar switch). > We use one subdev to describe each of them, then came the problem: How > can we express the planar formats with mediabus format code? Could you please explain more exactly what you mean? How are those your blocks connected? How do they exchange data? If they exchange data over a serial bus, then I don't think planar formats make sense, right? Or do your blocks really output planes one after another, reordering data internally? That would be odd... If OTOH your blocks output data to RAM, and the next block takes data from there, then you use V4L2_PIX_FMT_* formats to describe them and any further processing block should be a mem2mem device. Wouldn't this work? Thanks Guennadi > I understand at beginning, media-bus was designed to describe the data > link between camera sensor and camera controller, where sensor is > described in subdev. So interleave formats looks good enough at that > time. But now as Media-controller is introduced, subdev can describe a > much wider range of hardware, which is not limited to camera sensor. > So now planar formats are possible to be passed between subdevs. > > I think the problem we meet can be very common for SoC based ISP > solutions, what do you think about it? > > there are many possible solution for it: > > 1> change the definition of v4l2_subdev_format::format, use v4l2_format; > > 2> extend the mediabus format code, add planar format code; > > 3> use a extra bit to tell the meaning of v4l2_mbus_framefmt::code, is > it in mediabus-format or in fourcc > > Do you have any suggestions? > > Thanks a lot! > -- > To unsubscribe from this list: send the line "unsubscribe linux-media" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > --- Guennadi Liakhovetski, Ph.D. Freelance Open-Source Software Developer http://www.open-technology.de/ -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
How to express planar formats with mediabus format code?
Hi, I know the title looks crazy, but here is our problem: In our SoC based ISP, the hardware can be divide to several blocks. Some blocks can do color space conversion(raw to YUV interleave/planar), others can do the pixel re-order(interleave/planar/semi-planar conversion, UV planar switch). We use one subdev to describe each of them, then came the problem: How can we express the planar formats with mediabus format code? I understand at beginning, media-bus was designed to describe the data link between camera sensor and camera controller, where sensor is described in subdev. So interleave formats looks good enough at that time. But now as Media-controller is introduced, subdev can describe a much wider range of hardware, which is not limited to camera sensor. So now planar formats are possible to be passed between subdevs. I think the problem we meet can be very common for SoC based ISP solutions, what do you think about it? there are many possible solution for it: 1> change the definition of v4l2_subdev_format::format, use v4l2_format; 2> extend the mediabus format code, add planar format code; 3> use a extra bit to tell the meaning of v4l2_mbus_framefmt::code, is it in mediabus-format or in fourcc Do you have any suggestions? Thanks a lot! -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
