Hi Vaibhav, On Mon, 2009-10-05 at 23:57 +0530, Hiremath, Vaibhav wrote: > > -----Original Message----- > > From: Marek Szyprowski [mailto:m.szyprow...@samsung.com] > > Sent: Monday, October 05, 2009 7:26 PM > > To: Hiremath, Vaibhav; 'Ivan T. Ivanov'; linux-media@vger.kernel.org > > Cc: kyungmin.p...@samsung.com; Tomasz Fujak; Pawel Osciak; Marek > > Szyprowski > > Subject: RE: Mem2Mem V4L2 devices [RFC] > > > > Hello, > > > > On Monday, October 05, 2009 7:59 AM Hiremath, Vaibhav wrote: > > > > > -----Original Message----- > > > From: linux-media-ow...@vger.kernel.org [mailto:linux-media- > > ow...@vger.kernel.org] On Behalf Of > > > Hiremath, Vaibhav > > > Sent: Monday, October 05, 2009 7:59 AM > > > To: Ivan T. Ivanov; Marek Szyprowski > > > Cc: linux-media@vger.kernel.org; kyungmin.p...@samsung.com; Tomasz > > Fujak; Pawel Osciak > > > Subject: RE: Mem2Mem V4L2 devices [RFC] > > > > > > > > > > -----Original Message----- > > > > From: linux-media-ow...@vger.kernel.org [mailto:linux-media- > > > > ow...@vger.kernel.org] On Behalf Of Ivan T. Ivanov > > > > Sent: Friday, October 02, 2009 9:55 PM > > > > To: Marek Szyprowski > > > > Cc: linux-media@vger.kernel.org; kyungmin.p...@samsung.com; > > Tomasz > > > > Fujak; Pawel Osciak > > > > Subject: Re: Mem2Mem V4L2 devices [RFC] > > > > > > > > > > > > Hi Marek, > > > > > > > > > > > > On Fri, 2009-10-02 at 13:45 +0200, Marek Szyprowski wrote: > > > > > Hello, > > > > > > > > <snip> > > > > > > > > image format and size, while the existing v4l2 ioctls would > > only > > > > refer > > > > > to the output buffer. Frankly speaking, we don't like this > > idea. > > > > > > > > I think that is not unusual one video device to define that it > > can > > > > support at the same time input and output operation. > > > > > > > > Lets take as example resizer device. it is always possible that > > it > > > > inform user space application that > > > > > > > > struct v4l2_capability.capabilities == > > > > (V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT) > > > > > > > > User can issue S_FMT ioctl supplying > > > > > > > > struct v4l2_format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE > > > > .pix = width x height > > > > > > > > which will instruct this device to prepare its output for this > > > > resolution. after that user can issue S_FMT ioctl supplying > > > > > > > > struct v4l2_format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT > > > > .pix = width x height > > > > > > > > using only these ioctls should be enough to device driver > > > > to know down/up scale factor required. > > > > > > > > regarding color space struct v4l2_pix_format have field > > > > 'pixelformat' > > > > which can be used to define input and output buffers content. > > > > so using only existing ioctl's user can have working resizer > > device. > > > > > > > > also please note that there is VIDIOC_S_CROP which can add > > > > additional > > > > flexibility of adding cropping on input or output. > > > > > > > [Hiremath, Vaibhav] I think this makes more sense in capture > > pipeline, for example, > > > > > > Sensor/decoder -> previewer -> resizer -> /dev/videoX > > > > > > > I don't get this. In strictly capture pipeline we will get one video > > node anyway. > > > > However the question is how we should support a bit more complicated > > pipeline. > > > > Just consider a resizer module and the pipeline: > > > > sensor/decoder -[bus]-> previewer -> [memory] -> resizer -> [memory] > > > [Hiremath, Vaibhav] For me this is not single pipeline, it has two separate > links - > > 1) sensor/decoder -[bus]-> previewer -> [memory] > > 2) [memory] -> resizer -> [memory] > > > > ([bus] means some kind of internal bus that is completely > > interdependent from the system memory) > > > > Mapping to video nodes is not so trivial. In fact this pipeline > > consist of 2 independent (sub)pipelines connected by user space > > application: > > > > sensor/decoder -[bus]-> previewer -> [memory] -[user application]-> > > [memory] -> resizer -> [memory] > > > > For further analysis it should be cut into 2 separate pipelines: > > > > a. sensor/decoder -[bus]-> previewer -> [memory] > > b. [memory] -> resizer -> [memory] > > > [Hiremath, Vaibhav] Correct, I wouldn't call them as sub-pipeline. > Application is linking them, so from driver point of view they are completely > separate. > > > Again, mapping the first subpipeline is trivial: > > > > sensor/decoder -[bus]-> previewer -> /dev/video0 > > > [Hiremath, Vaibhav] Correct, it is separate streaming device. > > > But the last, can be mapped either as: > > > > /dev/video1 -> resizer -> /dev/video1 > > (one video node approach) > > > [Hiremath, Vaibhav] Please go through my last response where I have mentioned > about buffer queuing constraints with this approach. > > > or > > > > /dev/video1 -> resizer -> /dev/video2 > > (2 video nodes approach). > > > > > > So at the end the pipeline would look like this: > > > > sensor/decoder -[bus]-> previewer -> /dev/video0 -[user > > application]-> /dev/video1 -> resizer -> /dev/video2 > > > > or > > > > sensor/decoder -[bus]-> previewer -> /dev/video0 -[user > > application]-> /dev/video1 -> resizer -> /dev/video1 > > > > > > last thing which should be done is to QBUF 2 buffers and call > > > > STREAMON. > > > > > > > [Hiremath, Vaibhav] IMO, this implementation is not streaming > > model, we are trying to fit mem-to-mem > > > forcefully to streaming. > > > > Why this does not fit streaming? I see no problems with streaming > > over mem2mem device with only one video node. You just queue input > > and output buffers (they are distinguished by 'type' parameter) on > > the same video node. > > > [Hiremath, Vaibhav] Do we create separate queue of buffers based on type? I > think we don't. > > App1 App2 App3 ... AppN > | | | | | > ----------------------------------------------- > | > /dev/video0 > | > Resizer Driver
why not? they can be per file handler input/output queue. and we can do time sharing use of resizer driver like Marek suggests. > > Everyone will be doing streamon, and in normal use case every application > must be getting buffers from another module (another driver, codecs, DSP, > etc...) in multiple streams, 0, 1,2,3,4....N > > Every application will start streaming with (mostly) fixed scaling factor > which mostly never changes. This one video node approach is possible only > with constraint that, the application will always queue only 2 buffers with > one CAPTURE and one with OUTPUT type. i don't see how 2 device node approach can help with this case. even in "normal" video capture device you should stop streaming when change buffer sizes. > He has to wait till first/second gets finished, you can't queue multiple > buffers (input and output) simultaneously. actually this should be possible. iivanov > > I do agree here with you that we need to investigate on whether we really > have such use-case. Does it make sense to put such constraint on application? > What is the impact? Again in case of down-scaling, application may want to > use same buffer as input, which is easily possible with single node approach. > > Thanks, > Vaibhav > > > > We have to put some constraints - > > > > > > - Driver will treat index 0 as input always, irrespective of > > number of buffers queued. > > > - Or, application should not queue more that 2 buffers. > > > - Multi-channel use-case???? > > > > > > I think we have to have 2 device nodes which are capable of > > streaming multiple buffers, both are > > > queuing the buffers. > > > > In one video node approach there can be 2 buffer queues in one video > > node, for input and output respectively. > > > > > The constraint would be the buffers must be mapped one-to-one. > > > > Right, each queued input buffer must have corresponding output > > buffer. > > > > Best regards > > -- > > Marek Szyprowski > > Samsung Poland R&D Center > > > > > -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
