On Tue, Dec 06, 2011 at 02:20:32PM -0200, Mauro Carvalho Chehab wrote: > >>>>>1) After all the frames with the old resolution are dequeued a buffer > >>>>>with the > >>>>>following flags V4L2_BUF_FLAG_ERROR | V4L2_BUF_FLAG_WRONGFORMAT is > >>>>>returned. > >>>>>2) To acknowledge the resolution change the application should > >>>>>STREAMOFF, check > >>>>>what has changed and then STREAMON. > >>>> > >>>>I don't think this is needed, if the buffer size is enough to support the > >>>>new > >>>>format. > >>> > >>>Sometimes not, but sometimes there are buffer line alignment requirements > >>>which must be communicated to the driver using S_FMT. If the frames are > >>>decoded using a driver-decided format, it might be impossible to actually > >>>use these decoded frames. > >>> > >>>That's why there's streamoff and streamon. > >> > >>Don't get me wrong. What I'm saying is that there are valid cases where > >>there's no need to streamoff/streamon. What I'm saying is that, when > >>there's no need to do it, just don't rise the V4L2_BUF_FLAG_ERROR flag. > >>The V4L2_BUF_FLAG_FORMATCHANGED still makes sense. > > > >I try not to. :) > > > >The issue is that it's the user space which knows how it is going to use the > >buffers it dequeues from a device. It's not possible for the driver know > >that --- unless explicitly told by the user. This could be a new flag, if we > >need differing behaviour as it seems here. > > > >The user may queue the same memory buffers to another device, which I > >consider to be a common use case in embedded systems. The line alignment > >requirements of the two (or more) devices often are not the same. > > So what? > > You're probably not referring to queuing the same buffer at the same time > to two separate devices. If they're being queued at different times, I can't > see any issue.
The buffers are used in separate devices at different times. If the decoder has a line alignment requirement of 32, but the display output device has 128, chances are good that the new bytesperline won't be suitable to be displayed. > >>>>Btw, a few drivers (bttv comes into my mind) properly handles format > >>>>changes. > >>>>This were there in order to support a bad behavior found on a few V4L1 > >>>>applications, > >>>>where the applications were first calling STREAMON and then setting the > >>>>buffer. > >>> > >>>The buffers do not have a format, the video device queue has. If the format > >>>changes during streaming it is impossible to find that out using the > >>>current > >>>API. > >> > >>Yes, but extending it to proper support it is the scope of this RFC. Yet, > >>several > >>drivers allow to resize the "format" (e. g. the resolution of the image) > >>without > >>streamon/streamoff, via an explicit call to S_FMT. > >> > >>So, whatever change at the API is done, it should keep supporting format > >>changes > >>(in practice, resolution changes) without the need of re-initializing the > >>DMA engine, > >>of course when such change won't break the capability for userspace to > >>decode > >>the new frames. > > > >Stopping and starting the queue does not involve additional penalty: a > >memory-to-memory device processes buffers one at a time, and most of the > >hardware stops between the buffers in any case before being started by the > >software again. The buffers are not affected either; they stay mapped and > >pinned to memory. > > This is true only on memory-to-memory devices (although the V4L2 spec is > incomplete > with regards to this specific type of device). That's true. Would this be a candidate for a new V4L2 profile? > >>>>If I'm not mistaken, the old vlc V4L1 driver used to do that. > >>>> > >>>>What bttv used to do is to allocate a buffer big enough to support the > >>>>max resolution. > >>>>So, any format changes (size increase or decrease) would fit into the > >>>>allocated > >>>>buffers. > >>>> > >>>>Depending on how applications want to handle format changes, and how big > >>>>is the > >>>>amount of memory on the system, a similar approach may be done with > >>>>CREATE_BUFFERS: > >>>>just allocate enough space for the maximum supported resolution for that > >>>>stream, > >>>>and let the resolution changes, as required. > >>> > >>>I'm not fully certain it is always possible to find out the largest stream > >>>resolution. I'd like an answer from someone knowing more about video codecs > >>>than I do. > >> > >>When the input or output is some hardware device, there is a maximum > >>resolution > >>(sensor resolution, monitor resolution, pixel sampling rate, etc). A pure > >>DSP block > >>doesn't have it, but, anyway, it should be up to the userspace application > >>to decide > >>if it wants to over-allocate a buffer, in order to cover a scenario where > >>the > >>resolution may change or not. > > > >In this case, allocating bigger buffers than necessary is a big > >disadvantage, and forcing the application to requeue the compressed data > >to the OUTPUT queue doesn't sound very appealing either. This could involve > >the application having to queue several buffers which already have been > >decoded earlier on. Also figuring out how many is a task for the decoder, > >not the application. > > Sorry, but it seems I missed what scenario you're considering... M2M, Output, > Capture? All? > some specific pipeline? In this caase it's memory-to-memory device. ... > >>>>>3) The application should check with G_FMT how did the format change and > >>>>>the > >>>>>V4L2_CID_MIN_BUFFERS_FOR_CAPTURE control to check how many buffers are > >>>>>required. > >>>>>4) Now it is necessary to resume processing: > >>>>> A. If there is no need to change the size of buffers or their number > >>>>> the > >>>>>application needs only to STREAMON. > >>>>> B. If it is necessary to allocate bigger buffers the application > >>>>> should use > >>>>>CREATE_BUFS to allocate new buffers, the old should be left until the > >>>>>application is finished and frees them with the DESTROY_BUFS call. S_FMT > >>>>>should be used to adjust the new format (if necessary and possible in > >>>>>HW). > >>>> > >>>>If the application already cleaned the DMA transfers with STREAMOFF, it > >>>>can > >>>>also just re-queue the buffers with REQBUFS, e. g. vb2 should be smart > >>>>enough to > >>>>accept both ways to allocate buffers. > >>> > >>>No need to REQBUFS after streaming has been stopped. STREAMOFF won't harm > >>>the buffers in any way anymore --- as it did in videobuf1. > >> > >>OK, but userspace applications may use REQBUFS instead of CREATE_BUFFERS, > >>as REQBUFS > >>is part of the API. > > > >True. But new REQBUFS, as far as I remember, re-allocates the buffers, > >causing a glitch to the stream. That's why CREATE_BUFS was proposed. > > True, but also STREAMOFF will glitch the stream, in the general case (capture > or output > devices). Right. For those buffers to be affected, they'd have to be mmap buffers. The buffers are displayed elsewhere using different APIs. With CREATE_BUFS it's possible to keep the old buffers around as long as they are needed. > >>>That's a good point. It's more related to changes in stream properties --- > >>>the frame rate of the stream could change, too. That might be when you > >>>could > >>>like to have more buffers in the queue. I don't think this is critical > >>>either. > >>> > >>>This could also depend on the properties of the codec. Again, I'd wish a > >>>comment from someone who knows codecs well. Some codecs need to be able to > >>>access buffers which have already been decoded to decode more buffers. Key > >>>frames, simply. > >> > >>Ok, but let's not add unneeded things at the API if you're not sure. If we > >>have > >>such need for a given hardware, then add it. Otherwise, keep it simple. > > > >This is not so much dependent on hardware but on the standards which the > >cdoecs implement. > > Could you please elaborate it? On what scenario this is needed? This is a property of the stream, not a property of the decoder. To reconstruct each frame, a part of the stream is required and already decoded frames may be used to accelerate the decoding. What those parts are. depends on the codec, not a particular implementation. Anyone with more knowledge of codecs than myself might be able to give a concrete example. Sebastian? -- Sakari Ailus e-mail: [email protected] jabber/XMPP/Gmail: [email protected] -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
