On Wednesday, October 20, 2010 08:41:06 Marek Szyprowski wrote:
> Hello,
>
> As I promissed I continue the development of the VideoBuf2 at Samsung
> until Pawel finds some spare time to help us. This is a third version of
> the framework. Besides the minor bugfixes here and there I've added a
> complete read() callback emulator. This emulator provides 2 types of
> read() operation - 'streaming' and 'one shot'. It is suitable to replace
> both videobuf_read_stream() and videobuf_read_one() methods from the old
> videobuf.
One thing I never understood: what is the point of supporting 'one shot' read
mode? Why not support just streaming? Does anyone know?
Another question: how hard is it to support write mode as well? I think
vb2 should support both. I suspect that once we have a read emulator it isn't
difficult to make a write emulator too.
A last remark: the locking has changed recently in videobuf due to the work
done on eliminating the BKL. It's probably a good idea to incorporate those
changes as well in vb2.
Regards,
Hans
>
> Taking into account the size of the patches and the number of lines I've
> changed, I've decided to keep the Pawel signed-off attribute and
> authorship to correctly credit him. If this is against kernel rules,
> feel free to let me know.
>
> Best regards
> --
> Marek Szyprowski
> Samsung Poland R&D Center
>
>
> Changes since V2:
> =================
> - added read() emulator (see patch #5/7)
> - fixed lack of parentheses in macro definitions (caused side effects
> in some places)
> - added a separate check for VM_READ or VM_WRITE in vb2_mmap()
> - added vb2_is_streaming(), vb2_lock and vb2_unlock inlines
> - updated vivi driver with the new read() emulator
>
> Changes since V1:
> =================
> - removed drv_lock, added start_streaming and stop_streaming callbacks
>
>
> Here is the original Videobuf2 introduction prepared by Pawel:
> =======================================================================
>
> These patches add a new driver framework for Video for Linux 2 driver
> - Videobuf2.
>
> Videobuf2 is intended as a replacement for videobuf, the current driver
> framework, which will be referred to as "videobuf1" for the remainder
> of this document.
>
> ================================
> What is videobuf2?
> ================================
> Videobuf2 is a Video for Linux 2 API-compatible driver framework for
> multimedia devices. It acts as an intermediate layer between userspace
> applications and device drivers. It also provides low-level, modular
> memory management functions for drivers.
>
> Videobuf2 eases driver development, reduces drivers' code size and aids in
> proper and consistent implementation of V4L2 API in drivers.
>
> Videobuf2 memory management backend is fully modular. This allows custom
> memory management routines for devices and platforms with non-standard
> memory management requirements to be plugged in, without changing the
> high-level buffer management functions and API.
>
> The framework provides:
> - implementations of streaming I/O V4L2 ioctls and file operations
> - high-level video buffer, video queue and state management functions
> - video buffer memory allocation and management
>
> ================================
> Why a new framework?
> ================================
> There have been many discussions in the V4L2 community about the feasibility
> of writing a new framework, as opposed to fixing the existing one. It has been
> agreed though that:
> - videobuf1 has major flaws and an attempt to fix it would end up in rewriting
> most of the code
> - many drivers depend on videobuf1 and since the changes would be major,
> an effort to adapt and test them all would not be realistically possible
>
> Due to the problems with videobuf most new drivers cannot use it. This leads
> to code replication and overcomplicated drivers.
>
> ================================
> What is wrong with videobuf1?
> ================================
> There are many problems with the current videobuf implementation. During a
> V4L2
> mini-summit in Helsinki in June 2010, two presentations were delivered
> on this topic:
> - Laurent Pinchart "videobuf - the good, the bad and the ugly"
> http://linuxtv.org/downloads/presentations/summit_jun_2010/20100614-v4l2_summit-videobuf.pdf
> - Pawel Osciak "Future of the videobuf framework"
> http://linuxtv.org/downloads/presentations/summit_jun_2010/Videobuf_Helsinki_June2010.pdf
>
> These presentations highlighted many problems with videobuf. The most
> prominent
> include:
>
> - V4L2 API violations and wrong memory management design
> - it is impossible to pause streaming (buffers are freed on streamoff)
> - VIDIOC_REQBUFS(0) does not free memory
> - it is impossible to reallocate memory with VIDIOC_REQBUFS
> - video memory is allocated on mmap, qbuf or even on page fault,
> freed on unmap, streamoff or explicitly by drivers
> - per-buffer waitqueues
> - not extensible enough and thus not ready for new platforms and uses,
> especially considering embedded multimedia devices
> - very hard to add new memory handling routines and custom memory allocators
> - no or poor support for handling cache coherency, IOMMUs,
> - poor flexibility - only one do-it-all function for handling memory
> pinning,
> cache, sg-list creation, etc...
> - unused fields, code duplication, vague/inconsistent naming, obscure usage in
> some places...
>
> Many driver authors expressed their frustration with videobuf. Developers
> acknowledge its merits and would like to use it, but due mostly to its
> inflexible memory allocation schemes they are unable to do so.
>
> ================================
> Main goals of the redesign
> ================================
> - correct V4L2 API implementation, fixing videobuf1 problems and shortcomings
> - full separation between queue management and memory management
> - fully flexible, pluggable memory allocators and memory handling routines
> - more specialized driver callbacks, called at different points
> - support for new V4L2 API extensions, such as multi-planar video buffers
>
> ================================
> Driver callbacks
> ================================
> Driver callbacks have been redesigned for symmetry:
> - buf_init - called once, after memory is allocated or after a new USERPTR
> buffer is queued; can be used e.g. to pin pages, verify contiguity, set up
> IOMMU mappings, etc.
> - buf_prepare - called on each QBUF; can be used e.g. for cache sync, copying
> to bounce buffers, etc.
> - buf_finish - called on each DQBUF; can be used e.g. for cache sync, copying
> back from bounce buffers, etc.
> - buf_cleanup - called before freeing/releasing memory; can be used e.g. for
> unmapping memory, etc.
>
> The remaining driver callbacks have been slightly redesigned:
> - queue_negotiate - now incorporates multi-planar extensions; drivers return
> required number of buffers and planes per buffer
> - plane_setup - drivers return plane sizes
> Those two callbacks replace the old buf_setup.
>
> - buf_queue - basically stays the same
>
> ================================
> Memory allocators and handling
> ================================
> Memory handling has been designed to allow more customization than in the
> original videobuf. For this memory allocation ops have been slightly
> redesigned,
> and have become fully replaceable and an allocator context struct have been
> introduced.
>
> Allocator context is intended to provide memory operations to videobuf and
> also
> for storing allocator private data, if required, although simpler allocators
> do not have to use this feature. Private data can be added by embedding the
> context struct inside their own structures:
>
> struct vb2_alloc_ctx {
> const struct vb2_mem_ops *mem_ops;
> };
>
> struct vb2_foo_alloc_conf {
> struct vb2_alloc_ctx alloc_ctx;
> /* Allocator private data here */
> };
>
> Moreover, a buffer context structure concept has been introduced. Allocators
> return their own, custom, per-buffer structures on every allocation. This
> structure is then used as a "cookie" and passed to other memory handling
> methods called for its corresponding buffer.
>
> Memory operations, stored in the allocator context, can be replaced if
> needed by drivers on a per-function basis and functions from other allocators
> or drivers can be reused as well. A full list with documentation can be found
> in the videobuf2-core.h file.
>
> It is also possible, although not required, to assign different contexts per
> plane. This may be useful for drivers that need to use different memory types
> for different planes. An example may be a driver that stores video data in the
> first plane, which has to be allocated from a device-accessible memory area,
> and metadata in the second plane, which does not have to be stored in
> a device-accessible memory.
>
> An good example of integrating a more advanced allocator, the recently
> discussed
> on this list CMA (contiguous memory allocator), can be found in
> videobuf2-cma.*.
>
> ================================
> Other changes
> ================================
> The changes described above are the main changes in videobuf2. Most of the
> core
> API has remained the same or very similar, although its implementation has
> been
> fully rewritten. Some more visible changes include:
>
> - Memory is now properly allocated on REQBUFS and can be freed and reallocated
> there as well.
> - It is now possible to pause and resume streaming with streamon/streamoff,
> without freeing the buffers.
> - V4L2 API-related, userspace-visible metadata, such as inputs, timestamps,
> etc.
> are no longer stored in videobuf buffer structure, but in an actual
> v4l2_buffer struct (idea borrowed from Laurent Pinchart). I felt that driver
> authors would prefer to use V4L2 API-based structures of videobuf custom
> structures where possible. It also eases copying v4l2_buffer-related data
> from/to userspace.
> - Buffers do not include waitqueues anymore. One, global, per-queue waitqueue
> for done buffers has been introduced instead. Per-buffer waitqueues were not
> very useful and introduced additional complications in code.
> With this, drivers have gained the ability of dequeuing buffers out-of-order
> as well.
> - Buffer states are not handled jointly by both videobuf and driver anymore,
> I felt it was not required and confusing for driver authors
> - Some fields that were less useful have been removed and naming of others
> have been changed to better reflect their function.
> - Other then reqbufs, ioctl implementations have remained almost the same
> and behave in the same way,
>
>
> Please see documentation in videobuf2-core.c and videobuf2-core.h for more
> details and the patch porting vivi to videobuf2 for how to port existing
> drivers to videobuf2.
>
> This is a preliminary version intended for review, but has already been tested
> for multiple drivers and different memory handling implementations. DMA-SG
> implementation is not included yet.
>
> The CMA allocator patches are attached for reference, to show how a more
> complicated allocator can be integrated into the framework.
>
> Any comments will be very much appreciated!
>
> Best regards,
> Pawel Osciak
> Linux Platform Group
> Samsung Poland R&D Center
>
> =======================================================================
>
> Patch summary:
>
> Pawel Osciak (6):
> v4l: add videobuf2 Video for Linux 2 driver framework
> v4l: videobuf2: add generic memory handling routines
> v4l: videobuf2: add vmalloc allocator
> v4l: videobuf2: add DMA coherent allocator
> v4l: vivi: port to videobuf2
> v4l: videobuf2: add CMA allocator
>
> Marek Szyprowski (1):
> v4l: videobuf2: add read() emulator
>
> drivers/media/video/Kconfig | 23 +-
> drivers/media/video/Makefile | 6 +
> drivers/media/video/videobuf2-cma.c | 250 ++++
> drivers/media/video/videobuf2-core.c | 1726
> ++++++++++++++++++++++++++
> drivers/media/video/videobuf2-dma-coherent.c | 208 ++++
> drivers/media/video/videobuf2-memops.c | 199 +++
> drivers/media/video/videobuf2-vmalloc.c | 177 +++
> drivers/media/video/vivi.c | 348 +++---
> include/media/videobuf2-cma.h | 25 +
> include/media/videobuf2-core.h | 392 ++++++
> include/media/videobuf2-dma-coherent.h | 21 +
> include/media/videobuf2-memops.h | 31 +
> include/media/videobuf2-vmalloc.h | 16 +
> 13 files changed, 3258 insertions(+), 164 deletions(-)
> create mode 100644 drivers/media/video/videobuf2-cma.c
> create mode 100644 drivers/media/video/videobuf2-core.c
> create mode 100644 drivers/media/video/videobuf2-dma-coherent.c
> create mode 100644 drivers/media/video/videobuf2-memops.c
> create mode 100644 drivers/media/video/videobuf2-vmalloc.c
> create mode 100644 include/media/videobuf2-cma.h
> create mode 100644 include/media/videobuf2-core.h
> create mode 100644 include/media/videobuf2-dma-coherent.h
> create mode 100644 include/media/videobuf2-memops.h
> create mode 100644 include/media/videobuf2-vmalloc.h
>
>
--
Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
