Re: [RFCv11 PATCH 24/29] Documentation: v4l: document request API

2018-04-12 Thread Tomasz Figa 
Hi Hans,

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil  wrote:
[snip]
> @@ -514,6 +517,11 @@ Buffer Flags
>  streaming may continue as normal and the buffer may be reused
>  normally. Drivers set this flag when the ``VIDIOC_DQBUF`` ioctl is
>  called.
> +* .. _`V4L2-BUF-FLAG-IN-REQUEST`:
> +
> +  - ``V4L2_BUF_FLAG_IN_REQUEST``
> +  - 0x0080
> +  - This buffer is part of a request the hasn't been queued yet.

typo: s/the hasn't/that hasn't/

[snip]
> +Usage
> +=
> +
> +The Request API is used on top of standard media controller and V4L2
calls,
> +which are augmented with an extra ``request_fd`` parameter. Request
themselves
> +are allocated from either a supporting V4L2 device node, or a supporting
media
> +controller node. The origin of requests determine their scope: requests
> +allocated from a V4L2 device node can only act on that device, whereas
requests
> +allocated from a media controller node can control the whole pipeline of
the
> +controller.

Current patch set doesn't support allocating requests from V4L2 device
nodes.

> +
> +Request Allocation
> +--
> +
> +User-space allocates requests using the ``VIDIOC_NEW_REQUEST`` (for V4L2
device

VIDIOC_NEW_REQUEST doesn't exist in current patch set. (We should add this
bit to documentation, when we add respective functionality.)

> +requests) or ``MEDIA_IOC_NEW_REQUEST`` (for media controller requests)
on an

s/MEDIA_IOC_NEW_REQUEST/MEDIA_IOC_REQUEST_ALLOC/

> +opened device or media node. This returns a file descriptor representing
the
> +request. Typically, several such requests will be allocated.
> +
> +Request Preparation
> +---
> +
> +Standard V4L2 ioctls can then receive a request file descriptor to
express the
> +fact that the ioctl is part of said request, and is not to be applied
> +immediately. V4L2 ioctls supporting this are
:c:func:`VIDIOC_S_EXT_CTRLS` and
> +:c:func:`VIDIOC_QBUF`. Controls set with a request parameter are stored
instead
> +of being immediately applied, and queued buffers not enter the regular
buffer
> +queue until the request is submitted. Only one buffer can be queued to a
given
> +queue for a given request.
> +
> +Request Submission
> +--
> +
> +Once the parameters and buffers of the request are specified, it can be
> +submitted by calling the ``MEDIA_REQUEST_IOC_SUBMIT`` ioctl on the
request FD.

s/MEDIA_REQUEST_IOC_SUBMIT/MEDIA_REQUEST_IOC_QUEUE/

> +This will make the buffers associated to the request available to their
driver,
> +which can then apply the saved controls as buffers are processed. A
submitted
> +request cannot be modified anymore.
> +
> +If several devices are part of the request, individual drivers may
synchronize
> +so the requested pipeline's topology is applied before the buffers are
> +processed. This is at the discretion of media controller drivers and is
not a
> +requirement.
> +
> +Buffers queued without an associated request after a request-bound
buffer will
> +be processed using the state of the hardware at the time of the request
> +completion. All the same, controls set without a request are applied
> +immediately, regardless of whether a request is in use or not.
> +
> +User-space can ``poll()`` a request FD in order to wait until the request
> +completes. A request is considered complete once all its associated
buffers are
> +available for dequeing. Note that user-space does not need to wait for
the
> +request to complete to dequeue its buffers: buffers that are available
halfway
> +through a request can be dequeued independently of the request's state.
> +
> +A completed request includes the state of all devices that had queued
buffers

Why storing state is gated on queued buffers? Perhaps more like "all
devices, which had their state affected by the request"?

> +associated with it at the time of the request completion. User-space can
query
> +that state by calling :c:func:`VIDIOC_G_EXT_CTRLS` with the request FD.
> +
> +Recycling and Destruction
> +-
> +
> +Finally, completed request can either be discarded or be reused. Calling
> +``close()`` on a request FD will make that FD unusable, freeing the
request if
> +it is not referenced elsewhere. The ``MEDIA_REQUEST_IOC_SUBMIT`` ioctl
will
> +clear a request's state and make it available again. No state is
retained by
> +this operation: the request is as if it had just been allocated.
> +
> +Example for a M2M Device
> +
> +
> +M2M devices are single-node V4L2 devices providing one OUTPUT queue (for
> +user-space
> +to provide input buffers) and one CAPTURE queue (to retrieve processed
data).
> +They are perfectly symetric, i.e. one buffer of input will produce one
buffer of
> +output. These devices are commonly used for frame processors or stateless
> +codecs.
> +
> +In this use-case, the request API can be used to associate specific
controls to
> +be applied by the driver before processing an OUTPUT

Re: [RFCv11 PATCH 24/29] Documentation: v4l: document request API

2018-04-10 Thread Mauro Carvalho Chehab 
Em Mon,  9 Apr 2018 16:20:21 +0200
Hans Verkuil  escreveu:

> From: Alexandre Courbot 
> 
> Document the request API for V4L2 devices, and amend the documentation
> of system calls influenced by it.

As said before, I would be expecting this patch to be split,
with the initial contents coming together with patch 01/29,
adding things incrementally together with the code.

I can't really analyze it the way it is, as it is hard to identify
if all changes at the V4L2 API were covered here.

> 
> Signed-off-by: Alexandre Courbot 
> ---
>  Documentation/media/uapi/v4l/buffer.rst|  19 +-
>  Documentation/media/uapi/v4l/common.rst|   1 +
>  Documentation/media/uapi/v4l/request-api.rst   | 199 
> +
>  Documentation/media/uapi/v4l/user-func.rst |   1 +
>  .../media/uapi/v4l/vidioc-g-ext-ctrls.rst  |  22 ++-
>  .../media/uapi/v4l/vidioc-new-request.rst  |  64 +++
>  Documentation/media/uapi/v4l/vidioc-qbuf.rst   |   8 +
>  7 files changed, 308 insertions(+), 6 deletions(-)
>  create mode 100644 Documentation/media/uapi/v4l/request-api.rst
>  create mode 100644 Documentation/media/uapi/v4l/vidioc-new-request.rst
> 
> diff --git a/Documentation/media/uapi/v4l/buffer.rst 
> b/Documentation/media/uapi/v4l/buffer.rst
> index e2c85ddc990b..e23eae12905c 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -306,10 +306,13 @@ struct v4l2_buffer
>- A place holder for future extensions. Drivers and applications
>   must set this to 0.
>  * - __u32
> -  - ``reserved``
> +  - ``request_fd``

As I said before, on my comments to patch 02/29, u32 doesn't seem big enough.
The get_unused_fd_flags() function returns an integer, with can have up to
63 bits + sign.

>-
> -  - A place holder for future extensions. Drivers and applications
> - must set this to 0.
> +  - The file descriptor of the request to queue the buffer to. If 
> specified
> +and flag ``V4L2_BUF_FLAG_REQUEST_FD`` is set, then the buffer will be
> + queued to that request. This is set by the user when calling
> + :ref:`VIDIOC_QBUF` and :ref:`VIDIOC_PREPARE_BUF` and ignored by other
> + ioctls.
>  
>  
>  
> @@ -514,6 +517,11 @@ Buffer Flags
>   streaming may continue as normal and the buffer may be reused
>   normally. Drivers set this flag when the ``VIDIOC_DQBUF`` ioctl is
>   called.
> +* .. _`V4L2-BUF-FLAG-IN-REQUEST`:
> +
> +  - ``V4L2_BUF_FLAG_IN_REQUEST``
> +  - 0x0080
> +  - This buffer is part of a request the hasn't been queued yet.
>  * .. _`V4L2-BUF-FLAG-KEYFRAME`:
>  
>- ``V4L2_BUF_FLAG_KEYFRAME``
> @@ -589,6 +597,11 @@ Buffer Flags
>   the format. Any Any subsequent call to the
>   :ref:`VIDIOC_DQBUF ` ioctl will not block anymore,
>   but return an ``EPIPE`` error code.
> +* .. _`V4L2-BUF-FLAG-REQUEST-FD`:
> +
> +  - ``V4L2_BUF_FLAG_REQUEST_FD``
> +  - 0x0080
> +  - The ``request_fd`` field contains a valid file descriptor.
>  * .. _`V4L2-BUF-FLAG-TIMESTAMP-MASK`:
>  
>- ``V4L2_BUF_FLAG_TIMESTAMP_MASK``
> diff --git a/Documentation/media/uapi/v4l/common.rst 
> b/Documentation/media/uapi/v4l/common.rst
> index 13f2ed3fc5a6..a4aa0059d45a 100644
> --- a/Documentation/media/uapi/v4l/common.rst
> +++ b/Documentation/media/uapi/v4l/common.rst
> @@ -44,3 +44,4 @@ applicable to all devices.
>  crop
>  selection-api
>  streaming-par
> +request-api
> diff --git a/Documentation/media/uapi/v4l/request-api.rst 
> b/Documentation/media/uapi/v4l/request-api.rst
> new file mode 100644
> index ..0c1f2896e197
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/request-api.rst
> @@ -0,0 +1,199 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _media-request-api:
> +
> +Request API
> +===
> +
> +The Request API has been designed to allow V4L2 to deal with requirements of
> +modern devices (stateless codecs, MIPI cameras, ...) and APIs (Android Codec
> +v2). One such requirement is the ability for devices belonging to the same
> +pipeline to reconfigure and collaborate closely on a per-frame basis. 
> Another is

Another one

> +efficient support of stateless codecs, which need per-frame controls to be 
> set
> +asynchronously in order to be efficiently used.
> +
> +Supporting these features without the Request API is possible but terribly
> +inefficient: user-space would have to flush all activity on the media 
> pipeline,
> +reconfigure it for the next frame, queue the buffers to be processed with 
> that
> +configuration, and wait until they are all available for dequeing before
> +considering the next frame. This defeats the purpose of having buffer queues
> +since in practice only one buffer would be queued at a time.
> +
> +The Request API allows a specific configuration of the pipeline (media
> +controller topology + controls for each device) to be

[RFCv11 PATCH 24/29] Documentation: v4l: document request API

2018-04-09 Thread Hans Verkuil 
From: Alexandre Courbot 

Document the request API for V4L2 devices, and amend the documentation
of system calls influenced by it.

Signed-off-by: Alexandre Courbot 
---
 Documentation/media/uapi/v4l/buffer.rst|  19 +-
 Documentation/media/uapi/v4l/common.rst|   1 +
 Documentation/media/uapi/v4l/request-api.rst   | 199 +
 Documentation/media/uapi/v4l/user-func.rst |   1 +
 .../media/uapi/v4l/vidioc-g-ext-ctrls.rst  |  22 ++-
 .../media/uapi/v4l/vidioc-new-request.rst  |  64 +++
 Documentation/media/uapi/v4l/vidioc-qbuf.rst   |   8 +
 7 files changed, 308 insertions(+), 6 deletions(-)
 create mode 100644 Documentation/media/uapi/v4l/request-api.rst
 create mode 100644 Documentation/media/uapi/v4l/vidioc-new-request.rst

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index e2c85ddc990b..e23eae12905c 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -306,10 +306,13 @@ struct v4l2_buffer
   - A place holder for future extensions. Drivers and applications
must set this to 0.
 * - __u32
-  - ``reserved``
+  - ``request_fd``
   -
-  - A place holder for future extensions. Drivers and applications
-   must set this to 0.
+  - The file descriptor of the request to queue the buffer to. If specified
+and flag ``V4L2_BUF_FLAG_REQUEST_FD`` is set, then the buffer will be
+   queued to that request. This is set by the user when calling
+   :ref:`VIDIOC_QBUF` and :ref:`VIDIOC_PREPARE_BUF` and ignored by other
+   ioctls.
 
 
 
@@ -514,6 +517,11 @@ Buffer Flags
streaming may continue as normal and the buffer may be reused
normally. Drivers set this flag when the ``VIDIOC_DQBUF`` ioctl is
called.
+* .. _`V4L2-BUF-FLAG-IN-REQUEST`:
+
+  - ``V4L2_BUF_FLAG_IN_REQUEST``
+  - 0x0080
+  - This buffer is part of a request the hasn't been queued yet.
 * .. _`V4L2-BUF-FLAG-KEYFRAME`:
 
   - ``V4L2_BUF_FLAG_KEYFRAME``
@@ -589,6 +597,11 @@ Buffer Flags
the format. Any Any subsequent call to the
:ref:`VIDIOC_DQBUF ` ioctl will not block anymore,
but return an ``EPIPE`` error code.
+* .. _`V4L2-BUF-FLAG-REQUEST-FD`:
+
+  - ``V4L2_BUF_FLAG_REQUEST_FD``
+  - 0x0080
+  - The ``request_fd`` field contains a valid file descriptor.
 * .. _`V4L2-BUF-FLAG-TIMESTAMP-MASK`:
 
   - ``V4L2_BUF_FLAG_TIMESTAMP_MASK``
diff --git a/Documentation/media/uapi/v4l/common.rst 
b/Documentation/media/uapi/v4l/common.rst
index 13f2ed3fc5a6..a4aa0059d45a 100644
--- a/Documentation/media/uapi/v4l/common.rst
+++ b/Documentation/media/uapi/v4l/common.rst
@@ -44,3 +44,4 @@ applicable to all devices.
 crop
 selection-api
 streaming-par
+request-api
diff --git a/Documentation/media/uapi/v4l/request-api.rst 
b/Documentation/media/uapi/v4l/request-api.rst
new file mode 100644
index ..0c1f2896e197
--- /dev/null
+++ b/Documentation/media/uapi/v4l/request-api.rst
@@ -0,0 +1,199 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _media-request-api:
+
+Request API
+===
+
+The Request API has been designed to allow V4L2 to deal with requirements of
+modern devices (stateless codecs, MIPI cameras, ...) and APIs (Android Codec
+v2). One such requirement is the ability for devices belonging to the same
+pipeline to reconfigure and collaborate closely on a per-frame basis. Another 
is
+efficient support of stateless codecs, which need per-frame controls to be set
+asynchronously in order to be efficiently used.
+
+Supporting these features without the Request API is possible but terribly
+inefficient: user-space would have to flush all activity on the media pipeline,
+reconfigure it for the next frame, queue the buffers to be processed with that
+configuration, and wait until they are all available for dequeing before
+considering the next frame. This defeats the purpose of having buffer queues
+since in practice only one buffer would be queued at a time.
+
+The Request API allows a specific configuration of the pipeline (media
+controller topology + controls for each device) to be associated with specific
+buffers. The parameters are applied by each participating device as buffers
+associated to a request flow in. This allows user-space to schedule several
+tasks ("requests") with different parameters in advance, knowing that the
+parameters will be applied when needed to get the expected result. Controls
+values at the time of request completion are also available for reading.
+
+Usage
+=
+
+The Request API is used on top of standard media controller and V4L2 calls,
+which are augmented with an extra ``request_fd`` parameter. Request themselves
+are allocated from either a supporting V4L2 device node, or a supporting media
+controller node. The origin of requests determine their scope: requests
+allocat

[PATCH 24/29] Documentation: v4l: document request API

2018-03-28 Thread Hans Verkuil 
From: Alexandre Courbot 

Document the request API for V4L2 devices, and amend the documentation
of system calls influenced by it.

Signed-off-by: Alexandre Courbot 
---
 Documentation/media/uapi/v4l/buffer.rst|  19 +-
 Documentation/media/uapi/v4l/common.rst|   1 +
 Documentation/media/uapi/v4l/request-api.rst   | 199 +
 Documentation/media/uapi/v4l/user-func.rst |   1 +
 .../media/uapi/v4l/vidioc-g-ext-ctrls.rst  |  22 ++-
 .../media/uapi/v4l/vidioc-new-request.rst  |  64 +++
 Documentation/media/uapi/v4l/vidioc-qbuf.rst   |   8 +
 7 files changed, 308 insertions(+), 6 deletions(-)
 create mode 100644 Documentation/media/uapi/v4l/request-api.rst
 create mode 100644 Documentation/media/uapi/v4l/vidioc-new-request.rst

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index e2c85ddc990b..e23eae12905c 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -306,10 +306,13 @@ struct v4l2_buffer
   - A place holder for future extensions. Drivers and applications
must set this to 0.
 * - __u32
-  - ``reserved``
+  - ``request_fd``
   -
-  - A place holder for future extensions. Drivers and applications
-   must set this to 0.
+  - The file descriptor of the request to queue the buffer to. If specified
+and flag ``V4L2_BUF_FLAG_REQUEST_FD`` is set, then the buffer will be
+   queued to that request. This is set by the user when calling
+   :ref:`VIDIOC_QBUF` and :ref:`VIDIOC_PREPARE_BUF` and ignored by other
+   ioctls.
 
 
 
@@ -514,6 +517,11 @@ Buffer Flags
streaming may continue as normal and the buffer may be reused
normally. Drivers set this flag when the ``VIDIOC_DQBUF`` ioctl is
called.
+* .. _`V4L2-BUF-FLAG-IN-REQUEST`:
+
+  - ``V4L2_BUF_FLAG_IN_REQUEST``
+  - 0x0080
+  - This buffer is part of a request the hasn't been queued yet.
 * .. _`V4L2-BUF-FLAG-KEYFRAME`:
 
   - ``V4L2_BUF_FLAG_KEYFRAME``
@@ -589,6 +597,11 @@ Buffer Flags
the format. Any Any subsequent call to the
:ref:`VIDIOC_DQBUF ` ioctl will not block anymore,
but return an ``EPIPE`` error code.
+* .. _`V4L2-BUF-FLAG-REQUEST-FD`:
+
+  - ``V4L2_BUF_FLAG_REQUEST_FD``
+  - 0x0080
+  - The ``request_fd`` field contains a valid file descriptor.
 * .. _`V4L2-BUF-FLAG-TIMESTAMP-MASK`:
 
   - ``V4L2_BUF_FLAG_TIMESTAMP_MASK``
diff --git a/Documentation/media/uapi/v4l/common.rst 
b/Documentation/media/uapi/v4l/common.rst
index 13f2ed3fc5a6..a4aa0059d45a 100644
--- a/Documentation/media/uapi/v4l/common.rst
+++ b/Documentation/media/uapi/v4l/common.rst
@@ -44,3 +44,4 @@ applicable to all devices.
 crop
 selection-api
 streaming-par
+request-api
diff --git a/Documentation/media/uapi/v4l/request-api.rst 
b/Documentation/media/uapi/v4l/request-api.rst
new file mode 100644
index ..0c1f2896e197
--- /dev/null
+++ b/Documentation/media/uapi/v4l/request-api.rst
@@ -0,0 +1,199 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _media-request-api:
+
+Request API
+===
+
+The Request API has been designed to allow V4L2 to deal with requirements of
+modern devices (stateless codecs, MIPI cameras, ...) and APIs (Android Codec
+v2). One such requirement is the ability for devices belonging to the same
+pipeline to reconfigure and collaborate closely on a per-frame basis. Another 
is
+efficient support of stateless codecs, which need per-frame controls to be set
+asynchronously in order to be efficiently used.
+
+Supporting these features without the Request API is possible but terribly
+inefficient: user-space would have to flush all activity on the media pipeline,
+reconfigure it for the next frame, queue the buffers to be processed with that
+configuration, and wait until they are all available for dequeing before
+considering the next frame. This defeats the purpose of having buffer queues
+since in practice only one buffer would be queued at a time.
+
+The Request API allows a specific configuration of the pipeline (media
+controller topology + controls for each device) to be associated with specific
+buffers. The parameters are applied by each participating device as buffers
+associated to a request flow in. This allows user-space to schedule several
+tasks ("requests") with different parameters in advance, knowing that the
+parameters will be applied when needed to get the expected result. Controls
+values at the time of request completion are also available for reading.
+
+Usage
+=
+
+The Request API is used on top of standard media controller and V4L2 calls,
+which are augmented with an extra ``request_fd`` parameter. Request themselves
+are allocated from either a supporting V4L2 device node, or a supporting media
+controller node. The origin of requests determine their scope: requests
+allocat