RE: [PATCH v5 0/3] Multi-planar video format and buffer support for the V4L2 API
Hi, thanks for the review. Karicheri, Muralidharan m-kariche...@ti.com wrote: 7. Format conversion -- v4l2 core ioctl handling includes a simple conversion layer that allows translation - when possible - between multi-planar and single-planar APIs, transparently to drivers and applications. The table below summarizes conversion behavior for cases when driver and application use different API versions: --- | Application MP -- Driver SP -- Application MP G_FMT |always OK | always OK S_FMT |-EINVAL | always OK TRY_FMT |-EINVAL | always OK --- | Application SP -- Driver MP -- Application SP G_FMT |always OK | -EBUSY S_FMT |always OK | -EBUSY and WARN() TRY_FMT |always OK | -EBUSY and WARN() What is the logic behind using -EBUSY for this? Why not -EINVAL? Also why use WARN()? We think that it's better to return EBUSY from a G_FMT call to say: Your call is valid, but I cannot tell you the format now, but may be able to later (i.e. after I switch back to a single-planar format). EINVAL just says Your call is invalid, i.e. no point in retrying. Also, EINVAL doesn't really make sense for G_FMT. WARN because, as explained below, we think of it as a bug in driver to adjust a 1-plane format to a 1-plane format on S/TRY. Legend: - SP - single-planar API used (NOT format!) - MP - multi-planar API used (NOT format!) - always OK - conversion is always valid irrespective of number of planes - -EINVAL - if an MP application tries to TRY or SET a format with more than 1 plane, EINVAL is returned from the conversion function (of course, 1-plane multi-planar formats work and are converted) - -EBUSY - if an MP driver returns a more than 1-plane format to an SP application, the conversion layer returns EBUSY to the application; this is useful in cases when the driver is currently set to a more than 1-plane format, SP application would not be able to understand it) - -EBUSY and WARN() - there is only one reason for which SET or TRY from an SP application would result in a driver returning a more than 1- plane format - when the driver adjusts a 1-plane format to a more than 1-plane format. This should not happen and is a bug in driver, hence a WARN_ON(). Application receives EBUSY. Suppose, S_FMT/TRY_FMT is called with UYVY format and driver supports only NV12 (2 plane) only, then for Application SP -- Driver MP -- Application SP I guess in this case, new drivers that supports multi-plane format would have to return old NV12 format code (for backward compatibility). Is this handled by driver or translation layer? Not really. If a driver supports a 2-plane format only, an SP application won't be able to use it (unless we copy video data from two separate memory buffers into one and back on each QBUF/DQBUF in core ioctl handling, I don't think we want to go that far). The app expects one buffer, not two. So this is an EINVAL. Application MP -- Driver SP -- Application MP In this case, new driver would return new NV12 format code and have no issue. Not sure what translation layer does in this scenario. Again, not really. If an MP application requests a 2-plane format, the driver cannot support it (it can support 1-plane formats only). So this is also an EINVAL. Overall, we do not convert v4l2_buffers, we only convert format structs (not the actual formats). snip === V. Using ioctl()s and mmap() === * Format calls (VIDIOC_S/TRY/G_FMT) are converted transparently across APIs by the ioctl handling code, where possible. Conversion from single-planar to multi-planar cannot fail, but the other way around is possible only for 1-plane formats. Possible errors in conversion are described below. Could you explain what you mean by conversion of formats? The idea of the translation layer functionality is not clear to me. An example would help. I guess this needs to be rephrased: It doesn't convert the actual formats, it only converts between APIs, i.e. between different format structs. So it's a conversion between struct v4l2_pix_format_mplane and struct v4l2_pix_format. It's not a conversion of formats, the format stays completely identical. So basically an NV12 format can be passed as well in a v4l2_pix_format struct as in v4l2_pix_format_mplane. Fourccs and other fields will be exactly the same. Only the location of those fields will differ, a v4l2_pix_format_mplane struct will have an array of v4l2_plane_pix_format of size one. Look at the fmt_sp_to_mp and fmt_mp_to_sp functions in
[PATCH v5 0/3] Multi-planar video format and buffer support for the V4L2 API
Hello, After 9 months since the first proposal, lots of discussion and many changes, including an almost full redesign between versions 3 and 4, I present the patches that add the fifth version of the multi-planar API for V4L2. I am posting patches first for everyone to be able to take a look early and would be grateful for some indication of your acceptance. More documentation (DocBook) is on the way. Documentation in textual form can be found below. Changes since v4: - struct v4l2_pix_format_mplane: * does not include a struct v4l2_pix_format anymore, replaced with concrete fields * new field: num_planes - renamed fields in struct v4l2_plane (to prevent ambiguity): * mem_off - mem_offset * data_off - data_offset - renamed field in struct v4l2_format (for consistency): * mp_pix - pix_mp Contents: [PATCH v5 1/3] v4l: Add multi-planar API definitions to the V4L2 API [PATCH v5 2/3] v4l: Add multi-planar ioctl handling code [PATCH v5 3/3] v4l: Add compat functions for the multi-planar API === I. Rationale === Some embedded devices (including Samsung S3C/S5P SoC series) require physically separate memory buffers for placing video components. For example, S5P SoC video codec uses one buffer for Y and another for interleaved CbCr components. This cannot be achieved with the current V4L2 API. One of the reasons is that v4l2_buffer struct can only hold one pointer/offset to a video buffer. As the proposal evolved, we found more uses for separate planes per buffer, for non-embedded systems as well. Some examples include: - applications may receive (or choose to store) video data of one video buffer in separate memory buffers; such data would have to be temporarily copied together into one buffer before passing it to a V4L2 device; - applications or drivers may want to pass metadata related to a buffer and it may not be possible to place it in the same buffer together with video data; Example features to be implemented in the future: - allowing video data to be stored in driver-provided memory (MMAP type) while metadata in application-provided buffers (USEPTR type) - useful for drivers that require coefficient matrices, that take or return header/metadata, etc. - allowing variable number of planes passed to each QBUF/DQBUF operations, differing between calls. === II. Short introduction === To establish a consistent nomenclature, for the remainder of this document: - multi-planar indicates a call/format used with multi-planar API, irrespective of the number of planes; - 1-plane format, n-plane formats indicate the number of planes in a format and have nothing to do with the API used; 1-plane formats can be used with the multi-planar API; * The changes are fully backwards-compatible with the current V4L2 API. * All multi-planar calls and types can be recognized by their utilization of new buffer type defines (see below). * Multi-planar API can be used as a superset of both APIs and can replace the single-planar API; old formats can be used as 1-plane multi-planar formats. * A format translation layer is also introduced, new drivers and applications do not have to implement both API versions. A driver that only implements the multi-planar version will still be able to transparently communicate with applications that only use single-planar calls (but those applications will only be able to use the driver's 1-plane formats). The other way around is also possible - a driver that only implements the single-planar API can be used by a multi-planar-API-only application fully. * Applications can query for multi-planar capabilities by means of the standard VIDIOC_QUERYCAP call. * Affected ioctls are those that operate on either pix formats or buffers, which are interpreted in a different way when one of the new buffer types is passed in their corresponding fields: - VIDIOC_G_FMT, VIDIOC_S_FMT, VIDIOC_TRY_FMT, VIDIOC_ENUM_FMT; - VIDIOC_QBUF, VIDIOC_DQBUF, VIDIOC_QUERYBUF - VIDIOC_REQBUF (accepts new buffer types, behavior unchanged) * Fourcc codes differ across plane counts, e.g. a 1-plane YCbCr fourcc is different from that of an, otherwise identical, 2-plane YCbCr. On the other hand, a 1-plane format uses the same fourcc code in both versions of the API. * Applications do not have to support both APIs, it is enough to just use the multi-planar version, as it will be transparently converted to single-planar API for 1-plane-format-only drivers. Of course, it is not possible to set a format with more than 1-plane for a single-planar-only driver, but applications should not try this in the first place. They should use formats returned from ENUM_FMT only and those will be 1-plane only. === III. Multi-planar API format
RE: [PATCH v5 0/3] Multi-planar video format and buffer support for the V4L2 API
Snip
struct v4l2_format {
enum v4l2_buf_type type;
union {
struct v4l2_pix_format pix; /*
V4L2_BUF_TYPE_VIDEO_CAPTURE */
+ struct v4l2_pix_format_mplane pix_mp; /*
V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE */
struct v4l2_window win; /*
V4L2_BUF_TYPE_VIDEO_OVERLAY */
struct v4l2_vbi_format vbi; /*
V4L2_BUF_TYPE_VBI_CAPTURE */
struct v4l2_sliced_vbi_format sliced; /*
V4L2_BUF_TYPE_SLICED_VBI_CAPTURE */
__u8raw_data[200]; /* user-defined */
} fmt;
};
For the new buffer types mp_pix member is chosen. For those buffer types,
struct v4l2_pix_format_mplane is used:
Typo. replae mp_pix with pix_mp
/**
* struct v4l2_pix_format_mplane - multiplanar format definition
* @width: image width in pixels
* @height: image height in pixels
* @pixelformat:little endian four character code (fourcc)
* @field: field order (for interlaced video)
* @colorspace: supplemental to pixelformat
* @plane_fmt: per-plane information
* @num_planes: number of planes for this format and number of
valid
Snip
7. Format conversion
--
v4l2 core ioctl handling includes a simple conversion layer that allows
translation - when possible - between multi-planar and single-planar APIs,
transparently to drivers and applications.
The table below summarizes conversion behavior for cases when driver and
application use different API versions:
---
| Application MP -- Driver SP -- Application MP
G_FMT |always OK | always OK
S_FMT |-EINVAL | always OK
TRY_FMT |-EINVAL | always OK
---
| Application SP -- Driver MP -- Application SP
G_FMT |always OK | -EBUSY
S_FMT |always OK | -EBUSY and WARN()
TRY_FMT |always OK | -EBUSY and WARN()
What is the logic behind using -EBUSY for this? Why not -EINVAL? Also why use
WARN()?
Legend:
- SP - single-planar API used (NOT format!)
- MP - multi-planar API used (NOT format!)
- always OK - conversion is always valid irrespective of number of planes
- -EINVAL - if an MP application tries to TRY or SET a format with more
than 1 plane, EINVAL is returned from the conversion function
(of course, 1-plane multi-planar formats work and are
converted)
- -EBUSY - if an MP driver returns a more than 1-plane format to an SP
application, the conversion layer returns EBUSY to the
application;
this is useful in cases when the driver is currently set to a
more
than 1-plane format, SP application would not be able to
understand
it)
- -EBUSY and WARN() - there is only one reason for which SET or TRY from an
SP
application would result in a driver returning a more than 1-
plane
format - when the driver adjusts a 1-plane format to a more than
1-plane format. This should not happen and is a bug in driver,
hence
a WARN_ON(). Application receives EBUSY.
Suppose, S_FMT/TRY_FMT is called with UYVY format and driver supports only NV12
(2 plane) only, then for
Application SP -- Driver MP -- Application SP
I guess in this case, new drivers that supports multi-plane format would have
to return old NV12 format code (for backward compatibility). Is
this handled by driver or translation layer?
Application MP -- Driver SP -- Application MP
In this case, new driver would return new NV12 format code and have no issue.
Not sure what translation layer does in this scenario.
snip
===
V. Using ioctl()s and mmap()
===
* Format calls (VIDIOC_S/TRY/G_FMT) are converted transparently across APIs
by the ioctl handling code, where possible. Conversion from single-planar
to multi-planar cannot fail, but the other way around is possible only
for
1-plane formats.
Possible errors in conversion are described below.
Could you explain what you mean by conversion of formats? The idea of the
translation layer functionality is not clear to me. An example would help.
Murali
--
To unsubscribe from this list: send the line unsubscribe linux-media in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
