Re: [PATCH 1/2] media: videobuf2: use dmabuf size for length
On 3/26/21 10:03 AM, John Cox wrote:
Hi Helen
Hi John,
On 3/25/21 7:20 AM, John Cox wrote:
Hi
Always use dmabuf size when considering the length of the buffer.
Discard userspace provided length.
Fix length check error in _verify_length(), which was handling single and
multiplanar diferently, and also not catching the case where userspace
provides a bigger length and bytesused then the underlying buffer.
Suggested-by: Hans Verkuil
Signed-off-by: Helen Koike
---
Hello,
As discussed on
https://patchwork.linuxtv.org/project/linux-media/patch/gh5kef5bkeel3o6b2dkgc2dfagu9klj...@4ax.com/
This patch also helps the conversion layer of the Ext API patchset,
where we are not exposing the length field.
It was discussed that userspace might use a smaller length field to
limit the usage of the underlying buffer, but I'm not sure if this is
really usefull and just complicates things.
If this is usefull, then we should also expose a length field in the Ext
API, and document this feature properly.
What do you think?
---
.../media/common/videobuf2/videobuf2-core.c | 21 ---
.../media/common/videobuf2/videobuf2-v4l2.c | 8 +++
include/uapi/linux/videodev2.h| 7 +--
3 files changed, 27 insertions(+), 9 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
b/drivers/media/common/videobuf2/videobuf2-core.c
index 02281d13505f..2cbde14af051 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1205,6 +1205,7 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
for (plane = 0; plane < vb->num_planes; ++plane) {
struct dma_buf *dbuf = dma_buf_get(planes[plane].m.fd);
+ unsigned int bytesused;
if (IS_ERR_OR_NULL(dbuf)) {
dprintk(q, 1, "invalid dmabuf fd for plane %d\n",
@@ -1213,9 +1214,23 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
goto err;
}
- /* use DMABUF size if length is not provided */
- if (planes[plane].length == 0)
- planes[plane].length = dbuf->size;
+ planes[plane].length = dbuf->size;
+ bytesused = planes[plane].bytesused ?
+ planes[plane].bytesused : dbuf->size;
+
+ if (planes[plane].bytesused > planes[plane].length) {
+ dprintk(q, 1, "bytesused is bigger then dmabuf length for
plane %d\n",
+ plane);
+ ret = -EINVAL;
+ goto err;
+ }
+
+ if (planes[plane].data_offset >= bytesused) {
+ dprintk(q, 1, "data_offset >= bytesused for plane %d\n",
+ plane);
+ ret = -EINVAL;
+ goto err;
+ }
if (planes[plane].length < vb->planes[plane].min_length) {
dprintk(q, 1, "invalid dmabuf length %u for plane %d,
minimum length %u\n",
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 7e96f67c60ba..ffc7ed46f74a 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -98,14 +98,14 @@ static int __verify_length(struct vb2_buffer *vb, const
struct v4l2_buffer *b)
unsigned int bytesused;
unsigned int plane;
- if (V4L2_TYPE_IS_CAPTURE(b->type))
+ /* length check for dmabuf is performed in _prepare_dmabuf() */
+ if (V4L2_TYPE_IS_CAPTURE(b->type) || b->memory == VB2_MEMORY_DMABUF)
return 0;
if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
for (plane = 0; plane < vb->num_planes; ++plane) {
- length = (b->memory == VB2_MEMORY_USERPTR ||
- b->memory == VB2_MEMORY_DMABUF)
- ? b->m.planes[plane].length
+ length = b->memory == VB2_MEMORY_USERPTR
+ ? b->m.planes[plane].length
: vb->planes[plane].length;
bytesused = b->m.planes[plane].bytesused
? b->m.planes[plane].bytesused : length;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 8d15f6ccc4b4..79b3b2893513 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -968,7 +968,9 @@ struct v4l2_requestbuffers {
/**
* struct v4l2_plane - plane info for multi-planar buffers
* @bytesused:number of bytes occupied by data in the plane
(payload)
- * @length:size of this plane (NOT the payload) in bytes
+ * @length:
Re: [PATCH 1/2] media: videobuf2: use dmabuf size for length
Hi Laurent,
On 3/25/21 7:53 AM, Laurent Pinchart wrote:
Hi Helen,
On Wed, Mar 24, 2021 at 09:17:11PM -0300, Helen Koike wrote:
Always use dmabuf size when considering the length of the buffer.
Discard userspace provided length.
Fix length check error in _verify_length(), which was handling single and
multiplanar diferently, and also not catching the case where userspace
provides a bigger length and bytesused then the underlying buffer.
Suggested-by: Hans Verkuil
Signed-off-by: Helen Koike
---
Hello,
As discussed on
https://patchwork.linuxtv.org/project/linux-media/patch/gh5kef5bkeel3o6b2dkgc2dfagu9klj...@4ax.com/
This patch also helps the conversion layer of the Ext API patchset,
where we are not exposing the length field.
It was discussed that userspace might use a smaller length field to
limit the usage of the underlying buffer, but I'm not sure if this is
really usefull and just complicates things.
If this is usefull, then we should also expose a length field in the Ext
API, and document this feature properly.
What do you think?
I think a limit could be useful, as a single dmabuf object could hold
multiple planes, which should be addressed by an offset from the
beginning of the buffer. Giving a length to the kernel could help
catching errors. As the existing API doesn't support offsets, a length
limit is likely not very useful at the moment, but should I believe be
included at least in the new API.
For the new API, If there are no users, we can leave space to add it later
if such a feature becomes interesting for userspace in the future.
For the existing implementation, I'd say that we should be pragmatic. If
using the provided length as a maximum boundary makes the implementation
more complex for very little gain, let's not do it. But on the other
hand, considering existing userspace, would there be added value in
implementing such a mechanism ?
I'm guessing that userspace doesn't use this field as a boundary, since this
usage is not documented. So I'm guessing it makes things more complex for
little gain, so it doesn't seem we are adding much value to userspace.
And with this patch, it will be much easier to implement Ext API with a
conversion layer to/from the existing API.
Regards,
Helen
---
.../media/common/videobuf2/videobuf2-core.c | 21 ---
.../media/common/videobuf2/videobuf2-v4l2.c | 8 +++
include/uapi/linux/videodev2.h| 7 +--
3 files changed, 27 insertions(+), 9 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
b/drivers/media/common/videobuf2/videobuf2-core.c
index 02281d13505f..2cbde14af051 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1205,6 +1205,7 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
for (plane = 0; plane < vb->num_planes; ++plane) {
struct dma_buf *dbuf = dma_buf_get(planes[plane].m.fd);
+ unsigned int bytesused;
if (IS_ERR_OR_NULL(dbuf)) {
dprintk(q, 1, "invalid dmabuf fd for plane %d\n",
@@ -1213,9 +1214,23 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
goto err;
}
- /* use DMABUF size if length is not provided */
- if (planes[plane].length == 0)
- planes[plane].length = dbuf->size;
+ planes[plane].length = dbuf->size;
+ bytesused = planes[plane].bytesused ?
+ planes[plane].bytesused : dbuf->size;
+
+ if (planes[plane].bytesused > planes[plane].length) {
+ dprintk(q, 1, "bytesused is bigger then dmabuf length for
plane %d\n",
+ plane);
+ ret = -EINVAL;
+ goto err;
+ }
+
+ if (planes[plane].data_offset >= bytesused) {
+ dprintk(q, 1, "data_offset >= bytesused for plane %d\n",
+ plane);
+ ret = -EINVAL;
+ goto err;
+ }
if (planes[plane].length < vb->planes[plane].min_length) {
dprintk(q, 1, "invalid dmabuf length %u for plane %d,
minimum length %u\n",
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 7e96f67c60ba..ffc7ed46f74a 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -98,14 +98,14 @@ static int __verify_length(struct vb2_buffer *vb, const
struct v4l2_buffer *b)
unsigned int bytesused;
unsigned int plane;
- if (V4L2_TYPE_IS_CAPTURE(b->type))
+ /* length check for dmabuf is performed in _prepare_dmabuf() *
Re: [PATCH 1/2] media: videobuf2: use dmabuf size for length
Hi John,
On 3/25/21 7:20 AM, John Cox wrote:
Hi
Always use dmabuf size when considering the length of the buffer.
Discard userspace provided length.
Fix length check error in _verify_length(), which was handling single and
multiplanar diferently, and also not catching the case where userspace
provides a bigger length and bytesused then the underlying buffer.
Suggested-by: Hans Verkuil
Signed-off-by: Helen Koike
---
Hello,
As discussed on
https://patchwork.linuxtv.org/project/linux-media/patch/gh5kef5bkeel3o6b2dkgc2dfagu9klj...@4ax.com/
This patch also helps the conversion layer of the Ext API patchset,
where we are not exposing the length field.
It was discussed that userspace might use a smaller length field to
limit the usage of the underlying buffer, but I'm not sure if this is
really usefull and just complicates things.
If this is usefull, then we should also expose a length field in the Ext
API, and document this feature properly.
What do you think?
---
.../media/common/videobuf2/videobuf2-core.c | 21 ---
.../media/common/videobuf2/videobuf2-v4l2.c | 8 +++
include/uapi/linux/videodev2.h| 7 +--
3 files changed, 27 insertions(+), 9 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
b/drivers/media/common/videobuf2/videobuf2-core.c
index 02281d13505f..2cbde14af051 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1205,6 +1205,7 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
for (plane = 0; plane < vb->num_planes; ++plane) {
struct dma_buf *dbuf = dma_buf_get(planes[plane].m.fd);
+ unsigned int bytesused;
if (IS_ERR_OR_NULL(dbuf)) {
dprintk(q, 1, "invalid dmabuf fd for plane %d\n",
@@ -1213,9 +1214,23 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
goto err;
}
- /* use DMABUF size if length is not provided */
- if (planes[plane].length == 0)
- planes[plane].length = dbuf->size;
+ planes[plane].length = dbuf->size;
+ bytesused = planes[plane].bytesused ?
+ planes[plane].bytesused : dbuf->size;
+
+ if (planes[plane].bytesused > planes[plane].length) {
+ dprintk(q, 1, "bytesused is bigger then dmabuf length for
plane %d\n",
+ plane);
+ ret = -EINVAL;
+ goto err;
+ }
+
+ if (planes[plane].data_offset >= bytesused) {
+ dprintk(q, 1, "data_offset >= bytesused for plane %d\n",
+ plane);
+ ret = -EINVAL;
+ goto err;
+ }
if (planes[plane].length < vb->planes[plane].min_length) {
dprintk(q, 1, "invalid dmabuf length %u for plane %d,
minimum length %u\n",
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 7e96f67c60ba..ffc7ed46f74a 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -98,14 +98,14 @@ static int __verify_length(struct vb2_buffer *vb, const
struct v4l2_buffer *b)
unsigned int bytesused;
unsigned int plane;
- if (V4L2_TYPE_IS_CAPTURE(b->type))
+ /* length check for dmabuf is performed in _prepare_dmabuf() */
+ if (V4L2_TYPE_IS_CAPTURE(b->type) || b->memory == VB2_MEMORY_DMABUF)
return 0;
if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
for (plane = 0; plane < vb->num_planes; ++plane) {
- length = (b->memory == VB2_MEMORY_USERPTR ||
- b->memory == VB2_MEMORY_DMABUF)
- ? b->m.planes[plane].length
+ length = b->memory == VB2_MEMORY_USERPTR
+ ? b->m.planes[plane].length
: vb->planes[plane].length;
bytesused = b->m.planes[plane].bytesused
? b->m.planes[plane].bytesused : length;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 8d15f6ccc4b4..79b3b2893513 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -968,7 +968,9 @@ struct v4l2_requestbuffers {
/**
* struct v4l2_plane - plane info for multi-planar buffers
* @bytesused: number of bytes occupied by data in the plane (payload)
- * @length:size of this plane (NOT the payload) in bytes
+ * @length:size of this plane (NOT the payload) in bytes.
[PATCH 1/2] media: videobuf2: use dmabuf size for length
Always use dmabuf size when considering the length of the buffer.
Discard userspace provided length.
Fix length check error in _verify_length(), which was handling single and
multiplanar diferently, and also not catching the case where userspace
provides a bigger length and bytesused then the underlying buffer.
Suggested-by: Hans Verkuil
Signed-off-by: Helen Koike
---
Hello,
As discussed on
https://patchwork.linuxtv.org/project/linux-media/patch/gh5kef5bkeel3o6b2dkgc2dfagu9klj...@4ax.com/
This patch also helps the conversion layer of the Ext API patchset,
where we are not exposing the length field.
It was discussed that userspace might use a smaller length field to
limit the usage of the underlying buffer, but I'm not sure if this is
really usefull and just complicates things.
If this is usefull, then we should also expose a length field in the Ext
API, and document this feature properly.
What do you think?
---
.../media/common/videobuf2/videobuf2-core.c | 21 ---
.../media/common/videobuf2/videobuf2-v4l2.c | 8 +++
include/uapi/linux/videodev2.h| 7 +--
3 files changed, 27 insertions(+), 9 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
b/drivers/media/common/videobuf2/videobuf2-core.c
index 02281d13505f..2cbde14af051 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1205,6 +1205,7 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
for (plane = 0; plane < vb->num_planes; ++plane) {
struct dma_buf *dbuf = dma_buf_get(planes[plane].m.fd);
+ unsigned int bytesused;
if (IS_ERR_OR_NULL(dbuf)) {
dprintk(q, 1, "invalid dmabuf fd for plane %d\n",
@@ -1213,9 +1214,23 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
goto err;
}
- /* use DMABUF size if length is not provided */
- if (planes[plane].length == 0)
- planes[plane].length = dbuf->size;
+ planes[plane].length = dbuf->size;
+ bytesused = planes[plane].bytesused ?
+ planes[plane].bytesused : dbuf->size;
+
+ if (planes[plane].bytesused > planes[plane].length) {
+ dprintk(q, 1, "bytesused is bigger then dmabuf length
for plane %d\n",
+ plane);
+ ret = -EINVAL;
+ goto err;
+ }
+
+ if (planes[plane].data_offset >= bytesused) {
+ dprintk(q, 1, "data_offset >= bytesused for plane %d\n",
+ plane);
+ ret = -EINVAL;
+ goto err;
+ }
if (planes[plane].length < vb->planes[plane].min_length) {
dprintk(q, 1, "invalid dmabuf length %u for plane %d,
minimum length %u\n",
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 7e96f67c60ba..ffc7ed46f74a 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -98,14 +98,14 @@ static int __verify_length(struct vb2_buffer *vb, const
struct v4l2_buffer *b)
unsigned int bytesused;
unsigned int plane;
- if (V4L2_TYPE_IS_CAPTURE(b->type))
+ /* length check for dmabuf is performed in _prepare_dmabuf() */
+ if (V4L2_TYPE_IS_CAPTURE(b->type) || b->memory == VB2_MEMORY_DMABUF)
return 0;
if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
for (plane = 0; plane < vb->num_planes; ++plane) {
- length = (b->memory == VB2_MEMORY_USERPTR ||
- b->memory == VB2_MEMORY_DMABUF)
- ? b->m.planes[plane].length
+ length = b->memory == VB2_MEMORY_USERPTR
+ ? b->m.planes[plane].length
: vb->planes[plane].length;
bytesused = b->m.planes[plane].bytesused
? b->m.planes[plane].bytesused : length;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 8d15f6ccc4b4..79b3b2893513 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -968,7 +968,9 @@ struct v4l2_requestbuffers {
/**
* struct v4l2_plane - plane info for multi-planar buffers
* @bytesused: number of bytes occupied by data in the plane (payload)
- * @length:size of this plane (NOT the payload) in bytes
+ * @length:size of this plane (NOT the payload) in bytes. Filled
+ * by userspace for USER
[PATCH 2/2] media: videobuf2: cleanup size argument from attach_dmabuf()
Since we always use the size of the underlying buffer for dmabuf, remove
the size parameter from the attach_dmabuf() callback.
Suggested-by: Hans Verkuil
Signed-off-by: Helen Koike
---
drivers/media/common/videobuf2/videobuf2-core.c | 2 +-
drivers/media/common/videobuf2/videobuf2-dma-contig.c | 7 ++-
drivers/media/common/videobuf2/videobuf2-dma-sg.c | 7 ++-
drivers/media/common/videobuf2/videobuf2-vmalloc.c| 7 ++-
include/media/videobuf2-core.h| 1 -
5 files changed, 7 insertions(+), 17 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
b/drivers/media/common/videobuf2/videobuf2-core.c
index 2cbde14af051..86af4f3c72eb 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1266,7 +1266,7 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
/* Acquire each plane's memory */
mem_priv = call_ptr_memop(vb, attach_dmabuf,
q->alloc_devs[plane] ? : q->dev,
- dbuf, planes[plane].length, q->dma_dir);
+ dbuf, q->dma_dir);
if (IS_ERR(mem_priv)) {
dprintk(q, 1, "failed to attach dmabuf\n");
ret = PTR_ERR(mem_priv);
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
index a7f61ba85440..a26aa52f954b 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
@@ -661,14 +661,11 @@ static void vb2_dc_detach_dmabuf(void *mem_priv)
}
static void *vb2_dc_attach_dmabuf(struct device *dev, struct dma_buf *dbuf,
- unsigned long size, enum dma_data_direction dma_dir)
+ enum dma_data_direction dma_dir)
{
struct vb2_dc_buf *buf;
struct dma_buf_attachment *dba;
- if (dbuf->size < size)
- return ERR_PTR(-EFAULT);
-
if (WARN_ON(!dev))
return ERR_PTR(-EINVAL);
@@ -686,7 +683,7 @@ static void *vb2_dc_attach_dmabuf(struct device *dev,
struct dma_buf *dbuf,
}
buf->dma_dir = dma_dir;
- buf->size = size;
+ buf->size = dbuf->size;
buf->db_attach = dba;
return buf;
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
index c5b06a509566..8c006f79bed4 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
@@ -606,7 +606,7 @@ static void vb2_dma_sg_detach_dmabuf(void *mem_priv)
}
static void *vb2_dma_sg_attach_dmabuf(struct device *dev, struct dma_buf *dbuf,
- unsigned long size, enum dma_data_direction dma_dir)
+ enum dma_data_direction dma_dir)
{
struct vb2_dma_sg_buf *buf;
struct dma_buf_attachment *dba;
@@ -614,9 +614,6 @@ static void *vb2_dma_sg_attach_dmabuf(struct device *dev,
struct dma_buf *dbuf,
if (WARN_ON(!dev))
return ERR_PTR(-EINVAL);
- if (dbuf->size < size)
- return ERR_PTR(-EFAULT);
-
buf = kzalloc(sizeof(*buf), GFP_KERNEL);
if (!buf)
return ERR_PTR(-ENOMEM);
@@ -631,7 +628,7 @@ static void *vb2_dma_sg_attach_dmabuf(struct device *dev,
struct dma_buf *dbuf,
}
buf->dma_dir = dma_dir;
- buf->size = size;
+ buf->size = dmabuf->size;
buf->db_attach = dba;
return buf;
diff --git a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
index 83f95258ec8c..c2d41b375c10 100644
--- a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
+++ b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
@@ -404,20 +404,17 @@ static void vb2_vmalloc_detach_dmabuf(void *mem_priv)
}
static void *vb2_vmalloc_attach_dmabuf(struct device *dev, struct dma_buf
*dbuf,
- unsigned long size, enum dma_data_direction dma_dir)
+ enum dma_data_direction dma_dir)
{
struct vb2_vmalloc_buf *buf;
- if (dbuf->size < size)
- return ERR_PTR(-EFAULT);
-
buf = kzalloc(sizeof(*buf), GFP_KERNEL);
if (!buf)
return ERR_PTR(-ENOMEM);
buf->dbuf = dbuf;
buf->dma_dir = dma_dir;
- buf->size = size;
+ buf->size = dbuf->size;
return buf;
}
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 12955cb460d2..db07001cada8 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -134,7 +134,6 @@ struct vb2_mem_ops {
void*(*attach_dmabuf)(struct device *dev,
Re: [RFC PATCH v6 02/11] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
Hi Hans,
Thank you for your comment, please see my reply below.
On 2/23/21 9:35 AM, Hans Verkuil wrote:
Hi Helen,
On 14/01/2021 19:07, Helen Koike wrote:
This is part of the multiplanar and singleplanar unification process.
v4l2_ext_pix_format is supposed to work for both cases.
We also add the concept of modifiers already employed in DRM to expose
HW-specific formats (like tiled or compressed formats) and allow
exchanging this information with the DRM subsystem in a consistent way.
Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in
v4l2_ext_format, other types will be rejected if you use the
{G,S,TRY}_EXT_PIX_FMT ioctls.
New hooks have been added to v4l2_ioctl_ops to support those new ioctls
in drivers, but, in the meantime, the core takes care of converting
{S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can
still work if the userspace app/lib uses the new ioctls.
The conversion is also done the other around to allow userspace
apps/libs using {S,G,TRY}_FMT to work with drivers implementing the
_ext_ hooks.
I have some small comments below, but also one high level comment:
Regarding M variants of pixelformats: this patch 'normalizes' them to
regular pixelformats in the extended API. This makes life complicated,
and I wonder if this is the right approach.
Currently there are two reasons for a driver to support e.g. NV12M:
either luma and chroma need to be in two memory banks, or the luma
and chroma planes cannot be contiguous due to alignment requirements.
The first requirement is still valid for drivers that support the extended API.
The second requirement is no longer a reason to support NV12M. But I
don't think we should just drop NV12M support if it was already supported
before the conversion to this extended API. Since NV12M allocates two buffers
instead of one, it is still different from a regular NV12.
I don't see what would be the difference when using NV12 and NV12M in
Ext API.
NV12 could be used for both requirements. It would even allow the second
requirement with a single memory buffer.
I would prefer that such drivers support both NV12 and NV12M, so no
automatic conversion.
A related question is how to handle pixelformat enumeration: with the
extended API an NV12 format might work, but not with the old API (e.g.
due to memory alignment requirements). I wonder if a VIDIOC_ENUM_EXT_PIX_FMT
isn't needed.
We need VIDIOC_ENUM_EXT_PIX_FMT for modifiers, but we can add it later.
If the driver reports NV12M, userspace can use it and the framework
normilizes it.
VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while VIDIOC_ENUM_FMT
would just report NV12M.
If NV12 and NV12M are equivalent in Ext API, I don't see why we would
report both (unless I'm missing something, which is probably the case).
The idea was to deprecate the M-variants one day.
Regards,
Helen
Signed-off-by: Boris Brezillon
Signed-off-by: Helen Koike
---
Changes in v6:
The main change here was fixing the conversion, so planes reflects color
planes,
and to implement this properly I made major refactors compared to the previous
version.
- struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format
instead (Tomasz)
- refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans)
- reorder colorimetry fields in struct v4l2_ext_pix_format (Hans)
- do not set Ext ioctls as valid for vid_out_overlay (Tomasz)
- refactor conversion functions, so planes are color planes (Tomasz)
- Don't explicitly check for e->modifier != 0 in
v4l2_ext_pix_format_to_format() (Tomasz)
- Use "ef" for extended formats in the framework for consistency (Tomasz)
- Handle xfer_func field in conversions (Tomasz)
- Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz)
- Refactor format functions to use v4l_fmt_ioctl_via_ext()
- Several fixes/refactoring/changes
- Remove EXT API for touch devices
Changes in v5:
- change sizes and reorder fields to avoid holes in the struct and make
it the same for 32 and 64 bits
- removed __attribute__ ((packed)) from uapi structs
- Fix doc warning from make htmldocs
- Updated commit message with EXT_PIX prefix for the ioctls.
Changes in v4:
- Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
- Add reserved fields
- Removed num_planes from struct v4l2_ext_pix_format
- Removed flag field from struct v4l2_ext_pix_format, since the only
defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1,
where we can use modifiers, or add it back later through the reserved
bits.
- In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR &&
!= MOD_INVALID
- Fix type assignment in v4l_g_fmt_ext_pix()
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
-
Re: [PATCH] test-media: wrap vivid code around $vivid variable
On 2/17/21 3:22 PM, Hans Verkuil wrote: On 17/02/2021 19:11, Helen Koike wrote: The script was trying to load vivid and run some commands on top of it even when $vivid = 0. Wrap all vivid code under $vivid variable. Signed-off-by: Helen Koike --- contrib/test/test-media | 66 - 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/contrib/test/test-media b/contrib/test/test-media index 10b7e89d..8cd8bc37 100755 --- a/contrib/test/test-media +++ b/contrib/test/test-media @@ -146,29 +146,29 @@ if [ $kmemleak -eq 1 ]; then echo clear >/sys/kernel/debug/kmemleak fi -rmmod vivid 2&>/dev/null -modprobe vivid n_devs=3 multiplanar=1,2,2 cache_hints=1,0,0 #allocators=0,1,1 Ah, no. Vivid is also used to test dmabuf for vim2m, vimc and vicodec tests. It functions as the allocator for the dma buffers in that case. So even if vivid isn't given, but only vim2m, vimc or vicodec, it should still be loaded. I see, thanks, ignore this patch then. At some point I had a doubt if the script was testing vimc or vivid for some nodes, but this was my mistake. Thanks Helen It isn't needed for vidtv, so I guess it could be disabled if only vidtv is tested. Regards, Hans -sleep 1 +if [ $vivid -eq 1 ]; then + rmmod vivid 2&>/dev/null + modprobe vivid n_devs=3 multiplanar=1,2,2 cache_hints=1,0,0 #allocators=0,1,1 + sleep 1 -tmp=`mktemp` + tmp=`mktemp` -if ! $v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-cap ; then - echo "FAIL: the vivid module failed to load" | tee -a $tmp - echo "Grand Total for vivid: Succeeded: 0, Failed: 1, Warnings: 0" | tee -a $tmp - echo "Final Summary: 1, Succeeded: 0, Failed: 1, Warnings: 0" - exit 0 -fi + if ! $v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-cap ; then + echo "FAIL: the vivid module failed to load" | tee -a $tmp + echo "Grand Total for vivid: Succeeded: 0, Failed: 1, Warnings: 0" | tee -a $tmp + echo "Final Summary: 1, Succeeded: 0, Failed: 1, Warnings: 0" + exit 0 + fi -$v4l2_ctl -z platform:vivid-000 -d vivid-000-vid-cap -i3 -v width=3840,height=2160,pixelformat=NV24 -$v4l2_ctl -z platform:vivid-000 -d vivid-000-vid-out -o1 -x width=3840,height=2160,pixelformat=NV24 -$v4l2_ctl -z platform:vivid-001 -d vivid-001-vid-cap -i3 -v width=3840,height=2160,pixelformat=NM16 -$v4l2_ctl -z platform:vivid-001 -d vivid-001-vid-out -o1 -x width=3840,height=2160,pixelformat=NM16 -$v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-cap -i3 -v width=3840,height=2160,pixelformat=NV24 -$v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-out -o1 -x width=3840,height=2160,pixelformat=NM16 + $v4l2_ctl -z platform:vivid-000 -d vivid-000-vid-cap -i3 -v width=3840,height=2160,pixelformat=NV24 + $v4l2_ctl -z platform:vivid-000 -d vivid-000-vid-out -o1 -x width=3840,height=2160,pixelformat=NV24 + $v4l2_ctl -z platform:vivid-001 -d vivid-001-vid-cap -i3 -v width=3840,height=2160,pixelformat=NM16 + $v4l2_ctl -z platform:vivid-001 -d vivid-001-vid-out -o1 -x width=3840,height=2160,pixelformat=NM16 + $v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-cap -i3 -v width=3840,height=2160,pixelformat=NV24 + $v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-out -o1 -x width=3840,height=2160,pixelformat=NM16 -echo + echo -if [ $vivid -eq 1 ]; then dmesg -n notice echo echo vivid compliance tests, contiguous planes | tee /dev/kmsg @@ -287,6 +287,18 @@ if [ $vivid -eq 1 ]; then echo echo echo + + date + echo + echo unbind vivid | tee /dev/kmsg + echo + echo -n vivid.0 >/sys/bus/platform/drivers/vivid/unbind + sleep $unbind_time + echo + echo rmmod vivid | tee /dev/kmsg + echo + rmmod vivid + sleep $rmmod_time fi if [ $vim2m -eq 1 ]; then @@ -300,7 +312,7 @@ if [ $vim2m -eq 1 ]; then echo "FAIL: the vim2m module failed to load" | tee -a $tmp echo "Grand Total for vim2m: Succeeded: 0, Failed: 1, Warnings: 0" | tee -a $tmp echo "Final Summary: 1, Succeeded: 0, Failed: 1, Warnings: 0" - rmmod vivid + rmmod vim2m exit 0 fi @@ -373,7 +385,7 @@ if [ $vimc -eq 1 ]; then echo "FAIL: the vimc module failed to load" | tee -a $tmp echo "Grand Total for vimc: Succeeded: 0, Failed: 1, Warnings: 0" | tee -a $tmp echo "Final Summary: 1, Succeeded: 0, Failed: 1, Warnings: 0" - rmmod vivid + rmmod vimc exit 0 fi @@ -467,7 +479,7 @@ if [ $vicodec -eq 1 ]; then echo "
[PATCH] test-media: wrap vivid code around $vivid variable
The script was trying to load vivid and run some commands on top of it even when $vivid = 0. Wrap all vivid code under $vivid variable. Signed-off-by: Helen Koike --- contrib/test/test-media | 66 - 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/contrib/test/test-media b/contrib/test/test-media index 10b7e89d..8cd8bc37 100755 --- a/contrib/test/test-media +++ b/contrib/test/test-media @@ -146,29 +146,29 @@ if [ $kmemleak -eq 1 ]; then echo clear >/sys/kernel/debug/kmemleak fi -rmmod vivid 2&>/dev/null -modprobe vivid n_devs=3 multiplanar=1,2,2 cache_hints=1,0,0 #allocators=0,1,1 -sleep 1 +if [ $vivid -eq 1 ]; then + rmmod vivid 2&>/dev/null + modprobe vivid n_devs=3 multiplanar=1,2,2 cache_hints=1,0,0 #allocators=0,1,1 + sleep 1 -tmp=`mktemp` + tmp=`mktemp` -if ! $v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-cap ; then - echo "FAIL: the vivid module failed to load" | tee -a $tmp - echo "Grand Total for vivid: Succeeded: 0, Failed: 1, Warnings: 0" | tee -a $tmp - echo "Final Summary: 1, Succeeded: 0, Failed: 1, Warnings: 0" - exit 0 -fi + if ! $v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-cap ; then + echo "FAIL: the vivid module failed to load" | tee -a $tmp + echo "Grand Total for vivid: Succeeded: 0, Failed: 1, Warnings: 0" | tee -a $tmp + echo "Final Summary: 1, Succeeded: 0, Failed: 1, Warnings: 0" + exit 0 + fi -$v4l2_ctl -z platform:vivid-000 -d vivid-000-vid-cap -i3 -v width=3840,height=2160,pixelformat=NV24 -$v4l2_ctl -z platform:vivid-000 -d vivid-000-vid-out -o1 -x width=3840,height=2160,pixelformat=NV24 -$v4l2_ctl -z platform:vivid-001 -d vivid-001-vid-cap -i3 -v width=3840,height=2160,pixelformat=NM16 -$v4l2_ctl -z platform:vivid-001 -d vivid-001-vid-out -o1 -x width=3840,height=2160,pixelformat=NM16 -$v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-cap -i3 -v width=3840,height=2160,pixelformat=NV24 -$v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-out -o1 -x width=3840,height=2160,pixelformat=NM16 + $v4l2_ctl -z platform:vivid-000 -d vivid-000-vid-cap -i3 -v width=3840,height=2160,pixelformat=NV24 + $v4l2_ctl -z platform:vivid-000 -d vivid-000-vid-out -o1 -x width=3840,height=2160,pixelformat=NV24 + $v4l2_ctl -z platform:vivid-001 -d vivid-001-vid-cap -i3 -v width=3840,height=2160,pixelformat=NM16 + $v4l2_ctl -z platform:vivid-001 -d vivid-001-vid-out -o1 -x width=3840,height=2160,pixelformat=NM16 + $v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-cap -i3 -v width=3840,height=2160,pixelformat=NV24 + $v4l2_ctl -z platform:vivid-002 -d vivid-002-vid-out -o1 -x width=3840,height=2160,pixelformat=NM16 -echo + echo -if [ $vivid -eq 1 ]; then dmesg -n notice echo echo vivid compliance tests, contiguous planes | tee /dev/kmsg @@ -287,6 +287,18 @@ if [ $vivid -eq 1 ]; then echo echo echo + + date + echo + echo unbind vivid | tee /dev/kmsg + echo + echo -n vivid.0 >/sys/bus/platform/drivers/vivid/unbind + sleep $unbind_time + echo + echo rmmod vivid | tee /dev/kmsg + echo + rmmod vivid + sleep $rmmod_time fi if [ $vim2m -eq 1 ]; then @@ -300,7 +312,7 @@ if [ $vim2m -eq 1 ]; then echo "FAIL: the vim2m module failed to load" | tee -a $tmp echo "Grand Total for vim2m: Succeeded: 0, Failed: 1, Warnings: 0" | tee -a $tmp echo "Final Summary: 1, Succeeded: 0, Failed: 1, Warnings: 0" - rmmod vivid + rmmod vim2m exit 0 fi @@ -373,7 +385,7 @@ if [ $vimc -eq 1 ]; then echo "FAIL: the vimc module failed to load" | tee -a $tmp echo "Grand Total for vimc: Succeeded: 0, Failed: 1, Warnings: 0" | tee -a $tmp echo "Final Summary: 1, Succeeded: 0, Failed: 1, Warnings: 0" - rmmod vivid + rmmod vimc exit 0 fi @@ -467,7 +479,7 @@ if [ $vicodec -eq 1 ]; then echo "FAIL: the vicodec module failed to load" | tee -a $tmp echo "Grand Total for vicodec: Succeeded: 0, Failed: 1, Warnings: 0" | tee -a $tmp echo "Final Summary: 1, Succeeded: 0, Failed: 1, Warnings: 0" - rmmod vivid + rmmod vicodec exit 0 fi @@ -603,18 +615,6 @@ if [ $vicodec -eq 1 ]; then echo fi -date -echo -echo unbind vivid | tee /dev/kmsg -echo -echo -n vivid.0 >/sys/bus/platform/drivers/vivid/unbind -sleep $unbind_time -echo -echo rmmod vivid | tee /dev/kmsg -echo -rmmod vivid -sleep $rmmod_time - if [ $vidtv -eq 1 ]; then rmmod dvb_vidtv_bridge dvb_vidtv_tuner dvb_vidtv_demod 2&>/dev/null modprobe vidtv -- 2.30.1
Re: [PATCH 3/6] drm/rockchip: dsi: add ability to work as a phy instead of full dsi
ret;
+ mutex_lock(&dsi->usage_mutex);
+
+ if (dsi->usage_mode != DW_DSI_USAGE_IDLE) {
+ DRM_DEV_ERROR(dsi->dev, "dsi controller already in use\n");
+ mutex_unlock(&dsi->usage_mutex);
+ return -EBUSY;
+ }
+
+ dsi->usage_mode = DW_DSI_USAGE_DSI;
+ mutex_unlock(&dsi->usage_mutex);
+
ret = component_add(dsi->dev, &dw_mipi_dsi_rockchip_ops);
if (ret) {
DRM_DEV_ERROR(dsi->dev, "Failed to register component: %d\n",
@@ -1000,6 +1035,10 @@ static int dw_mipi_dsi_rockchip_host_detach(void
*priv_data,
component_del(dsi->dev, &dw_mipi_dsi_rockchip_ops);
+ mutex_lock(&dsi->usage_mutex);
+ dsi->usage_mode = DW_DSI_USAGE_IDLE;
+ mutex_unlock(&dsi->usage_mutex);
+
return 0;
}
@@ -1008,11 +1047,227 @@ static const struct dw_mipi_dsi_host_ops dw_mipi_dsi_rockchip_host_ops = {
.detach = dw_mipi_dsi_rockchip_host_detach,
};
+static int dw_mipi_dsi_rockchip_dphy_bind(struct device *dev,
+ struct device *master,
+ void *data)
+{
+ /*
+* Nothing to do when used as a dphy.
+* Just make the rest of Rockchip-DRM happy
+* by being here.
+*/
+
+ return 0;
+}
+
+static void dw_mipi_dsi_rockchip_dphy_unbind(struct device *dev,
+struct device *master,
+void *data)
+{
+ /* Nothing to do when used as a dphy. */
+}
+
+static const struct component_ops dw_mipi_dsi_rockchip_dphy_ops = {
+ .bind = dw_mipi_dsi_rockchip_dphy_bind,
+ .unbind = dw_mipi_dsi_rockchip_dphy_unbind,
+};
+
+static int dw_mipi_dsi_dphy_init(struct phy *phy)
+{
+ struct dw_mipi_dsi_rockchip *dsi = phy_get_drvdata(phy);
+ int ret;
+
+ mutex_lock(&dsi->usage_mutex);
+
+ if (dsi->usage_mode != DW_DSI_USAGE_IDLE) {
+ DRM_DEV_ERROR(dsi->dev, "dsi controller already in use\n");
+ mutex_unlock(&dsi->usage_mutex);
+ return -EBUSY;
+ }
+
+ dsi->usage_mode = DW_DSI_USAGE_PHY;
+ mutex_unlock(&dsi->usage_mutex);
+
+ ret = component_add(dsi->dev, &dw_mipi_dsi_rockchip_dphy_ops);
+ if (ret < 0)
+ goto err_graph;
+
+ if (dsi->cdata->dphy_rx_init) {
+ ret = clk_prepare_enable(dsi->pclk);
+ if (ret < 0)
+ goto err_init;
+
+ ret = clk_prepare_enable(dsi->grf_clk);
+ if (ret) {
+ clk_disable_unprepare(dsi->pclk);
+ goto err_init;
+ }
+
+ ret = dsi->cdata->dphy_rx_init(phy);
+ clk_disable_unprepare(dsi->grf_clk);
+ clk_disable_unprepare(dsi->pclk);
+ if (ret < 0)
+ goto err_init;
+ }
+
+ return 0;
+
+err_init:
+ component_del(dsi->dev, &dw_mipi_dsi_rockchip_dphy_ops);
+err_graph:
+ mutex_lock(&dsi->usage_mutex);
+ dsi->usage_mode = DW_DSI_USAGE_IDLE;
+ mutex_unlock(&dsi->usage_mutex);
+
+ return ret;
+}
+
+static int dw_mipi_dsi_dphy_exit(struct phy *phy)
+{
+ struct dw_mipi_dsi_rockchip *dsi = phy_get_drvdata(phy);
+
+ component_del(dsi->dev, &dw_mipi_dsi_rockchip_dphy_ops);
+
+ mutex_lock(&dsi->usage_mutex);
+ dsi->usage_mode = DW_DSI_USAGE_IDLE;
+ mutex_unlock(&dsi->usage_mutex);
+
+ return 0;
+}
+
+static int dw_mipi_dsi_dphy_configure(struct phy *phy, union
phy_configure_opts *opts)
+{
+ struct phy_configure_opts_mipi_dphy *config = &opts->mipi_dphy;
+ struct dw_mipi_dsi_rockchip *dsi = phy_get_drvdata(phy);
+ int ret;
+
+ ret = phy_mipi_dphy_config_validate(&opts->mipi_dphy);
+ if (ret)
+ return ret;
+
+ dsi->dphy_config = *config;
+ dsi->lane_mbps = div_u64(config->hs_clk_rate, 1000 * 1000 * 1);
+
+ return 0;
+}
+
+static int dw_mipi_dsi_dphy_power_on(struct phy *phy)
+{
+ struct dw_mipi_dsi_rockchip *dsi = phy_get_drvdata(phy);
+ int i, ret;
It seems "i" could be removed, use ret instead.
In general, the patch doesn't look wrong to me.
For the whole serie:
Acked-by: Helen Koike
Thanks
Helen
+
+ DRM_DEV_DEBUG(dsi->dev, "lanes %d - data_rate_mbps %u\n",
+ dsi->dphy_config.lanes, dsi->lane_mbps);
+
+ i = max_mbps_to_parameter(dsi->lane_mbps);
+ if (i < 0) {
+ DRM_DEV_ERROR(dsi->dev, "failed to get parameter for %dmbps
clock\n",
+ dsi->lane_mbps);
+ return i
Re: [RFC PATCH v6 00/11] media: v4l2: Add extended fmt and buffer ioctls
Hello,
On 1/14/21 3:07 PM, Helen Koike wrote:
> Hello,
>
> This is v6 of the Extended API for formats and buffers (see below the new
> API).
>
> The new API comes for free for old drivers through the conversion layer, which
> is independent of vb2.
>
> I completly refactored several patches. I would like to request comments not
> only in the uAPI, but also the kAPI for drivers, and I would appreciate any
> ideas on improving the quality of the code (in short: please review
> everything).
>
> NOTE: The Ext API wans't tested yet. My next step is to patch v4l2-compliance.
I implemented on libcamera to test it, please check:
https://lists.libcamera.org/pipermail/libcamera-devel/2021-February/017169.html
Thanks,
Helen
>
> Regression tests - v4l2-compliance with test-media script:
> vivid: http://ix.io/2M0G - Final Summary: 1856, Succeeded: 1856,
> Failed: 0, Warnings: 0)
> vimc: http://ix.io/2M0I - Final Summary: 488, Succeeded: 488, Failed:
> 0, Warnings: 0
>
> Git: https://gitlab.collabora.com/koike/linux/-/tree/v4l2/ext-api/v6
>
> v5:
> https://patchwork.linuxtv.org/project/linux-media/cover/20200804192939.2251988-1-helen.ko...@collabora.com/
> v4:
> https://patchwork.linuxtv.org/project/linux-media/cover/20200717115435.2632623-1-helen.ko...@collabora.com/
> v3: https://patchwork.linuxtv.org/cover/59345/
> v2: https://patchwork.kernel.org/project/linux-media/list/?series=101153
> v1: https://patchwork.kernel.org/project/linux-media/list/?series=93707
>
> Conversion layer:
> =
>
> * Old drivers implementing only ops->vidioc_*_fmt_vid_cap supports
> VIDIOC_*_EXT_PIX_FMT automatically with limitations[1].
>
> * New drivers implementing only ops->vidioc_*_ext_pix_fmt_vid_cap supports
> VIDIOC_*_FMT automatically.
>
> * Old drivers implementing only ops->vidioc_*buf support
> VIDIOC_EXT_*BUF automatically with limitations[2].
>
> * New drivers should implement both ops->vidioc_*buf and ops->vidioc_*buf
> to overcome limitations[2] and support both APIs.
> Which is easy with vb2:
> static const struct v4l2_ioctl_ops ioctl_ops = {
> ...
> + .vidioc_ext_qbuf = vb2_ioctl_ext_qbuf,
> + .vidioc_ext_dqbuf = vb2_ioctl_ext_dqbuf,
> ...
> }
> ...
> + /* Inform vb2 how to split the memory buffer in case a single one
> is used */
> + vb2_set_pixelformat(dev->pixelformat)
>
> [1] There are some limitations in the conversion such as modifiers that are
> ignored when converting v4l2_ext_pix_format to v4l_format
>
> [2] Ext API allows a single buffer with planes placed in random locations,
> which is not possible with v4l2_buffer.
>
>
> Major changes in v6:
>
>
> Fixed color planes vs memory planes handling.
>
> Removed VIDIOC_EXT_PREPARE_BUF, since this is an optimization, it doesn't
> blocks
> the API, we can add it later (my goal was to simplify this patchset).
>
> Removed VIDIOC_EXT_CREATE_BUFS, since this is useful only to MMAP (thus low
> priority)
> with the new format.
> Classic VIDIOC_CREATE_BUFS and VIDIOC_REQBUFS can still be used.
>
> Reformulated conversion layer as per above.
>
> Removed conversions in vb2, it is easier to add hooks to drivers.
>
> Fixed vb2 to allow Ext API only to Video types.
>
> API updates:
> * remove buffer and plane lengths
> * move `memory` field to v4l2_ext_buffer instead of v4l2_ext_plane
> * remove struct v4l2_plane_ext_pix_format
> * reordering
>
> Make Ext API valid only for Video types, and not for touch, vbi, meta, etc.
>
> Sereval code refactoring, simplification, fixes and applied suggestions from
> v5.
>
> New API (for convenience):
> ==
>
> int ioctl(int fd, VIDIOC_G_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp)
> int ioctl(int fd, VIDIOC_S_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp)
> int ioctl(int fd, VIDIOC_TRY_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp)
> int ioctl(int fd, VIDIOC_EXT_QBUF, struct v4l2_ext_buffer *argp)
> int ioctl(int fd, VIDIOC_EXT_DQBUF, struct v4l2_ext_buffer *argp)
>
> struct v4l2_ext_pix_format {
> __u32 type;
> __u32 width;
> __u32 height;
> __u32 field;
> struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES];
> __u32 pixelformat;
> __u64 modifier;
> __u32 colorspace;
> __u32 xfer_func;
> union {
> __u32 ycbcr_enc;
> __u32 hsv_enc;
> };
> __u32 quantization;
> __u32 reserved[9];
> };
>
> struct v4l2_ext_buffer {
> __u32 index;
>
Re: [PATCH 2/3] media: videodev2.h: clarify v4l2_pix_format_mplane.sizeimage docs when to set to zero
On 1/25/21 6:31 AM, Hans Verkuil wrote:
> On 14/01/2021 19:01, Helen Koike wrote:
>> sizeimage field should be set to zero for unused planes, even when
>> v4l2_pix_format_mplane.num_planes is smaller then the index of planes.
>
> then -> than
Ack.
>
>>
>> Signed-off-by: Helen Koike
>>
>> ---
>>
>> I caught this with v4l2-compliance, which throws an error if we dirty
>> planes, even if invalid, so I would like to make it clear in the docs.
>
> What is the error? And with which driver?
I was implementing conversions to/from Ext API, and I thought v4l2-compliance
wasn't happy if I didn't zero the other entries, but I'm trying to reproduce
it now by adding a non-zero value to sizeimage and I can't reproduce it, so
it was probably my mistake.
Please ignore this patch and sorry for the noise.
>
> I wonder if this isn't a v4l2-compliance bug. And if we want this to be
> zeroed, then it wouldn't it be better to do that in the V4L2 core rather
> than bother drivers with this?
>
>> ---
>> include/uapi/linux/videodev2.h | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index 79dbde3bcf8d..d9b7c9177605 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2227,6 +2227,7 @@ struct v4l2_mpeg_vbi_fmt_ivtv {
>> * struct v4l2_plane_pix_format - additional, per-plane format definition
>> * @sizeimage: maximum size in bytes required for data, for
>> which
>> * this plane will be used
>> + * Drivers should be set it zero for unused planes.
>
> This sentence is a bit garbled.
>
> You probably meant: Drivers must set this to zero for unused planes.
>
> But it makes no sense to just zero this field. I would zero the whole struct
> contents for the unused planes.
>
>> * @bytesperline: distance in bytes between the leftmost pixels in two
>> * adjacent lines
>> */
>>
>
> The API doesn't mention whether unused plane formats should be zeroed or not,
> but it does make sense that they are. I don't think that the userspace API
> should be changed (esp. since there are apparently already drivers that do
> not zero these unused plane formats), but it makes sense that the compliance
> test does verify this, and that the V4L2 core would zero unused plane formats.
>
> I never like it when undefined values are allowed in an API, so it makes sense
> that this is done.
Ack.
Thanks
Helen
>
> Regards,
>
> Hans
>
[PATCH v2] media: doc: pixfmt-yuv: Fix 4:4:4 subsampling info
YUV 4:4:4 is not subsampled, fix this in the docs.
Fixes: da785536e007 ("media: doc: pixfmt-yuv: Move all semi-planar YUV formats
to common file")
Signed-off-by: Helen Koike
---
Changes in v2:
- s/No sub-sampling/The chroma plane is not subsampled/ (Laurent)
- Fixed description regarding the number of bytes (Laurent)
---
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
index 7d4d39201a3f..1e0db602cc1b 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
@@ -396,9 +396,9 @@ number of lines as the luma plane.
NV24 and NV42
-
-Semi-planar YUV 4:4:4 formats. The chroma plane is subsampled by 2 in the
-horizontal direction. Chroma lines contain half the number of pixels and the
-same number of bytes as luma lines, and the chroma plane contains the same
+Semi-planar YUV 4:4:4 formats. The chroma plane is not subsampled.
+Chroma lines contain the same number of pixels and twice the
+number of bytes as luma lines, and the chroma plane contains the same
number of lines as the luma plane.
.. flat-table:: Sample 4x4 NV24 Image
--
2.30.0
Re: [PATCH] media: doc: pixfmt-yuv: Fix 4:4:4 subsampling info
On 1/25/21 10:57 AM, Helen Koike wrote:
>
>
> On 1/23/21 6:56 AM, Laurent Pinchart wrote:
>> Hi Helen,
>>
>> Thank you for the patch.
>>
>> On Fri, Jan 22, 2021 at 03:27:23PM -0300, Helen Koike wrote:
>>> YUV 4:4:4 is not subsampled, fix this in the docs.
>>>
>>> Fixes: da785536e007 ("media: doc: pixfmt-yuv: Move all semi-planar YUV
>>> formats to common file")
>>> Signed-off-by: Helen Koike
>>> ---
>>> Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst | 4 ++--
>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>>> b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>>> index 7d4d39201a3f..bcb4ef24c334 100644
>>> --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>>> @@ -396,8 +396,8 @@ number of lines as the luma plane.
>>> NV24 and NV42
>>> -
>>>
>>> -Semi-planar YUV 4:4:4 formats. The chroma plane is subsampled by 2 in the
>>> -horizontal direction. Chroma lines contain half the number of pixels and
>>> the
>>> +Semi-planar YUV 4:4:4 formats. No sub-sampling.
>>
>> "The chroma plane is not subsampled." ?
>
> Ack.
>
>>
>>> +Chroma lines contain the same number of pixels and the
>>> same number of bytes as luma lines, and the chroma plane contains the same
>>> number of lines as the luma plane.
>>
>> That's not quite right, the chroma lines contain twice the number of
>> pixels and bytes, as there's one Cb and one Cr value in the chroma line
>> for each Y value in the luma line.
Actually, it is the same number o pixels, but twice the number o bytes.
Since a trio (YCbCr) compose a pixel.
At least this is how I understand comparing the logic of the text description
of NV16 YUV4:2:2.
Regards,
Helen
>>
>>
>> Maybe the text could be reflowed ?
>>
>
> Ack.
>
> I'll submit v2 updating the text.
>
> Thanks,
> Helen
>
Re: [PATCH] media: doc: pixfmt-yuv: Fix 4:4:4 subsampling info
On 1/23/21 6:56 AM, Laurent Pinchart wrote:
> Hi Helen,
>
> Thank you for the patch.
>
> On Fri, Jan 22, 2021 at 03:27:23PM -0300, Helen Koike wrote:
>> YUV 4:4:4 is not subsampled, fix this in the docs.
>>
>> Fixes: da785536e007 ("media: doc: pixfmt-yuv: Move all semi-planar YUV
>> formats to common file")
>> Signed-off-by: Helen Koike
>> ---
>> Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>> b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>> index 7d4d39201a3f..bcb4ef24c334 100644
>> --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
>> @@ -396,8 +396,8 @@ number of lines as the luma plane.
>> NV24 and NV42
>> -
>>
>> -Semi-planar YUV 4:4:4 formats. The chroma plane is subsampled by 2 in the
>> -horizontal direction. Chroma lines contain half the number of pixels and the
>> +Semi-planar YUV 4:4:4 formats. No sub-sampling.
>
> "The chroma plane is not subsampled." ?
Ack.
>
>> +Chroma lines contain the same number of pixels and the
>> same number of bytes as luma lines, and the chroma plane contains the same
>> number of lines as the luma plane.
>
> That's not quite right, the chroma lines contain twice the number of
> pixels and bytes, as there's one Cb and one Cr value in the chroma line
> for each Y value in the luma line.
>
>
> Maybe the text could be reflowed ?
>
Ack.
I'll submit v2 updating the text.
Thanks,
Helen
[PATCH] media: doc: pixfmt-yuv: Fix 4:4:4 subsampling info
YUV 4:4:4 is not subsampled, fix this in the docs.
Fixes: da785536e007 ("media: doc: pixfmt-yuv: Move all semi-planar YUV formats
to common file")
Signed-off-by: Helen Koike
---
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
index 7d4d39201a3f..bcb4ef24c334 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
@@ -396,8 +396,8 @@ number of lines as the luma plane.
NV24 and NV42
-
-Semi-planar YUV 4:4:4 formats. The chroma plane is subsampled by 2 in the
-horizontal direction. Chroma lines contain half the number of pixels and the
+Semi-planar YUV 4:4:4 formats. No sub-sampling.
+Chroma lines contain the same number of pixels and the
same number of bytes as luma lines, and the chroma plane contains the same
number of lines as the luma plane.
--
2.30.0
[RFC PATCH v6 09/11] media: vivid: Convert to v4l2_ext_pix_format
Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
Duplicate v4l2_ioctl_ops for touch devices. This is done to force the
framework to use the ext hooks when the classic Api is used from
userspace in Vid devices, and to keep touch devices with classic hook.
Signed-off-by: Boris Brezillon
Signed-off-by: Helen Koike
---
Changes in v6:
- Update with new format and buffer structs
- duplicate v4l2_ioctl_ops for touch devices.
Changes in v4:
- Update with new format and buffer structs
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- New patch
---
---
drivers/media/test-drivers/vivid/vivid-core.c | 207 ++
.../media/test-drivers/vivid/vivid-vid-cap.c | 202 ++---
.../media/test-drivers/vivid/vivid-vid-cap.h | 15 +-
.../media/test-drivers/vivid/vivid-vid-out.c | 197 ++---
.../media/test-drivers/vivid/vivid-vid-out.h | 15 +-
5 files changed, 271 insertions(+), 365 deletions(-)
diff --git a/drivers/media/test-drivers/vivid/vivid-core.c
b/drivers/media/test-drivers/vivid/vivid-core.c
index eacb23808c7e..cebd2032e209 100644
--- a/drivers/media/test-drivers/vivid/vivid-core.c
+++ b/drivers/media/test-drivers/vivid/vivid-core.c
@@ -477,76 +477,6 @@ static int vivid_s_input(struct file *file, void *priv,
unsigned int i)
return vidioc_s_input(file, priv, i);
}
-static int vivid_enum_fmt_cap(struct file *file, void *priv,
- struct v4l2_fmtdesc *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_enum_fmt_tch(file, priv, f);
- return vivid_enum_fmt_vid(file, priv, f);
-}
-
-static int vivid_g_fmt_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch(file, priv, f);
- return vidioc_g_fmt_vid_cap(file, priv, f);
-}
-
-static int vivid_try_fmt_cap(struct file *file, void *priv,
-struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch(file, priv, f);
- return vidioc_try_fmt_vid_cap(file, priv, f);
-}
-
-static int vivid_s_fmt_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch(file, priv, f);
- return vidioc_s_fmt_vid_cap(file, priv, f);
-}
-
-static int vivid_g_fmt_cap_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch_mplane(file, priv, f);
- return vidioc_g_fmt_vid_cap_mplane(file, priv, f);
-}
-
-static int vivid_try_fmt_cap_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch_mplane(file, priv, f);
- return vidioc_try_fmt_vid_cap_mplane(file, priv, f);
-}
-
-static int vivid_s_fmt_cap_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch_mplane(file, priv, f);
- return vidioc_s_fmt_vid_cap_mplane(file, priv, f);
-}
-
static bool vivid_is_in_use(bool valid, struct video_device *vdev)
{
unsigned long flags;
@@ -656,24 +586,129 @@ static const struct v4l2_file_operations
vivid_radio_fops = {
.unlocked_ioctl = video_ioctl2,
};
+static const struct v4l2_ioctl_ops vivid_ioctl_ops_tch = {
+ .vidioc_querycap= vidioc_querycap,
+
+ .vidioc_enum_fmt_vid_cap = vivid_enum_fmt_tch,
+ .vidioc_g_fmt_vid_cap = vivid_g_fmt_tch,
+ .vidioc_try_fmt_vid_cap = vivid_g_fmt_tch,
+ .vidioc_s_fmt_vid_cap = vivid_g_fmt_tch,
+ .vidioc_g_fmt_vid_cap_mplane = vivid_g_fmt_tch_mplane,
+ .vidioc_try_fmt_vid_cap_mplane = vivid_g_fmt_tch_mplane,
+ .vidioc_s_fmt_vid_cap_mplane = vivid_g_fmt_tch_mplane,
+
+ .vidioc_g_selection = vidioc_g_selection,
+ .vidioc_s_selection = vidioc_s_selection,
+ .vidioc_g_pixelaspect = vidioc_g_pixelaspect,
+
+ .vidioc_g_fmt_vbi_cap = vidioc_g_fmt_vbi_cap,
+ .vidioc_try_fmt_vbi_cap = vidioc_g_fmt_vbi_cap,
+ .vidioc_s_fmt_vbi_cap =
[RFC PATCH v6 11/11] media: docs: add documentation for the Extended API
Add documentation and update references in current documentation for the Extended API. Signed-off-by: Helen Koike --- Changes in v6: - Update note saying ext_api should be used for new applications on newer kernels (Tomasz and Hans) - Fix typos pointed in v5 (Hand and Tomasz) - Change order, mention Ext first in format.rst (Tomasz) - Mention planes[i].offset should be set to zero for userptr - Remove ext_create_buf and ext_prep_buf from the docs - s/displayed/consumed for output (Tomasz) - Remove references for plane length - Drop EIO sentence mentioning signal loss (Hans) - Removed first half of the note in EIO (Tomas and Hans) - Update text to mention EXT_TRY_FMT is mandatory (Hans and Tomasz) - Remove requirement to fill `memory` field for dqbuf (Tomasz) - EXT_DQBUF sets `m` field to zero (Tomasz for DMA-fd) Changes in v5: - new patch --- .../userspace-api/media/v4l/buffer.rst| 5 + .../userspace-api/media/v4l/common.rst| 1 + .../userspace-api/media/v4l/dev-capture.rst | 6 + .../userspace-api/media/v4l/dev-output.rst| 6 + .../userspace-api/media/v4l/ext-api.rst | 89 + .../userspace-api/media/v4l/format.rst| 18 +- .../userspace-api/media/v4l/user-func.rst | 5 + .../media/v4l/vidioc-ext-qbuf.rst | 188 ++ .../media/v4l/vidioc-g-ext-pix-fmt.rst| 116 +++ .../userspace-api/media/v4l/vidioc-qbuf.rst | 2 +- 10 files changed, 431 insertions(+), 5 deletions(-) create mode 100644 Documentation/userspace-api/media/v4l/ext-api.rst create mode 100644 Documentation/userspace-api/media/v4l/vidioc-ext-qbuf.rst create mode 100644 Documentation/userspace-api/media/v4l/vidioc-g-ext-pix-fmt.rst diff --git a/Documentation/userspace-api/media/v4l/buffer.rst b/Documentation/userspace-api/media/v4l/buffer.rst index 1b0fdc160533..89652fa7a098 100644 --- a/Documentation/userspace-api/media/v4l/buffer.rst +++ b/Documentation/userspace-api/media/v4l/buffer.rst @@ -21,6 +21,11 @@ such as pointers and sizes for each plane, are stored in struct :c:type:`v4l2_plane` instead. In that case, struct :c:type:`v4l2_buffer` contains an array of plane structures. +.. note:: + +For modern applications on newer kernels, these operations are replaced +by their :ref:`ext_api` counterparts, which should be used instead. + Dequeued video buffers come with timestamps. The driver decides at which part of the frame and with which clock the timestamp is taken. Please see flags in the masks ``V4L2_BUF_FLAG_TIMESTAMP_MASK`` and diff --git a/Documentation/userspace-api/media/v4l/common.rst b/Documentation/userspace-api/media/v4l/common.rst index 8c263c5a85d8..61a64065f9f3 100644 --- a/Documentation/userspace-api/media/v4l/common.rst +++ b/Documentation/userspace-api/media/v4l/common.rst @@ -53,6 +53,7 @@ applicable to all devices. ext-ctrls-detect fourcc format +ext-api planar-apis selection-api crop diff --git a/Documentation/userspace-api/media/v4l/dev-capture.rst b/Documentation/userspace-api/media/v4l/dev-capture.rst index fe58fd450e2f..73580e16057c 100644 --- a/Documentation/userspace-api/media/v4l/dev-capture.rst +++ b/Documentation/userspace-api/media/v4l/dev-capture.rst @@ -93,6 +93,12 @@ and :ref:`VIDIOC_S_FMT ` ioctl, even if :ref:`VIDIOC_S_FMT ` does. :ref:`VIDIOC_TRY_FMT ` is optional. +.. note:: + +For modern applications on newer kernels, these operations are replaced +by their :ref:`ext_api` counterparts, which should be used instead. + + Reading Images == diff --git a/Documentation/userspace-api/media/v4l/dev-output.rst b/Documentation/userspace-api/media/v4l/dev-output.rst index eadcb4aa813b..676578c099b6 100644 --- a/Documentation/userspace-api/media/v4l/dev-output.rst +++ b/Documentation/userspace-api/media/v4l/dev-output.rst @@ -90,6 +90,12 @@ and :ref:`VIDIOC_S_FMT ` ioctl, even if :ref:`VIDIOC_S_FMT ` does. :ref:`VIDIOC_TRY_FMT ` is optional. +.. note:: + +For modern applications on newer kernels, these operations are replaced +by their :ref:`ext_api` counterparts, which should be used instead. + + Writing Images == diff --git a/Documentation/userspace-api/media/v4l/ext-api.rst b/Documentation/userspace-api/media/v4l/ext-api.rst new file mode 100644 index ..e73d5b77a550 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/ext-api.rst @@ -0,0 +1,89 @@ +.. Permission is granted to copy, distribute and/or modify this +.. document under the terms of the GNU Free Documentation License, +.. Version 1.1 or any later version published by the Free Software +.. Foundation, with no Invariant Sections, no Front-Cover Texts +.. and no Back-Cover Texts. A copy of the license is included at +.. Documentation/userspace-api/media/fdl-appendix.rst. +.. +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections + +.. _ext_api: + +* +Extended API +* + +Introduction
[RFC PATCH v6 10/11] media: vimc: Convert to v4l2_ext_pix_format
Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
Signed-off-by: Boris Brezillon
Signed-off-by: Helen Koike
---
Changes in v6:
- Update with new format and buffer structs
Changes in v4:
- Update with new format and buffer structs
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- New patch
---
.../media/test-drivers/vimc/vimc-capture.c| 54 ++-
drivers/media/test-drivers/vimc/vimc-common.c | 6 +--
drivers/media/test-drivers/vimc/vimc-common.h | 2 +-
3 files changed, 32 insertions(+), 30 deletions(-)
diff --git a/drivers/media/test-drivers/vimc/vimc-capture.c
b/drivers/media/test-drivers/vimc/vimc-capture.c
index 729614d19002..51191dc9661b 100644
--- a/drivers/media/test-drivers/vimc/vimc-capture.c
+++ b/drivers/media/test-drivers/vimc/vimc-capture.c
@@ -15,7 +15,7 @@
struct vimc_cap_device {
struct vimc_ent_device ved;
struct video_device vdev;
- struct v4l2_pix_format format;
+ struct v4l2_ext_pix_format format;
struct vb2_queue queue;
struct list_head buf_list;
/*
@@ -32,7 +32,8 @@ struct vimc_cap_device {
struct media_pad pad;
};
-static const struct v4l2_pix_format fmt_default = {
+static const struct v4l2_ext_pix_format fmt_default = {
+ .type = V4L2_BUF_TYPE_VIDEO_CAPTURE,
.width = 640,
.height = 480,
.pixelformat = V4L2_PIX_FMT_RGB24,
@@ -63,7 +64,7 @@ static int vimc_cap_querycap(struct file *file, void *priv,
}
static void vimc_cap_get_format(struct vimc_ent_device *ved,
- struct v4l2_pix_format *fmt)
+ struct v4l2_ext_pix_format *fmt)
{
struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
ved);
@@ -72,19 +73,18 @@ static void vimc_cap_get_format(struct vimc_ent_device *ved,
}
static int vimc_cap_g_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
struct vimc_cap_device *vcap = video_drvdata(file);
- f->fmt.pix = vcap->format;
+ *f = vcap->format;
return 0;
}
static int vimc_cap_try_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *format)
{
- struct v4l2_pix_format *format = &f->fmt.pix;
const struct vimc_pix_map *vpix;
format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
@@ -99,8 +99,10 @@ static int vimc_cap_try_fmt_vid_cap(struct file *file, void
*priv,
vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
}
/* TODO: Add support for custom bytesperline values */
- format->bytesperline = format->width * vpix->bpp;
- format->sizeimage = format->bytesperline * format->height;
+ memset(format->plane_fmt, 0, sizeof(format->plane_fmt));
+ format->plane_fmt[0].bytesperline = format->width * vpix->bpp;
+ format->plane_fmt[0].sizeimage = format->plane_fmt[0].bytesperline *
+format->height;
if (format->field == V4L2_FIELD_ANY)
format->field = fmt_default.field;
@@ -114,7 +116,7 @@ static int vimc_cap_try_fmt_vid_cap(struct file *file, void
*priv,
}
static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
struct vimc_cap_device *vcap = video_drvdata(file);
int ret;
@@ -136,12 +138,10 @@ static int vimc_cap_s_fmt_vid_cap(struct file *file, void
*priv,
vcap->format.quantization, vcap->format.xfer_func,
vcap->format.ycbcr_enc,
/* new */
- f->fmt.pix.width, f->fmt.pix.height,
- f->fmt.pix.pixelformat, f->fmt.pix.colorspace,
- f->fmt.pix.quantization, f->fmt.pix.xfer_func,
- f->fmt.pix.ycbcr_enc);
+ f->width, f->height, f->pixelformat, f->colorspace,
+ f->quantization, f->xfer_func, f->ycbcr_enc);
- vcap->format = f->fmt.pix;
+ vcap->format = *f;
return 0;
}
@@ -205,9 +205,9 @@ static const struct v4l2_file_operations vimc_cap_fops = {
static const struct v4l2_ioctl_ops vimc_cap_ioctl_ops = {
.vidioc_querycap = vimc_cap_querycap,
- .vidioc_g_fmt_vid_cap = vimc_cap_g_fmt_vid_cap,
- .vidioc_s_fmt_vid_cap = vimc_cap_s_fmt_vid_cap,
- .vidioc_try_fmt_vid_cap = vimc_cap_try_fmt_vid_cap,
+ .vidioc_g_ext_pix_fmt_vid_
[RFC PATCH v6 08/11] media: mediabus: Add helpers to convert a ext_pix format to/from a mbus_fmt
Just a new version of v4l2_fill_mbus_format() and v4l2_fill_ext_pix_format()
to deal with the new v4l2_ext_pix_format struct.
This is needed to convert the VIMC driver to the EXT_FMT/EXT_BUF iocts.
Signed-off-by: Boris Brezillon
Signed-off-by: Helen Koike
---
Changes in v6:
- Rename v4l2_fill_ext_pix_format() to v4l2_fill_ext_pix_format_from_mbus()
(Tomasz)
Changes in v4:
- Add helper v4l2_fill_ext_pix_format()
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- New patch
---
include/media/v4l2-mediabus.h | 42 +++
1 file changed, 42 insertions(+)
diff --git a/include/media/v4l2-mediabus.h b/include/media/v4l2-mediabus.h
index 841e190aedd9..055e2abbc1dd 100644
--- a/include/media/v4l2-mediabus.h
+++ b/include/media/v4l2-mediabus.h
@@ -209,4 +209,46 @@ v4l2_fill_mbus_format_mplane(struct v4l2_mbus_framefmt
*mbus_fmt,
mbus_fmt->xfer_func = pix_mp_fmt->xfer_func;
}
+/**
+ * v4l2_fill_ext_pix_format_from_mbus - Ancillary routine that fills a &struct
+ * v4l2_ext_pix_format fields from a &struct v4l2_mbus_framefmt.
+ *
+ * @pix_fmt: pointer to &struct v4l2_ext_pix_format to be filled
+ * @mbus_fmt: pointer to &struct v4l2_mbus_framefmt to be used as model
+ */
+static inline void
+v4l2_fill_ext_pix_format_from_mbus(struct v4l2_ext_pix_format *pix_fmt,
+ const struct v4l2_mbus_framefmt *mbus_fmt)
+{
+ pix_fmt->width = mbus_fmt->width;
+ pix_fmt->height = mbus_fmt->height;
+ pix_fmt->field = mbus_fmt->field;
+ pix_fmt->colorspace = mbus_fmt->colorspace;
+ pix_fmt->ycbcr_enc = mbus_fmt->ycbcr_enc;
+ pix_fmt->quantization = mbus_fmt->quantization;
+ pix_fmt->xfer_func = mbus_fmt->xfer_func;
+}
+
+/**
+ * v4l2_fill_mbus_format_ext - Ancillary routine that fills a &struct
+ * v4l2_mbus_framefmt from a &struct v4l2_ext_pix_format.
+ *
+ * @mbus_fmt: pointer to &struct v4l2_mbus_framefmt to be filled
+ * @pix_fmt: pointer to &struct v4l2_ext_pix_format to be used as model
+ * @code: data format code (from &enum v4l2_mbus_pixelcode)
+ */
+static inline void
+v4l2_fill_mbus_format_ext(struct v4l2_mbus_framefmt *mbus_fmt,
+ const struct v4l2_ext_pix_format *pix_fmt, u32 code)
+{
+ mbus_fmt->width = pix_fmt->width;
+ mbus_fmt->height = pix_fmt->height;
+ mbus_fmt->field = pix_fmt->field;
+ mbus_fmt->colorspace = pix_fmt->colorspace;
+ mbus_fmt->ycbcr_enc = pix_fmt->ycbcr_enc;
+ mbus_fmt->quantization = pix_fmt->quantization;
+ mbus_fmt->xfer_func = pix_fmt->xfer_func;
+ mbus_fmt->code = code;
+}
+
#endif
--
2.29.2
[RFC PATCH v6 07/11] media: vimc: use vb2_ioctls_ext_{d}qbuf hooks
Add vb2 ext hooks and call vb2_set_pixelformat().
This allows more flexibility with buffer handling.
Signed-off-by: Helen Koike
---
Changes in v6:
- New patch to exemplify how drivers would easily support features from Ext Buf
---
drivers/media/test-drivers/vimc/vimc-capture.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/media/test-drivers/vimc/vimc-capture.c
b/drivers/media/test-drivers/vimc/vimc-capture.c
index 5e9fd902cd37..729614d19002 100644
--- a/drivers/media/test-drivers/vimc/vimc-capture.c
+++ b/drivers/media/test-drivers/vimc/vimc-capture.c
@@ -217,6 +217,8 @@ static const struct v4l2_ioctl_ops vimc_cap_ioctl_ops = {
.vidioc_querybuf = vb2_ioctl_querybuf,
.vidioc_qbuf = vb2_ioctl_qbuf,
.vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_ext_qbuf = vb2_ioctl_ext_qbuf,
+ .vidioc_ext_dqbuf = vb2_ioctl_ext_dqbuf,
.vidioc_expbuf = vb2_ioctl_expbuf,
.vidioc_streamon = vb2_ioctl_streamon,
.vidioc_streamoff = vb2_ioctl_streamoff,
@@ -389,6 +391,7 @@ static void *vimc_cap_process_frame(struct vimc_ent_device
*ved,
/* Set it as ready */
vb2_set_plane_payload(&vimc_buf->vb2.vb2_buf, 0,
vcap->format.sizeimage);
+ vb2_set_pixelformat(&vimc_buf->vb2.vb2_buf, vcap->format.pixelformat);
vb2_buffer_done(&vimc_buf->vb2.vb2_buf, VB2_BUF_STATE_DONE);
return NULL;
}
--
2.29.2
[RFC PATCH v6 05/11] media: videobuf2: Expose helpers for Ext qbuf/dqbuf
To overcome the limitations of Ext ioctls, that is being converted to
classic hooks, add helpers to allow applications support layouts such as
using the same buffer with planes in different offsets.
To use the new hooks, drivers should:
static const struct v4l2_ioctl_ops ioctl_ops = {
...
+ .vidioc_ext_qbuf = vb2_ioctl_ext_qbuf,
+ .vidioc_ext_dqbuf = vb2_ioctl_ext_dqbuf,
...
}
+ vb2_set_pixelformat(dev->pixelformat)
The old hooks should be kept to keep the driver compatible with classic
Api.
Add mem_offset field to struct vb2_plane, to allow tracking where the plane
starts in a buffer, as defined from userspace.
When returning the buffer to userspace, this offset can be adjusted
depending on the data_offset returned from the driver.
Add pixelformat field to struct vb2_buffer, to allow vb2 to know how to
decompose the payload set with vb2_set_plane_payload() into color planes
when a single memory buffer is used.
Signed-off-by: Boris Brezillon
Signed-off-by: Helen Koike
---
Changes in v6:
This patch is based on original "media: videobuf2: Expose helpers to implement
the _ext_fmt and _ext_buf hooks"
- This patch was completly refactored
- Conversions from v4l2_buffer to v4l2_ext_buffer was removed from vb2.
Both v4l2_buffer and v4l2_ext_buffer need to be supported, since Ext is only
valid
for video devices, v4l2_buffer needs to be supported for vbi, meta, and
others.
- Zero v4l2_ext_buffer.planes.m field (Tomasz for DMA-fd)
Changes in v5:
- Update with new format and buffer structs
- Updated commit message with the uAPI prefix
Changes in v4:
- Update with new format and buffer structs
- Fix some bugs caught by v4l2-compliance
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- New patch
---
.../media/common/videobuf2/videobuf2-core.c | 46 ++-
.../media/common/videobuf2/videobuf2-v4l2.c | 326 +-
include/media/videobuf2-core.h| 33 +-
include/media/videobuf2-v4l2.h| 8 +-
4 files changed, 388 insertions(+), 25 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
b/drivers/media/common/videobuf2/videobuf2-core.c
index 02281d13505f..b034bd525ea4 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1569,8 +1569,10 @@ static int vb2_start_streaming(struct vb2_queue *q)
return ret;
}
-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
- struct media_request *req)
+static int
+vb2_core_qbuf_common(struct vb2_queue *q, unsigned int index,
+struct v4l2_buffer *pb, struct v4l2_ext_buffer *eb,
+struct media_request *req)
{
struct vb2_buffer *vb;
int ret;
@@ -1642,6 +1644,9 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int
index, void *pb,
if (pb) {
call_void_bufop(q, copy_timestamp, vb, pb);
call_void_bufop(q, fill_user_buffer, vb, pb);
+ } else if (eb) {
+ call_void_bufop(q, copy_timestamp_ext, vb, eb);
+ call_void_bufop(q, fill_user_ext_buffer, vb, eb);
}
dprintk(q, 2, "qbuf of buffer %d succeeded\n", vb->index);
@@ -1680,6 +1685,8 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int
index, void *pb,
if (pb)
call_void_bufop(q, copy_timestamp, vb, pb);
+ else if (eb)
+ call_void_bufop(q, copy_timestamp_ext, vb, eb);
trace_vb2_qbuf(q, vb);
@@ -1693,6 +1700,8 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int
index, void *pb,
/* Fill buffer information for the userspace */
if (pb)
call_void_bufop(q, fill_user_buffer, vb, pb);
+ else if (eb)
+ call_void_bufop(q, fill_user_ext_buffer, vb, eb);
/*
* If streamon has been called, and we haven't yet called
@@ -1710,8 +1719,21 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int
index, void *pb,
dprintk(q, 2, "qbuf of buffer %d succeeded\n", vb->index);
return 0;
}
+
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+ struct media_request *req)
+{
+ return vb2_core_qbuf_common(q, index, pb, NULL, req);
+}
EXPORT_SYMBOL_GPL(vb2_core_qbuf);
+int vb2_core_ext_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+ struct media_request *req)
+{
+ return vb2_core_qbuf_common(q, index, NULL, pb, req);
+}
+EXPORT_SYMBOL_GPL(vb2_core_ext_qbuf);
+
/*
* __vb2_wait_for_done_vb() - wait for a buffer to become available
* for dequeuing
@@ -1861,8 +1883,10 @@ static void __vb2_dqbuf(struct vb2_buffer *vb)
call_void_bufop(q, init_buffer, vb);
}
-int vb2_core_dqbuf(struct
[RFC PATCH v6 06/11] media: vivid: use vb2_ioctls_ext_{d}qbuf hooks
Add vb2 ext hooks and call vb2_set_pixelformat().
This allows more flexibility with buffer handling.
Signed-off-by: Helen Koike
---
Changes in v6:
- New patch to exemplify how drivers would easily support features from Ext Buf
---
drivers/media/test-drivers/vivid/vivid-core.c| 2 ++
drivers/media/test-drivers/vivid/vivid-vid-cap.c | 1 +
drivers/media/test-drivers/vivid/vivid-vid-out.c | 1 +
3 files changed, 4 insertions(+)
diff --git a/drivers/media/test-drivers/vivid/vivid-core.c
b/drivers/media/test-drivers/vivid/vivid-core.c
index 0dc65ef3aa14..eacb23808c7e 100644
--- a/drivers/media/test-drivers/vivid/vivid-core.c
+++ b/drivers/media/test-drivers/vivid/vivid-core.c
@@ -723,6 +723,8 @@ static const struct v4l2_ioctl_ops vivid_ioctl_ops = {
.vidioc_querybuf= vb2_ioctl_querybuf,
.vidioc_qbuf= vb2_ioctl_qbuf,
.vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_ext_qbuf= vb2_ioctl_ext_qbuf,
+ .vidioc_ext_dqbuf = vb2_ioctl_ext_dqbuf,
.vidioc_expbuf = vb2_ioctl_expbuf,
.vidioc_streamon= vb2_ioctl_streamon,
.vidioc_streamoff = vb2_ioctl_streamoff,
diff --git a/drivers/media/test-drivers/vivid/vivid-vid-cap.c
b/drivers/media/test-drivers/vivid/vivid-vid-cap.c
index b9caa4b26209..882b0c4a6591 100644
--- a/drivers/media/test-drivers/vivid/vivid-vid-cap.c
+++ b/drivers/media/test-drivers/vivid/vivid-vid-cap.c
@@ -169,6 +169,7 @@ static int vid_cap_buf_prepare(struct vb2_buffer *vb)
}
vb2_set_plane_payload(vb, p, size);
+ vb2_set_pixelformat(vb, dev->fmt_cap->fourcc);
vb->planes[p].data_offset = dev->fmt_cap->data_offset[p];
}
diff --git a/drivers/media/test-drivers/vivid/vivid-vid-out.c
b/drivers/media/test-drivers/vivid/vivid-vid-out.c
index ac1e981e8342..6be61112065d 100644
--- a/drivers/media/test-drivers/vivid/vivid-vid-out.c
+++ b/drivers/media/test-drivers/vivid/vivid-vid-out.c
@@ -136,6 +136,7 @@ static int vid_out_buf_prepare(struct vb2_buffer *vb)
return -EINVAL;
}
}
+ vb2_set_pixelformat(vb, dev->fmt_out->fourcc);
return 0;
}
--
2.29.2
[RFC PATCH v6 04/11] media: videobuf2-v4l2: reorganize flags handling
Reorganize flags handling to be easily reuseble when Ext functions get
added.
No logic is changed, just moving around code.
- Two new functions:
v4l2_clear_buffer_flags()
vb2_fill_vb2_v4l2_buffer_flags()
- set_buffer_cache_hints() receives a pointer to flags instead of the
v4l2_buffer object, making it undependent of this struct.
Signed-off-by: Helen Koike
---
Changes in v6:
- New patch
---
.../media/common/videobuf2/videobuf2-v4l2.c | 176 ++
1 file changed, 97 insertions(+), 79 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index bb642c0775d1..aa4f17ec0982 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -174,6 +174,43 @@ static void vb2_warn_zero_bytesused(struct vb2_buffer *vb)
pr_warn("use the actual size instead.\n");
}
+static void
+vb2_fill_vb2_v4l2_buffer_flags(struct vb2_buffer *vb,
+ u32 type, u32 field, u32 flags)
+{
+ struct vb2_queue *q = vb->vb2_queue;
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+ /* Zero flags that we handle */
+ vbuf->flags = flags & ~V4L2_BUFFER_MASK_FLAGS;
+ if (!vb->vb2_queue->copy_timestamp || V4L2_TYPE_IS_CAPTURE(type)) {
+ /*
+* Non-COPY timestamps and non-OUTPUT queues will get
+* their timestamp and timestamp source flags from the
+* queue.
+*/
+ vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
+ }
+
+ if (V4L2_TYPE_IS_OUTPUT(type)) {
+ /*
+* For output buffers mask out the timecode flag:
+* this will be handled later in vb2_qbuf().
+* The 'field' is valid metadata for this output buffer
+* and so that needs to be copied here.
+*/
+ vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
+ vbuf->field = field;
+ if (!(q->subsystem_flags &
VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF))
+ vbuf->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF;
+ } else {
+ /* Zero any output buffer flags as this is a capture buffer */
+ vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
+ /* Zero last flag, this is a signal from driver to userspace */
+ vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
+ }
+}
+
static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer
*b)
{
struct vb2_queue *q = vb->vb2_queue;
@@ -310,41 +347,14 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer
*vb, struct v4l2_buffer *b
}
- /* Zero flags that we handle */
- vbuf->flags = b->flags & ~V4L2_BUFFER_MASK_FLAGS;
- if (!vb->vb2_queue->copy_timestamp || V4L2_TYPE_IS_CAPTURE(b->type)) {
- /*
-* Non-COPY timestamps and non-OUTPUT queues will get
-* their timestamp and timestamp source flags from the
-* queue.
-*/
- vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
- }
-
- if (V4L2_TYPE_IS_OUTPUT(b->type)) {
- /*
-* For output buffers mask out the timecode flag:
-* this will be handled later in vb2_qbuf().
-* The 'field' is valid metadata for this output buffer
-* and so that needs to be copied here.
-*/
- vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
- vbuf->field = b->field;
- if (!(q->subsystem_flags &
VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF))
- vbuf->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF;
- } else {
- /* Zero any output buffer flags as this is a capture buffer */
- vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
- /* Zero last flag, this is a signal from driver to userspace */
- vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
- }
+ vb2_fill_vb2_v4l2_buffer_flags(vb, b->type, b->field, b->flags);
return 0;
}
static void set_buffer_cache_hints(struct vb2_queue *q,
struct vb2_buffer *vb,
- struct v4l2_buffer *b)
+ u32 *flags)
{
/*
* DMA exporter should take care of cache syncs, so we can avoid
@@ -370,8 +380,8 @@ static void set_buffer_cache_hints(struct vb2_queue *q,
* space hints. That's to indicate to userspace that these
* flags won't work.
*/
- b->flags &= ~V4L2_BUF_FLAG_NO_CACH
[RFC PATCH v6 03/11] media: v4l2: Add extended buffer (de)queue operations for video types
Those extended buffer ops have several purpose:
1/ Fix y2038 issues by converting the timestamp into an u64 counting
the number of ns elapsed since 1970
2/ Unify single/multiplanar handling
3/ Add a new start offset field to each v4l2 plane buffer info struct
to support the case where a single buffer object is storing all
planes data, each one being placed at a different offset
New hooks are created in v4l2_ioctl_ops so that drivers can start using
these new objects.
Note that the timecode field is gone, since there doesn't seem to be
in-kernel users. We can be added back in the reserved area if needed or
use the Request API to collect more metadata information from the
frame.
Signed-off-by: Hans Verkuil
Signed-off-by: Boris Brezillon
Signed-off-by: Helen Koike
---
Changes in v6:
This patch was completely refactored, and based on previous version from
Hans and Boris.
- Refactor conversions v4l2_buffer <-> v4l2_ext_buffer for (d)qbuf
- I removed EXT_CREATE_BUFS since it is basically only usefull to MMAP.
If this is going towards DMA-fd centric, then we can use the current
REQUESTBUF to switch to it, and we can think a better way to support
MMAP later if there are usecases.
I also moved memory field from v4l2_ext_plane to v4l2_ext_buffer,
since it is very unlikely to mix memory types, and REQUESTBUF can
switch the whole buffer object to a given type.
- I removed EXT_QUERYBUF, since it is only useful to MMAP, for the
same reason above.
- I removed EXT_PREPARE_BUF, since it is basically just an optimization,
we can add it later (my intention is to simplify this patchset).
- These ioctls are only valid for video types (and not for overlay,
vbi, touch, meta, etc).
- Refactor struct v4l2_ext_buffer and struct v4l2_ext_planes as
discussed with Tomasz:
- add bytesused back
- remove lenght field
- move memory field from planes to buffer object
- Fix order in documentation of struct v4l2_ext_buffer (Tomasz)
- Fix flags documentation of struct v4l2_ext_buffer, don't say when flags are
ignored (Tomasz)
- v4l_print_ext_buffer(): print request_fd and offset/userptr (Tomasz)
Changes in v5:
- migrate memory from v4l2_ext_buffer to v4l2_ext_plane
- return mem_offset to struct v4l2_ext_plane
- change sizes and reorder fields to avoid holes in the struct and make
it the same for 32 and 64 bits
Changes in v4:
- Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
- Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF
was that with VIDIOC_EXT_EXPBUF we could export multiple planes at once.
I think we can add this later, so I removed it from this RFC to simplify it.
- Remove num_planes field from struct v4l2_ext_buffer
- Add flags field to struct v4l2_ext_create_buffers
- Reformulate struct v4l2_ext_plane
- Fix some bugs caught by v4l2-compliance
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- Add reserved space to v4l2_ext_buffer so that new fields can be added
later on
Signed-off-by: Helen Koike
---
drivers/media/v4l2-core/v4l2-dev.c | 4 +
drivers/media/v4l2-core/v4l2-ioctl.c | 184 +++
include/media/v4l2-ioctl.h | 8 ++
include/uapi/linux/videodev2.h | 55
4 files changed, 251 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-dev.c
b/drivers/media/v4l2-core/v4l2-dev.c
index 5add58cb6d45..94c9f1e04704 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -664,6 +664,10 @@ static void determine_valid_ioctls(struct video_device
*vdev)
set_bit(_IOC_NR(VIDIOC_S_CROP), valid_ioctls);
SET_VALID_IOCTL(ops, VIDIOC_G_SELECTION, vidioc_g_selection);
SET_VALID_IOCTL(ops, VIDIOC_S_SELECTION, vidioc_s_selection);
+ SET_VALID_IOCTL(ops, VIDIOC_EXT_QBUF, vidioc_ext_qbuf);
+ SET_VALID_IOCTL(ops, VIDIOC_EXT_QBUF, vidioc_qbuf);
+ SET_VALID_IOCTL(ops, VIDIOC_EXT_DQBUF, vidioc_ext_dqbuf);
+ SET_VALID_IOCTL(ops, VIDIOC_EXT_DQBUF, vidioc_dqbuf);
}
if (is_meta && is_rx) {
/* metadata capture specific ioctls */
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index a9c07c0a73ec..ba633e7efd6d 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -533,6 +533,24 @@ static void v4l_print_buffer(const void *arg, bool
write_only)
tc->type, tc->flags, tc->frames, *(__u32
*)tc->userbits);
}
+static void v4l_print_ext_buffer(const void *arg, bool write_only)
+{
+ const struct v4l2_ext_buffer *e = arg;
+ unsigned int i;
+
+ pr_cont("%lld index=%d, type=%s, request_fd=%d, flags=0x%08llx,
field=%s, seq
[RFC PATCH v6 02/11] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
This is part of the multiplanar and singleplanar unification process.
v4l2_ext_pix_format is supposed to work for both cases.
We also add the concept of modifiers already employed in DRM to expose
HW-specific formats (like tiled or compressed formats) and allow
exchanging this information with the DRM subsystem in a consistent way.
Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in
v4l2_ext_format, other types will be rejected if you use the
{G,S,TRY}_EXT_PIX_FMT ioctls.
New hooks have been added to v4l2_ioctl_ops to support those new ioctls
in drivers, but, in the meantime, the core takes care of converting
{S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can
still work if the userspace app/lib uses the new ioctls.
The conversion is also done the other around to allow userspace
apps/libs using {S,G,TRY}_FMT to work with drivers implementing the
_ext_ hooks.
Signed-off-by: Boris Brezillon
Signed-off-by: Helen Koike
---
Changes in v6:
The main change here was fixing the conversion, so planes reflects color
planes,
and to implement this properly I made major refactors compared to the previous
version.
- struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format
instead (Tomasz)
- refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans)
- reorder colorimetry fields in struct v4l2_ext_pix_format (Hans)
- do not set Ext ioctls as valid for vid_out_overlay (Tomasz)
- refactor conversion functions, so planes are color planes (Tomasz)
- Don't explicitly check for e->modifier != 0 in
v4l2_ext_pix_format_to_format() (Tomasz)
- Use "ef" for extended formats in the framework for consistency (Tomasz)
- Handle xfer_func field in conversions (Tomasz)
- Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz)
- Refactor format functions to use v4l_fmt_ioctl_via_ext()
- Several fixes/refactoring/changes
- Remove EXT API for touch devices
Changes in v5:
- change sizes and reorder fields to avoid holes in the struct and make
it the same for 32 and 64 bits
- removed __attribute__ ((packed)) from uapi structs
- Fix doc warning from make htmldocs
- Updated commit message with EXT_PIX prefix for the ioctls.
Changes in v4:
- Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
- Add reserved fields
- Removed num_planes from struct v4l2_ext_pix_format
- Removed flag field from struct v4l2_ext_pix_format, since the only
defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1,
where we can use modifiers, or add it back later through the reserved
bits.
- In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR &&
!= MOD_INVALID
- Fix type assignment in v4l_g_fmt_ext_pix()
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- Move the modifier in v4l2_ext_format (was formerly placed in
v4l2_ext_plane)
- Fix a few bugs in the converters and add a strict parameter to
allow conversion of uninitialized/mis-initialized objects
---
drivers/media/v4l2-core/v4l2-dev.c | 27 +-
drivers/media/v4l2-core/v4l2-ioctl.c | 538 +--
include/media/v4l2-ioctl.h | 28 ++
include/uapi/linux/videodev2.h | 41 ++
4 files changed, 602 insertions(+), 32 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-dev.c
b/drivers/media/v4l2-core/v4l2-dev.c
index f9cff033d0dc..5add58cb6d45 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct video_device
*vdev)
ops->vidioc_enum_fmt_vid_overlay)) ||
(is_tx && ops->vidioc_enum_fmt_vid_out))
set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls);
+ if ((is_rx && ops->vidioc_g_fmt_vid_overlay) ||
+ (is_tx && ops->vidioc_g_fmt_vid_out_overlay))
+set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls);
if ((is_rx && (ops->vidioc_g_fmt_vid_cap ||
ops->vidioc_g_fmt_vid_cap_mplane ||
- ops->vidioc_g_fmt_vid_overlay)) ||
+ ops->vidioc_g_ext_pix_fmt_vid_cap)) ||
(is_tx && (ops->vidioc_g_fmt_vid_out ||
ops->vidioc_g_fmt_vid_out_mplane ||
- ops->vidioc_g_fmt_vid_out_overlay)))
+ ops->vidioc_g_ext_pix_fmt_vid_out))) {
set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls);
+set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls);
+ }
+ if ((is_rx && ops->vidioc_s_fmt_vid_overlay) ||
+
[RFC PATCH v6 01/11] media: v4l2-common: add normalized pixelformat field to struct v4l2_format_info
Add normalization to pixelformats, so we can fallback to it when using
Ext API, and eliminating the handling of two variantes (M and non-M
formats).
Signed-off-by: Helen Koike
---
Changes in v6:
- New patch
---
drivers/media/v4l2-core/v4l2-common.c | 16
include/media/v4l2-common.h | 3 +++
2 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-common.c
b/drivers/media/v4l2-core/v4l2-common.c
index 78007dba4677..002051b9dc0c 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -276,17 +276,17 @@ const struct v4l2_format_info *v4l2_format_info(u32
format)
{ .format = V4L2_PIX_FMT_GREY,.pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
.hdiv = 1, .vdiv = 1 },
/* YUV planar formats, non contiguous variant */
- { .format = V4L2_PIX_FMT_YUV420M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 2, .vdiv = 2 },
- { .format = V4L2_PIX_FMT_YVU420M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 2, .vdiv = 2 },
- { .format = V4L2_PIX_FMT_YUV422M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 2, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_YUV420M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 2, .vdiv = 2, .norm= V4L2_PIX_FMT_YUV420 },
+ { .format = V4L2_PIX_FMT_YVU420M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 2, .vdiv = 2, .norm= V4L2_PIX_FMT_YVU420 },
+ { .format = V4L2_PIX_FMT_YUV422M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 2, .vdiv = 1, .norm= V4L2_PIX_FMT_YUV422P },
{ .format = V4L2_PIX_FMT_YVU422M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 2, .vdiv = 1 },
- { .format = V4L2_PIX_FMT_YUV444M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_YUV444M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 1, .vdiv = 1, .norm= V4L2_PIX_FMT_YUV444 },
{ .format = V4L2_PIX_FMT_YVU444M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
.hdiv = 1, .vdiv = 1 },
- { .format = V4L2_PIX_FMT_NV12M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
.hdiv = 2, .vdiv = 2 },
- { .format = V4L2_PIX_FMT_NV21M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
.hdiv = 2, .vdiv = 2 },
- { .format = V4L2_PIX_FMT_NV16M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
.hdiv = 2, .vdiv = 1 },
- { .format = V4L2_PIX_FMT_NV61M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
.hdiv = 2, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_NV12M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
.hdiv = 2, .vdiv = 2, .norm = V4L2_PIX_FMT_NV12 },
+ { .format = V4L2_PIX_FMT_NV21M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
.hdiv = 2, .vdiv = 2, .norm = V4L2_PIX_FMT_NV21 },
+ { .format = V4L2_PIX_FMT_NV16M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
.hdiv = 2, .vdiv = 1, .norm = V4L2_PIX_FMT_NV16 },
+ { .format = V4L2_PIX_FMT_NV61M, .pixel_enc =
V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
.hdiv = 2, .vdiv = 1, .norm = V4L2_PIX_FMT_NV61 },
/* Bayer RGB formats */
{ .format = V4L2_PIX_FMT_SBGGR8,.pixel_enc =
V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
.hdiv = 1, .vdiv = 1 },
diff --git a/include/media/v4l2-common.h b/include/media/v4l2-common.h
index be36cbdcc1bd..7236af1cfa2f 100644
--- a/include/media/v4l2-common.h
+++ b/include/media/v4l2-common.h
@@ -483,6 +483,8 @@ enum v4l2_pixel_encoding {
* @vdiv: Vertical chroma subsampling factor
* @block_w: Per-plane macroblock pixel width (optional)
* @block_h: Per-plane macroblock pixel height (optional)
+ * @norm: The normalized format that should be used in Ext API. Should be set
+ * to zero if @format is already the normalized version.
*/
struct v4l2_format_info {
u32 format;
@@ -494,6 +496,7 @@ struct v4l2_format_info {
u8
[RFC PATCH v6 00/11] media: v4l2: Add extended fmt and buffer ioctls
Hello,
This is v6 of the Extended API for formats and buffers (see below the new API).
The new API comes for free for old drivers through the conversion layer, which
is independent of vb2.
I completly refactored several patches. I would like to request comments not
only in the uAPI, but also the kAPI for drivers, and I would appreciate any
ideas on improving the quality of the code (in short: please review everything).
NOTE: The Ext API wans't tested yet. My next step is to patch v4l2-compliance.
Regression tests - v4l2-compliance with test-media script:
vivid: http://ix.io/2M0G - Final Summary: 1856, Succeeded: 1856,
Failed: 0, Warnings: 0)
vimc: http://ix.io/2M0I - Final Summary: 488, Succeeded: 488, Failed:
0, Warnings: 0
Git: https://gitlab.collabora.com/koike/linux/-/tree/v4l2/ext-api/v6
v5:
https://patchwork.linuxtv.org/project/linux-media/cover/20200804192939.2251988-1-helen.ko...@collabora.com/
v4:
https://patchwork.linuxtv.org/project/linux-media/cover/20200717115435.2632623-1-helen.ko...@collabora.com/
v3: https://patchwork.linuxtv.org/cover/59345/
v2: https://patchwork.kernel.org/project/linux-media/list/?series=101153
v1: https://patchwork.kernel.org/project/linux-media/list/?series=93707
Conversion layer:
=
* Old drivers implementing only ops->vidioc_*_fmt_vid_cap supports
VIDIOC_*_EXT_PIX_FMT automatically with limitations[1].
* New drivers implementing only ops->vidioc_*_ext_pix_fmt_vid_cap supports
VIDIOC_*_FMT automatically.
* Old drivers implementing only ops->vidioc_*buf support
VIDIOC_EXT_*BUF automatically with limitations[2].
* New drivers should implement both ops->vidioc_*buf and ops->vidioc_*buf
to overcome limitations[2] and support both APIs.
Which is easy with vb2:
static const struct v4l2_ioctl_ops ioctl_ops = {
...
+ .vidioc_ext_qbuf = vb2_ioctl_ext_qbuf,
+ .vidioc_ext_dqbuf = vb2_ioctl_ext_dqbuf,
...
}
...
+ /* Inform vb2 how to split the memory buffer in case a single one
is used */
+ vb2_set_pixelformat(dev->pixelformat)
[1] There are some limitations in the conversion such as modifiers that are
ignored when converting v4l2_ext_pix_format to v4l_format
[2] Ext API allows a single buffer with planes placed in random locations,
which is not possible with v4l2_buffer.
Major changes in v6:
Fixed color planes vs memory planes handling.
Removed VIDIOC_EXT_PREPARE_BUF, since this is an optimization, it doesn't blocks
the API, we can add it later (my goal was to simplify this patchset).
Removed VIDIOC_EXT_CREATE_BUFS, since this is useful only to MMAP (thus low
priority)
with the new format.
Classic VIDIOC_CREATE_BUFS and VIDIOC_REQBUFS can still be used.
Reformulated conversion layer as per above.
Removed conversions in vb2, it is easier to add hooks to drivers.
Fixed vb2 to allow Ext API only to Video types.
API updates:
* remove buffer and plane lengths
* move `memory` field to v4l2_ext_buffer instead of v4l2_ext_plane
* remove struct v4l2_plane_ext_pix_format
* reordering
Make Ext API valid only for Video types, and not for touch, vbi, meta, etc.
Sereval code refactoring, simplification, fixes and applied suggestions from v5.
New API (for convenience):
==
int ioctl(int fd, VIDIOC_G_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp)
int ioctl(int fd, VIDIOC_S_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp)
int ioctl(int fd, VIDIOC_TRY_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp)
int ioctl(int fd, VIDIOC_EXT_QBUF, struct v4l2_ext_buffer *argp)
int ioctl(int fd, VIDIOC_EXT_DQBUF, struct v4l2_ext_buffer *argp)
struct v4l2_ext_pix_format {
__u32 type;
__u32 width;
__u32 height;
__u32 field;
struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES];
__u32 pixelformat;
__u64 modifier;
__u32 colorspace;
__u32 xfer_func;
union {
__u32 ycbcr_enc;
__u32 hsv_enc;
};
__u32 quantization;
__u32 reserved[9];
};
struct v4l2_ext_buffer {
__u32 index;
__u32 type;
__u32 field;
__u32 sequence;
__u64 flags;
__u64 timestamp;
__u32 memory;
__s32 request_fd;
struct v4l2_ext_plane planes[VIDEO_MAX_PLANES];
__u32 reserved[10];
};
struct v4l2_ext_plane {
__u32 offset;
__u32 bytesused;
union {
__u32 mmap_offset;
__u64 userptr;
__s32 dmabuf_fd;
} m;
__u32 reserved[6];
};
Helen Koike (11):
media: v4l2-common: add normalized pixelformat field to struct
v4l2_format_info
media: v4l2: Extend pixel formats to unify single/multi-planar
handling (and more)
media: v4l2: Add extended buffer (de)queue operations for video types
media: videobuf2-v4l2: reorganize flags handling
media: v
[PATCH 1/3] media: v4l2-ioctl: print capabilities in v4l_print_create_buffers()
Print capabilities field from struct v4l2_create_buffers for better
debugging.
Signed-off-by: Helen Koike
---
drivers/media/v4l2-core/v4l2-ioctl.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 3198abdd538c..848286a284f6 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -518,9 +518,9 @@ static void v4l_print_create_buffers(const void *arg, bool
write_only)
{
const struct v4l2_create_buffers *p = arg;
- pr_cont("index=%d, count=%d, memory=%s, ",
- p->index, p->count,
- prt_names(p->memory, v4l2_memory_names));
+ pr_cont("index=%d, count=%d, memory=%s, capabilities=0x%08x, ",
+ p->index, p->count, prt_names(p->memory, v4l2_memory_names),
+ p->capabilities);
v4l_print_format(&p->format, write_only);
}
--
2.29.2
[PATCH 3/3] media: videobuf2-v4l2: remove redundant error test
request_fd is validated under media_request_get_by_fd() just below this
check. Thus remove it.
Suggested-by: Tomasz Figa
Signed-off-by: Helen Koike
---
drivers/media/common/videobuf2/videobuf2-v4l2.c | 5 -
1 file changed, 5 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 96d3b2b2aa31..bb642c0775d1 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -488,11 +488,6 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q,
struct media_device *md
!q->ops->buf_out_validate))
return -EINVAL;
- if (b->request_fd < 0) {
- dprintk(q, 1, "%s: request_fd < 0\n", opname);
- return -EINVAL;
- }
-
req = media_request_get_by_fd(mdev, b->request_fd);
if (IS_ERR(req)) {
dprintk(q, 1, "%s: invalid request_fd\n", opname);
--
2.29.2
[PATCH 2/3] media: videodev2.h: clarify v4l2_pix_format_mplane.sizeimage docs when to set to zero
sizeimage field should be set to zero for unused planes, even when
v4l2_pix_format_mplane.num_planes is smaller then the index of planes.
Signed-off-by: Helen Koike
---
I caught this with v4l2-compliance, which throws an error if we dirty
planes, even if invalid, so I would like to make it clear in the docs.
---
include/uapi/linux/videodev2.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 79dbde3bcf8d..d9b7c9177605 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2227,6 +2227,7 @@ struct v4l2_mpeg_vbi_fmt_ivtv {
* struct v4l2_plane_pix_format - additional, per-plane format definition
* @sizeimage: maximum size in bytes required for data, for which
* this plane will be used
+ * Drivers should be set it zero for unused planes.
* @bytesperline: distance in bytes between the leftmost pixels in two
* adjacent lines
*/
--
2.29.2
Re: [PATCH v5 7/7] media: docs: add documentation for the Extended API
On 11/20/20 10:20 AM, Hans Verkuil wrote: > On 20/11/2020 13:40, Tomasz Figa wrote: >> On Fri, Nov 20, 2020 at 9:24 PM Hans Verkuil wrote: >>> >>> On 20/11/2020 12:06, Tomasz Figa wrote:z >>>> Hi Helen, >>>> >>>> On Tue, Aug 04, 2020 at 04:29:39PM -0300, Helen Koike wrote: >>>>> Add documentation and update references in current documentation for the >>>>> Extended API. >>>>> >>>> >>>> Thank you for the patch. Please see my comments inline. >>>> >>>>> Signed-off-by: Helen Koike >>>>> --- >>>>> Changes in v5: >>>>> - new patch >>>>> >>>>> .../userspace-api/media/v4l/buffer.rst| 5 + >>>>> .../userspace-api/media/v4l/common.rst| 1 + >>>>> .../userspace-api/media/v4l/dev-capture.rst | 5 + >>>>> .../userspace-api/media/v4l/dev-output.rst| 5 + >>>>> .../userspace-api/media/v4l/ext-api.rst | 107 + >>>>> .../userspace-api/media/v4l/format.rst| 16 +- >>>>> .../userspace-api/media/v4l/user-func.rst | 5 + >>>>> .../media/v4l/vidioc-ext-create-bufs.rst | 95 >>>>> .../media/v4l/vidioc-ext-prepare-buf.rst | 62 ++ >>>>> .../media/v4l/vidioc-ext-qbuf.rst | 204 ++ >>>>> .../media/v4l/vidioc-ext-querybuf.rst | 79 +++ >>>>> .../media/v4l/vidioc-g-ext-pix-fmt.rst| 117 ++ >>>>> 12 files changed, 697 insertions(+), 4 deletions(-) >>>>> create mode 100644 Documentation/userspace-api/media/v4l/ext-api.rst >>>>> create mode 100644 >>>>> Documentation/userspace-api/media/v4l/vidioc-ext-create-bufs.rst >>>>> create mode 100644 >>>>> Documentation/userspace-api/media/v4l/vidioc-ext-prepare-buf.rst >>>>> create mode 100644 >>>>> Documentation/userspace-api/media/v4l/vidioc-ext-qbuf.rst >>>>> create mode 100644 >>>>> Documentation/userspace-api/media/v4l/vidioc-ext-querybuf.rst >>>>> create mode 100644 >>>>> Documentation/userspace-api/media/v4l/vidioc-g-ext-pix-fmt.rst >>>>> >>>>> diff --git a/Documentation/userspace-api/media/v4l/buffer.rst >>>>> b/Documentation/userspace-api/media/v4l/buffer.rst >>>>> index 57e752aaf414a..c832bedd64e4c 100644 >>>>> --- a/Documentation/userspace-api/media/v4l/buffer.rst >>>>> +++ b/Documentation/userspace-api/media/v4l/buffer.rst >>>>> @@ -27,6 +27,11 @@ such as pointers and sizes for each plane, are stored >>>>> in >>>>> struct :c:type:`v4l2_plane` instead. In that case, >>>>> struct :c:type:`v4l2_buffer` contains an array of plane structures. >>>>> >>>>> +.. note:: >>>>> + >>>>> +The :ref:`ext_api` version can also be used, and it is >>>>> +preferable when applicable. >>>> >>>> Would rephrasing this as below making a bit more definitive? >>>> >>>> For modern applications, these operations are replaced by their >>>> :ref:`ext_api` counterparts, which should be used instead. >>> >>> You can't say that, since especially in the beginning userspace will be >>> running >>> on older kernels that do not support this. >>> >>> This will work: "should be used instead, if supported by the driver." >>> >> >> With the wrappers that the patches provide, all drivers would support >> the new API, so this boils down to the kernel version only, not >> specific drivers. >> >> Agreed, though, that application developers must be made aware that >> the new API is only available in new kernels and old API must be used >> for compatibility with old kernels. > > Is the conversion layer fully independent of vb2? We still have older > drivers that do not use vb2, and I'm not sure if the extended buffer > operations work with those drivers as well. (Sorry, it's been a while > since I last looked at this series, so I can't remember). The conversion layer should be independent of vb2 (please check v6, I think it is cleaner). > >> >>>> >>>>> + >>>>> Dequeued video buffers come with timestamps. The driver decides at which >>>>> part of the frame and with w
[PATCH 0/3] v4l2 framework minor improvements
Just minor things. Add capabilities to v4l_print_create_buffers(), clarify docs and remove a redundant check. Helen Koike (3): media: v4l2-ioctl: print capabilities in v4l_print_create_buffers() media: videodev2.h: clarify v4l2_pix_format_mplane.sizeimage docs when to set to zero media: videobuf2-v4l2: remove redundant error test drivers/media/common/videobuf2/videobuf2-v4l2.c | 5 - drivers/media/v4l2-core/v4l2-ioctl.c| 6 +++--- include/uapi/linux/videodev2.h | 1 + 3 files changed, 4 insertions(+), 8 deletions(-) -- 2.29.2
Re: [PATCH v5 2/7] media: v4l2: Add extended buffer operations
Hi Tomasz,
On 12/21/20 12:13 AM, Tomasz Figa wrote:
> On Thu, Dec 17, 2020 at 10:20 PM Helen Koike
> wrote:
>>
>> Hi Tomasz,
>>
>> Thanks for your comments, I have a few questions below.
>>
>> On 12/16/20 12:13 AM, Tomasz Figa wrote:
>>> On Tue, Dec 15, 2020 at 11:37 PM Helen Koike
>>> wrote:
>>>>
>>>> Hi Tomasz,
>>>>
>>>> On 12/14/20 7:46 AM, Tomasz Figa wrote:
>>>>> On Fri, Dec 4, 2020 at 4:52 AM Helen Koike
>>>>> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please see my 2 points below (about v4l2_ext_buffer and another about
>>>>>> timestamp).
>>>>>>
>>>>>> On 12/3/20 12:11 PM, Hans Verkuil wrote:
>>>>>>> On 23/11/2020 18:40, Helen Koike wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 11/23/20 12:46 PM, Tomasz Figa wrote:
>>>>>>>>> On Tue, Nov 24, 2020 at 12:08 AM Helen Koike
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Hans,
>>>>>>>>>>
>>>>>>>>>> Thank you for your review.
>>>>>>>>>>
>>>>>>>>>> On 9/9/20 9:27 AM, Hans Verkuil wrote:
>>>>>>>>>>> Hi Helen,
>>>>>>>>>>>
>>>>>>>>>>> Again I'm just reviewing the uAPI.
>>>>>>>>>>>
>>>>>>>>>>> On 04/08/2020 21:29, Helen Koike wrote:
> [snip]
>>>
>>>>
>>>> Output: userspace fills plane information, informing in which memory
>>>> buffer each
>>>> plane was placed (Or should this be pre-determined by the driver?)
>>>>
>>>> For MMAP
>>>> ---
>>>> userspace performs EXT_CREATE_BUF ioctl to reserve a buffer "index" range
>>>> in
>>>> that mode, to be used in EXT_QBUF and EXT_DQBUF
>>>>
>>>> Should the API allow userspace to select how many memory buffers it wants?
>>>> (maybe not)
>>>
>>> I think it does allow that - it accepts the v4l2_ext_format struct.
>>
>> hmmm, I thought v4l2_ext_format would describe color planes, and not memory
>> planes.
>> Should it describe memory planes instead? Since planes are defined by the
>> pixelformat.
>> But is this information relevant to ext_{set/get/try} format?
>>
>
> Good point. I ended up assuming the current convention, where giving
> an M format would imply num_memory_planes == num_color_planes and
> non-M format num_memory_planes == 1. Sounds like we might want
> something like a flags field and that could have bits defined to
> select that. I think it would actually be useful for S_FMT as well,
> because that's what REQBUFS would use.
Would this flag select between memory and color planes?
I didn't understand how this flag would be useful to S_FMT, could you
please clarify?
Thanks
Helen
>
>>>
>>>>
>>>> userspace performs EXT_QUERY_MMAP_BUF to get the mmap offset/cookie and
>>>> length
>>>> for each memory buffer.
>>>>
>>>> On EXT_QBUF, userspace doesn't need to fill membuf information. Should the
>>>> mmap offset and length be filled by the kernel and returned to userspace
>>>> here
>>>> as well? I'm leaning towards: no.
>>>
>>> Yeah, based on my comment above, I think the answer should be no.
>>>
>>>>
>>>> If the answer is no, then here is my proposal:
>>>> --
>>>>
>>>> /* If MMAP, drivers decide how many memory buffers to allocate */
>>>> int ioctl( int fd, VIDIOC_EXT_CREATE_BUFS, struct v4l2_ext_buffer *argp )
>>>>
>>>> /* Returns -EINVAL if not MMAP */
>>>> int ioctl( int fd, VIDIOC_EXT_MMAP_QUERYBUF, struct v4l2_ext_mmap_querybuf
>>>> *argp )
>>>>
>>>> /* userspace fills v4l2_ext_buffer.membufs if DMA-fd or Userptr, leave it
>>>> zero for MMAP
>>>> * Should userspace also fill v4l2_ext_buffer.planes?
>>>> */
>>>> int ioctl( int fd, VIDIOC_EXT_QBUF, struct v4l2_ext_buffer *argp )
>>>>
>>>> /* v4l2_ext_buffer.membufs is set to zero by the driver */
>>>&
Re: [PATCH v5 2/7] media: v4l2: Add extended buffer operations
Hi Tomasz, Thanks for your comments, I have a few questions below. On 12/16/20 12:13 AM, Tomasz Figa wrote: > On Tue, Dec 15, 2020 at 11:37 PM Helen Koike > wrote: >> >> Hi Tomasz, >> >> On 12/14/20 7:46 AM, Tomasz Figa wrote: >>> On Fri, Dec 4, 2020 at 4:52 AM Helen Koike >>> wrote: >>>> >>>> Hi, >>>> >>>> Please see my 2 points below (about v4l2_ext_buffer and another about >>>> timestamp). >>>> >>>> On 12/3/20 12:11 PM, Hans Verkuil wrote: >>>>> On 23/11/2020 18:40, Helen Koike wrote: >>>>>> >>>>>> >>>>>> On 11/23/20 12:46 PM, Tomasz Figa wrote: >>>>>>> On Tue, Nov 24, 2020 at 12:08 AM Helen Koike >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Hans, >>>>>>>> >>>>>>>> Thank you for your review. >>>>>>>> >>>>>>>> On 9/9/20 9:27 AM, Hans Verkuil wrote: >>>>>>>>> Hi Helen, >>>>>>>>> >>>>>>>>> Again I'm just reviewing the uAPI. >>>>>>>>> >>>>>>>>> On 04/08/2020 21:29, Helen Koike wrote: >>>>>>>>>> From: Hans Verkuil >>>>>>>>>> >>>>>>>>>> Those extended buffer ops have several purpose: >>>>>>>>>> 1/ Fix y2038 issues by converting the timestamp into an u64 counting >>>>>>>>>>the number of ns elapsed since 1970 >>>>>>>>>> 2/ Unify single/multiplanar handling >>>>>>>>>> 3/ Add a new start offset field to each v4l2 plane buffer info struct >>>>>>>>>>to support the case where a single buffer object is storing all >>>>>>>>>>planes data, each one being placed at a different offset >>>>>>>>>> >>>>>>>>>> New hooks are created in v4l2_ioctl_ops so that drivers can start >>>>>>>>>> using >>>>>>>>>> these new objects. >>>>>>>>>> >>>>>>>>>> The core takes care of converting new ioctls requests to old ones >>>>>>>>>> if the driver does not support the new hooks, and vice versa. >>>>>>>>>> >>>>>>>>>> Note that the timecode field is gone, since there doesn't seem to be >>>>>>>>>> in-kernel users. We can be added back in the reserved area if needed >>>>>>>>>> or >>>>>>>>>> use the Request API to collect more metadata information from the >>>>>>>>>> frame. >>>>>>>>>> >>>>>>>>>> Signed-off-by: Hans Verkuil >>>>>>>>>> Signed-off-by: Boris Brezillon >>>>>>>>>> Signed-off-by: Helen Koike >>>>>>>>>> --- >>>>>>>>>> Changes in v5: >>>>>>>>>> - migrate memory from v4l2_ext_buffer to v4l2_ext_plane >>>>>>>>>> - return mem_offset to struct v4l2_ext_plane >>>>>>>>>> - change sizes and reorder fields to avoid holes in the struct and >>>>>>>>>> make >>>>>>>>>> it the same for 32 and 64 bits >>>>>>>>>> >>>>>>>>>> Changes in v4: >>>>>>>>>> - Use v4l2_ext_pix_format directly in the ioctl, drop >>>>>>>>>> v4l2_ext_format, >>>>>>>>>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. >>>>>>>>>> - Drop VIDIOC_EXT_EXPBUF, since the only difference from >>>>>>>>>> VIDIOC_EXPBUF >>>>>>>>>> was that with VIDIOC_EXT_EXPBUF we could export multiple planes at >>>>>>>>>> once. >>>>>>>>>> I think we can add this later, so I removed it from this RFC to >>>>>>>>>> simplify it. >>>>>>>>>> - Remove num_planes field from struct v4l2_ext_buffer >>>>>>>>>> - Add flags field to struct v4l2_ext_create_buffers >>>>>>>>>&g
Re: [PATCH v5 2/7] media: v4l2: Add extended buffer operations
Hi Tomasz, On 12/14/20 7:46 AM, Tomasz Figa wrote: > On Fri, Dec 4, 2020 at 4:52 AM Helen Koike wrote: >> >> Hi, >> >> Please see my 2 points below (about v4l2_ext_buffer and another about >> timestamp). >> >> On 12/3/20 12:11 PM, Hans Verkuil wrote: >>> On 23/11/2020 18:40, Helen Koike wrote: >>>> >>>> >>>> On 11/23/20 12:46 PM, Tomasz Figa wrote: >>>>> On Tue, Nov 24, 2020 at 12:08 AM Helen Koike >>>>> wrote: >>>>>> >>>>>> Hi Hans, >>>>>> >>>>>> Thank you for your review. >>>>>> >>>>>> On 9/9/20 9:27 AM, Hans Verkuil wrote: >>>>>>> Hi Helen, >>>>>>> >>>>>>> Again I'm just reviewing the uAPI. >>>>>>> >>>>>>> On 04/08/2020 21:29, Helen Koike wrote: >>>>>>>> From: Hans Verkuil >>>>>>>> >>>>>>>> Those extended buffer ops have several purpose: >>>>>>>> 1/ Fix y2038 issues by converting the timestamp into an u64 counting >>>>>>>>the number of ns elapsed since 1970 >>>>>>>> 2/ Unify single/multiplanar handling >>>>>>>> 3/ Add a new start offset field to each v4l2 plane buffer info struct >>>>>>>>to support the case where a single buffer object is storing all >>>>>>>>planes data, each one being placed at a different offset >>>>>>>> >>>>>>>> New hooks are created in v4l2_ioctl_ops so that drivers can start using >>>>>>>> these new objects. >>>>>>>> >>>>>>>> The core takes care of converting new ioctls requests to old ones >>>>>>>> if the driver does not support the new hooks, and vice versa. >>>>>>>> >>>>>>>> Note that the timecode field is gone, since there doesn't seem to be >>>>>>>> in-kernel users. We can be added back in the reserved area if needed or >>>>>>>> use the Request API to collect more metadata information from the >>>>>>>> frame. >>>>>>>> >>>>>>>> Signed-off-by: Hans Verkuil >>>>>>>> Signed-off-by: Boris Brezillon >>>>>>>> Signed-off-by: Helen Koike >>>>>>>> --- >>>>>>>> Changes in v5: >>>>>>>> - migrate memory from v4l2_ext_buffer to v4l2_ext_plane >>>>>>>> - return mem_offset to struct v4l2_ext_plane >>>>>>>> - change sizes and reorder fields to avoid holes in the struct and make >>>>>>>> it the same for 32 and 64 bits >>>>>>>> >>>>>>>> Changes in v4: >>>>>>>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, >>>>>>>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. >>>>>>>> - Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF >>>>>>>> was that with VIDIOC_EXT_EXPBUF we could export multiple planes at >>>>>>>> once. >>>>>>>> I think we can add this later, so I removed it from this RFC to >>>>>>>> simplify it. >>>>>>>> - Remove num_planes field from struct v4l2_ext_buffer >>>>>>>> - Add flags field to struct v4l2_ext_create_buffers >>>>>>>> - Reformulate struct v4l2_ext_plane >>>>>>>> - Fix some bugs caught by v4l2-compliance >>>>>>>> - Rebased on top of media/master (post 5.8-rc1) >>>>>>>> >>>>>>>> Changes in v3: >>>>>>>> - Rebased on top of media/master (post 5.4-rc1) >>>>>>>> >>>>>>>> Changes in v2: >>>>>>>> - Add reserved space to v4l2_ext_buffer so that new fields can be added >>>>>>>> later on >>>>>>>> --- >>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 29 ++- >>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 353 +-- >>>>>>>> include/media/v4l2-ioctl.h | 26 ++ >>>>>>>> include/uapi/linux/videodev2.h | 90 +
Re: [PATCH v5 2/7] media: v4l2: Add extended buffer operations
Hi Tomasz,
Thank you for your comments,
On 12/14/20 7:36 AM, Tomasz Figa wrote:
> On Tue, Nov 24, 2020 at 5:33 AM Helen Koike wrote:
>>
>> Hi Tomasz,
>>
>>
>> On 11/20/20 8:14 AM, Tomasz Figa wrote:
>>> Hi Helen,
>>>
>>> On Tue, Aug 04, 2020 at 04:29:34PM -0300, Helen Koike wrote:
>>>> From: Hans Verkuil
>>>>
>>>> Those extended buffer ops have several purpose:
>>>> 1/ Fix y2038 issues by converting the timestamp into an u64 counting
>>>>the number of ns elapsed since 1970
>>>> 2/ Unify single/multiplanar handling
>>>> 3/ Add a new start offset field to each v4l2 plane buffer info struct
>>>>to support the case where a single buffer object is storing all
>>>>planes data, each one being placed at a different offset
>>>>
>>>> New hooks are created in v4l2_ioctl_ops so that drivers can start using
>>>> these new objects.
>>>>
>>>> The core takes care of converting new ioctls requests to old ones
>>>> if the driver does not support the new hooks, and vice versa.
>>>>
>>>> Note that the timecode field is gone, since there doesn't seem to be
>>>> in-kernel users. We can be added back in the reserved area if needed or
>>>> use the Request API to collect more metadata information from the
>>>> frame.
>>>>
>>>
>>> Thanks for the patch. Please see my comments inline.
>>
>> Thank you for your detailed review, please see my comments below.
>>
>>>
>>>> Signed-off-by: Hans Verkuil
>>>> Signed-off-by: Boris Brezillon
>>>> Signed-off-by: Helen Koike
>>>> ---
>>>> Changes in v5:
>>>> - migrate memory from v4l2_ext_buffer to v4l2_ext_plane
>>>> - return mem_offset to struct v4l2_ext_plane
>>>> - change sizes and reorder fields to avoid holes in the struct and make
>>>> it the same for 32 and 64 bits
>>>>
>>>> Changes in v4:
>>>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
>>>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
>>>> - Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF
>>>> was that with VIDIOC_EXT_EXPBUF we could export multiple planes at once.
>>>> I think we can add this later, so I removed it from this RFC to simplify
>>>> it.
>>>> - Remove num_planes field from struct v4l2_ext_buffer
>>>> - Add flags field to struct v4l2_ext_create_buffers
>>>> - Reformulate struct v4l2_ext_plane
>>>> - Fix some bugs caught by v4l2-compliance
>>>> - Rebased on top of media/master (post 5.8-rc1)
>>>>
>>>> Changes in v3:
>>>> - Rebased on top of media/master (post 5.4-rc1)
>>>>
>>>> Changes in v2:
>>>> - Add reserved space to v4l2_ext_buffer so that new fields can be added
>>>> later on
>>>> ---
>>>> drivers/media/v4l2-core/v4l2-dev.c | 29 ++-
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 353 +--
>>>> include/media/v4l2-ioctl.h | 26 ++
>>>> include/uapi/linux/videodev2.h | 90 +++
>>>> 4 files changed, 476 insertions(+), 22 deletions(-)
>>>>
>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>> index e1829906bc086..cb21ee8eb075c 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>> @@ -720,15 +720,34 @@ static void determine_valid_ioctls(struct
>>>> video_device *vdev)
>>>> SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_sdr_out);
>>>> }
>>>>
>>>> +if (is_vid || is_tch) {
>>>> +/* ioctls valid for video and touch */
>>>> +if (ops->vidioc_querybuf || ops->vidioc_ext_querybuf)
>>>> +set_bit(_IOC_NR(VIDIOC_EXT_QUERYBUF), valid_ioctls);
>>>> +if (ops->vidioc_qbuf || ops->vidioc_ext_qbuf)
>>>> +set_bit(_IOC_NR(VIDIOC_EXT_QBUF), valid_ioctls);
>>>> +if (ops->vidioc_dqbuf || ops->vidioc_ext_dqbuf)
>>>> +set_bit(_IOC_NR(VIDIOC_EXT_DQBUF), valid_ioctls);
>>>> +if (ops->vidioc_create_bufs || ops->vidioc_ext_create_bufs)
>>>
Re: [PATCH v5 3/7] media: videobuf2: Expose helpers to implement the _ext_fmt and _ext_buf hooks
Hi Tomasz,
Thank you for your comments,
On 12/14/20 5:52 AM, Tomasz Figa wrote:
> Hi Helen,
>
> On Tue, Aug 04, 2020 at 04:29:35PM -0300, Helen Koike wrote:
>> The VB2 layer is used by a lot of drivers. Patch it to support the
>> _EXT_PIX_FMT and _EXT_BUF ioctls in order to ease conversion of existing
>> drivers to these new APIs.
>>
>> Note that internally, the VB2 core is now only using ext structs and old
>> APIs are supported through conversion wrappers.
>
> We decided to only support V4L2_BUF_TYPE_VIDEO* for the ext structs. Still,
> existing drivers may use vb2 with the other, legacy, buf types. How would
> they be handled with this change?
I completly refactored this patch in my wip branch, I'll submit for comments
soon after finishing addressing the other comments.
The way I'm approaching this is to support both structures v4l2_buffer and
v4l2_ext_buffer, but only in the vb2 entry points, since all the
information we need is in vb2_buffer struct.
To implement this I had to use new hooks in the framework. I think it is easier
if you take a look in next version when I submited it.
>
>>
>> Signed-off-by: Boris Brezillon
>> Signed-off-by: Helen Koike
>> ---
>> Changes in v5:
>> - Update with new format and buffer structs
>> - Updated commit message with the uAPI prefix
>>
>> Changes in v4:
>> - Update with new format and buffer structs
>> - Fix some bugs caught by v4l2-compliance
>> - Rebased on top of media/master (post 5.8-rc1)
>>
>> Changes in v3:
>> - Rebased on top of media/master (post 5.4-rc1)
>>
>> Changes in v2:
>> - New patch
>> ---
>> .../media/common/videobuf2/videobuf2-core.c | 2 +
>> .../media/common/videobuf2/videobuf2-v4l2.c | 560 ++
>> include/media/videobuf2-core.h| 6 +-
>> include/media/videobuf2-v4l2.h| 21 +-
>> 4 files changed, 345 insertions(+), 244 deletions(-)
>>
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index f544d3393e9d6..d719b1e9c148b 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -1270,6 +1270,7 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
>> vb->planes[plane].length = 0;
>> vb->planes[plane].m.fd = 0;
>> vb->planes[plane].data_offset = 0;
>> +vb->planes[plane].dbuf_offset = 0;
>>
>> /* Acquire each plane's memory */
>> mem_priv = call_ptr_memop(vb, attach_dmabuf,
>> @@ -1313,6 +1314,7 @@ static int __prepare_dmabuf(struct vb2_buffer *vb)
>> vb->planes[plane].length = planes[plane].length;
>> vb->planes[plane].m.fd = planes[plane].m.fd;
>> vb->planes[plane].data_offset = planes[plane].data_offset;
>> +vb->planes[plane].dbuf_offset = planes[plane].dbuf_offset;
>> }
>>
>> if (reacquired) {
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 30caad27281e1..911681d24b3ae 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -29,6 +29,7 @@
>> #include
>> #include
>> #include
>> +#include
>>
>> #include
>>
>> @@ -56,72 +57,39 @@ module_param(debug, int, 0644);
>> V4L2_BUF_FLAG_TIMECODE | \
>> V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF)
>>
>> -/*
>> - * __verify_planes_array() - verify that the planes array passed in struct
>> - * v4l2_buffer from userspace can be safely used
>> - */
>> -static int __verify_planes_array(struct vb2_buffer *vb, const struct
>> v4l2_buffer *b)
>> -{
>> -if (!V4L2_TYPE_IS_MULTIPLANAR(b->type))
>> -return 0;
>> -
>> -/* Is memory for copying plane information present? */
>> -if (b->m.planes == NULL) {
>> -dprintk(vb->vb2_queue, 1,
>> -"multi-planar buffer passed but planes array not
>> provided\n");
>> -return -EINVAL;
>> -}
>> -
>> -if (b->length < vb->num_planes || b->length > VB2_MAX_PLANES) {
>> -dprintk(vb->vb2_queue, 1,
>> -"incorrect planes array length, expected %d, got %d\n",
>> -
Re: [PATCH v5 2/7] media: v4l2: Add extended buffer operations
Hi,
Please see my 2 points below (about v4l2_ext_buffer and another about
timestamp).
On 12/3/20 12:11 PM, Hans Verkuil wrote:
> On 23/11/2020 18:40, Helen Koike wrote:
>>
>>
>> On 11/23/20 12:46 PM, Tomasz Figa wrote:
>>> On Tue, Nov 24, 2020 at 12:08 AM Helen Koike
>>> wrote:
>>>>
>>>> Hi Hans,
>>>>
>>>> Thank you for your review.
>>>>
>>>> On 9/9/20 9:27 AM, Hans Verkuil wrote:
>>>>> Hi Helen,
>>>>>
>>>>> Again I'm just reviewing the uAPI.
>>>>>
>>>>> On 04/08/2020 21:29, Helen Koike wrote:
>>>>>> From: Hans Verkuil
>>>>>>
>>>>>> Those extended buffer ops have several purpose:
>>>>>> 1/ Fix y2038 issues by converting the timestamp into an u64 counting
>>>>>>the number of ns elapsed since 1970
>>>>>> 2/ Unify single/multiplanar handling
>>>>>> 3/ Add a new start offset field to each v4l2 plane buffer info struct
>>>>>>to support the case where a single buffer object is storing all
>>>>>>planes data, each one being placed at a different offset
>>>>>>
>>>>>> New hooks are created in v4l2_ioctl_ops so that drivers can start using
>>>>>> these new objects.
>>>>>>
>>>>>> The core takes care of converting new ioctls requests to old ones
>>>>>> if the driver does not support the new hooks, and vice versa.
>>>>>>
>>>>>> Note that the timecode field is gone, since there doesn't seem to be
>>>>>> in-kernel users. We can be added back in the reserved area if needed or
>>>>>> use the Request API to collect more metadata information from the
>>>>>> frame.
>>>>>>
>>>>>> Signed-off-by: Hans Verkuil
>>>>>> Signed-off-by: Boris Brezillon
>>>>>> Signed-off-by: Helen Koike
>>>>>> ---
>>>>>> Changes in v5:
>>>>>> - migrate memory from v4l2_ext_buffer to v4l2_ext_plane
>>>>>> - return mem_offset to struct v4l2_ext_plane
>>>>>> - change sizes and reorder fields to avoid holes in the struct and make
>>>>>> it the same for 32 and 64 bits
>>>>>>
>>>>>> Changes in v4:
>>>>>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
>>>>>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
>>>>>> - Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF
>>>>>> was that with VIDIOC_EXT_EXPBUF we could export multiple planes at once.
>>>>>> I think we can add this later, so I removed it from this RFC to simplify
>>>>>> it.
>>>>>> - Remove num_planes field from struct v4l2_ext_buffer
>>>>>> - Add flags field to struct v4l2_ext_create_buffers
>>>>>> - Reformulate struct v4l2_ext_plane
>>>>>> - Fix some bugs caught by v4l2-compliance
>>>>>> - Rebased on top of media/master (post 5.8-rc1)
>>>>>>
>>>>>> Changes in v3:
>>>>>> - Rebased on top of media/master (post 5.4-rc1)
>>>>>>
>>>>>> Changes in v2:
>>>>>> - Add reserved space to v4l2_ext_buffer so that new fields can be added
>>>>>> later on
>>>>>> ---
>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 29 ++-
>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 353 +--
>>>>>> include/media/v4l2-ioctl.h | 26 ++
>>>>>> include/uapi/linux/videodev2.h | 90 +++
>>>>>> 4 files changed, 476 insertions(+), 22 deletions(-)
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>> b/include/uapi/linux/videodev2.h
>>>>>> index 7123c6a4d9569..334cafdd2be97 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -996,6 +996,41 @@ struct v4l2_plane {
>>>>>> __u32 reserved[11];
>>>>>> };
>>>>>>
>>>>>> +/**
>>>>>> + * struct v4l2_ext_plane - extended plane bu
Re: [PATCH v2] media: rockchip: rkisp1: remove useless debugfs checks
On 12/1/20 11:30 AM, Dan Carpenter wrote:
> The debugfs_create_dir() function never returns NULLs so this code will
> never be executed. It's not intended that callers will check for
> debugfs errors in the normal case and it's not necessary in this driver,
> so we can just delete this code.
>
> Signed-off-by: Dan Carpenter
Since the debugfs_create_*() functions already check the parent
with IS_ERR(), this looks good to me.
Acked-by: Helen Koike
Thanks
Helen
> ---
> v2: Fix subject
>
> drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c | 4
> 1 file changed, 4 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
> b/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
> index 9af137e4967f..68da1eed753d 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
> @@ -430,10 +430,6 @@ static void rkisp1_debug_init(struct rkisp1_device
> *rkisp1)
> struct rkisp1_debug *debug = &rkisp1->debug;
>
> debug->debugfs_dir = debugfs_create_dir(RKISP1_DRIVER_NAME, NULL);
> - if (!debug->debugfs_dir) {
> - dev_dbg(rkisp1->dev, "failed to create debugfs directory\n");
> - return;
> - }
> debugfs_create_ulong("data_loss", 0444, debug->debugfs_dir,
>&debug->data_loss);
> debugfs_create_ulong("outform_size_err", 0444, debug->debugfs_dir,
>
Re: [PATCH] media: rockchip: rkisp1: remove some dead code
Hi Dan,
On 12/1/20 11:27 AM, Dan Carpenter wrote:
> On Mon, Nov 30, 2020 at 11:20:05AM -0300, Helen Koike wrote:
>> Hi Dan,
>>
>> Thank you for your patch.
>>
>> On 11/30/20 9:53 AM, Dan Carpenter wrote:
>>> The debugfs_create_dir() function never returns NULLs. It's not supposed
>>> to checked for errors in the normal case and there is no need to check
>>> in this function so let's just delete this dead code.
>>>
>>> Signed-off-by: Dan Carpenter
>>> ---
>>> drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c | 4
>>> 1 file changed, 4 deletions(-)
>>>
>>> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
>>> b/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
>>> index 9af137e4967f..68da1eed753d 100644
>>> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
>>> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
>>> @@ -430,10 +430,6 @@ static void rkisp1_debug_init(struct rkisp1_device
>>> *rkisp1)
>>> struct rkisp1_debug *debug = &rkisp1->debug;
>>>
>>> debug->debugfs_dir = debugfs_create_dir(RKISP1_DRIVER_NAME, NULL);
>>> - if (!debug->debugfs_dir) {
>>> - dev_dbg(rkisp1->dev, "failed to create debugfs directory\n");
>>> - return;
>>> - }
>>
>> I was taking a look at the debugfs_create_dir() code, and I saw it can
>> return ERR_PTR(), so ideally we should check for errors with IS_ERR() /
>> PTR_ERR().
>
> Debugfs functions aren't meant to be error checked in the normal case.
> There are some drivers which dereference the dentry pointer so those
> need to check it but that's not very common and isn't the case here.
right, I just saw the functions in inode.c already checks the parent with
IS_ERR(). the debugfs_create_*() function calls start_creating() which
already checks the parent.
ok, fair enough, I'll ack v2.
Regards,
Helen
>
> I'm really sure this must be documented somewhere but I can't find it
> at all. :P But look at commit 057e212eae72 ("media: usb: uvc: no need
> to check return value of debugfs_create functions") for example.
>
> regards,
> dan carpenter
>
Re: [PATCH] media: rockchip: rkisp1: remove some dead code
Hi Dan,
Thank you for your patch.
On 11/30/20 9:53 AM, Dan Carpenter wrote:
> The debugfs_create_dir() function never returns NULLs. It's not supposed
> to checked for errors in the normal case and there is no need to check
> in this function so let's just delete this dead code.
>
> Signed-off-by: Dan Carpenter
> ---
> drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c | 4
> 1 file changed, 4 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
> b/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
> index 9af137e4967f..68da1eed753d 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
> @@ -430,10 +430,6 @@ static void rkisp1_debug_init(struct rkisp1_device
> *rkisp1)
> struct rkisp1_debug *debug = &rkisp1->debug;
>
> debug->debugfs_dir = debugfs_create_dir(RKISP1_DRIVER_NAME, NULL);
> - if (!debug->debugfs_dir) {
> - dev_dbg(rkisp1->dev, "failed to create debugfs directory\n");
> - return;
> - }
I was taking a look at the debugfs_create_dir() code, and I saw it can
return ERR_PTR(), so ideally we should check for errors with IS_ERR() /
PTR_ERR().
Also from the docs:
* If an error occurs, ERR_PTR(-ERROR) will be
* returned.
*
* If debugfs is not enabled in the kernel, the value -%ENODEV will be
* returned.
> debugfs_create_ulong("data_loss", 0444, debug->debugfs_dir,
>&debug->data_loss);
> debugfs_create_ulong("outform_size_err", 0444, debug->debugfs_dir,
>
nit: I would change the name of the commit just to make it clear what it
does.
Example: Remove useless error check from debugfs_create_dir()
Thanks,
Helen
Re: [PATCH v5 0/7] media: v4l2: Add extended fmt and buffer ioctls
Hello, On 8/4/20 4:29 PM, Helen Koike wrote: > Hello, > > This is v5 of the Extended API for formats and buffers, which introduces > the following new ioctls: > > int ioctl(int fd, VIDIOC_G_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp) > int ioctl(int fd, VIDIOC_S_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp) > int ioctl(int fd, VIDIOC_TRY_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp) > > int ioctl(int fd, VIDIOC_EXT_CREATE_BUFS, struct v4l2_ext_create_buffers > *argp) > int ioctl(int fd, VIDIOC_EXT_QUERYBUF, struct v4l2_ext_buffer *argp) > int ioctl(int fd, VIDIOC_EXT_QBUF, struct v4l2_ext_buffer *argp) > int ioctl(int fd, VIDIOC_EXT_DQBUF, struct v4l2_ext_buffer *argp) > int ioctl(int fd, VIDIOC_EXT_PREPARE_BUF, struct v4l2_ext_buffer *argp) > > Please check v4 cover letter specific topic past discussions > https://patchwork.linuxtv.org/project/linux-media/cover/20200717115435.2632623-1-helen.ko...@collabora.com/ > > Documentation > = > I added a first version of the documentation. > I would appreciate some tips in how to organize this better, since there are > several parts of the docs which reference the "old" API, and for now > I just added a note pointing to the Extended API. > > Instead of duplicating documentation from the code to the Docs (as used by > most part of v4l2 documentation), I just added a reference to let Sphinx get > the structs documentation from the code, so we can avoid duplicating. > > For reviewing convenience, I uploaded the generated html docs at > https://people.collabora.com/~koike/ext-doc-v5/userspace-api/media/v4l/ext-api.html > > There is room for improvements, it would be great to get your suggestions. > > uAPI > > This version have some minor changes in the uAPI structs, highlight to the > mem_offset that was returned to struct v4l2_ext_plane, memory field that now > is per plane, and some adjustments in field sizes and re-ordering to make > structs the same for 32 and 64 bits (which I verified with pahole tool). > > I also updated v4l2-compliance: > https://gitlab.collabora.com/koike/v4l-utils/-/tree/ext-api/wip > > We need to decide the size of the reserved fields left, how much reserved > fields do you think we should leave for each struct? > > What is next > > I would like to improve implementation (for the kernel and v4l2-compliane) and > drop the RFC tag for next version, so please review the uAPI. > > > List of changes in v5: > == > * first version of Documentation > * migrate memory from v4l2_ext_buffer to v4l2_ext_plane > * return mem_offset to struct v4l2_ext_plane > * change sizes and reorder fields to avoid holes in the struct and make it > the same for 32 and 64 bits > * removed __attribute__ ((packed)) from uapi structs > * set request_fd to signed > * add documentation > * updated some commit messages > > Hans Verkuil (1): > media: v4l2: Add extended buffer operations > > Helen Koike (6): > media: v4l2: Extend pixel formats to unify single/multi-planar > handling (and more) > media: videobuf2: Expose helpers to implement the _ext_fmt and > _ext_buf hooks > media: mediabus: Add helpers to convert a ext_pix format to/from a > mbus_fmt > media: vivid: Convert the capture and output drivers to > EXT_FMT/EXT_BUF > media: vimc: Implement the ext_fmt and ext_buf hooks > media: docs: add documentation for the Extended API > > .../userspace-api/media/v4l/buffer.rst| 5 + > .../userspace-api/media/v4l/common.rst| 1 + > .../userspace-api/media/v4l/dev-capture.rst | 5 + > .../userspace-api/media/v4l/dev-output.rst| 5 + > .../userspace-api/media/v4l/ext-api.rst | 107 ++ > .../userspace-api/media/v4l/format.rst| 16 +- > .../userspace-api/media/v4l/user-func.rst | 5 + > .../media/v4l/vidioc-ext-create-bufs.rst | 95 ++ > .../media/v4l/vidioc-ext-prepare-buf.rst | 62 ++ > .../media/v4l/vidioc-ext-qbuf.rst | 204 > .../media/v4l/vidioc-ext-querybuf.rst | 79 ++ > .../media/v4l/vidioc-g-ext-pix-fmt.rst| 117 +++ > .../media/common/videobuf2/videobuf2-core.c | 2 + > .../media/common/videobuf2/videobuf2-v4l2.c | 560 ++- > .../media/test-drivers/vimc/vimc-capture.c| 61 +- > drivers/media/test-drivers/vimc/vimc-common.c | 6 +- > drivers/media/test-drivers/vimc/vimc-common.h | 2 +- > drivers/media/test-drivers/vivid/vivid-core.c | 70 +- > .../test-drivers/vivid/vivid-touch-cap.c | 26 +- > .../test-drivers/vivid/vivid-touch-cap.h | 3 +- > .../media/test-drivers/vivid/vivid-vid-cap.c | 169 +--- > .../media/test-
Re: [PATCH] media: rockchip: rkisp1: Fix typos in comments and macro definitions
Hi Peilin,
On 11/26/20 9:21 AM, Peilin Ye wrote:
> Fix 4 typos under drivers/media/platform/rockchip/rkisp1/ found by
> checkpatch, including the RKISP1_CIF_MI_{M,S}P_PINGPONG_ENABLE macro
> definitions.
>
> Signed-off-by: Peilin Ye
Thanks
Acked-by: Helen Koike
> ---
> Hi Helen, Dafna,
>
> I noticed that the RKISP1_CIF_MI_{M,S}P_PINGPONG_ENABLE macros are not
> being used yet, but according to page 12 of this developer guide [1] I
> think they are for *enabling* the ping-pong ("double buffers") mode?
Looks like. The documentation I have doesn't explain much about this
mode, it just mention that bit is used to enable it, and that
MI_{MP,SP}_{CR,CB}_BASE_AD_INIT2 are used to configure base address 2.
Regards,
Helen
>
> Based on linux-next 9d3e48f20e11 ("Add linux-next specific files for
> 20201125").
>
> Thanks,
> Peilin Ye
>
> [1]
> https://dl.vamrs.com/products/rock960/docs/sw/Rockchip%C2%A0Linux%20Camera%C2%A0Developer%20Guide%20V1.1.pdf#page=12
>
> drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c | 4 ++--
> drivers/media/platform/rockchip/rkisp1/rkisp1-regs.h| 4 ++--
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
> b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
> index b81235afd053..94b65680c4c1 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
> @@ -46,7 +46,7 @@ enum rkisp1_plane {
> /*
> * @fourcc: pixel format
> * @fmt_type: helper filed for pixel format
> - * @uv_swap: if cb cr swaped, for yuv
> + * @uv_swap: if cb cr swapped, for yuv
> * @write_format: defines how YCbCr self picture data is written to memory
> * @output_format: defines sp output format
> * @mbus: the mbus code on the src resizer pad that matches the pixel format
> @@ -870,7 +870,7 @@ static void rkisp1_cap_stream_disable(struct
> rkisp1_capture *cap)
> {
> int ret;
>
> - /* Stream should stop in interrupt. If it dosn't, stop it by force. */
> + /* Stream should stop in interrupt. If it doesn't, stop it by force. */
> cap->is_stopping = true;
> ret = wait_event_timeout(cap->done,
>!cap->is_streaming,
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-regs.h
> b/drivers/media/platform/rockchip/rkisp1/rkisp1-regs.h
> index 049f6c3a11df..8a8d960a679c 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-regs.h
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-regs.h
> @@ -106,8 +106,8 @@
> #define RKISP1_CIF_MI_SP_Y_FULL_YUV2RGB BIT(8)
> #define RKISP1_CIF_MI_SP_CBCR_FULL_YUV2RGB BIT(9)
> #define RKISP1_CIF_MI_SP_422NONCOSITEED BIT(10)
> -#define RKISP1_CIF_MI_MP_PINGPONG_ENABEL BIT(11)
> -#define RKISP1_CIF_MI_SP_PINGPONG_ENABEL BIT(12)
> +#define RKISP1_CIF_MI_MP_PINGPONG_ENABLE BIT(11)
> +#define RKISP1_CIF_MI_SP_PINGPONG_ENABLE BIT(12)
> #define RKISP1_CIF_MI_MP_AUTOUPDATE_ENABLE BIT(13)
> #define RKISP1_CIF_MI_SP_AUTOUPDATE_ENABLE BIT(14)
> #define RKISP1_CIF_MI_LAST_PIXEL_SIG_ENABLE BIT(15)
>
Re: [PATCH v5 2/7] media: v4l2: Add extended buffer operations
Hi Tomasz,
On 11/20/20 8:14 AM, Tomasz Figa wrote:
> Hi Helen,
>
> On Tue, Aug 04, 2020 at 04:29:34PM -0300, Helen Koike wrote:
>> From: Hans Verkuil
>>
>> Those extended buffer ops have several purpose:
>> 1/ Fix y2038 issues by converting the timestamp into an u64 counting
>>the number of ns elapsed since 1970
>> 2/ Unify single/multiplanar handling
>> 3/ Add a new start offset field to each v4l2 plane buffer info struct
>>to support the case where a single buffer object is storing all
>>planes data, each one being placed at a different offset
>>
>> New hooks are created in v4l2_ioctl_ops so that drivers can start using
>> these new objects.
>>
>> The core takes care of converting new ioctls requests to old ones
>> if the driver does not support the new hooks, and vice versa.
>>
>> Note that the timecode field is gone, since there doesn't seem to be
>> in-kernel users. We can be added back in the reserved area if needed or
>> use the Request API to collect more metadata information from the
>> frame.
>>
>
> Thanks for the patch. Please see my comments inline.
Thank you for your detailed review, please see my comments below.
>
>> Signed-off-by: Hans Verkuil
>> Signed-off-by: Boris Brezillon
>> Signed-off-by: Helen Koike
>> ---
>> Changes in v5:
>> - migrate memory from v4l2_ext_buffer to v4l2_ext_plane
>> - return mem_offset to struct v4l2_ext_plane
>> - change sizes and reorder fields to avoid holes in the struct and make
>> it the same for 32 and 64 bits
>>
>> Changes in v4:
>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
>> - Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF
>> was that with VIDIOC_EXT_EXPBUF we could export multiple planes at once.
>> I think we can add this later, so I removed it from this RFC to simplify it.
>> - Remove num_planes field from struct v4l2_ext_buffer
>> - Add flags field to struct v4l2_ext_create_buffers
>> - Reformulate struct v4l2_ext_plane
>> - Fix some bugs caught by v4l2-compliance
>> - Rebased on top of media/master (post 5.8-rc1)
>>
>> Changes in v3:
>> - Rebased on top of media/master (post 5.4-rc1)
>>
>> Changes in v2:
>> - Add reserved space to v4l2_ext_buffer so that new fields can be added
>> later on
>> ---
>> drivers/media/v4l2-core/v4l2-dev.c | 29 ++-
>> drivers/media/v4l2-core/v4l2-ioctl.c | 353 +--
>> include/media/v4l2-ioctl.h | 26 ++
>> include/uapi/linux/videodev2.h | 90 +++
>> 4 files changed, 476 insertions(+), 22 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>> b/drivers/media/v4l2-core/v4l2-dev.c
>> index e1829906bc086..cb21ee8eb075c 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -720,15 +720,34 @@ static void determine_valid_ioctls(struct video_device
>> *vdev)
>> SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_sdr_out);
>> }
>>
>> +if (is_vid || is_tch) {
>> +/* ioctls valid for video and touch */
>> +if (ops->vidioc_querybuf || ops->vidioc_ext_querybuf)
>> +set_bit(_IOC_NR(VIDIOC_EXT_QUERYBUF), valid_ioctls);
>> +if (ops->vidioc_qbuf || ops->vidioc_ext_qbuf)
>> +set_bit(_IOC_NR(VIDIOC_EXT_QBUF), valid_ioctls);
>> +if (ops->vidioc_dqbuf || ops->vidioc_ext_dqbuf)
>> +set_bit(_IOC_NR(VIDIOC_EXT_DQBUF), valid_ioctls);
>> +if (ops->vidioc_create_bufs || ops->vidioc_ext_create_bufs)
>> +set_bit(_IOC_NR(VIDIOC_EXT_CREATE_BUFS), valid_ioctls);
>> +if (ops->vidioc_prepare_buf || ops->vidioc_ext_prepare_buf)
>> +set_bit(_IOC_NR(VIDIOC_EXT_PREPARE_BUF), valid_ioctls);
>
> nit: Could we stick to the SET_VALID_IOCTL() macro and just call it twice,
> once for the new and once for the legacy callback?
Ack.
>
>> +}
>> +
>> if (is_vid || is_vbi || is_sdr || is_tch || is_meta) {
>> /* ioctls valid for video, vbi, sdr, touch and metadata */
>> SET_VALID_IOCTL(ops, VIDIOC_REQBUFS, vidioc_reqbufs);
>> -SET_VALID_IOCTL(ops, VIDIOC_QUERYBUF, vidioc_querybuf);
>> -SET_VALID_IOCTL(ops, VIDIOC_QBUF, vidioc_qbuf);
>> SET_VALID_IOCTL(ops, VIDIOC_E
Re: [PATCH v5 2/7] media: v4l2: Add extended buffer operations
On 11/23/20 12:46 PM, Tomasz Figa wrote:
> On Tue, Nov 24, 2020 at 12:08 AM Helen Koike
> wrote:
>>
>> Hi Hans,
>>
>> Thank you for your review.
>>
>> On 9/9/20 9:27 AM, Hans Verkuil wrote:
>>> Hi Helen,
>>>
>>> Again I'm just reviewing the uAPI.
>>>
>>> On 04/08/2020 21:29, Helen Koike wrote:
>>>> From: Hans Verkuil
>>>>
>>>> Those extended buffer ops have several purpose:
>>>> 1/ Fix y2038 issues by converting the timestamp into an u64 counting
>>>>the number of ns elapsed since 1970
>>>> 2/ Unify single/multiplanar handling
>>>> 3/ Add a new start offset field to each v4l2 plane buffer info struct
>>>>to support the case where a single buffer object is storing all
>>>>planes data, each one being placed at a different offset
>>>>
>>>> New hooks are created in v4l2_ioctl_ops so that drivers can start using
>>>> these new objects.
>>>>
>>>> The core takes care of converting new ioctls requests to old ones
>>>> if the driver does not support the new hooks, and vice versa.
>>>>
>>>> Note that the timecode field is gone, since there doesn't seem to be
>>>> in-kernel users. We can be added back in the reserved area if needed or
>>>> use the Request API to collect more metadata information from the
>>>> frame.
>>>>
>>>> Signed-off-by: Hans Verkuil
>>>> Signed-off-by: Boris Brezillon
>>>> Signed-off-by: Helen Koike
>>>> ---
>>>> Changes in v5:
>>>> - migrate memory from v4l2_ext_buffer to v4l2_ext_plane
>>>> - return mem_offset to struct v4l2_ext_plane
>>>> - change sizes and reorder fields to avoid holes in the struct and make
>>>> it the same for 32 and 64 bits
>>>>
>>>> Changes in v4:
>>>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
>>>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
>>>> - Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF
>>>> was that with VIDIOC_EXT_EXPBUF we could export multiple planes at once.
>>>> I think we can add this later, so I removed it from this RFC to simplify
>>>> it.
>>>> - Remove num_planes field from struct v4l2_ext_buffer
>>>> - Add flags field to struct v4l2_ext_create_buffers
>>>> - Reformulate struct v4l2_ext_plane
>>>> - Fix some bugs caught by v4l2-compliance
>>>> - Rebased on top of media/master (post 5.8-rc1)
>>>>
>>>> Changes in v3:
>>>> - Rebased on top of media/master (post 5.4-rc1)
>>>>
>>>> Changes in v2:
>>>> - Add reserved space to v4l2_ext_buffer so that new fields can be added
>>>> later on
>>>> ---
>>>> drivers/media/v4l2-core/v4l2-dev.c | 29 ++-
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 353 +--
>>>> include/media/v4l2-ioctl.h | 26 ++
>>>> include/uapi/linux/videodev2.h | 90 +++
>>>> 4 files changed, 476 insertions(+), 22 deletions(-)
>>>>
>>>
>>>
>>>
>>>> diff --git a/include/uapi/linux/videodev2.h
>>>> b/include/uapi/linux/videodev2.h
>>>> index 7123c6a4d9569..334cafdd2be97 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -996,6 +996,41 @@ struct v4l2_plane {
>>>> __u32 reserved[11];
>>>> };
>>>>
>>>> +/**
>>>> + * struct v4l2_ext_plane - extended plane buffer info
>>>> + * @buffer_length: size of the entire buffer in bytes, should fit
>>>> + * @offset + @plane_length
>>>> + * @plane_length: size of the plane in bytes.
>>>> + * @mem_offset: If V4L2_MEMORY_MMAP is used, then it can be a
>>>> "cookie"
>>>> + * that should be passed to mmap() called on the video
>>>> node.
>>>> + * @userptr:when memory is V4L2_MEMORY_USERPTR, a
>>>> userspace pointer pointing
>>>> + * to this plane.
>>>> + * @dmabuf_fd: when memory is V4L2_MEMORY_DMABUF, a
>>>> userspace file descriptor
>>>> + * associated with this plane.
>>>> + * @offset:
Re: [PATCH v5 2/7] media: v4l2: Add extended buffer operations
Hi Hans,
Thank you for your review.
On 9/9/20 9:27 AM, Hans Verkuil wrote:
> Hi Helen,
>
> Again I'm just reviewing the uAPI.
>
> On 04/08/2020 21:29, Helen Koike wrote:
>> From: Hans Verkuil
>>
>> Those extended buffer ops have several purpose:
>> 1/ Fix y2038 issues by converting the timestamp into an u64 counting
>>the number of ns elapsed since 1970
>> 2/ Unify single/multiplanar handling
>> 3/ Add a new start offset field to each v4l2 plane buffer info struct
>>to support the case where a single buffer object is storing all
>>planes data, each one being placed at a different offset
>>
>> New hooks are created in v4l2_ioctl_ops so that drivers can start using
>> these new objects.
>>
>> The core takes care of converting new ioctls requests to old ones
>> if the driver does not support the new hooks, and vice versa.
>>
>> Note that the timecode field is gone, since there doesn't seem to be
>> in-kernel users. We can be added back in the reserved area if needed or
>> use the Request API to collect more metadata information from the
>> frame.
>>
>> Signed-off-by: Hans Verkuil
>> Signed-off-by: Boris Brezillon
>> Signed-off-by: Helen Koike
>> ---
>> Changes in v5:
>> - migrate memory from v4l2_ext_buffer to v4l2_ext_plane
>> - return mem_offset to struct v4l2_ext_plane
>> - change sizes and reorder fields to avoid holes in the struct and make
>> it the same for 32 and 64 bits
>>
>> Changes in v4:
>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
>> - Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF
>> was that with VIDIOC_EXT_EXPBUF we could export multiple planes at once.
>> I think we can add this later, so I removed it from this RFC to simplify it.
>> - Remove num_planes field from struct v4l2_ext_buffer
>> - Add flags field to struct v4l2_ext_create_buffers
>> - Reformulate struct v4l2_ext_plane
>> - Fix some bugs caught by v4l2-compliance
>> - Rebased on top of media/master (post 5.8-rc1)
>>
>> Changes in v3:
>> - Rebased on top of media/master (post 5.4-rc1)
>>
>> Changes in v2:
>> - Add reserved space to v4l2_ext_buffer so that new fields can be added
>> later on
>> ---
>> drivers/media/v4l2-core/v4l2-dev.c | 29 ++-
>> drivers/media/v4l2-core/v4l2-ioctl.c | 353 +--
>> include/media/v4l2-ioctl.h | 26 ++
>> include/uapi/linux/videodev2.h | 90 +++
>> 4 files changed, 476 insertions(+), 22 deletions(-)
>>
>
>
>
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index 7123c6a4d9569..334cafdd2be97 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -996,6 +996,41 @@ struct v4l2_plane {
>> __u32 reserved[11];
>> };
>>
>> +/**
>> + * struct v4l2_ext_plane - extended plane buffer info
>> + * @buffer_length: size of the entire buffer in bytes, should fit
>> + * @offset + @plane_length
>> + * @plane_length: size of the plane in bytes.
>> + * @mem_offset: If V4L2_MEMORY_MMAP is used, then it can be a
>> "cookie"
>> + * that should be passed to mmap() called on the video
>> node.
>> + * @userptr:when memory is V4L2_MEMORY_USERPTR, a userspace
>> pointer pointing
>> + * to this plane.
>> + * @dmabuf_fd: when memory is V4L2_MEMORY_DMABUF, a userspace
>> file descriptor
>> + * associated with this plane.
>> + * @offset: offset in the memory buffer where the plane starts.
>> + * @memory: enum v4l2_memory; the method, in which the actual video
>> + * data is passed
>> + * @reserved: extra space reserved for future fields, must be
>> set to 0.
>> + *
>> + *
>> + * Buffers consist of one or more planes, e.g. an YCbCr buffer with two
>> planes
>> + * can have one plane for Y, and another for interleaved CbCr components.
>> + * Each plane can reside in a separate memory buffer, or even in
>> + * a completely separate memory node (e.g. in embedded devices).
>> + */
>> +struct v4l2_ext_plane {
>> +__u32 buffer_length;
>> +__u32 plane_length;
>> +union {
>> +__u32 mem_offset;
>> +__u64 userptr;
>> +
Re: [PATCH v2] char: tpm: add i2c driver for cr50
Hello Adrian, I just spotted small things (nothing major), please see below. On 11/20/20 2:23 PM, Adrian Ratiu wrote: > From: "dlau...@chromium.org" > > Add TPM 2.0 compatible I2C interface for chips with cr50 firmware. > > The firmware running on the currently supported H1 MCU requires a > special driver to handle its specific protocol, and this makes it > unsuitable to use tpm_tis_core_* and instead it must implement the > underlying TPM protocol similar to the other I2C TPM drivers. > > - All 4 byes of status register must be read/written at once. s/byes/bytes > - FIFO and burst count is limited to 63 and must be drained by AP. > - Provides an interrupt to indicate when read response data is ready > and when the TPM is finished processing write data. > > This driver is based on the existing infineon I2C TPM driver, which > most closely matches the cr50 i2c protocol behavior. > > Cc: Helen Koike > Signed-off-by: Duncan Laurie > [swb...@chromium.org: Depend on i2c even if it's a module, replace > boilier plate with SPDX tag, drop asm/byteorder.h include, simplify > return from probe] > Signed-off-by: Stephen Boyd > Signed-off-by: Fabien Lahoudere > Signed-off-by: Adrian Ratiu > --- > Changes in v2: > - Various small fixes all over (reorder includes, MAX_BUFSIZE, comments, > etc) > - Reworked return values of i2c_wait_tpm_ready() to fix timeout mis-handling > so ret == 0 now means success, the wait period jiffies is ignored because that > number is meaningless and return a proper timeout error in case jiffies == 0. > - Make i2c default to 1 message per transfer (requested by Helen) > - Move -EIO error reporting to transfer function to cleanup transfer() > itself > and its R/W callers > - Remove magic value hardcodings and introduce enum force_release. > > v1 posted at https://lkml.org/lkml/2020/2/25/349 > > Applies on next-20201120, tested on Chromebook EVE. > --- > drivers/char/tpm/Kconfig| 10 + > drivers/char/tpm/Makefile | 2 + > drivers/char/tpm/tpm_tis_i2c_cr50.c | 768 > 3 files changed, 780 insertions(+) > create mode 100644 drivers/char/tpm/tpm_tis_i2c_cr50.c > > diff --git a/drivers/char/tpm/Kconfig b/drivers/char/tpm/Kconfig > index a18c314da211..4308f9ca7a43 100644 > --- a/drivers/char/tpm/Kconfig > +++ b/drivers/char/tpm/Kconfig > @@ -86,6 +86,16 @@ config TCG_TIS_SYNQUACER > To compile this driver as a module, choose M here; > the module will be called tpm_tis_synquacer. > > +config TCG_TIS_I2C_CR50 > + tristate "TPM Interface Specification 2.0 Interface (I2C - CR50)" > + depends on I2C > + select TCG_CR50 > + help > + This is a driver for the Google cr50 I2C TPM interface which is a > + custom microcontroller and requires a custom i2c protocol interface > + to handle the limitations of the hardware. To compile this driver > + as a module, choose M here; the module will be called > tcg_tis_i2c_cr50. > + > config TCG_TIS_I2C_ATMEL > tristate "TPM Interface Specification 1.2 Interface (I2C - Atmel)" > depends on I2C > diff --git a/drivers/char/tpm/Makefile b/drivers/char/tpm/Makefile > index 84db4fb3a9c9..66d39ea6bd10 100644 > --- a/drivers/char/tpm/Makefile > +++ b/drivers/char/tpm/Makefile > @@ -27,6 +27,8 @@ obj-$(CONFIG_TCG_TIS_SPI) += tpm_tis_spi.o > tpm_tis_spi-y := tpm_tis_spi_main.o > tpm_tis_spi-$(CONFIG_TCG_TIS_SPI_CR50) += tpm_tis_spi_cr50.o > > +obj-$(CONFIG_TCG_TIS_I2C_CR50) += tpm_tis_i2c_cr50.o > + > obj-$(CONFIG_TCG_TIS_I2C_ATMEL) += tpm_i2c_atmel.o > obj-$(CONFIG_TCG_TIS_I2C_INFINEON) += tpm_i2c_infineon.o > obj-$(CONFIG_TCG_TIS_I2C_NUVOTON) += tpm_i2c_nuvoton.o > diff --git a/drivers/char/tpm/tpm_tis_i2c_cr50.c > b/drivers/char/tpm/tpm_tis_i2c_cr50.c > new file mode 100644 > index ..37555dafdca0 > --- /dev/null > +++ b/drivers/char/tpm/tpm_tis_i2c_cr50.c > @@ -0,0 +1,768 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Copyright 2016 Google Inc. > + * > + * Based on Linux Kernel TPM driver by > + * Peter Huewe > + * Copyright (C) 2011 Infineon Technologies > + */ > + > +/* > + * cr50 is a firmware for H1 secure modules that requires special > + * handling for the I2C interface. > + * > + * - Use an interrupt for transaction status instead of hardcoded delays > + * - Must use write+wait+read read protocol > + * - All 4 bytes of status register must be read/written at once > + * - Burst count max is 63 bytes, and burst count behaves > + * slightly differently than other I2C TPMs > + * - When reading from FIFO the full burstcnt must be
Re: [PATCH v5 1/7] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
Hi Alexandre,
Thanks for your review.
On 8/14/20 4:49 AM, Alexandre Courbot wrote:
> On Wed, Aug 5, 2020 at 4:32 AM Helen Koike wrote:
>>
>> This is part of the multiplanar and singleplanar unification process.
>> v4l2_ext_pix_format is supposed to work for both cases.
>>
>> We also add the concept of modifiers already employed in DRM to expose
>> HW-specific formats (like tiled or compressed formats) and allow
>> exchanging this information with the DRM subsystem in a consistent way.
>>
>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in
>> v4l2_ext_format, other types will be rejected if you use the
>> {G,S,TRY}_EXT_PIX_FMT ioctls.
>>
>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls
>> in drivers, but, in the meantime, the core takes care of converting
>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can
>> still work if the userspace app/lib uses the new ioctls.
>> The conversion is also done the other around to allow userspace
>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the
>> _ext_ hooks.
>>
>> Signed-off-by: Boris Brezillon
>> Signed-off-by: Helen Koike
>> ---
>> Changes in v5:
>> - change sizes and reorder fields to avoid holes in the struct and make
>> it the same for 32 and 64 bits
>> - removed __attribute__ ((packed)) from uapi structs
>> - Fix doc warning from make htmldocs
>> - Updated commit message with EXT_PIX prefix for the ioctls.
>>
>> Changes in v4:
>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
>> - Add reserved fields
>> - Removed num_planes from struct v4l2_ext_pix_format
>> - Removed flag field from struct v4l2_ext_pix_format, since the only
>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1,
>> where we can use modifiers, or add it back later through the reserved
>> bits.
>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR &&
>> != MOD_INVALID
>> - Fix type assignment in v4l_g_fmt_ext_pix()
>> - Rebased on top of media/master (post 5.8-rc1)
>>
>> Changes in v3:
>> - Rebased on top of media/master (post 5.4-rc1)
>>
>> Changes in v2:
>> - Move the modifier in v4l2_ext_format (was formerly placed in
>> v4l2_ext_plane)
>> - Fix a few bugs in the converters and add a strict parameter to
>> allow conversion of uninitialized/mis-initialized objects
>> ---
>> drivers/media/v4l2-core/v4l2-dev.c | 21 +-
>> drivers/media/v4l2-core/v4l2-ioctl.c | 585 +++
>> include/media/v4l2-ioctl.h | 34 ++
>> include/uapi/linux/videodev2.h | 56 +++
>> 4 files changed, 615 insertions(+), 81 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>> b/drivers/media/v4l2-core/v4l2-dev.c
>> index a593ea0598b55..e1829906bc086 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -607,25 +607,37 @@ static void determine_valid_ioctls(struct video_device
>> *vdev)
>> set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls);
>> if ((is_rx && (ops->vidioc_g_fmt_vid_cap ||
>>ops->vidioc_g_fmt_vid_cap_mplane ||
>> + ops->vidioc_g_ext_pix_fmt_vid_cap ||
>>ops->vidioc_g_fmt_vid_overlay)) ||
>> (is_tx && (ops->vidioc_g_fmt_vid_out ||
>>ops->vidioc_g_fmt_vid_out_mplane ||
>> - ops->vidioc_g_fmt_vid_out_overlay)))
>> + ops->vidioc_g_ext_pix_fmt_vid_out ||
>> + ops->vidioc_g_fmt_vid_out_overlay))) {
>> set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls);
>> +set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT),
>> valid_ioctls);
>> + }
>> if ((is_rx && (ops->vidioc_s_fmt_vid_cap ||
>>ops->vidioc_s_fmt_vid_cap_mplane ||
>> + ops->vidioc_s_ext_pix_fmt_vid_cap ||
>>ops->vidioc_s_fmt_vid_overlay)) ||
>> (is_tx && (ops->vidioc_s_fmt_vid_out ||
>>ops->vidioc_s_fmt_vid_out_mplane ||
>> -
Re: [PATCH v5 1/7] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
On 11/19/20 7:08 AM, Helen Koike wrote:
> Hi Tomasz,
>
> On 11/19/20 2:45 AM, Tomasz Figa wrote:
>> On Sat, Nov 14, 2020 at 11:21:26AM -0300, Helen Koike wrote:
>>> Hi Tomasz,
>>>
>>> On 10/2/20 4:49 PM, Tomasz Figa wrote:
>>>> Hi Helen,
>>>>
>>>> On Tue, Aug 04, 2020 at 04:29:33PM -0300, Helen Koike wrote:
>> [snip]
>>>>> +static void v4l_print_ext_pix_format(const void *arg, bool write_only)
>>>>> +{
>>>>> + const struct v4l2_ext_pix_format *pix = arg;
>>>>> + unsigned int i;
>>>>> +
>>>>> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx,
>>>>> field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n",
>>>>> + prt_names(pix->type, v4l2_type_names),
>>>>> + pix->width, pix->height,
>>>>> + (pix->pixelformat & 0xff),
>>>>> + (pix->pixelformat >> 8) & 0xff,
>>>>> + (pix->pixelformat >> 16) & 0xff,
>>>>> + (pix->pixelformat >> 24) & 0xff,
>>>>> + pix->modifier, prt_names(pix->field, v4l2_field_names),
>>>>> + pix->colorspace, pix->ycbcr_enc,
>>>>> + pix->quantization, pix->xfer_func);
>>>>> + for (i = 0; i < VIDEO_MAX_PLANES && pix->plane_fmt[i].sizeimage; i++)
>>>>
>>>> This is going to print 8 lines every time. Maybe we could skip 0-sized
>>>> planes or something?
>>>
>>> I'm already checking pix->plane_fmt[i].sizeimage in the loop, it shouldn't
>>> print 8 lines every time.
>>>
>>
>> Oops, how could I not notice it. Sorry for the noise.
>>
>> [snip]
>>>>> +int v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *e,
>>>>> + struct v4l2_format *f, bool mplane_cap,
>>>>> + bool strict)
>>>>> +{
>>>>> + const struct v4l2_plane_ext_pix_format *pe;
>>>>> + struct v4l2_plane_pix_format *p;
>>>>> + unsigned int i;
>>>>> +
>>>>> + memset(f, 0, sizeof(*f));
>>>>> +
>>>>> + /*
>>>>> + * Make sure no modifier is required before doing the
>>>>> + * conversion.
>>>>> + */
>>>>> + if (e->modifier && strict &&
>>>>
>>>> Do we need the explicit check for e->modifier != 0 if we have to check for
>>>> the 2 specific values below anyway?
>>>
>>> We don't, since DRM_FORMAT_MOD_LINEAR is zero.
>>>
>>> But I wanted to make it explicit we don't support modifiers in this
>>> conversion.
>>> But I can remove this check, no problem.
>>>
>>
>> Yes, please. I think the double checking is confusing for the reader.
>
> ok.
>
>>
>>>>
>>>>> + e->modifier != DRM_FORMAT_MOD_LINEAR &&
>>>>> + e->modifier != DRM_FORMAT_MOD_INVALID)
>>>>> + return -EINVAL;
>>>>> +
>>>>> + if (!e->plane_fmt[0].sizeimage && strict)
>>>>> + return -EINVAL;
>>>>
>>>> Why is this incorrect?
>>>
>>> !sizeimage for the first plane means that there are no planes in ef.
>>> strict is true if the result for the conversion should be returned to
>>> userspace
>>> and it is not some internal handling.
>>>
>>> So if there are no planes, we would return an incomplete v4l2_format struct
>>> to userspace.
>>>
>>> But this is not very clear, I'll improve this for the next version.
>>>
>>
>> So I can see 2 cases here:
>>
>> 1) Userspace gives ext struct and driver accepts legacy.
>>
>> In this case, the kernel needs to adjust the structure to be correct.
>> -EINVAL is only valid if
>>
>> "The struct v4l2_format type field is invalid or the requested buffer type
>> not supported."
>>
>> as per the current uAPI documentation.
>>
>> 2) Driver gives ext struct and userspace accepts legacy.
>>
>> If at this point we get a struct with no planes, that sounds like a
>> driver bug, so rather than signaling -EINVAL to the userspace, we should
>> probably WARN()?
>>
>> Or am I getting something
Re: [PATCH v5 7/7] media: docs: add documentation for the Extended API
On 8/4/20 4:29 PM, Helen Koike wrote: > Add documentation and update references in current documentation for the > Extended API. > > Signed-off-by: Helen Koike > --- > Changes in v5: > - new patch > > .../userspace-api/media/v4l/buffer.rst| 5 + > .../userspace-api/media/v4l/common.rst| 1 + > .../userspace-api/media/v4l/dev-capture.rst | 5 + > .../userspace-api/media/v4l/dev-output.rst| 5 + > .../userspace-api/media/v4l/ext-api.rst | 107 + > .../userspace-api/media/v4l/format.rst| 16 +- > .../userspace-api/media/v4l/user-func.rst | 5 + > .../media/v4l/vidioc-ext-create-bufs.rst | 95 > .../media/v4l/vidioc-ext-prepare-buf.rst | 62 ++ > .../media/v4l/vidioc-ext-qbuf.rst | 204 ++ > .../media/v4l/vidioc-ext-querybuf.rst | 79 +++ > .../media/v4l/vidioc-g-ext-pix-fmt.rst| 117 ++ > 12 files changed, 697 insertions(+), 4 deletions(-) > create mode 100644 Documentation/userspace-api/media/v4l/ext-api.rst > create mode 100644 > Documentation/userspace-api/media/v4l/vidioc-ext-create-bufs.rst > create mode 100644 > Documentation/userspace-api/media/v4l/vidioc-ext-prepare-buf.rst > create mode 100644 Documentation/userspace-api/media/v4l/vidioc-ext-qbuf.rst > create mode 100644 > Documentation/userspace-api/media/v4l/vidioc-ext-querybuf.rst > create mode 100644 > Documentation/userspace-api/media/v4l/vidioc-g-ext-pix-fmt.rst > > diff --git a/Documentation/userspace-api/media/v4l/buffer.rst > b/Documentation/userspace-api/media/v4l/buffer.rst > index 57e752aaf414a..c832bedd64e4c 100644 > --- a/Documentation/userspace-api/media/v4l/buffer.rst > +++ b/Documentation/userspace-api/media/v4l/buffer.rst > @@ -27,6 +27,11 @@ such as pointers and sizes for each plane, are stored in > struct :c:type:`v4l2_plane` instead. In that case, > struct :c:type:`v4l2_buffer` contains an array of plane structures. > > +.. note:: > + > +The :ref:`ext_api` version can also be used, and it is > +preferable when applicable. > + > Dequeued video buffers come with timestamps. The driver decides at which > part of the frame and with which clock the timestamp is taken. Please > see flags in the masks ``V4L2_BUF_FLAG_TIMESTAMP_MASK`` and > diff --git a/Documentation/userspace-api/media/v4l/common.rst > b/Documentation/userspace-api/media/v4l/common.rst > index 7d81c58a13cd7..3430e0bdad667 100644 > --- a/Documentation/userspace-api/media/v4l/common.rst > +++ b/Documentation/userspace-api/media/v4l/common.rst > @@ -59,6 +59,7 @@ applicable to all devices. > ext-ctrls-detect > fourcc > format > +ext-api > planar-apis > selection-api > crop > diff --git a/Documentation/userspace-api/media/v4l/dev-capture.rst > b/Documentation/userspace-api/media/v4l/dev-capture.rst > index 44d3094093ab6..5077639787d92 100644 > --- a/Documentation/userspace-api/media/v4l/dev-capture.rst > +++ b/Documentation/userspace-api/media/v4l/dev-capture.rst > @@ -102,6 +102,11 @@ and :ref:`VIDIOC_S_FMT ` ioctl, even if > :ref:`VIDIOC_S_FMT requests and always returns default parameters as :ref:`VIDIOC_G_FMT > ` does. > :ref:`VIDIOC_TRY_FMT ` is optional. > > +.. note:: > + > +The :ref:`ext_api` version can also be used, and it is > +preferable when applicable. > + > > Reading Images > == > diff --git a/Documentation/userspace-api/media/v4l/dev-output.rst > b/Documentation/userspace-api/media/v4l/dev-output.rst > index e4f2a1d8b0fc7..f8f40c708e49f 100644 > --- a/Documentation/userspace-api/media/v4l/dev-output.rst > +++ b/Documentation/userspace-api/media/v4l/dev-output.rst > @@ -99,6 +99,11 @@ and :ref:`VIDIOC_S_FMT ` ioctl, even if > :ref:`VIDIOC_S_FMT requests and always returns default parameters as :ref:`VIDIOC_G_FMT > ` does. > :ref:`VIDIOC_TRY_FMT ` is optional. > > +.. note:: > + > +The :ref:`ext_api` version can also be used, and it is > +preferable when applicable. > + > > Writing Images > == > diff --git a/Documentation/userspace-api/media/v4l/ext-api.rst > b/Documentation/userspace-api/media/v4l/ext-api.rst > new file mode 100644 > index 0..da2a82960d22f > --- /dev/null > +++ b/Documentation/userspace-api/media/v4l/ext-api.rst > @@ -0,0 +1,107 @@ > +.. Permission is granted to copy, distribute and/or modify this > +.. document under the terms of the GNU Free Documentation License, > +.. Version 1.1 or any later version published by the Free Software > +.. Foundation, with no Invariant Sections, no Front-Cover Texts > +.. and no Back-Cover Texts.
Re: [PATCH v5 1/7] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
Hi Tomasz,
On 11/19/20 2:45 AM, Tomasz Figa wrote:
> On Sat, Nov 14, 2020 at 11:21:26AM -0300, Helen Koike wrote:
>> Hi Tomasz,
>>
>> On 10/2/20 4:49 PM, Tomasz Figa wrote:
>>> Hi Helen,
>>>
>>> On Tue, Aug 04, 2020 at 04:29:33PM -0300, Helen Koike wrote:
> [snip]
>>>> +static void v4l_print_ext_pix_format(const void *arg, bool write_only)
>>>> +{
>>>> + const struct v4l2_ext_pix_format *pix = arg;
>>>> + unsigned int i;
>>>> +
>>>> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx,
>>>> field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n",
>>>> + prt_names(pix->type, v4l2_type_names),
>>>> + pix->width, pix->height,
>>>> + (pix->pixelformat & 0xff),
>>>> + (pix->pixelformat >> 8) & 0xff,
>>>> + (pix->pixelformat >> 16) & 0xff,
>>>> + (pix->pixelformat >> 24) & 0xff,
>>>> + pix->modifier, prt_names(pix->field, v4l2_field_names),
>>>> + pix->colorspace, pix->ycbcr_enc,
>>>> + pix->quantization, pix->xfer_func);
>>>> + for (i = 0; i < VIDEO_MAX_PLANES && pix->plane_fmt[i].sizeimage; i++)
>>>
>>> This is going to print 8 lines every time. Maybe we could skip 0-sized
>>> planes or something?
>>
>> I'm already checking pix->plane_fmt[i].sizeimage in the loop, it shouldn't
>> print 8 lines every time.
>>
>
> Oops, how could I not notice it. Sorry for the noise.
>
> [snip]
>>>> +int v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *e,
>>>> +struct v4l2_format *f, bool mplane_cap,
>>>> +bool strict)
>>>> +{
>>>> + const struct v4l2_plane_ext_pix_format *pe;
>>>> + struct v4l2_plane_pix_format *p;
>>>> + unsigned int i;
>>>> +
>>>> + memset(f, 0, sizeof(*f));
>>>> +
>>>> + /*
>>>> + * Make sure no modifier is required before doing the
>>>> + * conversion.
>>>> + */
>>>> + if (e->modifier && strict &&
>>>
>>> Do we need the explicit check for e->modifier != 0 if we have to check for
>>> the 2 specific values below anyway?
>>
>> We don't, since DRM_FORMAT_MOD_LINEAR is zero.
>>
>> But I wanted to make it explicit we don't support modifiers in this
>> conversion.
>> But I can remove this check, no problem.
>>
>
> Yes, please. I think the double checking is confusing for the reader.
ok.
>
>>>
>>>> + e->modifier != DRM_FORMAT_MOD_LINEAR &&
>>>> + e->modifier != DRM_FORMAT_MOD_INVALID)
>>>> + return -EINVAL;
>>>> +
>>>> + if (!e->plane_fmt[0].sizeimage && strict)
>>>> + return -EINVAL;
>>>
>>> Why is this incorrect?
>>
>> !sizeimage for the first plane means that there are no planes in ef.
>> strict is true if the result for the conversion should be returned to
>> userspace
>> and it is not some internal handling.
>>
>> So if there are no planes, we would return an incomplete v4l2_format struct
>> to userspace.
>>
>> But this is not very clear, I'll improve this for the next version.
>>
>
> So I can see 2 cases here:
>
> 1) Userspace gives ext struct and driver accepts legacy.
>
> In this case, the kernel needs to adjust the structure to be correct.
> -EINVAL is only valid if
>
> "The struct v4l2_format type field is invalid or the requested buffer type
> not supported."
>
> as per the current uAPI documentation.
>
> 2) Driver gives ext struct and userspace accepts legacy.
>
> If at this point we get a struct with no planes, that sounds like a
> driver bug, so rather than signaling -EINVAL to the userspace, we should
> probably WARN()?
>
> Or am I getting something wrong? :)
Make sense, I'll restructure this for the next version.
>
> [snip]
>>>> +{
>>>> + const struct v4l2_plane_pix_format *p;
>>>> + struct v4l2_plane_ext_pix_format *pe;
>>>> + unsigned int i;
>>>> +
>>>> + memset(e, 0, sizeof(*e));
>>>> +
>>>> + switch (f->type) {
>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
Re: linux-next: build warnings after merge of the v4l-dvb tree
Hi Stephen,
On 11/18/20 2:32 AM, Stephen Rothwell wrote:
> Hi all,
>
> After merging the v4l-dvb tree, today's linux-next build (htmldocs)
> produced these warnings:
>
> Documentation/output/videodev2.h.rst:6: WARNING: undefined label:
> v4l2-meta-fmt-rk-isp1-params (if the link has no caption the label must
> precede a section header)
> Documentation/output/videodev2.h.rst:6: WARNING: undefined label:
> v4l2-meta-fmt-rk-isp1-stat-3a (if the link has no caption the label must
> precede a section header)
>
> Introduced by commit
>
> df22026aebd8 ("media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer
> format")
>
Thanks for catching this, fix sent:
https://patchwork.linuxtv.org/project/linux-media/patch/20201118142400.3540109-1-helen.ko...@collabora.com/
Regards,
Helen
[PATCH] media: admin-guide/pixfmt-meta-rkisp1.rst: pixfmt reference conforming with macro
Fix warnings from make htmlddocs:
Documentation/output/videodev2.h.rst:6: WARNING: undefined label:
v4l2-meta-fmt-rk-isp1-params (if the link has no caption the label must precede
a section header)
Documentation/output/videodev2.h.rst:6: WARNING: undefined label:
v4l2-meta-fmt-rk-isp1-stat-3a (if the link has no caption the label must
precede a section header)
Fixes: df22026aebd8 ("media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer
format")
Signed-off-by: Helen Koike
Reported-by: Stephen Rothwell
---
Documentation/admin-guide/media/rkisp1.rst | 4 ++--
Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/Documentation/admin-guide/media/rkisp1.rst
b/Documentation/admin-guide/media/rkisp1.rst
index 42e37ed255f6..2267e4fb475e 100644
--- a/Documentation/admin-guide/media/rkisp1.rst
+++ b/Documentation/admin-guide/media/rkisp1.rst
@@ -86,7 +86,7 @@ the driver through the rkisp_params node to improve image
quality during a
video stream.
The buffer format is defined by struct :c:type:`rkisp1_stat_buffer`, and
userspace should set
-:ref:`V4L2_META_FMT_RK_ISP1_STAT_3A ` as the
+:ref:`V4L2_META_FMT_RK_ISP1_STAT_3A ` as the
dataformat.
.. _rkisp1_params:
@@ -100,7 +100,7 @@ and others.
The buffer format is defined by struct :c:type:`rkisp1_params_cfg`, and
userspace should set
-:ref:`V4L2_META_FMT_RK_ISP1_PARAMS ` as the
+:ref:`V4L2_META_FMT_RK_ISP1_PARAMS ` as the
dataformat.
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
b/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
index f3671472d410..922fa1d59898 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
@@ -1,7 +1,7 @@
.. SPDX-License-Identifier: GPL-2.0
-.. _v4l2-meta-fmt-params-rkisp1:
-.. _v4l2-meta-fmt-stat-rkisp1:
+.. _v4l2-meta-fmt-rk-isp1-params:
+.. _v4l2-meta-fmt-rk-isp1-stat-3a:
*
V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s')
--
2.29.2
Re: [PATCH v5 1/7] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
Hi Tomasz,
On 10/2/20 4:49 PM, Tomasz Figa wrote:
> Hi Helen,
>
> On Tue, Aug 04, 2020 at 04:29:33PM -0300, Helen Koike wrote:
>> This is part of the multiplanar and singleplanar unification process.
>> v4l2_ext_pix_format is supposed to work for both cases.
>>
>> We also add the concept of modifiers already employed in DRM to expose
>> HW-specific formats (like tiled or compressed formats) and allow
>> exchanging this information with the DRM subsystem in a consistent way.
>>
>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in
>> v4l2_ext_format, other types will be rejected if you use the
>> {G,S,TRY}_EXT_PIX_FMT ioctls.
>>
>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls
>> in drivers, but, in the meantime, the core takes care of converting
>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can
>> still work if the userspace app/lib uses the new ioctls.
>> The conversion is also done the other around to allow userspace
>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the
>> _ext_ hooks.
>>
>
> Thank you for the patch. Please see my comments inline.
Thanks for reviewing.
>
>> Signed-off-by: Boris Brezillon
>> Signed-off-by: Helen Koike
>> ---
>> Changes in v5:
>> - change sizes and reorder fields to avoid holes in the struct and make
>> it the same for 32 and 64 bits
>> - removed __attribute__ ((packed)) from uapi structs
>> - Fix doc warning from make htmldocs
>> - Updated commit message with EXT_PIX prefix for the ioctls.
>>
>> Changes in v4:
>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
>> - Add reserved fields
>> - Removed num_planes from struct v4l2_ext_pix_format
>> - Removed flag field from struct v4l2_ext_pix_format, since the only
>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1,
>> where we can use modifiers, or add it back later through the reserved
>> bits.
>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR &&
>> != MOD_INVALID
>> - Fix type assignment in v4l_g_fmt_ext_pix()
>> - Rebased on top of media/master (post 5.8-rc1)
>>
>> Changes in v3:
>> - Rebased on top of media/master (post 5.4-rc1)
>>
>> Changes in v2:
>> - Move the modifier in v4l2_ext_format (was formerly placed in
>> v4l2_ext_plane)
>> - Fix a few bugs in the converters and add a strict parameter to
>> allow conversion of uninitialized/mis-initialized objects
>> ---
>> drivers/media/v4l2-core/v4l2-dev.c | 21 +-
>> drivers/media/v4l2-core/v4l2-ioctl.c | 585 +++
>> include/media/v4l2-ioctl.h | 34 ++
>> include/uapi/linux/videodev2.h | 56 +++
>> 4 files changed, 615 insertions(+), 81 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>> b/drivers/media/v4l2-core/v4l2-dev.c
>> index a593ea0598b55..e1829906bc086 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -607,25 +607,37 @@ static void determine_valid_ioctls(struct video_device
>> *vdev)
>> set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls);
>> if ((is_rx && (ops->vidioc_g_fmt_vid_cap ||
>> ops->vidioc_g_fmt_vid_cap_mplane ||
>> + ops->vidioc_g_ext_pix_fmt_vid_cap ||
>> ops->vidioc_g_fmt_vid_overlay)) ||
>> (is_tx && (ops->vidioc_g_fmt_vid_out ||
>> ops->vidioc_g_fmt_vid_out_mplane ||
>> - ops->vidioc_g_fmt_vid_out_overlay)))
>> + ops->vidioc_g_ext_pix_fmt_vid_out ||
>> + ops->vidioc_g_fmt_vid_out_overlay))) {
>> set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls);
>> + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls);
>
> Is it expected to allow the new ioctls for drivers which implement the old
> vid_out_overlay callbacks?
Thanks for noticing it, I don't think so, I'll update this in next version.
>
>> +}
>> if ((is_rx && (ops->vidioc_s_fmt_vid_cap ||
>> ops->vidioc_s_fmt_vid_cap_mplane ||
>> + ops->vidioc_s_ext_pix_fmt_vid_cap ||
>>
[PATCH] media: staging: rkisp1: cap: fix timeout when stopping the stream
The dma engine should be stopped first.
The driver waits for an interrupt to stop the stream in a known state
after a frame.
If rkisp1_cap_stream_disable() is called after stopping the rest of the
pipeline, then most likely the interrupt won't arrive, we'll get a
timeout and debugfs variables mp_stop_timeout or sp_stop_timeout will
be incremented.
Fixes: 37db540bb9d1f ("media: staging: rkisp1: cap: refactor enable/disable
stream to allow multistreaming")
Signed-off-by: Helen Koike
---
drivers/staging/media/rkisp1/rkisp1-capture.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/rkisp1/rkisp1-capture.c
b/drivers/staging/media/rkisp1/rkisp1-capture.c
index 6d2fd27970f28..b81235afd0534 100644
--- a/drivers/staging/media/rkisp1/rkisp1-capture.c
+++ b/drivers/staging/media/rkisp1/rkisp1-capture.c
@@ -895,6 +895,8 @@ static void rkisp1_pipeline_stream_disable(struct
rkisp1_capture *cap)
{
struct rkisp1_device *rkisp1 = cap->rkisp1;
+ rkisp1_cap_stream_disable(cap);
+
/*
* If the other capture is streaming, isp and sensor nodes shouldn't
* be disabled, skip them.
@@ -907,8 +909,6 @@ static void rkisp1_pipeline_stream_disable(struct
rkisp1_capture *cap)
v4l2_subdev_call(&rkisp1->resizer_devs[cap->id].sd, video, s_stream,
false);
-
- rkisp1_cap_stream_disable(cap);
}
/*
--
2.29.2
[PATCH v3 0/2] destage Rockchip ISP1 driver
Hello,
Changes in v3:
- Moved Kconfig entry from M2M to Platform devices
- Rename description and comment to Parameters and Statistics.
- Patches squashed:
dt-bindings: media: rkisp1: move rockchip-isp1 bindings out of staging
media: MAINTAINERS: rkisp1: add path to dt-bindings
media: rockchip: rkisp1: destage Rockchip ISP1 driver
media: MAINTAINERS: Update rkisp1 files with new location
Changes in v2:
- New patch updating MAINTAINERS file
- No changes in other patches
> media-ctl -p
http://ix.io/2Cso
> media-ctl --print-dot
http://ix.io/2Csp
> v4l2-compliance -m0
http://ix.io/2Csk
> v4l2-compliance -v -d /dev/video0 -s10
http://ix.io/2Csq
> v4l2-compliance -v -d /dev/video1 -s10
http://ix.io/2Css
Helen Koike (1):
media: rockchip: rkisp1: destage Rockchip ISP1 driver
Shunqian Zheng (1):
media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format
.../bindings/media/rockchip-isp1.yaml | 0
.../media/v4l/pixfmt-meta-rkisp1.rst | 2 +-
MAINTAINERS | 5 -
drivers/media/platform/Kconfig| 18 ++
drivers/media/platform/Makefile | 1 +
.../platform/rockchip}/rkisp1/Makefile| 0
.../rockchip}/rkisp1/rkisp1-capture.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.h | 2 +-
.../platform/rockchip}/rkisp1/rkisp1-dev.c| 0
.../platform/rockchip}/rkisp1/rkisp1-isp.c| 0
.../platform/rockchip}/rkisp1/rkisp1-params.c | 0
.../platform/rockchip}/rkisp1/rkisp1-regs.h | 0
.../rockchip}/rkisp1/rkisp1-resizer.c | 0
.../platform/rockchip}/rkisp1/rkisp1-stats.c | 0
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
drivers/staging/media/Kconfig | 2 --
drivers/staging/media/Makefile| 1 -
drivers/staging/media/rkisp1/Kconfig | 19 ---
drivers/staging/media/rkisp1/TODO | 6 --
.../uapi/linux}/rkisp1-config.h | 4
include/uapi/linux/videodev2.h| 4
22 files changed, 31 insertions(+), 35 deletions(-)
rename {drivers/staging/media/rkisp1/Documentation =>
Documentation}/devicetree/bindings/media/rockchip-isp1.yaml (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/Makefile
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-capture.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.h (99%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-dev.c
(100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-isp.c
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-params.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-regs.h
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-resizer.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-stats.c (100%)
delete mode 100644 drivers/staging/media/rkisp1/Kconfig
delete mode 100644 drivers/staging/media/rkisp1/TODO
rename {drivers/staging/media/rkisp1/uapi =>
include/uapi/linux}/rkisp1-config.h (99%)
--
2.29.2
[PATCH v3 1/2] media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format
From: Shunqian Zheng
Add the Rockchip ISP1 specific processing parameter format
V4L2_META_FMT_RK_ISP1_PARAMS and metadata format
V4L2_META_FMT_RK_ISP1_STAT_3A for 3A.
Signed-off-by: Shunqian Zheng
Signed-off-by: Jacob Chen
Signed-off-by: Helen Koike
Reviewed-by: Laurent Pinchart
---
Hello,
This patch is a continuation of:
https://patchwork.kernel.org/project/linux-arm-kernel/patch/20191106120132.6876-2-helen.ko...@collabora.com/
These formats are already documented under
Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
Changes in V3:
- Rename description and comment to Parameters and Statistics.
Regards,
Helen
---
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
drivers/staging/media/rkisp1/uapi/rkisp1-config.h | 4
include/uapi/linux/videodev2.h| 4
3 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index f0f6906a879de..3198abdd538ce 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1403,6 +1403,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_UVC: descr = "UVC Payload Header Metadata";
break;
case V4L2_META_FMT_D4XX:descr = "Intel D4xx UVC Metadata";
break;
case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
+ case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A
Parameters"; break;
+ case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A
Statistics"; break;
default:
/* Compressed formats */
diff --git a/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
b/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
index 8d906cc7da8fc..6e449e7842605 100644
--- a/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
+++ b/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
@@ -9,10 +9,6 @@
#include
-/* Vendor specific - used for RK_ISP1 camera sub-system */
-#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /*
Rockchip ISP1 params */
-#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /*
Rockchip ISP1 3A statistics */
-
/* Defect Pixel Cluster Detection */
#define RKISP1_CIF_ISP_MODULE_DPCC (1U << 0)
/* Black Level Subtraction */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 927075fa9099e..761ac9da3ffda 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -768,6 +768,10 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_D4XXv4l2_fourcc('D', '4', 'X', 'X') /* D4XX
Payload Header metadata */
#define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid
Metadata */
+/* Vendor specific - used for RK_ISP1 camera sub-system */
+#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /*
Rockchip ISP1 3A Parameters */
+#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /*
Rockchip ISP1 3A Statistics */
+
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC0xfeedcafe
--
2.29.2
[PATCH v3 2/2] media: rockchip: rkisp1: destage Rockchip ISP1 driver
All the items in the TODO list were addressed, uapi was reviewed,
documentation written, checkpatch errors fixed, several bugs fixed.
There is no big reason to keep this driver in staging, so move it out.
Dt-bindings Verified with:
make ARCH=arm64 dt_binding_check
DT_SCHEMA_FILES=Documentation/devicetree/bindings/media/rockchip-isp1.yaml
Fields of MAINTAINERS file sorted according to output of
./scripts/parse-maintainers.pl --input=MAINTAINERS --output=MAINTAINERS
--order
Signed-off-by: Helen Koike
[dt-bindings: media: rkisp1: move rockchip-isp1 bindings out of staging]
Acked-by: Rob Herring
[dt-bindings: media: rkisp1: move rockchip-isp1 bindings out of staging]
Reviewed-by: Tomasz Figa
---
Changes in v3:
- Moved Kconfig entry from M2M to Platform devices
- Patches squashed:
dt-bindings: media: rkisp1: move rockchip-isp1 bindings out of staging
media: MAINTAINERS: rkisp1: add path to dt-bindings
media: rockchip: rkisp1: destage Rockchip ISP1 driver
media: MAINTAINERS: Update rkisp1 files with new location
---
.../bindings/media/rockchip-isp1.yaml | 0
.../media/v4l/pixfmt-meta-rkisp1.rst | 2 +-
MAINTAINERS | 5 -
drivers/media/platform/Kconfig| 18 ++
drivers/media/platform/Makefile | 1 +
.../platform/rockchip}/rkisp1/Makefile| 0
.../rockchip}/rkisp1/rkisp1-capture.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.h | 2 +-
.../platform/rockchip}/rkisp1/rkisp1-dev.c| 0
.../platform/rockchip}/rkisp1/rkisp1-isp.c| 0
.../platform/rockchip}/rkisp1/rkisp1-params.c | 0
.../platform/rockchip}/rkisp1/rkisp1-regs.h | 0
.../rockchip}/rkisp1/rkisp1-resizer.c | 0
.../platform/rockchip}/rkisp1/rkisp1-stats.c | 0
drivers/staging/media/Kconfig | 2 --
drivers/staging/media/Makefile| 1 -
drivers/staging/media/rkisp1/Kconfig | 19 ---
drivers/staging/media/rkisp1/TODO | 6 --
.../uapi/linux}/rkisp1-config.h | 0
20 files changed, 25 insertions(+), 31 deletions(-)
rename {drivers/staging/media/rkisp1/Documentation =>
Documentation}/devicetree/bindings/media/rockchip-isp1.yaml (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/Makefile
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-capture.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.h (99%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-dev.c
(100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-isp.c
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-params.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-regs.h
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-resizer.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-stats.c (100%)
delete mode 100644 drivers/staging/media/rkisp1/Kconfig
delete mode 100644 drivers/staging/media/rkisp1/TODO
rename {drivers/staging/media/rkisp1/uapi =>
include/uapi/linux}/rkisp1-config.h (100%)
diff --git
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
b/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
similarity index 100%
rename from
drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
rename to Documentation/devicetree/bindings/media/rockchip-isp1.yaml
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
b/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
index 7e43837ed260a..f3671472d4105 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
@@ -46,4 +46,4 @@ important tuning tools using software control loop.
rkisp1 uAPI data types
==
-.. kernel-doc:: drivers/staging/media/rkisp1/uapi/rkisp1-config.h
+.. kernel-doc:: include/uapi/linux/rkisp1-config.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 811db1d3ca33b..352b8eaa21f7c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15034,10 +15034,13 @@ ROCKCHIP ISP V1 DRIVER
M: Helen Koike
M: Dafna Hirschfeld
L: linux-me...@vger.kernel.org
+L: linux-rockc...@lists.infradead.org
S: Maintained
F: Documentation/admin-guide/media/rkisp1.rst
+F: Documentation/devicetree/bindings/media/rockchip-isp1.yaml
F: Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
-F: drivers/staging/media/rkisp1/
+F: drivers/media/platform/rockchip/rkisp1
+F: include/uapi/linux/rkisp1-config.h
ROCKCHIP RASTER 2D
Re: [PATCH 08/14] media: sunxi: Add support for the A31 MIPI CSI-2 controller
On 11/4/20 3:45 PM, Maxime Ripard wrote: > On Wed, Nov 04, 2020 at 01:38:08PM -0300, Helen Koike wrote: >> >> >> On 11/4/20 8:17 AM, Paul Kocialkowski wrote: >>> Hi, >>> >>> On Mon 02 Nov 20, 10:21, Maxime Ripard wrote: >>>> On Fri, Oct 30, 2020 at 07:45:18PM -0300, Helen Koike wrote: >>>>> On 10/23/20 2:45 PM, Paul Kocialkowski wrote: >>>>>> The A31 MIPI CSI-2 controller is a dedicated MIPI CSI-2 controller >>>>>> found on Allwinner SoCs such as the A31 and V3/V3s. >>>>>> >>>>>> It is a standalone block, connected to the CSI controller on one side >>>>>> and to the MIPI D-PHY block on the other. It has a dedicated address >>>>>> space, interrupt line and clock. >>>>>> >>>>>> Currently, the MIPI CSI-2 controller is hard-tied to a specific CSI >>>>>> controller (CSI0) but newer SoCs (such as the V5) may allow switching >>>>>> MIPI CSI-2 controllers between CSI controllers. >>>>>> >>>>>> It is represented as a V4L2 subdev to the CSI controller and takes a >>>>>> MIPI CSI-2 sensor as its own subdev, all using the fwnode graph and >>>>>> media controller API. >>>>> >>>>> Maybe this is a bad idea, but I was thinking: >>>>> This driver basically just turn on/off and catch some interrupts for >>>>> errors, >>>>> and all the rest of v4l2 config you just forward to the next subdevice >>>>> on the pipeline. >>>>> >>>>> So instead of exposing it as a subdevice, I was wondering if modeling >>>>> this driver also through the phy subsystem wouldn't be cleaner, so >>>>> you won't need all the v4l2 subdevice/topology boilerplate code that >>>>> it seems you are not using (unless you have plans to add controls or >>>>> some specific configuration on this node later). >>>>> >>>>> But this would require changes on the sun6i-csi driver. >>>>> >>>>> What do you think? >>>> >>>> Eventually we'll need to filter the virtual channels / datatypes I >>>> guess, so it's definitely valuable to have it in v4l2 >> >> Which kind of datatypes? > > MIPI-CSI datatypes. Each packet on the MIPI-CSI bus is assigned a > virtual channel and data type so that you can multiplex multiple streams > (like a 3d camera would send for example, through the virtual channels) > and data types (like frames and metadata) and MIPI-CSI controllers > usually allow to filter them based on what you want. > >> I ask to know if this shouldn't be configured through the video node >> instead of subdevice. > > Not really, some setups have a mux that can split the multiple virtual > channels to multiple video nodes for example. > >> Regarding channels, we had a discussion to implement it through the video >> node (and not subdevice) [1]. But we discussed about blitters and >> multi-scalers, >> so now I'm wondering if we could use the same API for mipi-csi virtual >> channels >> in the video entity device, or if it doesn't apply and we need another API >> for that in a subdevice instead. >> >> [1] >> https://patchwork.linuxtv.org/project/linux-media/cover/20200717115435.2632623-1-helen.ko...@collabora.com/ > > There's already an API to deal with MIPI-CSI virtual channels: > https://patchwork.kernel.org/project/linux-renesas-soc/cover/20190328200608.9463-1-jacopo+rene...@jmondi.org/ > > Maxime > Thanks for the explanation :) Helen
Re: [PATCH 08/14] media: sunxi: Add support for the A31 MIPI CSI-2 controller
On 11/4/20 8:17 AM, Paul Kocialkowski wrote: > Hi, > > On Mon 02 Nov 20, 10:21, Maxime Ripard wrote: >> On Fri, Oct 30, 2020 at 07:45:18PM -0300, Helen Koike wrote: >>> On 10/23/20 2:45 PM, Paul Kocialkowski wrote: >>>> The A31 MIPI CSI-2 controller is a dedicated MIPI CSI-2 controller >>>> found on Allwinner SoCs such as the A31 and V3/V3s. >>>> >>>> It is a standalone block, connected to the CSI controller on one side >>>> and to the MIPI D-PHY block on the other. It has a dedicated address >>>> space, interrupt line and clock. >>>> >>>> Currently, the MIPI CSI-2 controller is hard-tied to a specific CSI >>>> controller (CSI0) but newer SoCs (such as the V5) may allow switching >>>> MIPI CSI-2 controllers between CSI controllers. >>>> >>>> It is represented as a V4L2 subdev to the CSI controller and takes a >>>> MIPI CSI-2 sensor as its own subdev, all using the fwnode graph and >>>> media controller API. >>> >>> Maybe this is a bad idea, but I was thinking: >>> This driver basically just turn on/off and catch some interrupts for errors, >>> and all the rest of v4l2 config you just forward to the next subdevice >>> on the pipeline. >>> >>> So instead of exposing it as a subdevice, I was wondering if modeling >>> this driver also through the phy subsystem wouldn't be cleaner, so >>> you won't need all the v4l2 subdevice/topology boilerplate code that >>> it seems you are not using (unless you have plans to add controls or >>> some specific configuration on this node later). >>> >>> But this would require changes on the sun6i-csi driver. >>> >>> What do you think? >> >> Eventually we'll need to filter the virtual channels / datatypes I >> guess, so it's definitely valuable to have it in v4l2 Which kind of datatypes? I ask to know if this shouldn't be configured through the video node instead of subdevice. Regarding channels, we had a discussion to implement it through the video node (and not subdevice) [1]. But we discussed about blitters and multi-scalers, so now I'm wondering if we could use the same API for mipi-csi virtual channels in the video entity device, or if it doesn't apply and we need another API for that in a subdevice instead. [1] https://patchwork.linuxtv.org/project/linux-media/cover/20200717115435.2632623-1-helen.ko...@collabora.com/ > > Agreed and like I mentionned in the discussion on 00/14 I don't think it > would be a cleaner way to expose things. > > There's also the fact that newer SoCs like the V5 seem to allow connecting > any MIPI CSI-2 controller to any CSI controller, so the graph representation > is definitely welcome here. I'm not sure this is an advantage in userspace pov, because it means we'll have different topologies for basically the same end result to userspace. But as I mentioned, I don't mind keeping it in the media topology. Helen > > Paul >
Re: [PATCH 00/14] Allwinner MIPI CSI-2 support for A31/V3s/A83T
Hi Paul, On 11/4/20 8:11 AM, Paul Kocialkowski wrote: > Hi Helen, > > On Fri 30 Oct 20, 19:44, Helen Koike wrote: >> Hi Paul, >> >> I have some comments through the series, I hope this helps. > > Thanks for your comments :) > >> On 10/23/20 2:45 PM, Paul Kocialkowski wrote: >>> This series introduces support for MIPI CSI-2, with the A31 controller that >>> is >>> found on most SoCs (A31, V3s and probably V5) as well as the A83T-specific >>> controller. While the former uses the same MIPI D-PHY that is already >>> supported >>> for DSI, the latter embeds its own D-PHY. >>> >>> In order to distinguish the use of the D-PHY between Rx mode (for MIPI >>> CSI-2) >>> and Tx mode (for MIPI DSI), a submode is introduced for D-PHY in the PHY >>> API. >>> This allows adding Rx support in the A31 D-PHY driver. >>> >>> A few changes and fixes are applied to the A31 CSI controller driver, in >>> order >>> to support the MIPI CSI-2 use-case. >>> >>> Follows is the V4L2 device topology representing the interactions between >>> the MIPI CSI-2 sensor, the MIPI CSI-2 controller (which controls the D-PHY) >>> and the CSI controller: >>> - entity 1: sun6i-csi (1 pad, 1 link) >>> type Node subtype V4L flags 0 >>> device node name /dev/video0 >>> pad0: Sink >>> <- "sun6i-mipi-csi2":1 [ENABLED,IMMUTABLE] >>> >>> - entity 5: sun6i-mipi-csi2 (2 pads, 2 links) >>> type V4L2 subdev subtype Unknown flags 0 >>> pad0: Sink >>> <- "ov5648 0-0036":0 [ENABLED,IMMUTABLE] >>> pad1: Source >>> -> "sun6i-csi":0 [ENABLED,IMMUTABLE] >>> >>> - entity 8: ov5648 0-0036 (1 pad, 1 link) >>> type V4L2 subdev subtype Sensor flags 0 >>> device node name /dev/v4l-subdev0 >> >> Question: I noticed is that sun6i-mipi-csi2 doesn't expose a node under >> /dev/, but the sensor >> exposes it. Probably because it uses V4L2_SUBDEV_FL_HAS_DEVNODE and >> sun6i-csi() calls >> v4l2_device_register_subdev_nodes(). >> >> I find this weird from a userspace pov, since usually we don't mix manual >> and auto propagation >> of the configs, so I started wondering if sun6i-csi driver should be calling >> v4l2_device_register_subdev_nodes() in the first place. > > I must admit that I didn't really pay attention to that, but since > sun6i-mipi-csi2 is basically a bridge driver, it doesn't make sense to apply > manual configuration to it. It is actually designed to forward most subdev ops > to its own subdev so configuring it manually would actually result in > configuring the sensor. Ack, then maybe sun6i-csi needs a patch removing the call to v4l2_device_register_subdev_nodes() > > XXX > >> Also, sun6i-csi doesn't seem to be used by any board dts (it's declared on >> the dtsi, but I >> didn't find any dts enabling it), so I wonder if it would be a bad thing if >> we update it. >> >>> pad0: Source >>> [fmt:SBGGR8_1X8/640x480@1/30 field:none colorspace:raw >>> xfer:none ycbcr:601 quantization:full-range] >>> -> "sun6i-mipi-csi2":0 [ENABLED,IMMUTABLE] >> >> If I understand correctly, this is very similar to ipu3: >> sensor->bus->dma_engine >> >> in the case of ipu3-cio2: >> sensor->ipu3-csi2->ipu3-cio2 >> >> in this case: >> ov5648->sun6i-mipi-csi2->sun6i-csi > > Yes this is the correct picture. > >> On thing that is confusing me is the name csi2 with csi (that makes me think >> of csi >> version one, which is not the case), I would rename it to sun6i-video (or >> maybe >> it is just me who gets confused). > > So the CSI name comes from the Allwinner litterature and implementation for > that > controller. Since it supports parallel input on its own, it does in fact > support > parallel CSI. The DMA engine part alone from that controller is also used for > MIPI CSI-2, so in this case the name looses its relevance. > >> I know this driver is already upstream and not part of this series, but on >> the other hand it >> doesn't seem to be used. > > Personally I don't find a rename to be necessary and while I agree that > nothing would apparently prevent us from renaming it, I would prefer to keep > the naming in line with Allwinner&#x
Re: [PATCH 02/14] phy: allwinner: phy-sun6i-mipi-dphy: Support D-PHY Rx mode for MIPI CSI-2
Hello,
On 10/23/20 2:45 PM, Paul Kocialkowski wrote:
> The Allwinner A31 D-PHY supports both Rx and Tx modes. While the latter
> is already supported and used for MIPI DSI this adds support for the
> former, to be used with MIPI CSI-2.
>
> This implementation is inspired by the Allwinner BSP implementation.
>
> Signed-off-by: Paul Kocialkowski
> ---
> drivers/phy/allwinner/phy-sun6i-mipi-dphy.c | 164 +++-
> 1 file changed, 160 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/phy/allwinner/phy-sun6i-mipi-dphy.c
> b/drivers/phy/allwinner/phy-sun6i-mipi-dphy.c
> index 1fa761ba6cbb..8bcd4bb79f60 100644
> --- a/drivers/phy/allwinner/phy-sun6i-mipi-dphy.c
> +++ b/drivers/phy/allwinner/phy-sun6i-mipi-dphy.c
> @@ -24,6 +24,14 @@
> #define SUN6I_DPHY_TX_CTL_REG0x04
> #define SUN6I_DPHY_TX_CTL_HS_TX_CLK_CONT BIT(28)
>
> +#define SUN6I_DPHY_RX_CTL_REG0x08
> +#define SUN6I_DPHY_RX_CTL_EN_DBC BIT(31)
> +#define SUN6I_DPHY_RX_CTL_RX_CLK_FORCE BIT(24)
> +#define SUN6I_DPHY_RX_CTL_RX_D3_FORCEBIT(23)
> +#define SUN6I_DPHY_RX_CTL_RX_D2_FORCEBIT(22)
> +#define SUN6I_DPHY_RX_CTL_RX_D1_FORCEBIT(21)
> +#define SUN6I_DPHY_RX_CTL_RX_D0_FORCEBIT(20)
> +
> #define SUN6I_DPHY_TX_TIME0_REG 0x10
> #define SUN6I_DPHY_TX_TIME0_HS_TRAIL(n) (((n) & 0xff) << 24)
> #define SUN6I_DPHY_TX_TIME0_HS_PREPARE(n)(((n) & 0xff) << 16)
> @@ -44,12 +52,29 @@
> #define SUN6I_DPHY_TX_TIME4_HS_TX_ANA1(n)(((n) & 0xff) << 8)
> #define SUN6I_DPHY_TX_TIME4_HS_TX_ANA0(n)((n) & 0xff)
>
> +#define SUN6I_DPHY_RX_TIME0_REG 0x30
> +#define SUN6I_DPHY_RX_TIME0_HS_RX_SYNC(n)(((n) & 0xff) << 24)
> +#define SUN6I_DPHY_RX_TIME0_HS_RX_CLK_MISS(n)(((n) & 0xff) << 16)
> +#define SUN6I_DPHY_RX_TIME0_LP_RX(n) (((n) & 0xff) << 8)
> +
> +#define SUN6I_DPHY_RX_TIME1_REG 0x34
> +#define SUN6I_DPHY_RX_TIME1_RX_DLY(n)(((n) & 0xfff) << 20)
> +#define SUN6I_DPHY_RX_TIME1_LP_RX_ULPS_WP(n) ((n) & 0xf)
> +
> +#define SUN6I_DPHY_RX_TIME2_REG 0x38
> +#define SUN6I_DPHY_RX_TIME2_HS_RX_ANA1(n)(((n) & 0xff) << 8)
> +#define SUN6I_DPHY_RX_TIME2_HS_RX_ANA0(n)((n) & 0xff)
> +
> +#define SUN6I_DPHY_RX_TIME3_REG 0x40
> +#define SUN6I_DPHY_RX_TIME3_LPRST_DLY(n) (((n) & 0x) << 16)
> +
> #define SUN6I_DPHY_ANA0_REG 0x4c
> #define SUN6I_DPHY_ANA0_REG_PWS BIT(31)
> #define SUN6I_DPHY_ANA0_REG_DMPC BIT(28)
> #define SUN6I_DPHY_ANA0_REG_DMPD(n) (((n) & 0xf) << 24)
> #define SUN6I_DPHY_ANA0_REG_SLV(n) (((n) & 7) << 12)
> #define SUN6I_DPHY_ANA0_REG_DEN(n) (((n) & 0xf) << 8)
> +#define SUN6I_DPHY_ANA0_REG_SFB(n) (((n) & 3) << 2)
>
> #define SUN6I_DPHY_ANA1_REG 0x50
> #define SUN6I_DPHY_ANA1_REG_VTTMODE BIT(31)
> @@ -92,6 +117,8 @@ struct sun6i_dphy {
>
> struct phy *phy;
> struct phy_configure_opts_mipi_dphy config;
> +
> + int submode;
> };
>
> static int sun6i_dphy_init(struct phy *phy)
> @@ -105,6 +132,18 @@ static int sun6i_dphy_init(struct phy *phy)
> return 0;
> }
>
> +static int sun6i_dphy_set_mode(struct phy *phy, enum phy_mode mode, int
> submode)
> +{
> + struct sun6i_dphy *dphy = phy_get_drvdata(phy);
> +
> + if (mode != PHY_MODE_MIPI_DPHY)
> + return -EINVAL;
> +
> + dphy->submode = submode;
Shouldn't you check if the submode is valid here?
> +
> + return 0;
> +}
> +
> static int sun6i_dphy_configure(struct phy *phy, union phy_configure_opts
> *opts)
> {
> struct sun6i_dphy *dphy = phy_get_drvdata(phy);
> @@ -119,9 +158,8 @@ static int sun6i_dphy_configure(struct phy *phy, union
> phy_configure_opts *opts)
> return 0;
> }
>
> -static int sun6i_dphy_power_on(struct phy *phy)
> +static int sun6i_dphy_tx_power_on(struct sun6i_dphy *dphy)
> {
> - struct sun6i_dphy *dphy = phy_get_drvdata(phy);
> u8 lanes_mask = GENMASK(dphy->config.lanes - 1, 0);
>
> regmap_write(dphy->regs, SUN6I_DPHY_TX_CTL_REG,
> @@ -211,12 +249,129 @@ static int sun6i_dphy_power_on(struct phy *phy)
> return 0;
> }
>
> +static int sun6i_dphy_rx_power_on(struct sun6i_dphy *dphy)
> +{
> + /* Physical clock rate is actually half of symbol rate with DDR. */
> + unsigned long mipi_symbol_rate = dphy->config.hs_clk_rate;
> + unsigned long dphy_clk_rate;
> + unsigned int rx_dly;
> + unsigned int lprst_dly;
> + u32 value;
> +
> + dphy_clk_rate = clk_get_rate(dphy->mod_clk);
> + if (!dphy_clk_rate)
> + return -1;
> +
> + /* Hardcoded timing parameters from the Allwinner BSP. */
> + regmap_write(dphy->regs, SUN6I_DPHY_RX_TIME0_REG,
> + SUN6I_DPHY_RX_TIME0_HS_RX_SYNC(255) |
> + SUN6I_DPHY_RX_TIME0_HS_RX_CLK_MIS
Re: [PATCH 04/14] media: sun6i-csi: Fix the image storage bpp for 10/12-bit Bayer formats
Hi Paul,
On 10/23/20 2:45 PM, Paul Kocialkowski wrote:
> Both 10 and 12-bit Bayer formats are stored aligned as 16-bit values
> in memory, not unaligned 10 or 12 bits.
>
> Since the current code for retreiving the bpp is used only to
> calculate the memory storage size of the picture (which is what
> pixel formats describe, unlike media bus formats), fix it there.
>
> Fixes: 5cc7522d8965 ("media: sun6i: Add support for Allwinner CSI V3s")
> Co-developed-by: Kévin L'hôpital
> Signed-off-by: Kévin L'hôpital
> Signed-off-by: Paul Kocialkowski
> ---
> .../platform/sunxi/sun6i-csi/sun6i_csi.h | 20 +--
> 1 file changed, 10 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
> b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
> index c626821aaedb..7f2be70ae641 100644
> --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
> +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
> @@ -86,7 +86,7 @@ void sun6i_csi_update_buf_addr(struct sun6i_csi *csi,
> dma_addr_t addr);
> */
> void sun6i_csi_set_stream(struct sun6i_csi *csi, bool enable);
>
> -/* get bpp form v4l2 pixformat */
> +/* get memory storage bpp from v4l2 pixformat */
> static inline int sun6i_csi_get_bpp(unsigned int pixformat)
> {
> switch (pixformat) {
> @@ -96,15 +96,6 @@ static inline int sun6i_csi_get_bpp(unsigned int pixformat)
> case V4L2_PIX_FMT_SRGGB8:
> case V4L2_PIX_FMT_JPEG:
> return 8;
> - case V4L2_PIX_FMT_SBGGR10:
> - case V4L2_PIX_FMT_SGBRG10:
> - case V4L2_PIX_FMT_SGRBG10:
> - case V4L2_PIX_FMT_SRGGB10:
> - return 10;
> - case V4L2_PIX_FMT_SBGGR12:
> - case V4L2_PIX_FMT_SGBRG12:
> - case V4L2_PIX_FMT_SGRBG12:
> - case V4L2_PIX_FMT_SRGGB12:
> case V4L2_PIX_FMT_HM12:
> case V4L2_PIX_FMT_NV12:
> case V4L2_PIX_FMT_NV21:
> @@ -121,6 +112,15 @@ static inline int sun6i_csi_get_bpp(unsigned int
> pixformat)
> case V4L2_PIX_FMT_RGB565:
> case V4L2_PIX_FMT_RGB565X:
> return 16;
> + case V4L2_PIX_FMT_SBGGR10:
> + case V4L2_PIX_FMT_SGBRG10:
> + case V4L2_PIX_FMT_SGRBG10:
> + case V4L2_PIX_FMT_SRGGB10:
> + case V4L2_PIX_FMT_SBGGR12:
> + case V4L2_PIX_FMT_SGBRG12:
> + case V4L2_PIX_FMT_SGRBG12:
> + case V4L2_PIX_FMT_SRGGB12:
> + return 16;
> case V4L2_PIX_FMT_RGB24:
> case V4L2_PIX_FMT_BGR24:
> return 24;
>
Instead of updating this table, how about using v4l2_format_info() instead?
Regards,
Helen
Re: [PATCH 00/14] Allwinner MIPI CSI-2 support for A31/V3s/A83T
Hi Paul, I have some comments through the series, I hope this helps. On 10/23/20 2:45 PM, Paul Kocialkowski wrote: > This series introduces support for MIPI CSI-2, with the A31 controller that is > found on most SoCs (A31, V3s and probably V5) as well as the A83T-specific > controller. While the former uses the same MIPI D-PHY that is already > supported > for DSI, the latter embeds its own D-PHY. > > In order to distinguish the use of the D-PHY between Rx mode (for MIPI CSI-2) > and Tx mode (for MIPI DSI), a submode is introduced for D-PHY in the PHY API. > This allows adding Rx support in the A31 D-PHY driver. > > A few changes and fixes are applied to the A31 CSI controller driver, in order > to support the MIPI CSI-2 use-case. > > Follows is the V4L2 device topology representing the interactions between > the MIPI CSI-2 sensor, the MIPI CSI-2 controller (which controls the D-PHY) > and the CSI controller: > - entity 1: sun6i-csi (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video0 > pad0: Sink > <- "sun6i-mipi-csi2":1 [ENABLED,IMMUTABLE] > > - entity 5: sun6i-mipi-csi2 (2 pads, 2 links) > type V4L2 subdev subtype Unknown flags 0 > pad0: Sink > <- "ov5648 0-0036":0 [ENABLED,IMMUTABLE] > pad1: Source > -> "sun6i-csi":0 [ENABLED,IMMUTABLE] > > - entity 8: ov5648 0-0036 (1 pad, 1 link) > type V4L2 subdev subtype Sensor flags 0 > device node name /dev/v4l-subdev0 Question: I noticed is that sun6i-mipi-csi2 doesn't expose a node under /dev/, but the sensor exposes it. Probably because it uses V4L2_SUBDEV_FL_HAS_DEVNODE and sun6i-csi() calls v4l2_device_register_subdev_nodes(). I find this weird from a userspace pov, since usually we don't mix manual and auto propagation of the configs, so I started wondering if sun6i-csi driver should be calling v4l2_device_register_subdev_nodes() in the first place. Also, sun6i-csi doesn't seem to be used by any board dts (it's declared on the dtsi, but I didn't find any dts enabling it), so I wonder if it would be a bad thing if we update it. > pad0: Source > [fmt:SBGGR8_1X8/640x480@1/30 field:none colorspace:raw > xfer:none ycbcr:601 quantization:full-range] > -> "sun6i-mipi-csi2":0 [ENABLED,IMMUTABLE] If I understand correctly, this is very similar to ipu3: sensor->bus->dma_engine in the case of ipu3-cio2: sensor->ipu3-csi2->ipu3-cio2 in this case: ov5648->sun6i-mipi-csi2->sun6i-csi On thing that is confusing me is the name csi2 with csi (that makes me think of csi vesun6i-csirsion one, which is not the case), I would rename it to sun6i-video (or maybe it is just me who gets confused). I know this driver is already upstream and not part of this series, but on the other hand it doesn't seem to be used. On another note, I always wonder if we should expose the bus in the topology, I'm not sure if it provides any useful API or information for userspace, and you could have a cleaner code (maybe code could be under phy subsystem). But at the same time, it seems this is a pattern on v4l2. I'd like to hear what others think on the above. Regards, Helen > > Happy reviewing! > > Paul Kocialkowski (14): > phy: Distinguish between Rx and Tx for MIPI D-PHY with submodes > phy: allwinner: phy-sun6i-mipi-dphy: Support D-PHY Rx mode for MIPI > CSI-2 > media: sun6i-csi: Support an optional dedicated memory pool > media: sun6i-csi: Fix the image storage bpp for 10/12-bit Bayer > formats > media: sun6i-csi: Only configure the interface data width for parallel > media: sun6i-csi: Support feeding from the MIPI CSI-2 controller > dt-bindings: media: i2c: Add A31 MIPI CSI-2 bindings documentation > media: sunxi: Add support for the A31 MIPI CSI-2 controller > ARM: dts: sun8i: v3s: Add CSI0 camera interface node > ARM: dts: sun8i: v3s: Add MIPI D-PHY and MIPI CSI-2 interface nodes > dt-bindings: media: i2c: Add A83T MIPI CSI-2 bindings documentation > media: sunxi: Add support for the A83T MIPI CSI-2 controller > ARM: dts: sun8i: a83t: Add MIPI CSI-2 controller node > media: sunxi: sun8i-a83t-mipi-csi2: Avoid using the (unsolicited) > interrupt > > .../media/allwinner,sun6i-a31-mipi-csi2.yaml | 168 + > .../media/allwinner,sun8i-a83t-mipi-csi2.yaml | 158 + > arch/arm/boot/dts/sun8i-a83t.dtsi | 26 + > arch/arm/boot/dts/sun8i-v3s.dtsi | 62 ++ > drivers/media/platform/sunxi/Kconfig | 2 + > drivers/media/platform/sunxi/Makefile | 2 + > .../platform/sunxi/sun6i-csi/sun6i_csi.c | 54 +- > .../platform/sunxi/sun6i-csi/sun6i_csi.h | 20 +- > .../platform/sunxi/sun6i-mipi-csi2/Kconfig| 11 + > .../platform/sunxi/sun6i-mipi-csi2/Makefile | 4 + > .../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c | 635 + > .../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.
Re: [PATCH 08/14] media: sunxi: Add support for the A31 MIPI CSI-2 controller
Hi Paul,
On 10/23/20 2:45 PM, Paul Kocialkowski wrote:
> The A31 MIPI CSI-2 controller is a dedicated MIPI CSI-2 controller
> found on Allwinner SoCs such as the A31 and V3/V3s.
>
> It is a standalone block, connected to the CSI controller on one side
> and to the MIPI D-PHY block on the other. It has a dedicated address
> space, interrupt line and clock.
>
> Currently, the MIPI CSI-2 controller is hard-tied to a specific CSI
> controller (CSI0) but newer SoCs (such as the V5) may allow switching
> MIPI CSI-2 controllers between CSI controllers.
>
> It is represented as a V4L2 subdev to the CSI controller and takes a
> MIPI CSI-2 sensor as its own subdev, all using the fwnode graph and
> media controller API.
Maybe this is a bad idea, but I was thinking:
This driver basically just turn on/off and catch some interrupts for errors,
and all the rest of v4l2 config you just forward to the next subdevice
on the pipeline.
So instead of exposing it as a subdevice, I was wondering if modeling
this driver also through the phy subsystem wouldn't be cleaner, so
you won't need all the v4l2 subdevice/topology boilerplate code that
it seems you are not using (unless you have plans to add controls or
some specific configuration on this node later).
But this would require changes on the sun6i-csi driver.
What do you think?
Same comment for patch 12/14 (A83T MIPI CSI-2 controller).
>
> Signed-off-by: Paul Kocialkowski
> ---
> drivers/media/platform/sunxi/Kconfig | 1 +
> drivers/media/platform/sunxi/Makefile | 1 +
> .../platform/sunxi/sun6i-mipi-csi2/Kconfig| 11 +
> .../platform/sunxi/sun6i-mipi-csi2/Makefile | 4 +
> .../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c | 635 ++
> .../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h | 116
> 6 files changed, 768 insertions(+)
> create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig
> create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile
> create mode 100644
> drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
> create mode 100644
> drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h
>
> diff --git a/drivers/media/platform/sunxi/Kconfig
> b/drivers/media/platform/sunxi/Kconfig
> index 7151cc249afa..9684e07454ad 100644
> --- a/drivers/media/platform/sunxi/Kconfig
> +++ b/drivers/media/platform/sunxi/Kconfig
> @@ -2,3 +2,4 @@
>
> source "drivers/media/platform/sunxi/sun4i-csi/Kconfig"
> source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
> +source "drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig"
> diff --git a/drivers/media/platform/sunxi/Makefile
> b/drivers/media/platform/sunxi/Makefile
> index fc537c9f5ca9..887a7cae8fca 100644
> --- a/drivers/media/platform/sunxi/Makefile
> +++ b/drivers/media/platform/sunxi/Makefile
> @@ -2,5 +2,6 @@
>
> obj-y+= sun4i-csi/
> obj-y+= sun6i-csi/
> +obj-y+= sun6i-mipi-csi2/
> obj-y+= sun8i-di/
> obj-y+= sun8i-rotate/
> diff --git a/drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig
> b/drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig
> new file mode 100644
> index ..7033bda483b4
> --- /dev/null
> +++ b/drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig
> @@ -0,0 +1,11 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config VIDEO_SUN6I_MIPI_CSI2
> + tristate "Allwinner A31 MIPI CSI-2 Controller Driver"
> + depends on VIDEO_V4L2 && COMMON_CLK
> + depends on ARCH_SUNXI || COMPILE_TEST
> + select MEDIA_CONTROLLER
> + select VIDEO_V4L2_SUBDEV_API
> + select REGMAP_MMIO
> + select V4L2_FWNODE
> + help
> +Support for the Allwinner A31 MIPI CSI-2 Controller.
> diff --git a/drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile
> b/drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile
> new file mode 100644
> index ..14e4e03818b5
> --- /dev/null
> +++ b/drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile
> @@ -0,0 +1,4 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +sun6i-mipi-csi2-y += sun6i_mipi_csi2.o
> +
> +obj-$(CONFIG_VIDEO_SUN6I_MIPI_CSI2) += sun6i-mipi-csi2.o
> diff --git a/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
> b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
> new file mode 100644
> index ..ce89c35f5b86
> --- /dev/null
> +++ b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
> @@ -0,0 +1,635 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright 2020 Bootlin
> + * Author: Paul Kocialkowski
> + */
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +
> +#include "sun6i_mipi_csi2.h"
> +
> +#define MODULE_NAME "sun6i-mipi-csi2"
> +
> +/* Core */
> +
> +static irqreturn_t sun6i_mipi_csi2_isr(int irq, void *dev_id)
> +{
> + struct sun6i_mipi_csi2_dev *cdev = (str
Re: [PATCH 01/14] phy: Distinguish between Rx and Tx for MIPI D-PHY with submodes
Hi Paul,
On 10/23/20 2:45 PM, Paul Kocialkowski wrote:
> As some D-PHY controllers support both Rx and Tx mode, we need a way for
> users to explicitly request one or the other. For instance, Rx mode can
> be used along with MIPI CSI-2 while Tx mode can be used with MIPI DSI.
>
> Introduce new MIPI D-PHY PHY submodes to use with PHY_MODE_MIPI_DPHY.
> The default (zero value) is kept to Tx so only the rkisp1 driver, which
> uses D-PHY in Rx mode, needs to be adapted.
>
> Signed-off-by: Paul Kocialkowski
> ---
> drivers/staging/media/rkisp1/rkisp1-isp.c | 3 ++-
> include/linux/phy/phy-mipi-dphy.h | 13 +
> 2 files changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/staging/media/rkisp1/rkisp1-isp.c
> b/drivers/staging/media/rkisp1/rkisp1-isp.c
> index 6ec1e9816e9f..0afbce00121e 100644
> --- a/drivers/staging/media/rkisp1/rkisp1-isp.c
> +++ b/drivers/staging/media/rkisp1/rkisp1-isp.c
> @@ -902,7 +902,8 @@ static int rkisp1_mipi_csi2_start(struct rkisp1_isp *isp,
>
> phy_mipi_dphy_get_default_config(pixel_clock, isp->sink_fmt->bus_width,
>sensor->lanes, cfg);
> - phy_set_mode(sensor->dphy, PHY_MODE_MIPI_DPHY);
> + phy_set_mode_ext(cdev->dphy, PHY_MODE_MIPI_DPHY,
> + PHY_MIPI_DPHY_SUBMODE_RX);
>From rkisp1 pov, looks good to me
Acked-by: Helen Koike
Regards,
Helen
> phy_configure(sensor->dphy, &opts);
> phy_power_on(sensor->dphy);
>
> diff --git a/include/linux/phy/phy-mipi-dphy.h
> b/include/linux/phy/phy-mipi-dphy.h
> index a877ffee845d..0f57ef46a8b5 100644
> --- a/include/linux/phy/phy-mipi-dphy.h
> +++ b/include/linux/phy/phy-mipi-dphy.h
> @@ -6,6 +6,19 @@
> #ifndef __PHY_MIPI_DPHY_H_
> #define __PHY_MIPI_DPHY_H_
>
> +/**
> + * enum phy_mipi_dphy_submode - MIPI D-PHY sub-mode
> + *
> + * A MIPI D-PHY can be used to transmit or receive data.
> + * Since some controllers can support both, the direction to enable is
> specified
> + * with the PHY sub-mode. Transmit is assumed by default with phy_set_mode.
> + */
> +
> +enum phy_mipi_dphy_submode {
> + PHY_MIPI_DPHY_SUBMODE_TX = 0,
> + PHY_MIPI_DPHY_SUBMODE_RX,
> +};
> +
> /**
> * struct phy_configure_opts_mipi_dphy - MIPI D-PHY configuration set
> *
>
[PATCH v2 3/3] media: MAINTAINERS: Update rkisp1 files with new location
rkisp1 was moved out of staging, update MAINTAINERS file accordingly. Signed-off-by: Helen Koike --- MAINTAINERS | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index 9e23b20eb2a21..6691c5d3e974b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -15037,7 +15037,8 @@ S: Maintained F: Documentation/admin-guide/media/rkisp1.rst F: Documentation/devicetree/bindings/media/rockchip-isp1.yaml F: Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst -F: drivers/staging/media/rkisp1/ +F: drivers/media/platform/rockchip/rkisp1 +F: include/uapi/linux/rkisp1-config.h ROCKCHIP RASTER 2D GRAPHIC ACCELERATION UNIT DRIVER M: Jacob Chen -- 2.28.0
[PATCH v2 0/3] destage Rockchip ISP1 driver
Hello,
Changes in v2:
- New patch updating MAINTAINERS file
- No changes in other patches
I think it is time to move this driver out of staging.
Thanks all who contributed, specially to Dafna, who put a lot of
effort addressing all the items in the TODO list, fixing bugs,
cleaning the code, addressing past comments and testing.
Please, review the driver, see if there is any other thing that should
be addressed before this change.
> media-ctl -p
http://ix.io/2Cso
> media-ctl --print-dot
http://ix.io/2Csp
> v4l2-compliance -m0
http://ix.io/2Csk
> v4l2-compliance -v -d /dev/video0 -s10
http://ix.io/2Csq
> v4l2-compliance -v -d /dev/video1 -s10
http://ix.io/2Css
This patch depends on the following series:
* media: staging: rkisp1: uapi: add "WITH Linux-syscall-note"
https://patchwork.linuxtv.org/project/linux-media/patch/20201020132514.26651-1-dafna.hirschf...@collabora.com/
* [0/2] media: staging: rkisp1: Fix formats for metadata pads
https://patchwork.linuxtv.org/project/linux-media/cover/20200325212704.29862-1-dafna.hirschf...@collabora.com/
* [v2,1/2] media: uapi: add MEDIA_BUS_FMT_METADATA_FIXED media bus format.
[v2,2/2] media: staging: rkisp1: isp: set metadata pads to
MEDIA_BUS_FMT_METADATA_FIXED
https://patchwork.linuxtv.org/project/linux-media/patch/20201020154522.654-1-dafna.hirschf...@collabora.com/
* [0/6] media: staging: rkisp1: improvements
https://patchwork.linuxtv.org/project/linux-media/cover/20201002184222.7094-1-dafna.hirschf...@collabora.com/
* [0/4] media: staging: rkisp1: send cleanups and checkpatch fixes
https://patchwork.linuxtv.org/project/linux-media/cover/20201019205956.6980-1-dafna.hirschf...@collabora.com/
* media: staging: rkisp1: capture: set default quantization on 'set_fmt'
https://patchwork.linuxtv.org/project/linux-media/patch/20201026162848.18310-1-dafna.hirschf...@collabora.com/
* media: staging: rkisp1: remove TODO item to document quantization handling
https://patchwork.linuxtv.org/project/linux-media/patch/20200928152809.27490-1-dafna.hirschf...@collabora.com/
* [v2] media: staging: rkisp1: cap: refactor enable/disable stream to allow
multistreaming
https://patchwork.linuxtv.org/project/linux-media/patch/20201019160434.877568-1-helen.ko...@collabora.com/
* [v6,0/9] move Rockchip ISP bindings out of staging / add ISP DT nodes for
RK3399
https://patchwork.linuxtv.org/project/linux-media/patch/20201020193850.1460644-2-helen.ko...@collabora.com/
You can also see all of them applied in this branch:
https://gitlab.collabora.com/koike/linux/-/tree/rockchip/isp/destage
Thanks
Helen
Helen Koike (2):
media: rockchip: rkisp1: destage Rockchip ISP1 driver
media: MAINTAINERS: Update rkisp1 files with new location
Shunqian Zheng (1):
media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format
.../media/v4l/pixfmt-meta-rkisp1.rst | 2 +-
MAINTAINERS | 3 ++-
drivers/media/platform/Kconfig| 18 ++
drivers/media/platform/Makefile | 1 +
.../platform/rockchip}/rkisp1/Makefile| 0
.../rockchip}/rkisp1/rkisp1-capture.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.h | 2 +-
.../platform/rockchip}/rkisp1/rkisp1-dev.c| 0
.../platform/rockchip}/rkisp1/rkisp1-isp.c| 0
.../platform/rockchip}/rkisp1/rkisp1-params.c | 0
.../platform/rockchip}/rkisp1/rkisp1-regs.h | 0
.../rockchip}/rkisp1/rkisp1-resizer.c | 0
.../platform/rockchip}/rkisp1/rkisp1-stats.c | 0
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
drivers/staging/media/Kconfig | 2 --
drivers/staging/media/Makefile| 1 -
drivers/staging/media/rkisp1/Kconfig | 19 ---
drivers/staging/media/rkisp1/TODO | 6 --
.../uapi/linux}/rkisp1-config.h | 4
include/uapi/linux/videodev2.h| 4
21 files changed, 29 insertions(+), 35 deletions(-)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/Makefile
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-capture.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.h (99%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-dev.c
(100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-isp.c
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-params.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-regs.h
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-resizer.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp
[PATCH v2 2/3] media: rockchip: rkisp1: destage Rockchip ISP1 driver
All the items in the TODO list were addressed, uapi was reviewed,
documentation written, checkpatch errors fixed, several bugs fixed.
There is no big reason to keep this driver in staging, so move it out.
Signed-off-by: Helen Koike
---
.../media/v4l/pixfmt-meta-rkisp1.rst | 2 +-
drivers/media/platform/Kconfig| 18 ++
drivers/media/platform/Makefile | 1 +
.../platform/rockchip}/rkisp1/Makefile| 0
.../rockchip}/rkisp1/rkisp1-capture.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.h | 2 +-
.../platform/rockchip}/rkisp1/rkisp1-dev.c| 0
.../platform/rockchip}/rkisp1/rkisp1-isp.c| 0
.../platform/rockchip}/rkisp1/rkisp1-params.c | 0
.../platform/rockchip}/rkisp1/rkisp1-regs.h | 0
.../rockchip}/rkisp1/rkisp1-resizer.c | 0
.../platform/rockchip}/rkisp1/rkisp1-stats.c | 0
drivers/staging/media/Kconfig | 2 --
drivers/staging/media/Makefile| 1 -
drivers/staging/media/rkisp1/Kconfig | 19 ---
drivers/staging/media/rkisp1/TODO | 6 --
.../uapi/linux}/rkisp1-config.h | 0
18 files changed, 21 insertions(+), 30 deletions(-)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/Makefile
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-capture.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.h (99%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-dev.c
(100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-isp.c
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-params.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-regs.h
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-resizer.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-stats.c (100%)
delete mode 100644 drivers/staging/media/rkisp1/Kconfig
delete mode 100644 drivers/staging/media/rkisp1/TODO
rename {drivers/staging/media/rkisp1/uapi =>
include/uapi/linux}/rkisp1-config.h (100%)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
b/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
index 7e43837ed260a..f3671472d4105 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
@@ -46,4 +46,4 @@ important tuning tools using software control loop.
rkisp1 uAPI data types
==
-.. kernel-doc:: drivers/staging/media/rkisp1/uapi/rkisp1-config.h
+.. kernel-doc:: include/uapi/linux/rkisp1-config.h
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index a3cb104956d56..202d447759fd8 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -448,6 +448,24 @@ config VIDEO_RENESAS_VSP1
To compile this driver as a module, choose M here: the module
will be called vsp1.
+config VIDEO_ROCKCHIP_ISP1
+ tristate "Rockchip Image Signal Processing v1 Unit driver"
+ depends on VIDEO_V4L2 && OF
+ depends on ARCH_ROCKCHIP || COMPILE_TEST
+ select MEDIA_CONTROLLER
+ select VIDEO_V4L2_SUBDEV_API
+ select VIDEOBUF2_DMA_CONTIG
+ select VIDEOBUF2_VMALLOC
+ select V4L2_FWNODE
+ select GENERIC_PHY_MIPI_DPHY
+ default n
+ help
+ Enable this to support the Image Signal Processing (ISP) module
+ present in RK3399 SoCs.
+
+ To compile this driver as a module, choose M here: the module
+ will be called rockchip-isp1.
+
config VIDEO_ROCKCHIP_RGA
tristate "Rockchip Raster 2d Graphic Acceleration Unit"
depends on VIDEO_DEV && VIDEO_V4L2
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 62b6cdc8c7300..b342714228db4 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -52,6 +52,7 @@ obj-$(CONFIG_VIDEO_RENESAS_FDP1) += rcar_fdp1.o
obj-$(CONFIG_VIDEO_RENESAS_JPU)+= rcar_jpu.o
obj-$(CONFIG_VIDEO_RENESAS_VSP1) += vsp1/
+obj-$(CONFIG_VIDEO_ROCKCHIP_ISP1) += rockchip/rkisp1/
obj-$(CONFIG_VIDEO_ROCKCHIP_RGA) += rockchip/rga/
obj-y += omap/
diff --git a/drivers/staging/media/rkisp1/Makefile
b/drivers/media/platform/rockchip/rkisp1/Makefile
similarity index 100%
rename from drivers/staging/media/rkisp1/Makefile
rename to drivers/media/platform/rockchip/rkisp1/Makefile
diff --git a/drivers/staging/media/rkisp1/rkisp1-capture.c
b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
similarity index
[PATCH v2 1/3] media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format
From: Shunqian Zheng
Add the Rockchip ISP1 specific processing parameter format
V4L2_META_FMT_RK_ISP1_PARAMS and metadata format
V4L2_META_FMT_RK_ISP1_STAT_3A for 3A.
Signed-off-by: Shunqian Zheng
Signed-off-by: Jacob Chen
Signed-off-by: Helen Koike
Reviewed-by: Laurent Pinchart
---
Hello,
This patch is a continuation of:
https://patchwork.kernel.org/project/linux-arm-kernel/patch/20191106120132.6876-2-helen.ko...@collabora.com/
These formats are already documented under
Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
We had agreed to keep under
drivers/staging/media/rkisp1/uapi/rkisp1-config.h while the driver was
in staging, since we are moving it out of staging, I guess this is the
time :)
Regards,
Helen
---
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
drivers/staging/media/rkisp1/uapi/rkisp1-config.h | 4
include/uapi/linux/videodev2.h| 4
3 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index eeff398fbdcc1..202597d031c6b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1402,6 +1402,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_UVC: descr = "UVC Payload Header Metadata";
break;
case V4L2_META_FMT_D4XX:descr = "Intel D4xx UVC Metadata";
break;
case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
+ case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A
params"; break;
+ case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A
statistics"; break;
default:
/* Compressed formats */
diff --git a/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
b/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
index 8d906cc7da8fc..6e449e7842605 100644
--- a/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
+++ b/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
@@ -9,10 +9,6 @@
#include
-/* Vendor specific - used for RK_ISP1 camera sub-system */
-#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /*
Rockchip ISP1 params */
-#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /*
Rockchip ISP1 3A statistics */
-
/* Defect Pixel Cluster Detection */
#define RKISP1_CIF_ISP_MODULE_DPCC (1U << 0)
/* Black Level Subtraction */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 534eaa4d39bc8..c2e13ba81196b 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -770,6 +770,10 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_D4XXv4l2_fourcc('D', '4', 'X', 'X') /* D4XX
Payload Header metadata */
#define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid
Metadata */
+/* Vendor specific - used for RK_ISP1 camera sub-system */
+#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /*
Rockchip ISP1 params */
+#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /*
Rockchip ISP1 3A statistics */
+
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC0xfeedcafe
--
2.28.0
Re: [PATCH v2] media: staging: rkisp1: cap: refactor enable/disable stream to allow multistreaming
Hi Dafna,
On 10/26/20 2:50 PM, Dafna Hirschfeld wrote:
> Hi,
>
>
> Am 19.10.20 um 18:04 schrieb Helen Koike:
>> Allow streaming from self picture path and main picture path at the same
>> time.
>>
>> Take care for s_stream() callbacks to not be called twice.
>> When starting a stream, s_stream(true) shouldn't be called for the isp
>> and the sensor if the other stream is already enabled (since it was
>> already called).
>> When stopping a stream, s_stream(false) shouldn't be called for isp and
>> the sensor if the other stream is still enabled.
>>
>> Remove the callback function scheme for navigating through the topology,
>> simplifying the code, improving readability.
>>
>> Remove multistreaming item from the TODO list.
>>
>> Signed-off-by: Helen Koike
>> ---
>>
>> Hello,
>>
>> Since we didn't reach an agreement on the helpers in the core[1], I'm
>> sending this patch to fix this limitation only for rkisp1.
>>
>> [1]
>> https://patchwork.linuxtv.org/project/linux-media/cover/20200415013044.1778572-1-helen.ko...@collabora.com/
>>
>> If we decide to add the helpers in the future, we can clean up drivers
>> even more, but I don't want to block this feature.
>>
>> This Patch depends on patch:
>> "media: staging: rkisp1: validate links before powering and streaming"
>> https://patchwork.linuxtv.org/project/linux-media/patch/20201002184222.7094-2-dafna.hirschf...@collabora.com/
>>
>> Changes in V2:
>> ==
>> - Rebase on top of patch
>> "media: staging: rkisp1: validate links before powering and streaming"
>> which fixes media_pipeline_{start,stop}() calling order.
>> - Fix commit message
>> - Fix disable order
>> - Disable capture when s_stream(true) of the resizer fails
>>
>> Overview of the patch:
>> ==
>>
>> * Rename rkisp1_stream_{start,stop}() to
>> rkisp1_cap_stream_{enable,disable}() to clarify the difference between
>> other stream enable/disable functions
>>
>> * Implement rkisp1_pipeline_stream_{enable,disable}() to replace
>> rkisp1_pipeline_{enable,disable}_cb() and rkisp1_pipeline_sink_walk(),
>> which were removed.
>>
>> * Call rkisp1_cap_stream_{enable,disable}() from
>> rkisp1_pipeline_stream_{enable,disable}() for better
>> unwind handling and function name semantics.
>>
>> * Remove item from TODO list (I also reviewed the use of the
>> is_streaming var in the code and lgtm).
>>
>> This patch was tested on rockpi4 board with:
>>
>>
>> "media-ctl" "-d" "platform:rkisp1" "-r"
>> "media-ctl" "-d" "platform:rkisp1" "-l" "'imx219 4-0010':0 -> 'rkisp1_isp':0
>> [1]"
>> "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
>> 'rkisp1_resizer_selfpath':0 [1]"
>> "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
>> 'rkisp1_resizer_mainpath':0 [1]"
>>
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx219 4-0010":0
>> [fmt:SRGGB10_1X10/1640x1232]'
>>
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0
>> [fmt:SRGGB10_1X10/1640x1232 crop: (0,0)/1600x1200]'
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2
>> [fmt:YUYV8_2X8/1600x1200 crop: (0,0)/1500x1100]'
>>
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0
>> [fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]'
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":1
>> [fmt:YUYV8_2X8/900x800]'
>>
>> "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
>> "width=900,height=800,"
>> "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
>> "pixelformat=422P"
>>
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" &
[PATCH 2/2] media: rockchip: rkisp1: destage Rockchip ISP1 driver
All the items in the TODO list were addressed, uapi was reviewed,
documentation written, checkpatch errors fixed, several bugs fixed.
There is no big reason to keep this driver in staging, so move it out.
Signed-off-by: Helen Koike
---
.../media/v4l/pixfmt-meta-rkisp1.rst | 2 +-
drivers/media/platform/Kconfig| 18 ++
drivers/media/platform/Makefile | 1 +
.../platform/rockchip}/rkisp1/Makefile| 0
.../rockchip}/rkisp1/rkisp1-capture.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.h | 2 +-
.../platform/rockchip}/rkisp1/rkisp1-dev.c| 0
.../platform/rockchip}/rkisp1/rkisp1-isp.c| 0
.../platform/rockchip}/rkisp1/rkisp1-params.c | 0
.../platform/rockchip}/rkisp1/rkisp1-regs.h | 0
.../rockchip}/rkisp1/rkisp1-resizer.c | 0
.../platform/rockchip}/rkisp1/rkisp1-stats.c | 0
drivers/staging/media/Kconfig | 2 --
drivers/staging/media/Makefile| 1 -
drivers/staging/media/rkisp1/Kconfig | 19 ---
drivers/staging/media/rkisp1/TODO | 6 --
.../uapi/linux}/rkisp1-config.h | 0
18 files changed, 21 insertions(+), 30 deletions(-)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/Makefile
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-capture.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.h (99%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-dev.c
(100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-isp.c
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-params.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-regs.h
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-resizer.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-stats.c (100%)
delete mode 100644 drivers/staging/media/rkisp1/Kconfig
delete mode 100644 drivers/staging/media/rkisp1/TODO
rename {drivers/staging/media/rkisp1/uapi =>
include/uapi/linux}/rkisp1-config.h (100%)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
b/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
index 7e43837ed260a..f3671472d4105 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
@@ -46,4 +46,4 @@ important tuning tools using software control loop.
rkisp1 uAPI data types
==
-.. kernel-doc:: drivers/staging/media/rkisp1/uapi/rkisp1-config.h
+.. kernel-doc:: include/uapi/linux/rkisp1-config.h
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index a3cb104956d56..202d447759fd8 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -448,6 +448,24 @@ config VIDEO_RENESAS_VSP1
To compile this driver as a module, choose M here: the module
will be called vsp1.
+config VIDEO_ROCKCHIP_ISP1
+ tristate "Rockchip Image Signal Processing v1 Unit driver"
+ depends on VIDEO_V4L2 && OF
+ depends on ARCH_ROCKCHIP || COMPILE_TEST
+ select MEDIA_CONTROLLER
+ select VIDEO_V4L2_SUBDEV_API
+ select VIDEOBUF2_DMA_CONTIG
+ select VIDEOBUF2_VMALLOC
+ select V4L2_FWNODE
+ select GENERIC_PHY_MIPI_DPHY
+ default n
+ help
+ Enable this to support the Image Signal Processing (ISP) module
+ present in RK3399 SoCs.
+
+ To compile this driver as a module, choose M here: the module
+ will be called rockchip-isp1.
+
config VIDEO_ROCKCHIP_RGA
tristate "Rockchip Raster 2d Graphic Acceleration Unit"
depends on VIDEO_DEV && VIDEO_V4L2
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 62b6cdc8c7300..b342714228db4 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -52,6 +52,7 @@ obj-$(CONFIG_VIDEO_RENESAS_FDP1) += rcar_fdp1.o
obj-$(CONFIG_VIDEO_RENESAS_JPU)+= rcar_jpu.o
obj-$(CONFIG_VIDEO_RENESAS_VSP1) += vsp1/
+obj-$(CONFIG_VIDEO_ROCKCHIP_ISP1) += rockchip/rkisp1/
obj-$(CONFIG_VIDEO_ROCKCHIP_RGA) += rockchip/rga/
obj-y += omap/
diff --git a/drivers/staging/media/rkisp1/Makefile
b/drivers/media/platform/rockchip/rkisp1/Makefile
similarity index 100%
rename from drivers/staging/media/rkisp1/Makefile
rename to drivers/media/platform/rockchip/rkisp1/Makefile
diff --git a/drivers/staging/media/rkisp1/rkisp1-capture.c
b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
similarity index
[PATCH 0/2] destage Rockchip ISP1 driver
Hello,
I think it is time to move this driver out of staging.
Thanks all who contributed, specially to Dafna, who put a lot of
effort addressing all the items in the TODO list, fixing bugs,
cleaning the code, addressing past comments and testing.
Please, review the driver, see if there is any other thing that should
be addressed before this change.
> media-ctl -p
http://ix.io/2Cso
> media-ctl --print-dot
http://ix.io/2Csp
> v4l2-compliance -m0
http://ix.io/2Csk
> v4l2-compliance -v -d /dev/video0 -s10
http://ix.io/2Csq
> v4l2-compliance -v -d /dev/video1 -s10
http://ix.io/2Css
This patch depends on the following series:
* media: staging: rkisp1: uapi: add "WITH Linux-syscall-note"
https://patchwork.linuxtv.org/project/linux-media/patch/20201020132514.26651-1-dafna.hirschf...@collabora.com/
* [0/2] media: staging: rkisp1: Fix formats for metadata pads
https://patchwork.linuxtv.org/project/linux-media/cover/20200325212704.29862-1-dafna.hirschf...@collabora.com/
* [v2,1/2] media: uapi: add MEDIA_BUS_FMT_METADATA_FIXED media bus format.
[v2,2/2] media: staging: rkisp1: isp: set metadata pads to
MEDIA_BUS_FMT_METADATA_FIXED
https://patchwork.linuxtv.org/project/linux-media/patch/20201020154522.654-1-dafna.hirschf...@collabora.com/
* [0/6] media: staging: rkisp1: improvements
https://patchwork.linuxtv.org/project/linux-media/cover/20201002184222.7094-1-dafna.hirschf...@collabora.com/
* [0/4] media: staging: rkisp1: send cleanups and checkpatch fixes
https://patchwork.linuxtv.org/project/linux-media/cover/20201019205956.6980-1-dafna.hirschf...@collabora.com/
* media: staging: rkisp1: capture: set default quantization on 'set_fmt'
https://patchwork.linuxtv.org/project/linux-media/patch/20201026162848.18310-1-dafna.hirschf...@collabora.com/
* media: staging: rkisp1: remove TODO item to document quantization handling
https://patchwork.linuxtv.org/project/linux-media/patch/20200928152809.27490-1-dafna.hirschf...@collabora.com/
* [v2] media: staging: rkisp1: cap: refactor enable/disable stream to allow
multistreaming
https://patchwork.linuxtv.org/project/linux-media/patch/20201019160434.877568-1-helen.ko...@collabora.com/
* [v6,0/9] move Rockchip ISP bindings out of staging / add ISP DT nodes for
RK3399
https://patchwork.linuxtv.org/project/linux-media/patch/20201020193850.1460644-2-helen.ko...@collabora.com/
You can also see all of them applied in this branch:
https://gitlab.collabora.com/koike/linux/-/tree/rockchip/isp/destage
Thanks
Helen
Helen Koike (1):
media: rockchip: rkisp1: destage Rockchip ISP1 driver
Shunqian Zheng (1):
media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format
.../media/v4l/pixfmt-meta-rkisp1.rst | 2 +-
drivers/media/platform/Kconfig| 18 ++
drivers/media/platform/Makefile | 1 +
.../platform/rockchip}/rkisp1/Makefile| 0
.../rockchip}/rkisp1/rkisp1-capture.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.c | 0
.../platform/rockchip}/rkisp1/rkisp1-common.h | 2 +-
.../platform/rockchip}/rkisp1/rkisp1-dev.c| 0
.../platform/rockchip}/rkisp1/rkisp1-isp.c| 0
.../platform/rockchip}/rkisp1/rkisp1-params.c | 0
.../platform/rockchip}/rkisp1/rkisp1-regs.h | 0
.../rockchip}/rkisp1/rkisp1-resizer.c | 0
.../platform/rockchip}/rkisp1/rkisp1-stats.c | 0
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
drivers/staging/media/Kconfig | 2 --
drivers/staging/media/Makefile| 1 -
drivers/staging/media/rkisp1/Kconfig | 19 ---
drivers/staging/media/rkisp1/TODO | 6 --
.../uapi/linux}/rkisp1-config.h | 4
include/uapi/linux/videodev2.h| 4
20 files changed, 27 insertions(+), 34 deletions(-)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/Makefile
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-capture.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-common.h (99%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-dev.c
(100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-isp.c
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-params.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkisp1/rkisp1-regs.h
(100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-resizer.c (100%)
rename drivers/{staging/media =>
media/platform/rockchip}/rkisp1/rkisp1-stats.c (100%)
delete mode 100644 drivers/staging/media/rkisp1/Kconfig
delete mode 100644 drivers/staging/media/rkisp1/TODO
rename {drivers/staging/media/rkisp1/uapi =>
include/uapi/linux}/rkisp1-config.h (99%)
--
2.28.0
[PATCH 1/2] media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format
From: Shunqian Zheng
Add the Rockchip ISP1 specific processing parameter format
V4L2_META_FMT_RK_ISP1_PARAMS and metadata format
V4L2_META_FMT_RK_ISP1_STAT_3A for 3A.
Signed-off-by: Shunqian Zheng
Signed-off-by: Jacob Chen
Signed-off-by: Helen Koike
Reviewed-by: Laurent Pinchart
---
Hello,
This patch is a continuation of:
https://patchwork.kernel.org/project/linux-arm-kernel/patch/20191106120132.6876-2-helen.ko...@collabora.com/
These formats are already documented under
Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst
We had agreed to keep under
drivers/staging/media/rkisp1/uapi/rkisp1-config.h while the driver was
in staging, since we are moving it out of staging, I guess this is the
time :)
Regards,
Helen
---
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
drivers/staging/media/rkisp1/uapi/rkisp1-config.h | 4
include/uapi/linux/videodev2.h| 4
3 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index eeff398fbdcc1..202597d031c6b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1402,6 +1402,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_UVC: descr = "UVC Payload Header Metadata";
break;
case V4L2_META_FMT_D4XX:descr = "Intel D4xx UVC Metadata";
break;
case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
+ case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A
params"; break;
+ case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A
statistics"; break;
default:
/* Compressed formats */
diff --git a/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
b/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
index 8d906cc7da8fc..6e449e7842605 100644
--- a/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
+++ b/drivers/staging/media/rkisp1/uapi/rkisp1-config.h
@@ -9,10 +9,6 @@
#include
-/* Vendor specific - used for RK_ISP1 camera sub-system */
-#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /*
Rockchip ISP1 params */
-#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /*
Rockchip ISP1 3A statistics */
-
/* Defect Pixel Cluster Detection */
#define RKISP1_CIF_ISP_MODULE_DPCC (1U << 0)
/* Black Level Subtraction */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 534eaa4d39bc8..c2e13ba81196b 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -770,6 +770,10 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_D4XXv4l2_fourcc('D', '4', 'X', 'X') /* D4XX
Payload Header metadata */
#define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid
Metadata */
+/* Vendor specific - used for RK_ISP1 camera sub-system */
+#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /*
Rockchip ISP1 params */
+#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /*
Rockchip ISP1 3A statistics */
+
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC0xfeedcafe
--
2.28.0
Re: [PATCH v5 5/9] media: staging: rkisp1: remove unecessary clocks
Hi Rob, On 10/20/20 12:14 PM, Rob Herring wrote: > On Wed, Oct 14, 2020 at 11:46 AM Helen Koike > wrote: >> >> Hi Rob, >> >> Thnaks for your reply. >> >> On 9/22/20 11:24 AM, Rob Herring wrote: >>> On Wed, Jul 22, 2020 at 9:56 AM Helen Koike >>> wrote: >>>> >>>> aclk_isp_wrap is a child of aclk_isp, and hclk_isp_wrap is a child of >>>> hclk_isp, thus we can remove parents from the list. >>>> >>>> Also, for the isp0, we only need the ISP clock, ACLK and HCLK. >>>> In the future we'll need a pixel clock for RK3288 and RK3399, and a JPEG >>>> clock for RK3288. >>>> >>>> So with the goal to cleanup the dt-bindings and remove it from staging, >>>> simplify clock names to isp, aclk and hclk. >>>> >>>> Assigned clocks are meant to refer to the full path in the clock tree, >>>> i.e. the leaf in the tree. >>>> For instance, in RK3399, the clock responsible for ACLK (ISP AXI CLOCK) >>>> is aclk_isp0_wrapper. >>>> >>>> For reference, this is the isp clock topology on RK3399: >>>> >>>> xin24m >>>> pll_npll >>>>npll >>>> clk_isp1 >>>> clk_isp0 >>>> pll_cpll >>>>cpll >>>> aclk_isp1 >>>> aclk_isp1_noc >>>> hclk_isp1 >>>> aclk_isp1_wrapper >>>> hclk_isp1_noc >>>> aclk_isp0 >>>> hclk_isp1_wrapper >>>> aclk_isp0_wrapper >>>> aclk_isp0_noc >>>> hclk_isp0 >>>> hclk_isp0_wrapper >>>> hclk_isp0_noc >>>> pclkin_isp1_wrapper >>>> >>>> Signed-off-by: Helen Koike >>>> >>>> --- >>>> Changes in V5: >>>> - Use if/then schema as suggested by Rob Herring on >>>> https://patchwork.linuxtv.org/project/linux-media/patch/20200702191322.2639681-6-helen.ko...@collabora.com/#119729 >>>> >>>> Changes in V4: >>>> - update binding according to suggestion by Robin Murphy >>>> on https://patchwork.kernel.org/patch/11475007/ >>>> >>>> Changes in V3: >>>> - this is a new patch in the series >>>> --- >>>> .../bindings/media/rockchip-isp1.yaml | 50 --- >>>> drivers/staging/media/rkisp1/rkisp1-dev.c | 8 ++- >>>> 2 files changed, 36 insertions(+), 22 deletions(-) >>>> >>>> diff --git >>>> a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml >>>> >>>> b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml >>>> index 62a6b9c959498..23c677d15037a 100644 >>>> --- >>>> a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml >>>> +++ >>>> b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml >>>> @@ -24,20 +24,10 @@ properties: >>>> maxItems: 1 >>>> >>>>clocks: >>>> -items: >>>> - - description: ISP clock >>>> - - description: ISP AXI clock clock >>>> - - description: ISP AXI clock wrapper clock >>>> - - description: ISP AHB clock clock >>>> - - description: ISP AHB wrapper clock >>>> +minItems: 3 >>> >>> You need maxItems here too or it will always be 3. >>> >>>> >>>>clock-names: >>>> -items: >>>> - - const: clk_isp >>>> - - const: aclk_isp >>>> - - const: aclk_isp_wrap >>>> - - const: hclk_isp >>>> - - const: hclk_isp_wrap >>>> +minItems: 3 >>>> >>>>iommus: >>>> maxItems: 1 >>>> @@ -116,6 +106,34 @@ required: >>>>- power-domains >>>>- ports >>>> >>>> +if: >>>> + properties: >>>> +compatible: >>>> + contains: >>>> +const: rockchip,rk3399-cif-isp >>>> +then: >>>> + properties: >>>> +clocks: >>>> + maxItems: 4 >>>> + minItems: 3 >>>
[PATCH v6 8/9] arm64: dts: rockchip: add isp0 node for rk3399
From: Shunqian Zheng
RK3399 has two ISPs, but only isp0 was tested.
Add isp0 node in rk3399 dtsi
Verified with:
make ARCH=arm64 dtbs_check
DT_SCHEMA_FILES=Documentation/devicetree/bindings/media/rockchip-isp1.yaml
Signed-off-by: Shunqian Zheng
Signed-off-by: Jacob Chen
Signed-off-by: Helen Koike
---
Changes in v6:
- Add status = "disabled" in the isp0 node
---
arch/arm64/boot/dts/rockchip/rk3399.dtsi | 26
1 file changed, 26 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
index ada724b12f014..af5f8e2c5e64d 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
@@ -1723,6 +1723,32 @@ vopb_mmu: iommu@ff903f00 {
status = "disabled";
};
+ isp0: isp0@ff91 {
+ compatible = "rockchip,rk3399-cif-isp";
+ reg = <0x0 0xff91 0x0 0x4000>;
+ interrupts = ;
+ clocks = <&cru SCLK_ISP0>,
+<&cru ACLK_ISP0_WRAPPER>,
+<&cru HCLK_ISP0_WRAPPER>;
+ clock-names = "isp", "aclk", "hclk";
+ iommus = <&isp0_mmu>;
+ phys = <&mipi_dphy_rx0>;
+ phy-names = "dphy";
+ power-domains = <&power RK3399_PD_ISP0>;
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+ };
+ };
+
isp0_mmu: iommu@ff914000 {
compatible = "rockchip,iommu";
reg = <0x0 0xff914000 0x0 0x100>, <0x0 0xff915000 0x0 0x100>;
--
2.28.0
[PATCH v6 4/9] media: staging: dt-bindings: rkisp1: drop parent unit address
Fix the following error found with make ARCH=arm64 dt_binding_check:
Documentation/devicetree/bindings/media/rockchip-isp1.example.dts:24.27-101.11:
Warning (unit_address_vs_reg): /example-0/parent@0: node has a unit name, but
no reg or ranges property
Reported-by: Johan Jonker
Signed-off-by: Helen Koike
Acked-by: Rob Herring
Reviewed-by: Tomasz Figa
---
.../Documentation/devicetree/bindings/media/rockchip-isp1.yaml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
index 79ebacab83cf3..62a6b9c959498 100644
---
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
+++
b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
@@ -125,7 +125,7 @@ examples:
#include
#include
-parent0: parent@0 {
+parent0: parent {
#address-cells = <2>;
#size-cells = <2>;
--
2.28.0
[PATCH v6 9/9] arm64: dts: rockchip: add isp and sensors for Scarlet
From: Eddie Cai
Enable ISP and camera sensor ov2685 and ov5695 for Scarlet Chromebook
Verified with:
make ARCH=arm64 dtbs_check
Signed-off-by: Shunqian Zheng
Signed-off-by: Eddie Cai
Signed-off-by: Tomasz Figa
Signed-off-by: Helen Koike
Reviewed-by: Tomasz Figa
---
.../boot/dts/rockchip/rk3399-gru-scarlet.dtsi | 74 +++
1 file changed, 74 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet.dtsi
b/arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet.dtsi
index 60cd1c18cd4e0..beee5fbb34437 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet.dtsi
@@ -296,6 +296,52 @@ camera: &i2c7 {
/* 24M mclk is shared between world and user cameras */
pinctrl-0 = <&i2c7_xfer &test_clkout1>;
+
+ /* Rear-facing camera */
+ wcam: camera@36 {
+ compatible = "ovti,ov5695";
+ reg = <0x36>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&wcam_rst>;
+
+ clocks = <&cru SCLK_TESTCLKOUT1>;
+ clock-names = "xvclk";
+
+ avdd-supply = <&pp2800_cam>;
+ dvdd-supply = <&pp1250_cam>;
+ dovdd-supply = <&pp1800_s0>;
+ reset-gpios = <&gpio2 5 GPIO_ACTIVE_LOW>;
+
+ port {
+ wcam_out: endpoint {
+ remote-endpoint = <&mipi_in_wcam>;
+ data-lanes = <1 2>;
+ };
+ };
+ };
+
+ /* Front-facing camera */
+ ucam: camera@3c {
+ compatible = "ovti,ov2685";
+ reg = <0x3c>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&ucam_rst>;
+
+ clocks = <&cru SCLK_TESTCLKOUT1>;
+ clock-names = "xvclk";
+
+ avdd-supply = <&pp2800_cam>;
+ dovdd-supply = <&pp1800_s0>;
+ dvdd-supply = <&pp1800_s0>;
+ reset-gpios = <&gpio2 3 GPIO_ACTIVE_LOW>;
+
+ port {
+ ucam_out: endpoint {
+ remote-endpoint = <&mipi_in_ucam>;
+ data-lanes = <1>;
+ };
+ };
+ };
};
&cdn_dp {
@@ -353,10 +399,38 @@ &io_domains {
gpio1830-supply = <&pp1800_s0>; /* APIO4_VDD; 4c 4d */
};
+&isp0 {
+ status = "okay";
+
+ ports {
+ port@0 {
+ mipi_in_wcam: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&wcam_out>;
+ data-lanes = <1 2>;
+ };
+
+ mipi_in_ucam: endpoint@1 {
+ reg = <1>;
+ remote-endpoint = <&ucam_out>;
+ data-lanes = <1>;
+ };
+ };
+ };
+};
+
+&isp0_mmu {
+ status = "okay";
+};
+
&max98357a {
sdmode-gpios = <&gpio0 2 GPIO_ACTIVE_HIGH>;
};
+&mipi_dphy_rx0 {
+ status = "okay";
+};
+
&mipi_dsi {
status = "okay";
clock-master;
--
2.28.0
[PATCH v6 3/9] media: staging: dt-bindings: rkisp1: re-order properties
Organize properties order in dt-bindings to move it out of staging.
On top: compatible, reg and interrupts.
Then alphabetical order, then properties starting with '#'.
Signed-off-by: Helen Koike
Acked-by: Rob Herring
Reviewed-by: Tomasz Figa
---
.../bindings/media/rockchip-isp1.yaml | 32 +--
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
index 4c31cfaee2709..79ebacab83cf3 100644
---
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
+++
b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
@@ -23,19 +23,6 @@ properties:
interrupts:
maxItems: 1
- iommus:
-maxItems: 1
-
- power-domains:
-maxItems: 1
-
- phys:
-maxItems: 1
-description: phandle for the PHY port
-
- phy-names:
-const: dphy
-
clocks:
items:
- description: ISP clock
@@ -52,6 +39,19 @@ properties:
- const: hclk_isp
- const: hclk_isp_wrap
+ iommus:
+maxItems: 1
+
+ phys:
+maxItems: 1
+description: phandle for the PHY port
+
+ phy-names:
+const: dphy
+
+ power-domains:
+maxItems: 1
+
# See ./video-interfaces.txt for details
ports:
type: object
@@ -110,10 +110,10 @@ required:
- interrupts
- clocks
- clock-names
- - power-domains
- iommus
- phys
- phy-names
+ - power-domains
- ports
additionalProperties: false
@@ -139,19 +139,19 @@ examples:
clock-names = "clk_isp",
"aclk_isp", "aclk_isp_wrap",
"hclk_isp", "hclk_isp_wrap";
-power-domains = <&power RK3399_PD_ISP0>;
iommus = <&isp0_mmu>;
phys = <&dphy>;
phy-names = "dphy";
+power-domains = <&power RK3399_PD_ISP0>;
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
+reg = <0>;
#address-cells = <1>;
#size-cells = <0>;
-reg = <0>;
mipi_in_wcam: endpoint@0 {
reg = <0>;
--
2.28.0
[PATCH v6 6/9] dt-bindings: media: rkisp1: move rockchip-isp1 bindings out of staging
Move rkisp1 bindings to Documentation/devicetree/bindings/media
Verified with:
make ARCH=arm64 dt_binding_check
DT_SCHEMA_FILES=Documentation/devicetree/bindings/media/rockchip-isp1.yaml
Signed-off-by: Helen Koike
Acked-by: Rob Herring
Reviewed-by: Tomasz Figa
---
.../devicetree/bindings/media/rockchip-isp1.yaml | 0
1 file changed, 0 insertions(+), 0 deletions(-)
rename {drivers/staging/media/rkisp1/Documentation =>
Documentation}/devicetree/bindings/media/rockchip-isp1.yaml (100%)
diff --git
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
b/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
similarity index 100%
rename from
drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
rename to Documentation/devicetree/bindings/media/rockchip-isp1.yaml
--
2.28.0
[PATCH v6 7/9] media: MAINTAINERS: rkisp1: add path to dt-bindings
The Rockchip ISP bindings was moved out of staging. Update MAINTAINERS file with the new path. Fields sorted according to output of ./scripts/parse-maintainers.pl --input=MAINTAINERS --output=MAINTAINERS --order Signed-off-by: Helen Koike Reviewed-by: Tomasz Figa --- MAINTAINERS | 2 ++ 1 file changed, 2 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 7a12633747c80..df679b3626b9a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14889,8 +14889,10 @@ ROCKCHIP ISP V1 DRIVER M: Helen Koike M: Dafna Hirschfeld L: linux-me...@vger.kernel.org +L: linux-rockc...@lists.infradead.org S: Maintained F: Documentation/admin-guide/media/rkisp1.rst +F: Documentation/devicetree/bindings/media/rockchip-isp1.yaml F: Documentation/userspace-api/media/v4l/pixfmt-meta-rkisp1.rst F: drivers/staging/media/rkisp1/ -- 2.28.0
[PATCH v6 5/9] media: staging: rkisp1: remove unecessary clocks
aclk_isp_wrap is a child of aclk_isp, and hclk_isp_wrap is a child of
hclk_isp, thus we can remove parents from the list.
Also, for the isp0, we only need the ISP clock, ACLK and HCLK.
In the future we'll need a pixel clock for RK3288 and RK3399, and a JPEG
clock for RK3288.
So with the goal to cleanup the dt-bindings and remove it from staging,
simplify clock names to isp, aclk and hclk.
Assigned clocks are meant to refer to the full path in the clock tree,
i.e. the leaf in the tree.
For instance, in RK3399, the clock responsible for ACLK (ISP AXI CLOCK)
is aclk_isp0_wrapper.
For reference, this is the isp clock topology on RK3399:
xin24m
pll_npll
npll
clk_isp1
clk_isp0
pll_cpll
cpll
aclk_isp1
aclk_isp1_noc
hclk_isp1
aclk_isp1_wrapper
hclk_isp1_noc
aclk_isp0
hclk_isp1_wrapper
aclk_isp0_wrapper
aclk_isp0_noc
hclk_isp0
hclk_isp0_wrapper
hclk_isp0_noc
pclkin_isp1_wrapper
Signed-off-by: Helen Koike
Reviewed-by: Tomasz Figa
---
Changes in V6:
- Define clocks in the top level, and use if/else schema to define how
many for each compatible as sugested by Rob Herring on
https://patchwork.linuxtv.org/project/linux-media/patch/20200722155533.252844-6-helen.ko...@collabora.com/#122626
---
.../bindings/media/rockchip-isp1.yaml | 44 +--
drivers/staging/media/rkisp1/rkisp1-dev.c | 8 ++--
2 files changed, 33 insertions(+), 19 deletions(-)
diff --git
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
index 62a6b9c959498..2004c054ed1a0 100644
---
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
+++
b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
@@ -24,20 +24,24 @@ properties:
maxItems: 1
clocks:
+minItems: 3
items:
+ # isp0 and isp1
- description: ISP clock
- - description: ISP AXI clock clock
- - description: ISP AXI clock wrapper clock
- - description: ISP AHB clock clock
- - description: ISP AHB wrapper clock
+ - description: ISP AXI clock
+ - description: ISP AHB clock
+ # only for isp1
+ - description: ISP Pixel clock
clock-names:
+minItems: 3
items:
- - const: clk_isp
- - const: aclk_isp
- - const: aclk_isp_wrap
- - const: hclk_isp
- - const: hclk_isp_wrap
+ # isp0 and isp1
+ - const: isp
+ - const: aclk
+ - const: hclk
+ # only for isp1
+ - const: pclk_isp
iommus:
maxItems: 1
@@ -116,6 +120,20 @@ required:
- power-domains
- ports
+if:
+ properties:
+compatible:
+ contains:
+const: rockchip,rk3399-cif-isp
+then:
+ properties:
+clocks:
+ minItems: 3
+ maxItems: 4
+clock-names:
+ minItems: 3
+ maxItems: 4
+
additionalProperties: false
examples:
@@ -134,11 +152,9 @@ examples:
reg = <0x0 0xff91 0x0 0x4000>;
interrupts = ;
clocks = <&cru SCLK_ISP0>,
- <&cru ACLK_ISP0>, <&cru ACLK_ISP0_WRAPPER>,
- <&cru HCLK_ISP0>, <&cru HCLK_ISP0_WRAPPER>;
-clock-names = "clk_isp",
- "aclk_isp", "aclk_isp_wrap",
- "hclk_isp", "hclk_isp_wrap";
+ <&cru ACLK_ISP0_WRAPPER>,
+ <&cru HCLK_ISP0_WRAPPER>;
+clock-names = "isp", "aclk", "hclk";
iommus = <&isp0_mmu>;
phys = <&dphy>;
phy-names = "dphy";
diff --git a/drivers/staging/media/rkisp1/rkisp1-dev.c
b/drivers/staging/media/rkisp1/rkisp1-dev.c
index 91584695804bb..90d6543465562 100644
--- a/drivers/staging/media/rkisp1/rkisp1-dev.c
+++ b/drivers/staging/media/rkisp1/rkisp1-dev.c
@@ -405,11 +405,9 @@ static irqreturn_t rkisp1_isr(int irq, void *ctx)
}
static const char * const rk3399_isp_clks[] = {
- "clk_isp",
- "aclk_isp",
- "hclk_isp",
- "aclk_isp_wrap",
- "hclk_isp_wrap",
+ "isp",
+ "aclk",
+ "hclk",
};
static const struct rkisp1_match_data rk3399_isp_clk_data = {
--
2.28.0
[PATCH v6 2/9] media: staging: dt-bindings: rkisp1: drop i2c unit address
Add missing required items in Rockchip ISP1 dt-bindings example for
a complete i2c node.
Drop unit address to Fix error:
/example-0/parent/i2c@ff16: node has a unit name, but no reg or ranges
property
Remove unecessary fields for the example.
Signed-off-by: Helen Koike
Acked-by: Rob Herring
Reviewed-by: Tomasz Figa
---
.../Documentation/devicetree/bindings/media/rockchip-isp1.yaml | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
index a77b6ec500c95..4c31cfaee2709 100644
---
a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
+++
b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml
@@ -168,8 +168,7 @@ examples:
};
};
-i2c7: i2c@ff16 {
-clock-frequency = <40>;
+i2c7: i2c {
#address-cells = <1>;
#size-cells = <0>;
--
2.28.0
[PATCH v6 1/9] media: staging: dt-bindings: rkisp1: add missing required nodes
Add missing required nodes in json-schema yaml file for Rockchip ISP1 dt-bindings. Signed-off-by: Helen Koike Acked-by: Rob Herring Reviewed-by: Tomasz Figa --- .../devicetree/bindings/media/rockchip-isp1.yaml | 8 1 file changed, 8 insertions(+) diff --git a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml index af246b71eac6b..a77b6ec500c95 100644 --- a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml +++ b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml @@ -94,11 +94,19 @@ properties: remote-endpoint: true +required: + - reg + - "#address-cells" + - "#size-cells" + required: + - "#address-cells" + - "#size-cells" - port@0 required: - compatible + - reg - interrupts - clocks - clock-names -- 2.28.0
[PATCH v6 0/9] move Rockchip ISP bindings out of staging / add ISP DT nodes for RK3399
Move the bindings out of drivers/staging and place them in
Documentation/devicetree/bindings instead.
Also, add DT nodes for RK3399 and verify with make ARCH=arm64 dtbs_check
and make ARCH=arm64 dt_binding_check.
Tested by verifying images streamed from Scarlet Chromebook
Changes in v6:
- [PATCH v6 5/9] media: staging: rkisp1: remove unecessary clocks
Define clocks in the top level, and use if/else schema to define how
many for each compatible as sugested by Rob Herring on
https://patchwork.linuxtv.org/project/linux-media/patch/20200722155533.252844-6-helen.ko...@collabora.com/#122626
- [PATCH v6 8/9] arm64: dts: rockchip: add isp0 node for rk3399
Add status = "disabled" in the isp0 node
- Added Reviewed-by tags from Tomasz Figa
- Added Acked-by tags from Rog Herring
- No changes for other patches in the serie.
Changes in v5:
- Drop unit addresses in dt-bindings example for simplification and fix
errors as suggested by Rob Herring in previous version
- Fix typos
- Re-write clock organization with if/then schema
Changes in v4:
- simplify clocks with "isp", "aclk" and "hclk" as suggested by
Robin Murphy on https://patchwork.kernel.org/patch/11475007/
Changes in v3:
- dropped accepted patches
- cleanup clocks
- fix "no reg" error in dt-bindings parent@0 example
- add linux-rockchip list in MAINTAINERS and reorder items
- add Scarlet sensors dt nodes to the series
Changes in v2:
Add patches modifying bindings, as sugested by Johan Jonker in v1,
before moving them out of staging.
Eddie Cai (1):
arm64: dts: rockchip: add isp and sensors for Scarlet
Helen Koike (7):
media: staging: dt-bindings: rkisp1: add missing required nodes
media: staging: dt-bindings: rkisp1: drop i2c unit address
media: staging: dt-bindings: rkisp1: re-order properties
media: staging: dt-bindings: rkisp1: drop parent unit address
media: staging: rkisp1: remove unecessary clocks
dt-bindings: media: rkisp1: move rockchip-isp1 bindings out of staging
media: MAINTAINERS: rkisp1: add path to dt-bindings
Shunqian Zheng (1):
arm64: dts: rockchip: add isp0 node for rk3399
.../bindings/media/rockchip-isp1.yaml | 81 ---
MAINTAINERS | 2 +
.../boot/dts/rockchip/rk3399-gru-scarlet.dtsi | 74 +
arch/arm64/boot/dts/rockchip/rk3399.dtsi | 26 ++
drivers/staging/media/rkisp1/rkisp1-dev.c | 8 +-
5 files changed, 157 insertions(+), 34 deletions(-)
rename {drivers/staging/media/rkisp1/Documentation =>
Documentation}/devicetree/bindings/media/rockchip-isp1.yaml (81%)
--
2.28.0
[PATCH v2] media: staging: rkisp1: cap: refactor enable/disable stream to allow multistreaming
Allow streaming from self picture path and main picture path at the same
time.
Take care for s_stream() callbacks to not be called twice.
When starting a stream, s_stream(true) shouldn't be called for the isp
and the sensor if the other stream is already enabled (since it was
already called).
When stopping a stream, s_stream(false) shouldn't be called for isp and
the sensor if the other stream is still enabled.
Remove the callback function scheme for navigating through the topology,
simplifying the code, improving readability.
Remove multistreaming item from the TODO list.
Signed-off-by: Helen Koike
---
Hello,
Since we didn't reach an agreement on the helpers in the core[1], I'm
sending this patch to fix this limitation only for rkisp1.
[1]
https://patchwork.linuxtv.org/project/linux-media/cover/20200415013044.1778572-1-helen.ko...@collabora.com/
If we decide to add the helpers in the future, we can clean up drivers
even more, but I don't want to block this feature.
This Patch depends on patch:
"media: staging: rkisp1: validate links before powering and streaming"
https://patchwork.linuxtv.org/project/linux-media/patch/20201002184222.7094-2-dafna.hirschf...@collabora.com/
Changes in V2:
==
- Rebase on top of patch
"media: staging: rkisp1: validate links before powering and streaming"
which fixes media_pipeline_{start,stop}() calling order.
- Fix commit message
- Fix disable order
- Disable capture when s_stream(true) of the resizer fails
Overview of the patch:
==
* Rename rkisp1_stream_{start,stop}() to
rkisp1_cap_stream_{enable,disable}() to clarify the difference between
other stream enable/disable functions
* Implement rkisp1_pipeline_stream_{enable,disable}() to replace
rkisp1_pipeline_{enable,disable}_cb() and rkisp1_pipeline_sink_walk(),
which were removed.
* Call rkisp1_cap_stream_{enable,disable}() from
rkisp1_pipeline_stream_{enable,disable}() for better
unwind handling and function name semantics.
* Remove item from TODO list (I also reviewed the use of the
is_streaming var in the code and lgtm).
This patch was tested on rockpi4 board with:
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx219 4-0010':0 -> 'rkisp1_isp':0
[1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
'rkisp1_resizer_mainpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx219 4-0010":0
[fmt:SRGGB10_1X10/1640x1232]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0
[fmt:SRGGB10_1X10/1640x1232 crop: (0,0)/1600x1200]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2
[fmt:YUYV8_2X8/1600x1200 crop: (0,0)/1500x1100]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0
[fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":1
[fmt:YUYV8_2X8/900x800]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
"width=900,height=800,"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "pixelformat=422P"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_mainpath":0
[fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_mainpath":1
[fmt:YUYV8_2X8/900x800]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_mainpath" "-v"
"width=900,height=800,"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_mainpath" "-v" "pixelformat=422P"
sleep 1
time v4l2-ctl "-z" "platform:rkisp1" "-d" "rkisp1_mainpath" "--stream-mmap"
"--stream-count" "100" &
time v4l2-ctl "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap"
"--stream-count"
Re: [PATCH] media: staging: rkisp1: cap: refactor enable/disable stream to allow multistreaming
Hello,
On 10/16/20 3:11 PM, Helen Koike wrote:
>
>
> On 10/16/20 11:28 AM, Dafna Hirschfeld wrote:
>> Hi,
>>
>> Am 15.10.20 um 21:57 schrieb Helen Koike:
>>> Allow streaming from self picture path and main picture path at the same
>>> time.
>>>
>>> Take care for s_stream() callbacks to not be called twice.
>>> When starting a stream, s_stream(true) shouldn't be called for the isp
>>> and the sensor if the other stream is already enabled (since it was
>>> already called).
>>> When stopping a stream, s_stream(false) shouldn't be called for isp and
>>> the sensor if the other stream is still enabled.
>>>
>>> Remove the callback function scheme for navigating through the topology,
>>> simplifying the code, improving readability, while calling
>>> media_pipeline_{start,stop}() in the right order.
>>>
>>> Remove multistreaming item from the TODO list.
>>>
>>> Signed-off-by: Helen Koike
>>>
>>> ---
>>> Hello,
>>>
>>> Since we didn't reach an agreement on the helpers in the core[1], I'm
>>> sending this patch to fix this limitation only for rkisp1.
>>>
>>> [1]
>>> https://patchwork.linuxtv.org/project/linux-media/cover/20200415013044.1778572-1-helen.ko...@collabora.com/
>>>
>>> If we decide to add the helpers in the future, we can clean up drivers
>>> even more, but I don't want to block this feature.
>>>
>>> Overview of the patch:
>>> ==
>>>
>>> * Rename rkisp1_stream_{start,stop}() to
>>> rkisp1_cap_stream_{enable,disable}() to clarify the difference between
>>> other stream enable/disable functions
>>>
>>> * Implement rkisp1_pipeline_stream_{enable,disable}() to replace
>>> rkisp1_pipeline_{enable,disable}_cb() and rkisp1_pipeline_sink_walk(),
>>> which were removed.
>>>
>>> * Call rkisp1_cap_stream_{enable,disable}() from
>>> rkisp1_pipeline_stream_{enable,disable}() for better
>>> unwind handling and function name semantics.
>>>
>>> * Call media_pipeline_{start,stop}() in the right order.
>>>
>>> * Remove item from TODO list (I also reviewed the use of the
>>> is_streaming var in the code and lgtm).
>>>
>>> This patch was tested on rockpi4 board with:
>>>
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "-r"
>>> "media-ctl" "-d" "platform:rkisp1" "-l" "'imx219 4-0010':0 ->
>>> 'rkisp1_isp':0 [1]"
>>> "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
>>> 'rkisp1_resizer_selfpath':0 [1]"
>>> "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
>>> 'rkisp1_resizer_mainpath':0 [1]"
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx219 4-0010":0
>>> [fmt:SRGGB10_1X10/1640x1232]'
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0
>>> [fmt:SRGGB10_1X10/1640x1232 crop: (0,0)/1600x1200]'
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2
>>> [fmt:YUYV8_2X8/1600x1200 crop: (0,0)/1500x1100]'
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2"
>>> '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1500x1100 crop:
>>> (300,400)/1400x1000]'
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2"
>>> '"rkisp1_resizer_selfpath":1 [fmt:YUYV8_2X8/900x800]'
>>>
>>> "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
>>> "width=900,height=800,"
>>> "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
>>> "pixelformat=422P"
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2"
>>> '"rkisp1_resizer_mainpath":0 [fmt:YUYV8_2X8/1500x1100 crop:
&
Re: [PATCH] media: staging: rkisp1: cap: refactor enable/disable stream to allow multistreaming
Hi Tomasz.
On 10/16/20 11:53 AM, Tomasz Figa wrote:
> On Fri, Oct 16, 2020 at 4:28 PM Dafna Hirschfeld
> wrote:
>>
>> Hi,
>>
>> Am 15.10.20 um 21:57 schrieb Helen Koike:
>>> Allow streaming from self picture path and main picture path at the same
>>> time.
>>>
>>> Take care for s_stream() callbacks to not be called twice.
>>> When starting a stream, s_stream(true) shouldn't be called for the isp
>>> and the sensor if the other stream is already enabled (since it was
>>> already called).
>>> When stopping a stream, s_stream(false) shouldn't be called for isp and
>>> the sensor if the other stream is still enabled.
>>>
>>> Remove the callback function scheme for navigating through the topology,
>>> simplifying the code, improving readability, while calling
>>> media_pipeline_{start,stop}() in the right order.
>>>
>>> Remove multistreaming item from the TODO list.
>>>
>>> Signed-off-by: Helen Koike
>>>
>>> ---
>>> Hello,
>>>
>>> Since we didn't reach an agreement on the helpers in the core[1], I'm
>>> sending this patch to fix this limitation only for rkisp1.
>>>
>>> [1]
>>> https://patchwork.linuxtv.org/project/linux-media/cover/20200415013044.1778572-1-helen.ko...@collabora.com/
>>>
>>> If we decide to add the helpers in the future, we can clean up drivers
>>> even more, but I don't want to block this feature.
>>>
>>> Overview of the patch:
>>> ==
>>>
>>> * Rename rkisp1_stream_{start,stop}() to
>>>rkisp1_cap_stream_{enable,disable}() to clarify the difference between
>>>other stream enable/disable functions
>>>
>>> * Implement rkisp1_pipeline_stream_{enable,disable}() to replace
>>>rkisp1_pipeline_{enable,disable}_cb() and rkisp1_pipeline_sink_walk(),
>>>which were removed.
>>>
>>> * Call rkisp1_cap_stream_{enable,disable}() from
>>>rkisp1_pipeline_stream_{enable,disable}() for better
>>>unwind handling and function name semantics.
>>>
>>> * Call media_pipeline_{start,stop}() in the right order.
>>>
>>> * Remove item from TODO list (I also reviewed the use of the
>>>is_streaming var in the code and lgtm).
>>>
>>> This patch was tested on rockpi4 board with:
>>>
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "-r"
>>> "media-ctl" "-d" "platform:rkisp1" "-l" "'imx219 4-0010':0 ->
>>> 'rkisp1_isp':0 [1]"
>>> "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
>>> 'rkisp1_resizer_selfpath':0 [1]"
>>> "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
>>> 'rkisp1_resizer_mainpath':0 [1]"
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx219 4-0010":0
>>> [fmt:SRGGB10_1X10/1640x1232]'
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0
>>> [fmt:SRGGB10_1X10/1640x1232 crop: (0,0)/1600x1200]'
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2
>>> [fmt:YUYV8_2X8/1600x1200 crop: (0,0)/1500x1100]'
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2"
>>> '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1500x1100 crop:
>>> (300,400)/1400x1000]'
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2"
>>> '"rkisp1_resizer_selfpath":1 [fmt:YUYV8_2X8/900x800]'
>>>
>>> "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
>>> "width=900,height=800,"
>>> "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
>>> "pixelformat=422P"
>>>
>>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2"
>>> '"rkisp1_resizer_mainpath":0 [fmt:YUYV8_2X8/1
Re: [PATCH] media: staging: rkisp1: cap: refactor enable/disable stream to allow multistreaming
On 10/16/20 11:28 AM, Dafna Hirschfeld wrote:
> Hi,
>
> Am 15.10.20 um 21:57 schrieb Helen Koike:
>> Allow streaming from self picture path and main picture path at the same
>> time.
>>
>> Take care for s_stream() callbacks to not be called twice.
>> When starting a stream, s_stream(true) shouldn't be called for the isp
>> and the sensor if the other stream is already enabled (since it was
>> already called).
>> When stopping a stream, s_stream(false) shouldn't be called for isp and
>> the sensor if the other stream is still enabled.
>>
>> Remove the callback function scheme for navigating through the topology,
>> simplifying the code, improving readability, while calling
>> media_pipeline_{start,stop}() in the right order.
>>
>> Remove multistreaming item from the TODO list.
>>
>> Signed-off-by: Helen Koike
>>
>> ---
>> Hello,
>>
>> Since we didn't reach an agreement on the helpers in the core[1], I'm
>> sending this patch to fix this limitation only for rkisp1.
>>
>> [1]
>> https://patchwork.linuxtv.org/project/linux-media/cover/20200415013044.1778572-1-helen.ko...@collabora.com/
>>
>> If we decide to add the helpers in the future, we can clean up drivers
>> even more, but I don't want to block this feature.
>>
>> Overview of the patch:
>> ==
>>
>> * Rename rkisp1_stream_{start,stop}() to
>> rkisp1_cap_stream_{enable,disable}() to clarify the difference between
>> other stream enable/disable functions
>>
>> * Implement rkisp1_pipeline_stream_{enable,disable}() to replace
>> rkisp1_pipeline_{enable,disable}_cb() and rkisp1_pipeline_sink_walk(),
>> which were removed.
>>
>> * Call rkisp1_cap_stream_{enable,disable}() from
>> rkisp1_pipeline_stream_{enable,disable}() for better
>> unwind handling and function name semantics.
>>
>> * Call media_pipeline_{start,stop}() in the right order.
>>
>> * Remove item from TODO list (I also reviewed the use of the
>> is_streaming var in the code and lgtm).
>>
>> This patch was tested on rockpi4 board with:
>>
>>
>> "media-ctl" "-d" "platform:rkisp1" "-r"
>> "media-ctl" "-d" "platform:rkisp1" "-l" "'imx219 4-0010':0 -> 'rkisp1_isp':0
>> [1]"
>> "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
>> 'rkisp1_resizer_selfpath':0 [1]"
>> "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
>> 'rkisp1_resizer_mainpath':0 [1]"
>>
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx219 4-0010":0
>> [fmt:SRGGB10_1X10/1640x1232]'
>>
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0
>> [fmt:SRGGB10_1X10/1640x1232 crop: (0,0)/1600x1200]'
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2
>> [fmt:YUYV8_2X8/1600x1200 crop: (0,0)/1500x1100]'
>>
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0
>> [fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]'
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":1
>> [fmt:YUYV8_2X8/900x800]'
>>
>> "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
>> "width=900,height=800,"
>> "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
>> "pixelformat=422P"
>>
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_mainpath":0
>> [fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]'
>> "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_mainpath":1
>> [fmt:YUYV8_2X8/900x800]'
>>
>> "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_mainpath" "-v"
>> "width=900,height=800,"
>> "v4
[PATCH] media: staging: rkisp1: cap: refactor enable/disable stream to allow multistreaming
Allow streaming from self picture path and main picture path at the same
time.
Take care for s_stream() callbacks to not be called twice.
When starting a stream, s_stream(true) shouldn't be called for the isp
and the sensor if the other stream is already enabled (since it was
already called).
When stopping a stream, s_stream(false) shouldn't be called for isp and
the sensor if the other stream is still enabled.
Remove the callback function scheme for navigating through the topology,
simplifying the code, improving readability, while calling
media_pipeline_{start,stop}() in the right order.
Remove multistreaming item from the TODO list.
Signed-off-by: Helen Koike
---
Hello,
Since we didn't reach an agreement on the helpers in the core[1], I'm
sending this patch to fix this limitation only for rkisp1.
[1]
https://patchwork.linuxtv.org/project/linux-media/cover/20200415013044.1778572-1-helen.ko...@collabora.com/
If we decide to add the helpers in the future, we can clean up drivers
even more, but I don't want to block this feature.
Overview of the patch:
==
* Rename rkisp1_stream_{start,stop}() to
rkisp1_cap_stream_{enable,disable}() to clarify the difference between
other stream enable/disable functions
* Implement rkisp1_pipeline_stream_{enable,disable}() to replace
rkisp1_pipeline_{enable,disable}_cb() and rkisp1_pipeline_sink_walk(),
which were removed.
* Call rkisp1_cap_stream_{enable,disable}() from
rkisp1_pipeline_stream_{enable,disable}() for better
unwind handling and function name semantics.
* Call media_pipeline_{start,stop}() in the right order.
* Remove item from TODO list (I also reviewed the use of the
is_streaming var in the code and lgtm).
This patch was tested on rockpi4 board with:
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx219 4-0010':0 -> 'rkisp1_isp':0
[1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 ->
'rkisp1_resizer_mainpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx219 4-0010":0
[fmt:SRGGB10_1X10/1640x1232]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0
[fmt:SRGGB10_1X10/1640x1232 crop: (0,0)/1600x1200]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2
[fmt:YUYV8_2X8/1600x1200 crop: (0,0)/1500x1100]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0
[fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":1
[fmt:YUYV8_2X8/900x800]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v"
"width=900,height=800,"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "pixelformat=422P"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_mainpath":0
[fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_mainpath":1
[fmt:YUYV8_2X8/900x800]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_mainpath" "-v"
"width=900,height=800,"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_mainpath" "-v" "pixelformat=422P"
sleep 1
time v4l2-ctl "-z" "platform:rkisp1" "-d" "rkisp1_mainpath" "--stream-mmap"
"--stream-count" "100" &
time v4l2-ctl "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap"
"--stream-count" "100" &
wait
echo "Completed"
Thanks
Helen
---
drivers/staging/media/rkisp1/TODO | 3 -
drivers/staging/media/rkisp1/rkisp1-capture.c | 227 +-
2 files changed, 113 insertions(+), 117 deletions(-)
diff --git a/drivers/staging/media/rkisp1/TODO
b/drivers/staging/media/rkisp1/TODO
index e7c8398fc2cef..a2dd0ad951c25 100644
--- a/drivers/s
Re: [PATCH v5 8/9] arm64: dts: rockchip: add isp0 node for rk3399
Thank you Tomasz and Robin for your comments,
On 10/14/20 1:43 PM, Tomasz Figa wrote:
> On Wed, Oct 14, 2020 at 6:27 PM Helen Koike wrote:
>>
>> Hi Tomasz,
>>
>> On 9/26/20 10:00 AM, Tomasz Figa wrote:
>>> Hi Helen,
>>>
>>> On Wed, Jul 22, 2020 at 12:55:32PM -0300, Helen Koike wrote:
>>>> From: Shunqian Zheng
>>>>
>>>> RK3399 has two ISPs, but only isp0 was tested.
>>>> Add isp0 node in rk3399 dtsi
>>>>
>>>> Verified with:
>>>> make ARCH=arm64 dtbs_check
>>>> DT_SCHEMA_FILES=Documentation/devicetree/bindings/media/rockchip-isp1.yaml
>>>>
>>>> Signed-off-by: Shunqian Zheng
>>>> Signed-off-by: Jacob Chen
>>>> Signed-off-by: Helen Koike
>>>>
>>>> ---
>>>>
>>>> V4:
>>>> - update clock names
>>>>
>>>> V3:
>>>> - clean up clocks
>>>>
>>>> V2:
>>>> - re-order power-domains property
>>>>
>>>> V1:
>>>> This patch was originally part of this patchset:
>>>>
>>>> https://patchwork.kernel.org/patch/10267431/
>>>>
>>>> The only difference is:
>>>> - add phy properties
>>>> - add ports
>>>> ---
>>>> arch/arm64/boot/dts/rockchip/rk3399.dtsi | 25
>>>> 1 file changed, 25 insertions(+)
>>>>
>>>> diff --git a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>>>> b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>>>> index dba9641947a3a..ed8ba75dbbce8 100644
>>>> --- a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>>>> +++ b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>>>> @@ -1721,6 +1721,31 @@ vopb_mmu: iommu@ff903f00 {
>>>> status = "disabled";
>>>> };
>>>>
>>>> +isp0: isp0@ff91 {
>>>> +compatible = "rockchip,rk3399-cif-isp";
>>>> +reg = <0x0 0xff91 0x0 0x4000>;
>>>> +interrupts = ;
>>>> +clocks = <&cru SCLK_ISP0>,
>>>> + <&cru ACLK_ISP0_WRAPPER>,
>>>> + <&cru HCLK_ISP0_WRAPPER>;
>>>> +clock-names = "isp", "aclk", "hclk";
>>>> +iommus = <&isp0_mmu>;
>>>> +phys = <&mipi_dphy_rx0>;
>>>> +phy-names = "dphy";
>>>> +power-domains = <&power RK3399_PD_ISP0>;
>>>
>>> Should this have status = "disabled" too? The mipi_dphy_rx0 node is
>>> disabled by default too, so in the default configuration the driver
>>> would always fail to probe.
>>
>> I'm thinking what is the overall guideline here.
>> Since isp and mipi_dphy are always present in the rk3399, shouldn't they
>> always be enabled?
>> Or since they are only useful if a sensor is present, we should let the dts
>> of the board to
>> enable it?
>
> I don't have a strong opinion. I'm fine with enabling both by default
> as well, as it shouldn't hurt.
>
> That said, I recall some alternative CIF IP block being present on
> this SoC as well (and patches posted recently), which AFAIR can't be
> activated at the same time as the ISP, so perhaps both of the
> alternatives should be disabled by default?
>
> Best regards,
> Tomasz
>
Based on these two last emails, I think it make sense to disable them by
default.
I'll just wait for feedback on patch 5/9 to submit an updated version.
Thanks
Helen
Re: [PATCH v5 5/9] media: staging: rkisp1: remove unecessary clocks
Hi Rob, Thnaks for your reply. On 9/22/20 11:24 AM, Rob Herring wrote: > On Wed, Jul 22, 2020 at 9:56 AM Helen Koike wrote: >> >> aclk_isp_wrap is a child of aclk_isp, and hclk_isp_wrap is a child of >> hclk_isp, thus we can remove parents from the list. >> >> Also, for the isp0, we only need the ISP clock, ACLK and HCLK. >> In the future we'll need a pixel clock for RK3288 and RK3399, and a JPEG >> clock for RK3288. >> >> So with the goal to cleanup the dt-bindings and remove it from staging, >> simplify clock names to isp, aclk and hclk. >> >> Assigned clocks are meant to refer to the full path in the clock tree, >> i.e. the leaf in the tree. >> For instance, in RK3399, the clock responsible for ACLK (ISP AXI CLOCK) >> is aclk_isp0_wrapper. >> >> For reference, this is the isp clock topology on RK3399: >> >> xin24m >> pll_npll >>npll >> clk_isp1 >> clk_isp0 >> pll_cpll >>cpll >> aclk_isp1 >> aclk_isp1_noc >> hclk_isp1 >> aclk_isp1_wrapper >> hclk_isp1_noc >> aclk_isp0 >> hclk_isp1_wrapper >> aclk_isp0_wrapper >> aclk_isp0_noc >> hclk_isp0 >> hclk_isp0_wrapper >> hclk_isp0_noc >> pclkin_isp1_wrapper >> >> Signed-off-by: Helen Koike >> >> --- >> Changes in V5: >> - Use if/then schema as suggested by Rob Herring on >> https://patchwork.linuxtv.org/project/linux-media/patch/20200702191322.2639681-6-helen.ko...@collabora.com/#119729 >> >> Changes in V4: >> - update binding according to suggestion by Robin Murphy >> on https://patchwork.kernel.org/patch/11475007/ >> >> Changes in V3: >> - this is a new patch in the series >> --- >> .../bindings/media/rockchip-isp1.yaml | 50 --- >> drivers/staging/media/rkisp1/rkisp1-dev.c | 8 ++- >> 2 files changed, 36 insertions(+), 22 deletions(-) >> >> diff --git >> a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml >> >> b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml >> index 62a6b9c959498..23c677d15037a 100644 >> --- >> a/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml >> +++ >> b/drivers/staging/media/rkisp1/Documentation/devicetree/bindings/media/rockchip-isp1.yaml >> @@ -24,20 +24,10 @@ properties: >> maxItems: 1 >> >>clocks: >> -items: >> - - description: ISP clock >> - - description: ISP AXI clock clock >> - - description: ISP AXI clock wrapper clock >> - - description: ISP AHB clock clock >> - - description: ISP AHB wrapper clock >> +minItems: 3 > > You need maxItems here too or it will always be 3. > >> >>clock-names: >> -items: >> - - const: clk_isp >> - - const: aclk_isp >> - - const: aclk_isp_wrap >> - - const: hclk_isp >> - - const: hclk_isp_wrap >> +minItems: 3 >> >>iommus: >> maxItems: 1 >> @@ -116,6 +106,34 @@ required: >>- power-domains >>- ports >> >> +if: >> + properties: >> +compatible: >> + contains: >> +const: rockchip,rk3399-cif-isp >> +then: >> + properties: >> +clocks: >> + maxItems: 4 >> + minItems: 3 > > For a single compatible you shouldn't really have a variable number of clocks. I'm not entirely sure how to make this separation, since isp0 and isp1 (not yet supported) would use the same compatible. Unless if we separate in two compatibles, but maybe this is an overhead just for an extra clock. What do you think? > >> + items: >> +# isp0 and isp1 >> +- description: ISP clock >> +- description: ISP AXI clock >> +- description: ISP AHB clock >> +# only for isp1 >> +- description: ISP Pixel clock >> +clock-names: >> + maxItems: 4 >> + minItems: 3 >> + items: >> +# isp0 and isp1 >> +- const: isp >> +- const: aclk >> +- const: hclk >> +# only for isp1 >> +- const: pclk_isp > > Don't you need an 'else' clause. For not rockchip,rk3399-cif-isp, > there's no definition of what clocks there are. There is only one compatible defined for now, rk3288 will be added later. The idea to add if/then is to make it easier to add rk3288: https://patchwork.kernel.org/project/linux-media/patch/20200406073017.19462-4-karthik.podu...@gmail.com/ Regards, Helen > > Rob >
Re: [PATCH v5 8/9] arm64: dts: rockchip: add isp0 node for rk3399
Hi Tomasz,
On 9/26/20 10:00 AM, Tomasz Figa wrote:
> Hi Helen,
>
> On Wed, Jul 22, 2020 at 12:55:32PM -0300, Helen Koike wrote:
>> From: Shunqian Zheng
>>
>> RK3399 has two ISPs, but only isp0 was tested.
>> Add isp0 node in rk3399 dtsi
>>
>> Verified with:
>> make ARCH=arm64 dtbs_check
>> DT_SCHEMA_FILES=Documentation/devicetree/bindings/media/rockchip-isp1.yaml
>>
>> Signed-off-by: Shunqian Zheng
>> Signed-off-by: Jacob Chen
>> Signed-off-by: Helen Koike
>>
>> ---
>>
>> V4:
>> - update clock names
>>
>> V3:
>> - clean up clocks
>>
>> V2:
>> - re-order power-domains property
>>
>> V1:
>> This patch was originally part of this patchset:
>>
>> https://patchwork.kernel.org/patch/10267431/
>>
>> The only difference is:
>> - add phy properties
>> - add ports
>> ---
>> arch/arm64/boot/dts/rockchip/rk3399.dtsi | 25
>> 1 file changed, 25 insertions(+)
>>
>> diff --git a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> index dba9641947a3a..ed8ba75dbbce8 100644
>> --- a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> +++ b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> @@ -1721,6 +1721,31 @@ vopb_mmu: iommu@ff903f00 {
>> status = "disabled";
>> };
>>
>> +isp0: isp0@ff91 {
>> +compatible = "rockchip,rk3399-cif-isp";
>> +reg = <0x0 0xff91 0x0 0x4000>;
>> +interrupts = ;
>> +clocks = <&cru SCLK_ISP0>,
>> + <&cru ACLK_ISP0_WRAPPER>,
>> + <&cru HCLK_ISP0_WRAPPER>;
>> +clock-names = "isp", "aclk", "hclk";
>> +iommus = <&isp0_mmu>;
>> +phys = <&mipi_dphy_rx0>;
>> +phy-names = "dphy";
>> +power-domains = <&power RK3399_PD_ISP0>;
>
> Should this have status = "disabled" too? The mipi_dphy_rx0 node is
> disabled by default too, so in the default configuration the driver
> would always fail to probe.
I'm thinking what is the overall guideline here.
Since isp and mipi_dphy are always present in the rk3399, shouldn't they always
be enabled?
Or since they are only useful if a sensor is present, we should let the dts of
the board to
enable it?
Thanks
Helen
>
> Best regards,
> Tomasz
>
Re: [PATCH v5 1/7] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
Hi Hans,
On 9/9/20 8:41 AM, Hans Verkuil wrote:
> Hi Helen,
>
> Some review comments, concentrating on the uAPI.
>
> On 04/08/2020 21:29, Helen Koike wrote:
>> This is part of the multiplanar and singleplanar unification process.
>> v4l2_ext_pix_format is supposed to work for both cases.
>>
>> We also add the concept of modifiers already employed in DRM to expose
>> HW-specific formats (like tiled or compressed formats) and allow
>> exchanging this information with the DRM subsystem in a consistent way.
>>
>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in
>> v4l2_ext_format, other types will be rejected if you use the
>> {G,S,TRY}_EXT_PIX_FMT ioctls.
>>
>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls
>> in drivers, but, in the meantime, the core takes care of converting
>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can
>> still work if the userspace app/lib uses the new ioctls.
>> The conversion is also done the other around to allow userspace
>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the
>> _ext_ hooks.
>>
>> Signed-off-by: Boris Brezillon
>> Signed-off-by: Helen Koike
>> ---
>> Changes in v5:
>> - change sizes and reorder fields to avoid holes in the struct and make
>> it the same for 32 and 64 bits
>> - removed __attribute__ ((packed)) from uapi structs
>> - Fix doc warning from make htmldocs
>> - Updated commit message with EXT_PIX prefix for the ioctls.
>>
>> Changes in v4:
>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
>> - Add reserved fields
>> - Removed num_planes from struct v4l2_ext_pix_format
>> - Removed flag field from struct v4l2_ext_pix_format, since the only
>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1,
>> where we can use modifiers, or add it back later through the reserved
>> bits.
>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR &&
>> != MOD_INVALID
>> - Fix type assignment in v4l_g_fmt_ext_pix()
>> - Rebased on top of media/master (post 5.8-rc1)
>>
>> Changes in v3:
>> - Rebased on top of media/master (post 5.4-rc1)
>>
>> Changes in v2:
>> - Move the modifier in v4l2_ext_format (was formerly placed in
>> v4l2_ext_plane)
>> - Fix a few bugs in the converters and add a strict parameter to
>> allow conversion of uninitialized/mis-initialized objects
>> ---
>> drivers/media/v4l2-core/v4l2-dev.c | 21 +-
>> drivers/media/v4l2-core/v4l2-ioctl.c | 585 +++
>> include/media/v4l2-ioctl.h | 34 ++
>> include/uapi/linux/videodev2.h | 56 +++
>> 4 files changed, 615 insertions(+), 81 deletions(-)
>>
>
>
>
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index c7b70ff53bc1d..7123c6a4d9569 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2254,6 +2254,58 @@ struct v4l2_pix_format_mplane {
>> __u8reserved[7];
>> } __attribute__ ((packed));
>>
>> +/**
>> + * struct v4l2_plane_ext_pix_format - additional, per-plane format
>> definition
>> + * @sizeimage: maximum size in bytes required for data, for
>> which
>> + * this plane will be used.
>> + * Should be set to zero for unused planes.
>> + * @bytesperline: distance in bytes between the leftmost pixels in two
>> + * adjacent lines
>> + * @reserved: extra space reserved for future fields, must be
>> set to 0
>> + */
>> +struct v4l2_plane_ext_pix_format {
>> +__u32 sizeimage;
>> +__u32 bytesperline;
>> +__u32 reserved;
>
> I'd use reserved[4] here.
>
>> +};
>> +
>> +/**
>> + * struct v4l2_ext_pix_format - extended single/multiplanar format
>> definition
>> + * @type: type of the data stream; V4L2_BUF_TYPE_VIDEO_CAPTURE or
>> + * V4L2_BUF_TYPE_VIDEO_OUTPUT
>> + * @width: image width in pixels
>> + * @height: image height in pixels
>> + * @field: enum v4l2_field; field order (for interlaced video)
>> + * @pixelformat:little endian four character code (fourcc)
>> + * @modifier: modifier applied to the format (used for tiled
>> formats
>> + *
[PATCH] MAINTAINERS: add Dafna Hirschfeld for rkisp1
Add Dafna Hirschfeld to rkisp1 maintainers list Signed-off-by: Helen Koike --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index deaafb617361c..3deb954b2bb5d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14858,6 +14858,7 @@ F: include/linux/hid-roccat* ROCKCHIP ISP V1 DRIVER M: Helen Koike +M: Dafna Hirschfeld L: linux-me...@vger.kernel.org S: Maintained F: drivers/staging/media/rkisp1/ -- 2.28.0.rc2
Re: [PATCH v2 0/3] media: vimc: Allow multiple capture devices to use the same sensor
Hi, On 7/29/20 12:24 PM, Dafna Hirschfeld wrote: > > > On 29.07.20 15:27, Kieran Bingham wrote: >> Hi Dafna, Kaaira, >> >> On 29/07/2020 14:16, Dafna Hirschfeld wrote: >>> >>> >>> On 29.07.20 15:05, Kieran Bingham wrote: >>>> Hi Dafna, >>>> >>>> On 28/07/2020 15:00, Dafna Hirschfeld wrote: >>>>> >>>>> >>>>> On 28.07.20 14:07, Dafna Hirschfeld wrote: >>>>>> Hi >>>>>> >>>>>> On 28.07.20 13:39, Kaaira Gupta wrote: >>>>>>> On Mon, Jul 27, 2020 at 02:54:30PM -0300, Helen Koike wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 7/27/20 11:31 AM, Kieran Bingham wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> +Dafna for the thread discussion, as she's missed from the to/cc >>>>>>>>> list. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 24/07/2020 13:21, Kaaira Gupta wrote: >>>>>>>>>> On Fri, Jul 24, 2020 at 02:15:21PM +0200, Niklas Söderlund wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>>> Hi Kaaira, >>>>>>>>>>> >>>>>>>>>>> Thanks for your work. >>>>>>>>>> >>>>>>>>>> Thanks for yours :D >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020-07-24 17:32:10 +0530, Kaaira Gupta wrote: >>>>>>>>>>>> This is version 2 of the patch series posted by Niklas for >>>>>>>>>>>> allowing >>>>>>>>>>>> multiple streams in VIMC. >>>>>>>>>>>> The original series can be found here: >>>>>>>>>>>> https://patchwork.kernel.org/cover/10948831/ >>>>>>>>>>>> >>>>>>>>>>>> This series adds support for two (or more) capture devices to be >>>>>>>>>>>> connected to the same sensors and run simultaneously. Each >>>>>>>>>>>> capture device >>>>>>>>>>>> can be started and stopped independent of each other. >>>>>>>>>>>> >>>>>>>>>>>> Patch 1/3 and 2/3 deals with solving the issues that arises once >>>>>>>>>>>> two >>>>>>>>>>>> capture devices can be part of the same pipeline. While 3/3 >>>>>>>>>>>> allows for >>>>>>>>>>>> two capture devices to be part of the same pipeline and thus >>>>>>>>>>>> allows for >>>>>>>>>>>> simultaneously use. >>>>>> >>>>>> I wonder if these two patches are enough, since each vimc entity also >>>>>> have >>>>>> a 'process_frame' callback, but only one allocated frame. That means >>>>>> that the 'process_frame' can be called concurrently by two different >>>>>> streams >>>>>> on the same frame and cause corruption. >>>>>> >>>>> >>>>> I think we should somehow change the vimc-stream.c code so that we have >>>>> only >>>>> one stream process per pipe. So if one capture is already streaming, >>>>> then the new >>>>> capture that wants to stream uses the same thread so we don't have two >>>>> threads >>>>> both calling 'process_frame'. >>>> >>>> >>>> Yes, I think it looks and sounds like there are two threads running when >>>> there are two streams. >>>> >>>> so in effect, although they 'share a pipe', aren't they in effect just >>>> sending two separate buffers through their stream-path? >>>> >>>> If that's the case, then I don't think there's any frame corruption, >>>> because they would both have grabbed their own frame separately. >>> >>> But each entity allocates just one buffer. So the same buffer is used for >>> both stream. >> >>
Re: [PATCH v5 0/7] media: v4l2: Add extended fmt and buffer ioctls
On 8/4/20 4:29 PM, Helen Koike wrote: > Hello, > > This is v5 of the Extended API for formats and buffers, which introduces > the following new ioctls: > > int ioctl(int fd, VIDIOC_G_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp) > int ioctl(int fd, VIDIOC_S_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp) > int ioctl(int fd, VIDIOC_TRY_EXT_PIX_FMT, struct v4l2_ext_pix_format *argp) > > int ioctl(int fd, VIDIOC_EXT_CREATE_BUFS, struct v4l2_ext_create_buffers > *argp) > int ioctl(int fd, VIDIOC_EXT_QUERYBUF, struct v4l2_ext_buffer *argp) > int ioctl(int fd, VIDIOC_EXT_QBUF, struct v4l2_ext_buffer *argp) > int ioctl(int fd, VIDIOC_EXT_DQBUF, struct v4l2_ext_buffer *argp) > int ioctl(int fd, VIDIOC_EXT_PREPARE_BUF, struct v4l2_ext_buffer *argp) > > Please check v4 cover letter specific topic past discussions > https://patchwork.linuxtv.org/project/linux-media/cover/20200717115435.2632623-1-helen.ko...@collabora.com/ > > Documentation > = > I added a first version of the documentation. > I would appreciate some tips in how to organize this better, since there are > several parts of the docs which reference the "old" API, and for now > I just added a note pointing to the Extended API. > > Instead of duplicating documentation from the code to the Docs (as used by > most part of v4l2 documentation), I just added a reference to let Sphinx get > the structs documentation from the code, so we can avoid duplicating. > > For reviewing convenience, I uploaded the generated html docs at > https://people.collabora.com/~koike/ext-doc-v5/userspace-api/media/v4l/ext-api.html > > There is room for improvements, it would be great to get your suggestions. > > uAPI > > This version have some minor changes in the uAPI structs, highlight to the > mem_offset that was returned to struct v4l2_ext_plane, memory field that now > is per plane, and some adjustments in field sizes and re-ordering to make > structs the same for 32 and 64 bits (which I verified with pahole tool). > > I also updated v4l2-compliance: > https://gitlab.collabora.com/koike/v4l-utils/-/tree/ext-api/wip > > We need to decide the size of the reserved fields left, how much reserved > fields do you think we should leave for each struct? > > What is next > > I would like to improve implementation (for the kernel and v4l2-compliane) and > drop the RFC tag for next version, so please review the uAPI. Ops, sorry, I forgot to add RFC tag when submitting. Please consider RFC for this one. Regards, Helen > > > List of changes in v5: > == > * first version of Documentation > * migrate memory from v4l2_ext_buffer to v4l2_ext_plane > * return mem_offset to struct v4l2_ext_plane > * change sizes and reorder fields to avoid holes in the struct and make it > the same for 32 and 64 bits > * removed __attribute__ ((packed)) from uapi structs > * set request_fd to signed > * add documentation > * updated some commit messages > > Hans Verkuil (1): > media: v4l2: Add extended buffer operations > > Helen Koike (6): > media: v4l2: Extend pixel formats to unify single/multi-planar > handling (and more) > media: videobuf2: Expose helpers to implement the _ext_fmt and > _ext_buf hooks > media: mediabus: Add helpers to convert a ext_pix format to/from a > mbus_fmt > media: vivid: Convert the capture and output drivers to > EXT_FMT/EXT_BUF > media: vimc: Implement the ext_fmt and ext_buf hooks > media: docs: add documentation for the Extended API > > .../userspace-api/media/v4l/buffer.rst| 5 + > .../userspace-api/media/v4l/common.rst| 1 + > .../userspace-api/media/v4l/dev-capture.rst | 5 + > .../userspace-api/media/v4l/dev-output.rst| 5 + > .../userspace-api/media/v4l/ext-api.rst | 107 ++ > .../userspace-api/media/v4l/format.rst| 16 +- > .../userspace-api/media/v4l/user-func.rst | 5 + > .../media/v4l/vidioc-ext-create-bufs.rst | 95 ++ > .../media/v4l/vidioc-ext-prepare-buf.rst | 62 ++ > .../media/v4l/vidioc-ext-qbuf.rst | 204 > .../media/v4l/vidioc-ext-querybuf.rst | 79 ++ > .../media/v4l/vidioc-g-ext-pix-fmt.rst| 117 +++ > .../media/common/videobuf2/videobuf2-core.c | 2 + > .../media/common/videobuf2/videobuf2-v4l2.c | 560 ++- > .../media/test-drivers/vimc/vimc-capture.c| 61 +- > drivers/media/test-drivers/vimc/vimc-common.c | 6 +- > drivers/media/test-drivers/vimc/vimc-common.h | 2 +- > drivers/media/test-drivers/vivid/vivid-core.c | 70 +- > .../test-drivers/vivid/vivid-touch-cap.c | 26 +- > .../test-drivers/vivid/vivid-tou
[PATCH v5 2/7] media: v4l2: Add extended buffer operations
From: Hans Verkuil
Those extended buffer ops have several purpose:
1/ Fix y2038 issues by converting the timestamp into an u64 counting
the number of ns elapsed since 1970
2/ Unify single/multiplanar handling
3/ Add a new start offset field to each v4l2 plane buffer info struct
to support the case where a single buffer object is storing all
planes data, each one being placed at a different offset
New hooks are created in v4l2_ioctl_ops so that drivers can start using
these new objects.
The core takes care of converting new ioctls requests to old ones
if the driver does not support the new hooks, and vice versa.
Note that the timecode field is gone, since there doesn't seem to be
in-kernel users. We can be added back in the reserved area if needed or
use the Request API to collect more metadata information from the
frame.
Signed-off-by: Hans Verkuil
Signed-off-by: Boris Brezillon
Signed-off-by: Helen Koike
---
Changes in v5:
- migrate memory from v4l2_ext_buffer to v4l2_ext_plane
- return mem_offset to struct v4l2_ext_plane
- change sizes and reorder fields to avoid holes in the struct and make
it the same for 32 and 64 bits
Changes in v4:
- Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
- Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF
was that with VIDIOC_EXT_EXPBUF we could export multiple planes at once.
I think we can add this later, so I removed it from this RFC to simplify it.
- Remove num_planes field from struct v4l2_ext_buffer
- Add flags field to struct v4l2_ext_create_buffers
- Reformulate struct v4l2_ext_plane
- Fix some bugs caught by v4l2-compliance
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- Add reserved space to v4l2_ext_buffer so that new fields can be added
later on
---
drivers/media/v4l2-core/v4l2-dev.c | 29 ++-
drivers/media/v4l2-core/v4l2-ioctl.c | 353 +--
include/media/v4l2-ioctl.h | 26 ++
include/uapi/linux/videodev2.h | 90 +++
4 files changed, 476 insertions(+), 22 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-dev.c
b/drivers/media/v4l2-core/v4l2-dev.c
index e1829906bc086..cb21ee8eb075c 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -720,15 +720,34 @@ static void determine_valid_ioctls(struct video_device
*vdev)
SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_sdr_out);
}
+ if (is_vid || is_tch) {
+ /* ioctls valid for video and touch */
+ if (ops->vidioc_querybuf || ops->vidioc_ext_querybuf)
+ set_bit(_IOC_NR(VIDIOC_EXT_QUERYBUF), valid_ioctls);
+ if (ops->vidioc_qbuf || ops->vidioc_ext_qbuf)
+ set_bit(_IOC_NR(VIDIOC_EXT_QBUF), valid_ioctls);
+ if (ops->vidioc_dqbuf || ops->vidioc_ext_dqbuf)
+ set_bit(_IOC_NR(VIDIOC_EXT_DQBUF), valid_ioctls);
+ if (ops->vidioc_create_bufs || ops->vidioc_ext_create_bufs)
+ set_bit(_IOC_NR(VIDIOC_EXT_CREATE_BUFS), valid_ioctls);
+ if (ops->vidioc_prepare_buf || ops->vidioc_ext_prepare_buf)
+ set_bit(_IOC_NR(VIDIOC_EXT_PREPARE_BUF), valid_ioctls);
+ }
+
if (is_vid || is_vbi || is_sdr || is_tch || is_meta) {
/* ioctls valid for video, vbi, sdr, touch and metadata */
SET_VALID_IOCTL(ops, VIDIOC_REQBUFS, vidioc_reqbufs);
- SET_VALID_IOCTL(ops, VIDIOC_QUERYBUF, vidioc_querybuf);
- SET_VALID_IOCTL(ops, VIDIOC_QBUF, vidioc_qbuf);
SET_VALID_IOCTL(ops, VIDIOC_EXPBUF, vidioc_expbuf);
- SET_VALID_IOCTL(ops, VIDIOC_DQBUF, vidioc_dqbuf);
- SET_VALID_IOCTL(ops, VIDIOC_CREATE_BUFS, vidioc_create_bufs);
- SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
+ if (ops->vidioc_querybuf || ops->vidioc_ext_querybuf)
+ set_bit(_IOC_NR(VIDIOC_QUERYBUF), valid_ioctls);
+ if (ops->vidioc_qbuf || ops->vidioc_ext_qbuf)
+ set_bit(_IOC_NR(VIDIOC_QBUF), valid_ioctls);
+ if (ops->vidioc_dqbuf || ops->vidioc_ext_dqbuf)
+ set_bit(_IOC_NR(VIDIOC_DQBUF), valid_ioctls);
+ if (ops->vidioc_create_bufs || ops->vidioc_ext_create_bufs)
+ set_bit(_IOC_NR(VIDIOC_CREATE_BUFS), valid_ioctls);
+ if (ops->vidioc_prepare_buf || ops->vidioc_ext_prepare_buf)
+ set_bit(_IOC_NR(VIDIOC_PREPARE_BUF), valid_ioctls);
SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
}
diff -
[PATCH v5 7/7] media: docs: add documentation for the Extended API
Add documentation and update references in current documentation for the Extended API. Signed-off-by: Helen Koike --- Changes in v5: - new patch .../userspace-api/media/v4l/buffer.rst| 5 + .../userspace-api/media/v4l/common.rst| 1 + .../userspace-api/media/v4l/dev-capture.rst | 5 + .../userspace-api/media/v4l/dev-output.rst| 5 + .../userspace-api/media/v4l/ext-api.rst | 107 + .../userspace-api/media/v4l/format.rst| 16 +- .../userspace-api/media/v4l/user-func.rst | 5 + .../media/v4l/vidioc-ext-create-bufs.rst | 95 .../media/v4l/vidioc-ext-prepare-buf.rst | 62 ++ .../media/v4l/vidioc-ext-qbuf.rst | 204 ++ .../media/v4l/vidioc-ext-querybuf.rst | 79 +++ .../media/v4l/vidioc-g-ext-pix-fmt.rst| 117 ++ 12 files changed, 697 insertions(+), 4 deletions(-) create mode 100644 Documentation/userspace-api/media/v4l/ext-api.rst create mode 100644 Documentation/userspace-api/media/v4l/vidioc-ext-create-bufs.rst create mode 100644 Documentation/userspace-api/media/v4l/vidioc-ext-prepare-buf.rst create mode 100644 Documentation/userspace-api/media/v4l/vidioc-ext-qbuf.rst create mode 100644 Documentation/userspace-api/media/v4l/vidioc-ext-querybuf.rst create mode 100644 Documentation/userspace-api/media/v4l/vidioc-g-ext-pix-fmt.rst diff --git a/Documentation/userspace-api/media/v4l/buffer.rst b/Documentation/userspace-api/media/v4l/buffer.rst index 57e752aaf414a..c832bedd64e4c 100644 --- a/Documentation/userspace-api/media/v4l/buffer.rst +++ b/Documentation/userspace-api/media/v4l/buffer.rst @@ -27,6 +27,11 @@ such as pointers and sizes for each plane, are stored in struct :c:type:`v4l2_plane` instead. In that case, struct :c:type:`v4l2_buffer` contains an array of plane structures. +.. note:: + +The :ref:`ext_api` version can also be used, and it is +preferable when applicable. + Dequeued video buffers come with timestamps. The driver decides at which part of the frame and with which clock the timestamp is taken. Please see flags in the masks ``V4L2_BUF_FLAG_TIMESTAMP_MASK`` and diff --git a/Documentation/userspace-api/media/v4l/common.rst b/Documentation/userspace-api/media/v4l/common.rst index 7d81c58a13cd7..3430e0bdad667 100644 --- a/Documentation/userspace-api/media/v4l/common.rst +++ b/Documentation/userspace-api/media/v4l/common.rst @@ -59,6 +59,7 @@ applicable to all devices. ext-ctrls-detect fourcc format +ext-api planar-apis selection-api crop diff --git a/Documentation/userspace-api/media/v4l/dev-capture.rst b/Documentation/userspace-api/media/v4l/dev-capture.rst index 44d3094093ab6..5077639787d92 100644 --- a/Documentation/userspace-api/media/v4l/dev-capture.rst +++ b/Documentation/userspace-api/media/v4l/dev-capture.rst @@ -102,6 +102,11 @@ and :ref:`VIDIOC_S_FMT ` ioctl, even if :ref:`VIDIOC_S_FMT ` does. :ref:`VIDIOC_TRY_FMT ` is optional. +.. note:: + +The :ref:`ext_api` version can also be used, and it is +preferable when applicable. + Reading Images == diff --git a/Documentation/userspace-api/media/v4l/dev-output.rst b/Documentation/userspace-api/media/v4l/dev-output.rst index e4f2a1d8b0fc7..f8f40c708e49f 100644 --- a/Documentation/userspace-api/media/v4l/dev-output.rst +++ b/Documentation/userspace-api/media/v4l/dev-output.rst @@ -99,6 +99,11 @@ and :ref:`VIDIOC_S_FMT ` ioctl, even if :ref:`VIDIOC_S_FMT ` does. :ref:`VIDIOC_TRY_FMT ` is optional. +.. note:: + +The :ref:`ext_api` version can also be used, and it is +preferable when applicable. + Writing Images == diff --git a/Documentation/userspace-api/media/v4l/ext-api.rst b/Documentation/userspace-api/media/v4l/ext-api.rst new file mode 100644 index 0..da2a82960d22f --- /dev/null +++ b/Documentation/userspace-api/media/v4l/ext-api.rst @@ -0,0 +1,107 @@ +.. Permission is granted to copy, distribute and/or modify this +.. document under the terms of the GNU Free Documentation License, +.. Version 1.1 or any later version published by the Free Software +.. Foundation, with no Invariant Sections, no Front-Cover Texts +.. and no Back-Cover Texts. A copy of the license is included at +.. Documentation/userspace-api/media/fdl-appendix.rst. +.. +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections + +.. _ext_api: + +* +Extendend API +* + +Introduction + + +The Extended Format API was conceived to extend V4L2 capabilities and +to simplify certain mechanisms. +It unifies single- vs multi- planar handling, +brings the concept of pixelformat + modifiers, allows planes to be placed +in any offset inside a buffer and let userspace to change colorspace +attributes if supported by the driver. + +Data format negotiation and buffer handling works very similar to the classical +version, thus in this document we'll just mention the
[PATCH v5 4/7] media: mediabus: Add helpers to convert a ext_pix format to/from a mbus_fmt
Just a new version of v4l2_fill_mbus_format() and v4l2_fill_ext_pix_format()
to deal with the new v4l2_ext_pix_format struct.
This is needed to convert the VIMC driver to the EXT_FMT/EXT_BUF iocts.
Signed-off-by: Boris Brezillon
Signed-off-by: Helen Koike
---
Changes in v4:
- Add helper v4l2_fill_ext_pix_format()
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- New patch
---
include/media/v4l2-mediabus.h | 42 +++
1 file changed, 42 insertions(+)
diff --git a/include/media/v4l2-mediabus.h b/include/media/v4l2-mediabus.h
index 45f88f0248c4e..8133407377f7d 100644
--- a/include/media/v4l2-mediabus.h
+++ b/include/media/v4l2-mediabus.h
@@ -119,6 +119,26 @@ v4l2_fill_pix_format(struct v4l2_pix_format *pix_fmt,
pix_fmt->xfer_func = mbus_fmt->xfer_func;
}
+/**
+ * v4l2_fill_ext_pix_format - Ancillary routine that fills a &struct
+ * v4l2_ext_pix_format fields from a &struct v4l2_mbus_framefmt.
+ *
+ * @pix_fmt: pointer to &struct v4l2_ext_pix_format to be filled
+ * @mbus_fmt: pointer to &struct v4l2_mbus_framefmt to be used as model
+ */
+static inline void
+v4l2_fill_ext_pix_format(struct v4l2_ext_pix_format *pix_fmt,
+const struct v4l2_mbus_framefmt *mbus_fmt)
+{
+ pix_fmt->width = mbus_fmt->width;
+ pix_fmt->height = mbus_fmt->height;
+ pix_fmt->field = mbus_fmt->field;
+ pix_fmt->colorspace = mbus_fmt->colorspace;
+ pix_fmt->ycbcr_enc = mbus_fmt->ycbcr_enc;
+ pix_fmt->quantization = mbus_fmt->quantization;
+ pix_fmt->xfer_func = mbus_fmt->xfer_func;
+}
+
/**
* v4l2_fill_pix_format - Ancillary routine that fills a &struct
* v4l2_mbus_framefmt from a &struct v4l2_pix_format and a
@@ -182,4 +202,26 @@ v4l2_fill_mbus_format_mplane(struct v4l2_mbus_framefmt
*mbus_fmt,
mbus_fmt->xfer_func = pix_mp_fmt->xfer_func;
}
+/**
+ * v4l2_fill_mbus_format_ext - Ancillary routine that fills a &struct
+ * v4l2_mbus_framefmt from a &struct v4l2_ext_pix_format.
+ *
+ * @mbus_fmt: pointer to &struct v4l2_mbus_framefmt to be filled
+ * @pix_fmt: pointer to &struct v4l2_ext_pix_format to be used as model
+ * @code: data format code (from &enum v4l2_mbus_pixelcode)
+ */
+static inline void
+v4l2_fill_mbus_format_ext(struct v4l2_mbus_framefmt *mbus_fmt,
+ const struct v4l2_ext_pix_format *pix_fmt, u32 code)
+{
+ mbus_fmt->width = pix_fmt->width;
+ mbus_fmt->height = pix_fmt->height;
+ mbus_fmt->field = pix_fmt->field;
+ mbus_fmt->colorspace = pix_fmt->colorspace;
+ mbus_fmt->ycbcr_enc = pix_fmt->ycbcr_enc;
+ mbus_fmt->quantization = pix_fmt->quantization;
+ mbus_fmt->xfer_func = pix_fmt->xfer_func;
+ mbus_fmt->code = code;
+}
+
#endif
--
2.28.0.rc2
