Re: [PATCH 1/4] v4l: vsp1: Implement partition algorithm restrictions

[Kuninori Morimoto]

Hi Laurent, Kieran

> > Testing SRU-UDS scaling 768x576 - 768x576 - 640x480 in RGB24: fail
> > Testing SRU-UDS scaling 768x576 - 768x576 - 768x576 in RGB24: pass
> > Testing SRU-UDS scaling 768x576 - 768x576 - 1024x768 in RGB24: pass
> > Testing SRU-UDS scaling 768x576 - 1536x1152 - 1280x960 in RGB24: pass
> > Testing SRU-UDS scaling 768x576 - 1536x1152 - 1536x1152 in RGB24: pass
> > Testing SRU-UDS scaling 768x576 - 1536x1152 - 2048x1536 in RGB24: pass
> > Testing UDS-SRU scaling 768x576 - 640x480 - 640x480 in RGB24: pass
> > Testing UDS-SRU scaling 768x576 - 640x480 - 1280x960 in RGB24: pass
> > Testing UDS-SRU scaling 768x576 - 768x576 - 768x576 in RGB24: pass
> > Testing UDS-SRU scaling 768x576 - 768x576 - 1536x1152 in RGB24: pass
> > Testing UDS-SRU scaling 768x576 - 1024x768 - 1024x768 in RGB24: pass
> > Testing UDS-SRU scaling 768x576 - 1024x768 - 2048x1536 in RGB24: pass
> > Testing SRU-UDS scaling 768x576 - 768x576 - 640x480 in YUV444M: fail
> > Testing SRU-UDS scaling 768x576 - 768x576 - 768x576 in YUV444M: pass
> > Testing SRU-UDS scaling 768x576 - 768x576 - 1024x768 in YUV444M: pass
> > Testing SRU-UDS scaling 768x576 - 1536x1152 - 1280x960 in YUV444M: pass
> > Testing SRU-UDS scaling 768x576 - 1536x1152 - 1536x1152 in YUV444M: pass
> > Testing SRU-UDS scaling 768x576 - 1536x1152 - 2048x1536 in YUV444M: hangs
> > Testing UDS-SRU scaling 768x576 - 640x480 - 640x480 in YUV444M: pass
> > Testing UDS-SRU scaling 768x576 - 640x480 - 1280x960 in YUV444M: fail
> > Testing UDS-SRU scaling 768x576 - 768x576 - 768x576 in YUV444M: pass
> > Testing UDS-SRU scaling 768x576 - 768x576 - 1536x1152 in YUV444M: pass
> > Testing UDS-SRU scaling 768x576 - 1024x768 - 1024x768 in YUV444M: pass
> > Testing UDS-SRU scaling 768x576 - 1024x768 - 2048x1536 in YUV444M: hangs
> (snip)
> > However, from the above tests it looks like the hardware can live with more 
> > relaxed restrictions than the ones implemented here. I haven't tested all 
> > UDS 
> > scaling ratios, and certainly not under all memory bus load conditions, I 
> > might thus be too optimistic. Morimoto-san, would it be possible to get 
> > more 
> > information about this from the hardware team, to check whether the above 
> > two 
> > restrictions need to be honoured, or whether they come from an older 
> > hardware 
> > version ?
> 
> I asked it to HW team.
> Please wait

We still not yet get clear answer from HW team.
It is still researching

Best regards
---
Kuninori Morimoto

[PATCH] v4l: vsp1: Disable HSV formats on Gen3 hardware

[Laurent Pinchart]

While all VSP instances can process HSV internally, on Gen3 hardware
reading or writing HSV24 or HSV32 from/to memory causes the device to
hang. Disable those pixel formats on Gen3 hardware.

Signed-off-by: Laurent Pinchart 
---
 drivers/media/platform/vsp1/vsp1_pipe.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c 
b/drivers/media/platform/vsp1/vsp1_pipe.c
index 3f1acf68dc6e..35364f594e19 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -157,9 +157,15 @@ const struct vsp1_format_info *vsp1_get_format_info(struct 
vsp1_device *vsp1,
 {
unsigned int i;
 
-   /* Special case, the VYUY format is supported on Gen2 only. */
-   if (vsp1->info->gen != 2 && fourcc == V4L2_PIX_FMT_VYUY)
-   return NULL;
+   /* Special case, the VYUY and HSV formats are supported on Gen2 only. */
+   if (vsp1->info->gen != 2) {
+   switch (fourcc) {
+   case V4L2_PIX_FMT_VYUY:
+   case V4L2_PIX_FMT_HSV24:
+   case V4L2_PIX_FMT_HSV32:
+   return NULL;
+   }
+   }
 
for (i = 0; i < ARRAY_SIZE(vsp1_video_formats); ++i) {
const struct vsp1_format_info *info = _video_formats[i];
-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 3/3] v4l: vsp1: wpf: Implement rotation support

[Laurent Pinchart]

Hi Sakari,

On Tuesday 28 Feb 2017 23:13:34 Sakari Ailus wrote:
> On Tue, Feb 28, 2017 at 05:03:20PM +0200, Laurent Pinchart wrote:
> > Some WPF instances, on Gen3 devices, can perform 90° rotation when
> > writing frames to memory. Implement support for this using the
> > V4L2_CID_ROTATE control.
> > 
> > Signed-off-by: Laurent Pinchart
> > 
> > ---
> > 
> >  drivers/media/platform/vsp1/vsp1_rpf.c   |   2 +-
> >  drivers/media/platform/vsp1/vsp1_rwpf.c  |   5 +
> >  drivers/media/platform/vsp1/vsp1_rwpf.h  |   3 +-
> >  drivers/media/platform/vsp1/vsp1_video.c |  12 +-
> >  drivers/media/platform/vsp1/vsp1_wpf.c   | 205 ++
> >  5 files changed, 175 insertions(+), 52 deletions(-)

[snip]

> > diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h
> > b/drivers/media/platform/vsp1/vsp1_rwpf.h index
> > 1c98aff3da5d..b4ffc38f48af 100644
> > --- a/drivers/media/platform/vsp1/vsp1_rwpf.h
> > +++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
> > @@ -56,9 +56,10 @@ struct vsp1_rwpf {
> > 
> > struct {
> > spinlock_t lock;
> > -   struct v4l2_ctrl *ctrls[2];
> > +   struct v4l2_ctrl *ctrls[3];
> 
> At least what comes to this patch --- having a field for each control would
> look much nicer in the code. Is there a particular reason for having an
> array with all the controls in it?

Not really. I need to put the three controls in a cluster, so I stored them in 
an array to make sure they would be properly contiguous in memory. I can also 
use three separate pointers, it shouldn't hurt.

> > unsigned int pending;
> > unsigned int active;
> > +   bool rotate;
> > } flip;
> > 
> > struct vsp1_rwpf_memory mem;

-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 3/3] v4l: vsp1: wpf: Implement rotation support

[Sakari Ailus]

Hi Laurent,

On Tue, Feb 28, 2017 at 05:03:20PM +0200, Laurent Pinchart wrote:
> Some WPF instances, on Gen3 devices, can perform 90° rotation when
> writing frames to memory. Implement support for this using the
> V4L2_CID_ROTATE control.
> 
> Signed-off-by: Laurent Pinchart 
> ---
>  drivers/media/platform/vsp1/vsp1_rpf.c   |   2 +-
>  drivers/media/platform/vsp1/vsp1_rwpf.c  |   5 +
>  drivers/media/platform/vsp1/vsp1_rwpf.h  |   3 +-
>  drivers/media/platform/vsp1/vsp1_video.c |  12 +-
>  drivers/media/platform/vsp1/vsp1_wpf.c   | 205 
> +++
>  5 files changed, 175 insertions(+), 52 deletions(-)
> 
> diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c 
> b/drivers/media/platform/vsp1/vsp1_rpf.c
> index f5a9a4c8c74d..8feddd59cf8d 100644
> --- a/drivers/media/platform/vsp1/vsp1_rpf.c
> +++ b/drivers/media/platform/vsp1/vsp1_rpf.c
> @@ -106,7 +106,7 @@ static void rpf_configure(struct vsp1_entity *entity,
>* of the pipeline.
>*/
>   output = vsp1_entity_get_pad_format(wpf, wpf->config,
> - RWPF_PAD_SOURCE);
> + RWPF_PAD_SINK);
>  
>   crop.width = pipe->partition.width * input_width
>  / output->width;
> diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c 
> b/drivers/media/platform/vsp1/vsp1_rwpf.c
> index 7d52c88a583e..cfd8f1904fa6 100644
> --- a/drivers/media/platform/vsp1/vsp1_rwpf.c
> +++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
> @@ -121,6 +121,11 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev 
> *subdev,
>   RWPF_PAD_SOURCE);
>   *format = fmt->format;
>  
> + if (rwpf->flip.rotate) {
> + format->width = fmt->format.height;
> + format->height = fmt->format.width;
> + }
> +
>  done:
>   mutex_unlock(>entity.lock);
>   return ret;
> diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h 
> b/drivers/media/platform/vsp1/vsp1_rwpf.h
> index 1c98aff3da5d..b4ffc38f48af 100644
> --- a/drivers/media/platform/vsp1/vsp1_rwpf.h
> +++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
> @@ -56,9 +56,10 @@ struct vsp1_rwpf {
>  
>   struct {
>   spinlock_t lock;
> - struct v4l2_ctrl *ctrls[2];
> + struct v4l2_ctrl *ctrls[3];

At least what comes to this patch --- having a field for each control would
look much nicer in the code. Is there a particular reason for having an
array with all the controls in it?

>   unsigned int pending;
>   unsigned int active;
> + bool rotate;
>   } flip;
>  
>   struct vsp1_rwpf_memory mem;

-- 
Regards,

Sakari Ailus
e-mail: sakari.ai...@iki.fi XMPP: sai...@retiisi.org.uk

Re: [PATCH 1/1] clk: renesas: cpg-mssr: r8a7796: add IMR clocks

[Geert Uytterhoeven]

Hi Sergei,

On Tue, Feb 21, 2017 at 8:49 PM, Sergei Shtylyov
 wrote:
> On 02/21/2017 10:39 PM, Geert Uytterhoeven wrote:
>>> On 02/20/2017 11:38 AM, Geert Uytterhoeven wrote:
> Add the IMR[0-1] clocks to the R8A7796 CPG/MSSR driver.
>
> Signed-off-by: Sergei Shtylyov 

 Reviewed-by: Geert Uytterhoeven 
> --- linux.orig/drivers/clk/renesas/r8a7796-cpg-mssr.c
> +++ linux/drivers/clk/renesas/r8a7796-cpg-mssr.c
> @@ -179,6 +179,8 @@ static const struct mssr_mod_clk r8a7796
> DEF_MOD("vin1",  810,   R8A7796_CLK_S0D2),
> DEF_MOD("vin0",  811,   R8A7796_CLK_S0D2),
> DEF_MOD("etheravb",  812,   R8A7796_CLK_S0D6),
> +   DEF_MOD("imr1",  822,   R8A7796_CLK_S2D1),
> +   DEF_MOD("imr0",  823,   R8A7796_CLK_S2D1),

 According to R-Car Gen3 Hardware Users Manual Rev.0.53E, the parent
 clock is S0D2, like on H3 ES2.0.
>>>
>>>Not S0D4? I'd expect that what they call "register access" to be
>>> controlled by the MSSR bits...

I think in reality, both are gated by the same MSSR bit ;-)

However, if the driver needs to know the frequency of the clock, or control it
(e.g. it's driving some external interface), it's always interested in the
module's internal clock, not the clock used for register access.  The latter
is just an artifact of synchronous hardware.

>> Good question...

I've just finished going over all modules and their (known) parents on R-Car
H3 (ES1.x and ES2.0), and M3-W.
If multiple clock inputs are known, we always use the internal clock, which
is consistent with my comment above.

 Will queue in clk-renesas-for-v4.12 with corrected parent clock.
>>>
>>>So, what have you used for 7795 and 7796?
>>
>> S2D1 for r8a7795 (H3 ES1.0)
>> S0D2 for r8a7796 (M3-W)
>>
>> And I planned to use S0D2 for H3 ES2.0, too.
>
>Looks to me we should be using S0D4 for both. And perhaps the 2nd
> specifier for S0D2.

We typically don't list all other parents clocks, unless we need to
query/control them (especially if they can't be gated from Linux, like the
internal SxDy clocks).

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

[PATCH v3 0/8] R-Car VSP1 Histogram Support

[Laurent Pinchart]

Hello,

This patch series implements support for the Renesas R-Car VSP1 1-D and 2-D
histogram generators (HGO and HGT). It is based on top of the VSP1 rotation
patches and available for convenience at

git://linuxtv.org/pinchartl/media.git vsp1-histogram-v3-20170228

The series starts with the implementation and documentation of the new V4L2
metadata API (1/8), followed by three VSP1 patches that add generic histogram
support to the driver (2/8 to 4/8). Patches 5/8 and 7/8 then add new pixel
formats for the R-Car VSP1 1-D and 2-D histograms, and patches 6/8 and 8/8
implement support for those histogram generators in the VSP1 driver.

Laurent Pinchart (6):
  v4l: Add metadata buffer type and format
  v4l: vsp1: Add histogram support
  v4l: vsp1: Support histogram generators in pipeline configuration
  v4l: vsp1: Fix HGO and HGT routing register addresses
  v4l: Define a pixel format for the R-Car VSP1 1-D histogram engine
  v4l: vsp1: Add HGO support

Niklas Söderlund (2):
  v4l: Define a pixel format for the R-Car VSP1 2-D histogram engine
  v4l: vsp1: Add HGT support

 Documentation/media/uapi/v4l/buffer.rst|   3 +
 Documentation/media/uapi/v4l/dev-meta.rst  |  62 ++
 Documentation/media/uapi/v4l/devices.rst   |   1 +
 Documentation/media/uapi/v4l/meta-formats.rst  |  16 +
 .../media/uapi/v4l/pixfmt-meta-vsp1-hgo.rst| 168 ++
 .../media/uapi/v4l/pixfmt-meta-vsp1-hgt.rst| 120 
 Documentation/media/uapi/v4l/pixfmt.rst|   1 +
 Documentation/media/uapi/v4l/vidioc-querycap.rst   |   3 +
 Documentation/media/videodev2.h.rst.exceptions |   2 +
 drivers/media/platform/Kconfig |   1 +
 drivers/media/platform/vsp1/Makefile   |   1 +
 drivers/media/platform/vsp1/vsp1.h |   6 +
 drivers/media/platform/vsp1/vsp1_drm.c |   2 +-
 drivers/media/platform/vsp1/vsp1_drv.c |  70 ++-
 drivers/media/platform/vsp1/vsp1_entity.c  | 154 -
 drivers/media/platform/vsp1/vsp1_entity.h  |   8 +-
 drivers/media/platform/vsp1/vsp1_hgo.c | 228 
 drivers/media/platform/vsp1/vsp1_hgo.h |  45 ++
 drivers/media/platform/vsp1/vsp1_hgt.c | 222 +++
 drivers/media/platform/vsp1/vsp1_hgt.h |  42 ++
 drivers/media/platform/vsp1/vsp1_histo.c   | 646 +
 drivers/media/platform/vsp1/vsp1_histo.h   |  84 +++
 drivers/media/platform/vsp1/vsp1_pipe.c|  38 +-
 drivers/media/platform/vsp1/vsp1_pipe.h|   4 +
 drivers/media/platform/vsp1/vsp1_regs.h|  33 +-
 drivers/media/platform/vsp1/vsp1_video.c   |  30 +-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c  |  19 +
 drivers/media/v4l2-core/v4l2-dev.c |  16 +-
 drivers/media/v4l2-core/v4l2-ioctl.c   |  36 ++
 drivers/media/v4l2-core/videobuf2-v4l2.c   |   3 +
 include/media/v4l2-ioctl.h |  17 +
 include/trace/events/v4l2.h|   1 +
 include/uapi/linux/videodev2.h |  17 +
 33 files changed, 2051 insertions(+), 48 deletions(-)
 create mode 100644 Documentation/media/uapi/v4l/dev-meta.rst
 create mode 100644 Documentation/media/uapi/v4l/meta-formats.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgo.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgt.rst
 create mode 100644 drivers/media/platform/vsp1/vsp1_hgo.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_hgo.h
 create mode 100644 drivers/media/platform/vsp1/vsp1_hgt.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_hgt.h
 create mode 100644 drivers/media/platform/vsp1/vsp1_histo.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_histo.h

-- 
Regards,

Laurent Pinchart

[PATCH v3 5/8] v4l: Define a pixel format for the R-Car VSP1 1-D histogram engine

[Laurent Pinchart]

The format is used on the R-Car VSP1 video queues that carry
1-D histogram statistics data.

Signed-off-by: Laurent Pinchart 
Acked-by: Sakari Ailus 
---
 Documentation/media/uapi/v4l/meta-formats.rst  |  15 ++
 .../media/uapi/v4l/pixfmt-meta-vsp1-hgo.rst| 168 +
 Documentation/media/uapi/v4l/pixfmt.rst|   1 +
 drivers/media/v4l2-core/v4l2-ioctl.c   |   1 +
 include/uapi/linux/videodev2.h |   3 +
 5 files changed, 188 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/meta-formats.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgo.rst

diff --git a/Documentation/media/uapi/v4l/meta-formats.rst 
b/Documentation/media/uapi/v4l/meta-formats.rst
new file mode 100644
index ..05ab91e12f10
--- /dev/null
+++ b/Documentation/media/uapi/v4l/meta-formats.rst
@@ -0,0 +1,15 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _meta-formats:
+
+
+Metadata Formats
+
+
+These formats are used for the :ref:`metadata` interface only.
+
+
+.. toctree::
+:maxdepth: 1
+
+pixfmt-meta-vsp1-hgo
diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgo.rst 
b/Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgo.rst
new file mode 100644
index ..8d37bb313493
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgo.rst
@@ -0,0 +1,168 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-meta-fmt-vsp1-hgo:
+
+***
+V4L2_META_FMT_VSP1_HGO ('VSPH')
+***
+
+Renesas R-Car VSP1 1-D Histogram Data
+
+
+Description
+===
+
+This format describes histogram data generated by the Renesas R-Car VSP1 1-D
+Histogram (HGO) engine.
+
+The VSP1 HGO is a histogram computation engine that can operate on RGB, YCrCb
+or HSV data. It operates on a possibly cropped and subsampled input image and
+computes the minimum, maximum and sum of all pixels as well as per-channel
+histograms.
+
+The HGO can compute histograms independently per channel, on the maximum of the
+three channels (RGB data only) or on the Y channel only (YCbCr only). It can
+additionally output the histogram with 64 or 256 bins, resulting in four
+possible modes of operation.
+
+- In *64 bins normal mode*, the HGO operates on the three channels 
independently
+  to compute three 64-bins histograms. RGB, YCbCr and HSV image formats are
+  supported.
+- In *64 bins maximum mode*, the HGO operates on the maximum of the (R, G, B)
+  channels to compute a single 64-bins histogram. Only the RGB image format is
+  supported.
+- In *256 bins normal mode*, the HGO operates on the Y channel to compute a
+  single 256-bins histogram. Only the YCbCr image format is supported.
+- In *256 bins maximum mode*, the HGO operates on the maximum of the (R, G, B)
+  channels to compute a single 256-bins histogram. Only the RGB image format is
+  supported.
+
+**Byte Order.**
+All data is stored in memory in little endian format. Each cell in the tables
+contains one byte.
+
+.. flat-table:: VSP1 HGO Data - 64 Bins, Normal Mode (792 bytes)
+:header-rows:  2
+:stub-columns: 0
+
+* - Offset
+  - :cspan:`4` Memory
+* -
+  - [31:24]
+  - [23:16]
+  - [15:8]
+  - [7:0]
+* - 0
+  - -
+  - R/Cr/H max [7:0]
+  - -
+  - R/Cr/H min [7:0]
+* - 4
+  - -
+  - G/Y/S max [7:0]
+  - -
+  - G/Y/S min [7:0]
+* - 8
+  - -
+  - B/Cb/V max [7:0]
+  - -
+  - B/Cb/V min [7:0]
+* - 12
+  - :cspan:`4` R/Cr/H sum [31:0]
+* - 16
+  - :cspan:`4` G/Y/S sum [31:0]
+* - 20
+  - :cspan:`4` B/Cb/V sum [31:0]
+* - 24
+  - :cspan:`4` R/Cr/H bin 0 [31:0]
+* -
+  - :cspan:`4` ...
+* - 276
+  - :cspan:`4` R/Cr/H bin 63 [31:0]
+* - 280
+  - :cspan:`4` G/Y/S bin 0 [31:0]
+* -
+  - :cspan:`4` ...
+* - 532
+  - :cspan:`4` G/Y/S bin 63 [31:0]
+* - 536
+  - :cspan:`4` B/Cb/V bin 0 [31:0]
+* -
+  - :cspan:`4` ...
+* - 788
+  - :cspan:`4` B/Cb/V bin 63 [31:0]
+
+.. flat-table:: VSP1 HGO Data - 64 Bins, Max Mode (264 bytes)
+:header-rows:  2
+:stub-columns: 0
+
+* - Offset
+  - :cspan:`4` Memory
+* -
+  - [31:24]
+  - [23:16]
+  - [15:8]
+  - [7:0]
+* - 0
+  - -
+  - max(R,G,B) max [7:0]
+  - -
+  - max(R,G,B) min [7:0]
+* - 4
+  - :cspan:`4` max(R,G,B) sum [31:0]
+* - 8
+  - :cspan:`4` max(R,G,B) bin 0 [31:0]
+* -
+  - :cspan:`4` ...
+* - 260
+  - :cspan:`4` max(R,G,B) bin 63 [31:0]
+
+.. flat-table:: VSP1 HGO Data - 256 Bins, Normal Mode (1032 bytes)
+:header-rows:  2
+:stub-columns: 0
+
+* - Offset
+  - :cspan:`4` Memory
+* -
+  - [31:24]
+  - [23:16]
+  - [15:8]
+  - [7:0]
+* - 0
+  - -
+  - Y max [7:0]
+  - -
+  

[PATCH v3 7/8] v4l: Define a pixel format for the R-Car VSP1 2-D histogram engine

[Laurent Pinchart]

From: Niklas Söderlund 

The format is used on the R-Car VSP1 video queues that carry
2-D histogram statistics data.

Signed-off-by: Niklas Söderlund 
Reviewed-by: Laurent Pinchart 
Signed-off-by: Laurent Pinchart 
---
 Documentation/media/uapi/v4l/meta-formats.rst  |   1 +
 .../media/uapi/v4l/pixfmt-meta-vsp1-hgt.rst| 120 +
 drivers/media/v4l2-core/v4l2-ioctl.c   |   1 +
 include/uapi/linux/videodev2.h |   3 +-
 4 files changed, 124 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgt.rst

diff --git a/Documentation/media/uapi/v4l/meta-formats.rst 
b/Documentation/media/uapi/v4l/meta-formats.rst
index 05ab91e12f10..01e24e3df571 100644
--- a/Documentation/media/uapi/v4l/meta-formats.rst
+++ b/Documentation/media/uapi/v4l/meta-formats.rst
@@ -13,3 +13,4 @@ These formats are used for the :ref:`metadata` interface only.
 :maxdepth: 1
 
 pixfmt-meta-vsp1-hgo
+pixfmt-meta-vsp1-hgt
diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgt.rst 
b/Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgt.rst
new file mode 100644
index ..fb9f79466319
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-meta-vsp1-hgt.rst
@@ -0,0 +1,120 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-meta-fmt-vsp1-hgt:
+
+***
+V4L2_META_FMT_VSP1_HGT ('VSPT')
+***
+
+Renesas R-Car VSP1 2-D Histogram Data
+
+
+Description
+===
+
+This format describes histogram data generated by the Renesas R-Car VSP1
+2-D Histogram (HGT) engine.
+
+The VSP1 HGT is a histogram computation engine that operates on HSV
+data. It operates on a possibly cropped and subsampled input image and
+computes the sum, maximum and minimum of the S component as well as a
+weighted frequency histogram based on the H and S components.
+
+The histogram is a matrix of 6 Hue and 32 Saturation buckets, 192 in
+total. Each HSV value is added to one or more buckets with a weight
+between 1 and 16 depending on the Hue areas configuration. Finding the
+corresponding buckets is done by inspecting the H and S value independently.
+
+The Saturation position **n** (0 - 31) of the bucket in the matrix is
+found by the expression:
+
+n = S / 8
+
+The Hue position **m** (0 - 5) of the bucket in the matrix depends on
+how the HGT Hue areas are configured. There are 6 user configurable Hue
+Areas which can be configured to cover overlapping Hue values:
+
+::
+
+ Area 0   Area 1   Area 2   Area 3   Area 4   Area 
5
+     

+   \   /|  |\   /|  |\   /|  |\   /|  |\   /|  |\   /| 
 |\   /
+\ / |  | \ / |  | \ / |  | \ / |  | \ / |  | \ / | 
 | \ /
+ X  |  |  X  |  |  X  |  |  X  |  |  X  |  |  X  | 
 |  X
+/ \ |  | / \ |  | / \ |  | / \ |  | / \ |  | / \ | 
 | / \
+   /   \|  |/   \|  |/   \|  |/   \|  |/   \|  |/   \| 
 |/   \
+  5U   0L  0U   1L  1U   2L  2U   3L  3U   4L  4U   5L 
 5U   0L
+<0..Hue 
Value255>
+
+When two consecutive areas don't overlap (n+1L is equal to nU) the boundary
+value is considered as part of the lower area.
+
+Pixels with a hue value included in the centre of an area (between nL and nU
+included) are attributed to that single area and given a weight of 16. Pixels
+with a hue value included in the overlapping region between two areas (between
+n+1L and nU excluded) are attributed to both areas and given a weight for each
+of these areas proportional to their position along the diagonal lines
+(rounded down).
+
+The Hue area setup must match one of the following constrains:
+
+::
+
+0L <= 0U <= 1L <= 1U <= 2L <= 2U <= 3L <= 3U <= 4L <= 4U <= 5L <= 5U
+
+::
+
+0U <= 1L <= 1U <= 2L <= 2U <= 3L <= 3U <= 4L <= 4U <= 5L <= 5U <= 0L
+
+**Byte Order.**
+All data is stored in memory in little endian format. Each cell in the tables
+contains one byte.
+
+.. flat-table:: VSP1 HGT Data - (776 bytes)
+:header-rows:  2
+:stub-columns: 0
+
+* - Offset
+  - :cspan:`4` Memory
+* -
+  - [31:24]
+  - [23:16]
+  - [15:8]
+  - [7:0]
+* - 0
+  - -
+  - S max [7:0]
+  - -
+  - S min [7:0]
+* - 4
+  - :cspan:`4` S sum [31:0]
+* - 8
+  - :cspan:`4` Histogram bucket (m=0, n=0) [31:0]
+* - 12
+  - :cspan:`4` Histogram bucket (m=0, n=1) [31:0]
+* -
+  - :cspan:`4` ...
+* - 132
+  - :cspan:`4` Histogram bucket (m=0, n=31) [31:0]
+* - 136
+  - :cspan:`4` Histogram bucket (m=1, n=0) [31:0]
+* -

[PATCH v3 6/8] v4l: vsp1: Add HGO support

[Laurent Pinchart]

The HGO is a Histogram Generator One-Dimension. It computes per-channel
histograms over a configurable region of the image with optional
subsampling.

Signed-off-by: Laurent Pinchart 
---
 drivers/media/platform/vsp1/Makefile  |   2 +-
 drivers/media/platform/vsp1/vsp1.h|   3 +
 drivers/media/platform/vsp1/vsp1_drv.c|  42 --
 drivers/media/platform/vsp1/vsp1_entity.c |  16 +++
 drivers/media/platform/vsp1/vsp1_hgo.c| 228 ++
 drivers/media/platform/vsp1/vsp1_hgo.h|  45 ++
 drivers/media/platform/vsp1/vsp1_pipe.c   |  16 +++
 drivers/media/platform/vsp1/vsp1_pipe.h   |   2 +
 drivers/media/platform/vsp1/vsp1_regs.h   |  20 ++-
 drivers/media/platform/vsp1/vsp1_video.c  |   6 +
 10 files changed, 367 insertions(+), 13 deletions(-)
 create mode 100644 drivers/media/platform/vsp1/vsp1_hgo.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_hgo.h

diff --git a/drivers/media/platform/vsp1/Makefile 
b/drivers/media/platform/vsp1/Makefile
index c559536f7867..8ab6a063569e 100644
--- a/drivers/media/platform/vsp1/Makefile
+++ b/drivers/media/platform/vsp1/Makefile
@@ -3,7 +3,7 @@ vsp1-y  += vsp1_dl.o vsp1_drm.o 
vsp1_video.o
 vsp1-y += vsp1_rpf.o vsp1_rwpf.o vsp1_wpf.o
 vsp1-y += vsp1_clu.o vsp1_hsit.o vsp1_lut.o
 vsp1-y += vsp1_bru.o vsp1_sru.o vsp1_uds.o
-vsp1-y += vsp1_histo.o
+vsp1-y += vsp1_hgo.o vsp1_histo.o
 vsp1-y += vsp1_lif.o
 
 obj-$(CONFIG_VIDEO_RENESAS_VSP1)   += vsp1.o
diff --git a/drivers/media/platform/vsp1/vsp1.h 
b/drivers/media/platform/vsp1/vsp1.h
index b23fa879a9aa..0ba7521c01b4 100644
--- a/drivers/media/platform/vsp1/vsp1.h
+++ b/drivers/media/platform/vsp1/vsp1.h
@@ -32,6 +32,7 @@ struct vsp1_entity;
 struct vsp1_platform_data;
 struct vsp1_bru;
 struct vsp1_clu;
+struct vsp1_hgo;
 struct vsp1_hsit;
 struct vsp1_lif;
 struct vsp1_lut;
@@ -50,6 +51,7 @@ struct vsp1_uds;
 #define VSP1_HAS_CLU   (1 << 4)
 #define VSP1_HAS_WPF_VFLIP (1 << 5)
 #define VSP1_HAS_WPF_HFLIP (1 << 6)
+#define VSP1_HAS_HGO   (1 << 7)
 
 struct vsp1_device_info {
u32 version;
@@ -73,6 +75,7 @@ struct vsp1_device {
 
struct vsp1_bru *bru;
struct vsp1_clu *clu;
+   struct vsp1_hgo *hgo;
struct vsp1_hsit *hsi;
struct vsp1_hsit *hst;
struct vsp1_lif *lif;
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c 
b/drivers/media/platform/vsp1/vsp1_drv.c
index 83a6669a6328..0acc8ed6ac59 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -30,6 +30,7 @@
 #include "vsp1_clu.h"
 #include "vsp1_dl.h"
 #include "vsp1_drm.h"
+#include "vsp1_hgo.h"
 #include "vsp1_hsit.h"
 #include "vsp1_lif.h"
 #include "vsp1_lut.h"
@@ -150,6 +151,16 @@ static int vsp1_uapi_create_links(struct vsp1_device *vsp1)
return ret;
}
 
+   if (vsp1->hgo) {
+   ret = 
media_create_pad_link(>hgo->histo.entity.subdev.entity,
+   HISTO_PAD_SOURCE,
+   >hgo->histo.video.entity, 0,
+   MEDIA_LNK_FL_ENABLED |
+   MEDIA_LNK_FL_IMMUTABLE);
+   if (ret < 0)
+   return ret;
+   }
+
if (vsp1->lif) {
ret = media_create_pad_link(>wpf[0]->entity.subdev.entity,
RWPF_PAD_SOURCE,
@@ -283,6 +294,17 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
 
list_add_tail(>hst->entity.list_dev, >entities);
 
+   if (vsp1->info->features & VSP1_HAS_HGO && vsp1->info->uapi) {
+   vsp1->hgo = vsp1_hgo_create(vsp1);
+   if (IS_ERR(vsp1->hgo)) {
+   ret = PTR_ERR(vsp1->hgo);
+   goto done;
+   }
+
+   list_add_tail(>hgo->histo.entity.list_dev,
+ >entities);
+   }
+
/*
 * The LIF is only supported when used in conjunction with the DU, in
 * which case the userspace API is disabled. If the userspace API is
@@ -568,8 +590,8 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
.version = VI6_IP_VERSION_MODEL_VSPS_H2,
.model = "VSP1-S",
.gen = 2,
-   .features = VSP1_HAS_BRU | VSP1_HAS_CLU | VSP1_HAS_LUT
- | VSP1_HAS_SRU | VSP1_HAS_WPF_VFLIP,
+   .features = VSP1_HAS_BRU | VSP1_HAS_CLU | VSP1_HAS_HGO
+ | VSP1_HAS_LUT | VSP1_HAS_SRU | VSP1_HAS_WPF_VFLIP,
.rpf_count = 5,
.uds_count = 3,
.wpf_count = 4,
@@ 

[PATCH v3 3/8] v4l: vsp1: Support histogram generators in pipeline configuration

[Laurent Pinchart]

Histogram generators are single-pad entities that branch as leaf nodes
at any point in the pipeline. Make sure that pipeline traversal and
routing configuration support them correctly.

Support for the actual HGO and HGT operation will come later.

Signed-off-by: Laurent Pinchart 
---
 drivers/media/platform/vsp1/vsp1_drm.c|   2 +-
 drivers/media/platform/vsp1/vsp1_drv.c|   4 +-
 drivers/media/platform/vsp1/vsp1_entity.c | 124 ++
 drivers/media/platform/vsp1/vsp1_entity.h |   8 +-
 drivers/media/platform/vsp1/vsp1_pipe.c   |   6 +-
 drivers/media/platform/vsp1/vsp1_video.c  |  18 ++---
 6 files changed, 134 insertions(+), 28 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drm.c 
b/drivers/media/platform/vsp1/vsp1_drm.c
index 8b982960ba8d..f656b96fb150 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -497,7 +497,7 @@ void vsp1_du_atomic_flush(struct device *dev)
}
}
 
-   vsp1_entity_route_setup(entity, dl);
+   vsp1_entity_route_setup(entity, pipe, dl);
 
if (entity->ops->configure) {
entity->ops->configure(entity, pipe, dl,
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c 
b/drivers/media/platform/vsp1/vsp1_drv.c
index 8d1e61b353bb..83a6669a6328 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -105,7 +105,9 @@ static int vsp1_create_sink_links(struct vsp1_device *vsp1,
if (source->type == sink->type)
continue;
 
-   if (source->type == VSP1_ENTITY_LIF ||
+   if (source->type == VSP1_ENTITY_HGO ||
+   source->type == VSP1_ENTITY_HGT ||
+   source->type == VSP1_ENTITY_LIF ||
source->type == VSP1_ENTITY_WPF)
continue;
 
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c 
b/drivers/media/platform/vsp1/vsp1_entity.c
index 12eca5660d6e..88a2aae182ba 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -21,6 +21,8 @@
 #include "vsp1.h"
 #include "vsp1_dl.h"
 #include "vsp1_entity.h"
+#include "vsp1_pipe.h"
+#include "vsp1_rwpf.h"
 
 static inline struct vsp1_entity *
 media_entity_to_vsp1_entity(struct media_entity *entity)
@@ -28,11 +30,14 @@ media_entity_to_vsp1_entity(struct media_entity *entity)
return container_of(entity, struct vsp1_entity, subdev.entity);
 }
 
-void vsp1_entity_route_setup(struct vsp1_entity *source,
+void vsp1_entity_route_setup(struct vsp1_entity *entity,
+struct vsp1_pipeline *pipe,
 struct vsp1_dl_list *dl)
 {
+   struct vsp1_entity *source;
struct vsp1_entity *sink;
 
+   source = entity;
if (source->route->reg == 0)
return;
 
@@ -283,25 +288,32 @@ int vsp1_subdev_enum_frame_size(struct v4l2_subdev 
*subdev,
  * Media Operations
  */
 
-int vsp1_entity_link_setup(struct media_entity *entity,
-  const struct media_pad *local,
-  const struct media_pad *remote, u32 flags)
+static int vsp1_entity_link_setup_source(const struct media_pad *source_pad,
+const struct media_pad *sink_pad,
+u32 flags)
 {
struct vsp1_entity *source;
 
-   if (!(local->flags & MEDIA_PAD_FL_SOURCE))
-   return 0;
-
-   source = media_entity_to_vsp1_entity(local->entity);
+   source = media_entity_to_vsp1_entity(source_pad->entity);
 
if (!source->route)
return 0;
 
if (flags & MEDIA_LNK_FL_ENABLED) {
-   if (source->sink)
-   return -EBUSY;
-   source->sink = remote->entity;
-   source->sink_pad = remote->index;
+   struct vsp1_entity *sink
+   = media_entity_to_vsp1_entity(sink_pad->entity);
+
+   /*
+* Fan-out is limited to one for the normal data path plus
+* optional HGO and HGT. We ignore the HGO and HGT here.
+*/
+   if (sink->type != VSP1_ENTITY_HGO &&
+   sink->type != VSP1_ENTITY_HGT) {
+   if (source->sink)
+   return -EBUSY;
+   source->sink = sink_pad->entity;
+   source->sink_pad = sink_pad->index;
+   }
} else {
source->sink = NULL;
source->sink_pad = 0;
@@ -310,6 +322,85 @@ int vsp1_entity_link_setup(struct media_entity *entity,
return 0;
 }
 
+static int vsp1_entity_link_setup_sink(const struct media_pad *source_pad,
+  const struct media_pad *sink_pad,
+   

[PATCH v3 2/8] v4l: vsp1: Add histogram support

[Laurent Pinchart]

The histogram common code will be used to implement support for both the
HGO and HGT histogram computation engines.

Signed-off-by: Laurent Pinchart 
---
 drivers/media/platform/Kconfig   |   1 +
 drivers/media/platform/vsp1/Makefile |   1 +
 drivers/media/platform/vsp1/vsp1_histo.c | 646 +++
 drivers/media/platform/vsp1/vsp1_histo.h |  84 
 4 files changed, 732 insertions(+)
 create mode 100644 drivers/media/platform/vsp1/vsp1_histo.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_histo.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index c9106e105bab..4c7420df9c25 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -405,6 +405,7 @@ config VIDEO_RENESAS_VSP1
depends on (ARCH_RENESAS && OF) || COMPILE_TEST
depends on (!ARM64 && !VIDEO_RENESAS_FCP) || VIDEO_RENESAS_FCP
select VIDEOBUF2_DMA_CONTIG
+   select VIDEOBUF2_VMALLOC
---help---
  This is a V4L2 driver for the Renesas VSP1 video processing engine.
 
diff --git a/drivers/media/platform/vsp1/Makefile 
b/drivers/media/platform/vsp1/Makefile
index 1328e1bd2143..c559536f7867 100644
--- a/drivers/media/platform/vsp1/Makefile
+++ b/drivers/media/platform/vsp1/Makefile
@@ -3,6 +3,7 @@ vsp1-y  += vsp1_dl.o vsp1_drm.o 
vsp1_video.o
 vsp1-y += vsp1_rpf.o vsp1_rwpf.o vsp1_wpf.o
 vsp1-y += vsp1_clu.o vsp1_hsit.o vsp1_lut.o
 vsp1-y += vsp1_bru.o vsp1_sru.o vsp1_uds.o
+vsp1-y += vsp1_histo.o
 vsp1-y += vsp1_lif.o
 
 obj-$(CONFIG_VIDEO_RENESAS_VSP1)   += vsp1.o
diff --git a/drivers/media/platform/vsp1/vsp1_histo.c 
b/drivers/media/platform/vsp1/vsp1_histo.c
new file mode 100644
index ..afab77cf4fa5
--- /dev/null
+++ b/drivers/media/platform/vsp1/vsp1_histo.c
@@ -0,0 +1,646 @@
+/*
+ * vsp1_histo.c  --  R-Car VSP1 Histogram API
+ *
+ * Copyright (C) 2016 Renesas Electronics Corporation
+ * Copyright (C) 2016 Laurent Pinchart
+ *
+ * Contact: Laurent Pinchart (laurent.pinch...@ideasonboard.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include 
+#include 
+
+#include 
+#include 
+#include 
+
+#include "vsp1.h"
+#include "vsp1_histo.h"
+#include "vsp1_pipe.h"
+
+#define HISTO_MIN_SIZE 4U
+#define HISTO_MAX_SIZE 8192U
+
+/* 
-
+ * Buffer Operations
+ */
+
+static inline struct vsp1_histogram_buffer *
+to_vsp1_histogram_buffer(struct vb2_v4l2_buffer *vbuf)
+{
+   return container_of(vbuf, struct vsp1_histogram_buffer, buf);
+}
+
+struct vsp1_histogram_buffer *
+vsp1_histogram_buffer_get(struct vsp1_histogram *histo)
+{
+   struct vsp1_histogram_buffer *buf = NULL;
+   unsigned long flags;
+
+   spin_lock_irqsave(>irqlock, flags);
+
+   if (list_empty(>irqqueue))
+   goto done;
+
+   buf = list_first_entry(>irqqueue, struct vsp1_histogram_buffer,
+  queue);
+   list_del(>queue);
+   histo->readout = true;
+
+done:
+   spin_unlock_irqrestore(>irqlock, flags);
+   return buf;
+}
+
+void vsp1_histogram_buffer_complete(struct vsp1_histogram *histo,
+   struct vsp1_histogram_buffer *buf,
+   size_t size)
+{
+   struct vsp1_pipeline *pipe = histo->pipe;
+   unsigned long flags;
+
+   /*
+* The pipeline pointer is guaranteed to be valid as this function is
+* called from the frame completion interrupt handler, which can only
+* occur when video streaming is active.
+*/
+   buf->buf.sequence = pipe->sequence;
+   buf->buf.vb2_buf.timestamp = ktime_get_ns();
+   vb2_set_plane_payload(>buf.vb2_buf, 0, size);
+   vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_DONE);
+
+   spin_lock_irqsave(>irqlock, flags);
+   histo->readout = false;
+   wake_up(>wait_queue);
+   spin_unlock_irqrestore(>irqlock, flags);
+}
+
+/* 
-
+ * videobuf2 Queue Operations
+ */
+
+static int histo_queue_setup(struct vb2_queue *vq, unsigned int *nbuffers,
+unsigned int *nplanes, unsigned int sizes[],
+struct device *alloc_devs[])
+{
+   struct vsp1_histogram *histo = vb2_get_drv_priv(vq);
+
+   if (*nplanes) {
+   if (*nplanes != 1)
+   return -EINVAL;
+
+   if (sizes[0] 

[PATCH v3 4/8] v4l: vsp1: Fix HGO and HGT routing register addresses

[Laurent Pinchart]

The addresses are incorrect, fix them.

Signed-off-by: Laurent Pinchart 
---
 drivers/media/platform/vsp1/vsp1_regs.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_regs.h 
b/drivers/media/platform/vsp1/vsp1_regs.h
index 47b1dee044fb..61369e267667 100644
--- a/drivers/media/platform/vsp1/vsp1_regs.h
+++ b/drivers/media/platform/vsp1/vsp1_regs.h
@@ -328,8 +328,8 @@
 #define VI6_DPR_ROUTE_RT_MASK  (0x3f << 0)
 #define VI6_DPR_ROUTE_RT_SHIFT 0
 
-#define VI6_DPR_HGO_SMPPT  0x2050
-#define VI6_DPR_HGT_SMPPT  0x2054
+#define VI6_DPR_HGO_SMPPT  0x2054
+#define VI6_DPR_HGT_SMPPT  0x2058
 #define VI6_DPR_SMPPT_TGW_MASK (7 << 8)
 #define VI6_DPR_SMPPT_TGW_SHIFT8
 #define VI6_DPR_SMPPT_PT_MASK  (0x3f << 0)
-- 
Regards,

Laurent Pinchart

[PATCH v3 1/8] v4l: Add metadata buffer type and format

[Laurent Pinchart]

The metadata buffer type is used to transfer metadata between userspace
and kernelspace through a V4L2 buffers queue. It comes with a new
metadata capture capability and format description.

Signed-off-by: Laurent Pinchart 
Tested-by: Guennadi Liakhovetski 
Acked-by: Sakari Ailus 
Acked-by: Hans Verkuil 
---
 Documentation/media/uapi/v4l/buffer.rst  |  3 ++
 Documentation/media/uapi/v4l/dev-meta.rst| 62 
 Documentation/media/uapi/v4l/devices.rst |  1 +
 Documentation/media/uapi/v4l/vidioc-querycap.rst |  3 ++
 Documentation/media/videodev2.h.rst.exceptions   |  2 +
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c| 19 
 drivers/media/v4l2-core/v4l2-dev.c   | 16 +++---
 drivers/media/v4l2-core/v4l2-ioctl.c | 34 +
 drivers/media/v4l2-core/videobuf2-v4l2.c |  3 ++
 include/media/v4l2-ioctl.h   | 17 +++
 include/trace/events/v4l2.h  |  1 +
 include/uapi/linux/videodev2.h   | 13 +
 12 files changed, 168 insertions(+), 6 deletions(-)
 create mode 100644 Documentation/media/uapi/v4l/dev-meta.rst

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index 5c58db98ab7a..02834ce7fa4d 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -418,6 +418,9 @@ enum v4l2_buf_type
   - 12
   - Buffer for Software Defined Radio (SDR) output stream, see
:ref:`sdr`.
+* - ``V4L2_BUF_TYPE_META_CAPTURE``
+  - 13
+  - Buffer for metadata capture, see :ref:`metadata`.
 
 
 
diff --git a/Documentation/media/uapi/v4l/dev-meta.rst 
b/Documentation/media/uapi/v4l/dev-meta.rst
new file mode 100644
index ..b6044c54082a
--- /dev/null
+++ b/Documentation/media/uapi/v4l/dev-meta.rst
@@ -0,0 +1,62 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _metadata:
+
+**
+Metadata Interface
+**
+
+Metadata refers to any non-image data that supplements video frames with
+additional information. This may include statistics computed over the image
+or frame capture parameters supplied by the image source. This interface is
+intended for transfer of metadata to userspace and control of that operation.
+
+The metadata interface is implemented on video capture device nodes. The device
+can be dedicated to metadata or can implement both video and metadata capture
+as specified in its reported capabilities.
+
+.. note::
+
+This is an :ref:`experimental` interface and may
+change in the future.
+
+Querying Capabilities
+=
+
+Device nodes supporting the metadata interface set the 
``V4L2_CAP_META_CAPTURE``
+flag in the ``device_caps`` field of the
+:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP`
+ioctl. That flag means the device can capture metadata to memory.
+
+At least one of the read/write or streaming I/O methods must be supported.
+
+
+Data Format Negotiation
+===
+
+The metadata device uses the :ref:`format` ioctls to select the capture format.
+The metadata buffer content format is bound to that selected format. In 
addition
+to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be
+supported as well.
+
+To use the :ref:`format` ioctls applications set the ``type`` field of the
+:c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_META_CAPTURE`` and use the
+:c:type:`v4l2_meta_format` ``meta`` member of the ``fmt`` union as needed per
+the desired operation. Both drivers and applications must set the remainder of
+the :c:type:`v4l2_format` structure to 0.
+
+.. _v4l2-meta-format:
+.. flat-table:: struct v4l2_meta_format
+:header-rows:  0
+:stub-columns: 0
+:widths:   1 1 2
+
+* - __u32
+  - ``dataformat``
+  - The data format, set by the application. This is a little endian
+:ref:`four character code `. V4L2 defines metadata formats
+in :ref:`meta-formats`.
+* - __u32
+  - ``buffersize``
+  - Maximum buffer size in bytes required for data. The value is set by the
+driver.
diff --git a/Documentation/media/uapi/v4l/devices.rst 
b/Documentation/media/uapi/v4l/devices.rst
index 5c3d6c29e12c..fb7f8c26cf09 100644
--- a/Documentation/media/uapi/v4l/devices.rst
+++ b/Documentation/media/uapi/v4l/devices.rst
@@ -25,3 +25,4 @@ Interfaces
 dev-touch
 dev-event
 dev-subdev
+dev-meta
diff --git a/Documentation/media/uapi/v4l/vidioc-querycap.rst 
b/Documentation/media/uapi/v4l/vidioc-querycap.rst
index 165d8314327e..12e0d9a63cd8 100644
--- a/Documentation/media/uapi/v4l/vidioc-querycap.rst
+++ b/Documentation/media/uapi/v4l/vidioc-querycap.rst
@@ -236,6 +236,9 @@ specification the ioctl returns an ``EINVAL`` error code.
 * - 

[PATCH v2 0/3] Renesas R-Car VSP1 rotation support

[Laurent Pinchart]

Hello everybody,

This patch series implement support for rotation in the VSP1 driver. It
contains an update of the rotation support part of the "[PATCH 00/13]
Renesas R-Car VSP: Scaling and rotation support on Gen3" series.

Patch 1/3 starts by fixing the multi-line comment style through the whole
driver to avoid adding new offenders in patch 3/3.

Patch 2/3 is to biggest addition since v1. It clarifies interactions between
formats, controls and buffers in the V4L2 API documentation. The proposal is
based on an IRC meeting held on 2016-09-15 in the #v4l channel on freenode
which I have tried to translate as accurately as possible into documentation.

Patch 3/3 implements rotation support in the vsp1 driver. Beside being rebased
on top of the current vsp1/next branch, it has mostly been kept unchanged
compared to v1.

For your convenience the code is available at

git://linuxtv.org/pinchartl/media.git vsp1-rotation-v2-20170228

Laurent Pinchart (3):
  v4l: vsp1: Fix multi-line comment style
  v4l: Clearly document interactions between formats, controls and
buffers
  v4l: vsp1: wpf: Implement rotation support

 Documentation/media/uapi/v4l/buffer.rst   |  88 
 drivers/media/platform/vsp1/vsp1_bru.c|  27 ++--
 drivers/media/platform/vsp1/vsp1_dl.c |  27 ++--
 drivers/media/platform/vsp1/vsp1_drm.c|  21 ++-
 drivers/media/platform/vsp1/vsp1_drv.c|  12 +-
 drivers/media/platform/vsp1/vsp1_entity.c |   9 +-
 drivers/media/platform/vsp1/vsp1_hsit.c   |   3 +-
 drivers/media/platform/vsp1/vsp1_lif.c|   6 +-
 drivers/media/platform/vsp1/vsp1_pipe.c   |   9 +-
 drivers/media/platform/vsp1/vsp1_rpf.c|  11 +-
 drivers/media/platform/vsp1/vsp1_rwpf.c   |  11 +-
 drivers/media/platform/vsp1/vsp1_rwpf.h   |   3 +-
 drivers/media/platform/vsp1/vsp1_sru.c|   3 +-
 drivers/media/platform/vsp1/vsp1_uds.c|   3 +-
 drivers/media/platform/vsp1/vsp1_video.c  |  45 +--
 drivers/media/platform/vsp1/vsp1_wpf.c| 215 +++---
 16 files changed, 382 insertions(+), 111 deletions(-)

-- 
Regards,

Laurent Pinchart

[PATCH v2 1/3] v4l: vsp1: Fix multi-line comment style

[Laurent Pinchart]

Fix all multi-line comments to comply with the kernel coding style.

Signed-off-by: Laurent Pinchart 
---
 drivers/media/platform/vsp1/vsp1_bru.c| 27 -
 drivers/media/platform/vsp1/vsp1_dl.c | 27 -
 drivers/media/platform/vsp1/vsp1_drm.c| 21 +---
 drivers/media/platform/vsp1/vsp1_drv.c| 12 +++
 drivers/media/platform/vsp1/vsp1_entity.c |  9 ++---
 drivers/media/platform/vsp1/vsp1_hsit.c   |  3 ++-
 drivers/media/platform/vsp1/vsp1_lif.c|  6 --
 drivers/media/platform/vsp1/vsp1_pipe.c   |  9 ++---
 drivers/media/platform/vsp1/vsp1_rpf.c|  9 ++---
 drivers/media/platform/vsp1/vsp1_rwpf.c   |  6 --
 drivers/media/platform/vsp1/vsp1_sru.c|  3 ++-
 drivers/media/platform/vsp1/vsp1_uds.c|  3 ++-
 drivers/media/platform/vsp1/vsp1_video.c  | 33 ---
 drivers/media/platform/vsp1/vsp1_wpf.c| 12 +++
 14 files changed, 120 insertions(+), 60 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c 
b/drivers/media/platform/vsp1/vsp1_bru.c
index ee8355c28f94..85362c5ef57a 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -251,7 +251,8 @@ static int bru_set_selection(struct v4l2_subdev *subdev,
sel->r.left = clamp_t(unsigned int, sel->r.left, 0, format->width - 1);
sel->r.top = clamp_t(unsigned int, sel->r.top, 0, format->height - 1);
 
-   /* Scaling isn't supported, the compose rectangle size must be identical
+   /*
+* Scaling isn't supported, the compose rectangle size must be identical
 * to the sink format size.
 */
format = vsp1_entity_get_pad_format(>entity, config, sel->pad);
@@ -300,13 +301,15 @@ static void bru_configure(struct vsp1_entity *entity,
format = vsp1_entity_get_pad_format(>entity, bru->entity.config,
bru->entity.source_pad);
 
-   /* The hardware is extremely flexible but we have no userspace API to
+   /*
+* The hardware is extremely flexible but we have no userspace API to
 * expose all the parameters, nor is it clear whether we would have use
 * cases for all the supported modes. Let's just harcode the parameters
 * to sane default values for now.
 */
 
-   /* Disable dithering and enable color data normalization unless the
+   /*
+* Disable dithering and enable color data normalization unless the
 * format at the pipeline output is premultiplied.
 */
flags = pipe->output ? pipe->output->format.flags : 0;
@@ -314,7 +317,8 @@ static void bru_configure(struct vsp1_entity *entity,
   flags & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA ?
   0 : VI6_BRU_INCTRL_NRM);
 
-   /* Set the background position to cover the whole output image and
+   /*
+* Set the background position to cover the whole output image and
 * configure its color.
 */
vsp1_bru_write(bru, dl, VI6_BRU_VIRRPF_SIZE,
@@ -325,7 +329,8 @@ static void bru_configure(struct vsp1_entity *entity,
vsp1_bru_write(bru, dl, VI6_BRU_VIRRPF_COL, bru->bgcolor |
   (0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
 
-   /* Route BRU input 1 as SRC input to the ROP unit and configure the ROP
+   /*
+* Route BRU input 1 as SRC input to the ROP unit and configure the ROP
 * unit with a NOP operation to make BRU input 1 available as the
 * Blend/ROP unit B SRC input.
 */
@@ -337,7 +342,8 @@ static void bru_configure(struct vsp1_entity *entity,
bool premultiplied = false;
u32 ctrl = 0;
 
-   /* Configure all Blend/ROP units corresponding to an enabled BRU
+   /*
+* Configure all Blend/ROP units corresponding to an enabled BRU
 * input for alpha blending. Blend/ROP units corresponding to
 * disabled BRU inputs are used in ROP NOP mode to ignore the
 * SRC input.
@@ -352,13 +358,15 @@ static void bru_configure(struct vsp1_entity *entity,
 |  VI6_BRU_CTRL_AROP(VI6_ROP_NOP);
}
 
-   /* Select the virtual RPF as the Blend/ROP unit A DST input to
+   /*
+* Select the virtual RPF as the Blend/ROP unit A DST input to
 * serve as a background color.
 */
if (i == 0)
ctrl |= VI6_BRU_CTRL_DSTSEL_VRPF;
 
-   /* Route BRU inputs 0 to 3 as SRC inputs to Blend/ROP units A to
+   /*
+* Route BRU inputs 0 to 3 as SRC inputs to Blend/ROP units A to
 * D in that order. The Blend/ROP unit B SRC is hardwired to the
 * ROP unit output, the corresponding register 

[PATCH v2 2/3] v4l: Clearly document interactions between formats, controls and buffers

[Laurent Pinchart]

V4L2 exposes parameters that influence buffers sizes through the format
ioctls (VIDIOC_G_FMT, VIDIOC_TRY_FMT and VIDIO_S_FMT). Other parameters
not part of the format structure may also influence buffer sizes or
buffer layout in general. One existing such parameter is rotation, which
is implemented by the VIDIOC_ROTATE control and thus exposed through the
V4L2 control ioctls.

The interaction between those parameters and buffers is currently only
partially specified by the V4L2 API. In particular interactions between
controls and buffers isn't specified at all. The behaviour of the
VIDIOC_S_FMT ioctl when buffers are allocated is also not fully
specified.

This commit clearly defines and documents the interactions between
formats, controls and buffers.

The preparatory discussions for the documentation change considered
completely disallowing controls that change the buffer size or layout,
in favour of extending the format API with a new ioctl that would bundle
those controls with format information. The idea has been rejected, as
this would essentially be a restricted version of the upcoming request
API that wouldn't bring any additional value.

Another option we have considered was to mandate the use of the request
API to modify controls that influence buffer size or layout. This has
also been rejected on the grounds that requiring the request API to
change rotation even when streaming is stopped would significantly
complicate implementation of drivers and usage of the V4L2 API for
applications.

Applications will however be required to use the upcoming request API to
change at runtime formats or controls that influence the buffer size or
layout, because of the need to synchronize buffers with the formats and
controls. Otherwise there would be no way to interpret the content of a
buffer correctly.

Signed-off-by: Laurent Pinchart 
---
 Documentation/media/uapi/v4l/buffer.rst | 88 +
 1 file changed, 88 insertions(+)

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index ac58966ccb9b..5c58db98ab7a 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -34,6 +34,94 @@ flags are copied from the OUTPUT video buffer to the CAPTURE 
video
 buffer.
 
 
+Interactions between formats, controls and buffers
+==
+
+V4L2 exposes parameters that influence the buffer size, or the way data is
+laid out in the buffer. Those parameters are exposed through both formats and
+controls. One example of such a control is the ``V4L2_CID_ROTATE`` control
+that modifies the direction in which pixels are stored in the buffer, as well
+as the buffer size when the selected format includes padding at the end of
+lines.
+
+The set of information needed to interpret the content of a buffer (e.g. the
+pixel format, the line stride, the tiling orientation or the rotation) is
+collectively referred to in the rest of this section as the buffer layout.
+
+Modifying formats or controls that influence the buffer size or layout require
+the stream to be stopped. Any attempt at such a modification while the stream
+is active shall cause the format or control set ioctl to return the ``EBUSY``
+error code.
+
+Controls that only influence the buffer layout can be modified at any time
+when the stream is stopped. As they don't influence the buffer size, no
+special handling is needed to synchronize those controls with buffer
+allocation.
+
+Formats and controls that influence the buffer size interact with buffer
+allocation. As buffer allocation is an expensive operation, drivers should
+allow format or controls that influence the buffer size to be changed with
+buffers allocated. A typical ioctl sequence to modify format and controls is
+
+ #. VIDIOC_STREAMOFF
+ #. VIDIOC_S_FMT
+ #. VIDIOC_S_EXT_CTRLS
+ #. VIDIOC_QBUF
+ #. VIDIOC_STREAMON
+
+Queued buffers must be large enough for the new format or controls.
+
+Drivers shall return a ``ENOSPC`` error in response to format change
+(:c:func:`VIDIOC_S_FMT`) or control changes (:c:func:`VIDIOC_S_CTRL` or
+:c:func:`VIDIOC_S_EXT_CTRLS`) if buffers too small for the new format are
+currently queued. As a simplification, drivers are allowed to return an error
+from these ioctls if any buffer is currently queued, without checking the
+queued buffers sizes. Drivers shall also return a ``ENOSPC`` error from the
+:c:func:`VIDIOC_QBUF` ioctl if the buffer being queued is too small for the
+current format or controls. Together, these requirements ensure that queued
+buffers will always be large enough for the configured format and controls.
+
+Userspace applications can query the buffer size required for a given format
+and controls by first setting the desired control values and then trying the
+desired format. The :c:func:`VIDIOC_TRY_FMT` ioctl will return the required
+buffer size.
+
+ #. 

[PATCH v2 3/3] v4l: vsp1: wpf: Implement rotation support

[Laurent Pinchart]

Some WPF instances, on Gen3 devices, can perform 90° rotation when
writing frames to memory. Implement support for this using the
V4L2_CID_ROTATE control.

Signed-off-by: Laurent Pinchart 
---
 drivers/media/platform/vsp1/vsp1_rpf.c   |   2 +-
 drivers/media/platform/vsp1/vsp1_rwpf.c  |   5 +
 drivers/media/platform/vsp1/vsp1_rwpf.h  |   3 +-
 drivers/media/platform/vsp1/vsp1_video.c |  12 +-
 drivers/media/platform/vsp1/vsp1_wpf.c   | 205 +++
 5 files changed, 175 insertions(+), 52 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c 
b/drivers/media/platform/vsp1/vsp1_rpf.c
index f5a9a4c8c74d..8feddd59cf8d 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -106,7 +106,7 @@ static void rpf_configure(struct vsp1_entity *entity,
 * of the pipeline.
 */
output = vsp1_entity_get_pad_format(wpf, wpf->config,
-   RWPF_PAD_SOURCE);
+   RWPF_PAD_SINK);
 
crop.width = pipe->partition.width * input_width
   / output->width;
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c 
b/drivers/media/platform/vsp1/vsp1_rwpf.c
index 7d52c88a583e..cfd8f1904fa6 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -121,6 +121,11 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
RWPF_PAD_SOURCE);
*format = fmt->format;
 
+   if (rwpf->flip.rotate) {
+   format->width = fmt->format.height;
+   format->height = fmt->format.width;
+   }
+
 done:
mutex_unlock(>entity.lock);
return ret;
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h 
b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 1c98aff3da5d..b4ffc38f48af 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -56,9 +56,10 @@ struct vsp1_rwpf {
 
struct {
spinlock_t lock;
-   struct v4l2_ctrl *ctrls[2];
+   struct v4l2_ctrl *ctrls[3];
unsigned int pending;
unsigned int active;
+   bool rotate;
} flip;
 
struct vsp1_rwpf_memory mem;
diff --git a/drivers/media/platform/vsp1/vsp1_video.c 
b/drivers/media/platform/vsp1/vsp1_video.c
index 5239e08fabc3..795a3ca9ca03 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -187,9 +187,13 @@ static void vsp1_video_pipeline_setup_partitions(struct 
vsp1_pipeline *pipe)
struct vsp1_entity *entity;
unsigned int div_size;
 
+   /*
+* Partitions are computed on the size before rotation, use the format
+* at the WPF sink.
+*/
format = vsp1_entity_get_pad_format(>output->entity,
pipe->output->entity.config,
-   RWPF_PAD_SOURCE);
+   RWPF_PAD_SINK);
div_size = format->width;
 
/* Gen2 hardware doesn't require image partitioning. */
@@ -229,9 +233,13 @@ static struct v4l2_rect vsp1_video_partition(struct 
vsp1_pipeline *pipe,
struct v4l2_rect partition;
unsigned int modulus;
 
+   /*
+* Partitions are computed on the size before rotation, use the format
+* at the WPF sink.
+*/
format = vsp1_entity_get_pad_format(>output->entity,
pipe->output->entity.config,
-   RWPF_PAD_SOURCE);
+   RWPF_PAD_SINK);
 
/* A single partition simply processes the output size in full. */
if (pipe->partitions <= 1) {
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c 
b/drivers/media/platform/vsp1/vsp1_wpf.c
index 25a2ed6e2e18..82935d97cad9 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -43,32 +43,94 @@ static inline void vsp1_wpf_write(struct vsp1_rwpf *wpf,
 enum wpf_flip_ctrl {
WPF_CTRL_VFLIP = 0,
WPF_CTRL_HFLIP = 1,
-   WPF_CTRL_MAX,
+   WPF_CTRL_ROTATE = 2,
 };
 
+static int vsp1_wpf_set_rotation(struct vsp1_rwpf *wpf, unsigned int rotation)
+{
+   struct vsp1_video *video = wpf->video;
+   struct v4l2_mbus_framefmt *sink_format;
+   struct v4l2_mbus_framefmt *source_format;
+   bool rotate;
+   int ret = 0;
+
+   /*
+* Only consider the 0°/180° from/to 90°/270° modifications, the rest
+* is taken care of by the flipping configuration.
+*/
+   rotate = rotation == 90 || rotation == 270;
+   if (rotate == wpf->flip.rotate)
+