Re: [PATCH v2 1/4] arm64: dts: rockchip: Split out common nodes for Rock960 based boards

[Ezequiel Garcia]

On Tue, 2018-09-11 at 08:00 +0530, Manivannan Sadhasivam wrote:
> Since the same family members of Rock960 boards (Rock960 and Ficus)
> share the same configuration, split out the common nodes into a common
> dtsi file for reducing code duplication. The board specific nodes for
> Ficus boards are then placed in corresponding board DTS file.
> 

I think it should be possible to move the common USB nodes to the dtsi file,
and keep the board-specific (phy-supply property) in the dts files:

_host {
phy-supply = <_host>;
};

_host {
phy-supply = <_host>;
};

Also, I believe it would be good to have some more details
in this commit log. The information on the cover letter is great,
so I'd just repeat some of that here.

Other than that, for the ficus bits:

Reviewed-by: Ezequiel Garcia 

Thanks very much for this work!
Ezequiel


> Signed-off-by: Manivannan Sadhasivam 
> ---
>  arch/arm64/boot/dts/rockchip/rk3399-ficus.dts | 429 +
>  .../boot/dts/rockchip/rk3399-rock960.dtsi | 439 ++
>  2 files changed, 440 insertions(+), 428 deletions(-)
>  create mode 100644 arch/arm64/boot/dts/rockchip/rk3399-rock960.dtsi
> 
> diff --git a/arch/arm64/boot/dts/rockchip/rk3399-ficus.dts 
> b/arch/arm64/boot/dts/rockchip/rk3399-ficus.dts
> index 8978d924eb83..7f6ec37d5a69 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3399-ficus.dts
> +++ b/arch/arm64/boot/dts/rockchip/rk3399-ficus.dts
> @@ -7,8 +7,7 @@
>   */
>  
>  /dts-v1/;
> -#include "rk3399.dtsi"
> -#include "rk3399-opp.dtsi"
> +#include "rk3399-rock960.dtsi"
>  
>  / {
>   model = "96boards RK3399 Ficus";
> @@ -25,31 +24,6 @@
>   #clock-cells = <0>;
>   };
>  
> - vcc1v8_s0: vcc1v8-s0 {
> - compatible = "regulator-fixed";
> - regulator-name = "vcc1v8_s0";
> - regulator-min-microvolt = <180>;
> - regulator-max-microvolt = <180>;
> - regulator-always-on;
> - };
> -
> - vcc_sys: vcc-sys {
> - compatible = "regulator-fixed";
> - regulator-name = "vcc_sys";
> - regulator-min-microvolt = <500>;
> - regulator-max-microvolt = <500>;
> - regulator-always-on;
> - };
> -
> - vcc3v3_sys: vcc3v3-sys {
> - compatible = "regulator-fixed";
> - regulator-name = "vcc3v3_sys";
> - regulator-min-microvolt = <330>;
> - regulator-max-microvolt = <330>;
> - regulator-always-on;
> - vin-supply = <_sys>;
> - };
> -
>   vcc3v3_pcie: vcc3v3-pcie-regulator {
>   compatible = "regulator-fixed";
>   enable-active-high;
> @@ -75,46 +49,6 @@
>   regulator-always-on;
>   vin-supply = <_sys>;
>   };
> -
> - vdd_log: vdd-log {
> - compatible = "pwm-regulator";
> - pwms = < 0 25000 0>;
> - regulator-name = "vdd_log";
> - regulator-min-microvolt = <80>;
> - regulator-max-microvolt = <140>;
> - regulator-always-on;
> - regulator-boot-on;
> - vin-supply = <_sys>;
> - };
> -
> -};
> -
> -_l0 {
> - cpu-supply = <_cpu_l>;
> -};
> -
> -_l1 {
> - cpu-supply = <_cpu_l>;
> -};
> -
> -_l2 {
> - cpu-supply = <_cpu_l>;
> -};
> -
> -_l3 {
> - cpu-supply = <_cpu_l>;
> -};
> -
> -_b0 {
> - cpu-supply = <_cpu_b>;
> -};
> -
> -_b1 {
> - cpu-supply = <_cpu_b>;
> -};
> -
> -_phy {
> - status = "okay";
>  };
>  
>   {
> @@ -133,263 +67,6 @@
>   status = "okay";
>  };
>  
> - {
> - ddc-i2c-bus = <>;
> - pinctrl-names = "default";
> - pinctrl-0 = <_cec>;
> - status = "okay";
> -};
> -
> - {
> - clock-frequency = <40>;
> - i2c-scl-rising-time-ns = <168>;
> - i2c-scl-falling-time-ns = <4>;
> - status = "okay";
> -
> - vdd_cpu_b: regulator@40 {
> - compatible = "silergy,syr827";
> - reg = <0x40>;
> - fcs,suspend-voltage-selector = <1>;
> - regulator-name = "vdd_cpu_b";
> - regulator-min-microvolt = <712500>;
> - regulator-max-microvolt = <150>;
> -  

Re: [PATCH v2 1/4] arm64: dts: rockchip: Split out common nodes for Rock960 based boards

[Ezequiel Garcia]

On Tue, 2018-09-11 at 08:00 +0530, Manivannan Sadhasivam wrote:
> Since the same family members of Rock960 boards (Rock960 and Ficus)
> share the same configuration, split out the common nodes into a common
> dtsi file for reducing code duplication. The board specific nodes for
> Ficus boards are then placed in corresponding board DTS file.
> 

I think it should be possible to move the common USB nodes to the dtsi file,
and keep the board-specific (phy-supply property) in the dts files:

_host {
phy-supply = <_host>;
};

_host {
phy-supply = <_host>;
};

Also, I believe it would be good to have some more details
in this commit log. The information on the cover letter is great,
so I'd just repeat some of that here.

Other than that, for the ficus bits:

Reviewed-by: Ezequiel Garcia 

Thanks very much for this work!
Ezequiel


> Signed-off-by: Manivannan Sadhasivam 
> ---
>  arch/arm64/boot/dts/rockchip/rk3399-ficus.dts | 429 +
>  .../boot/dts/rockchip/rk3399-rock960.dtsi | 439 ++
>  2 files changed, 440 insertions(+), 428 deletions(-)
>  create mode 100644 arch/arm64/boot/dts/rockchip/rk3399-rock960.dtsi
> 
> diff --git a/arch/arm64/boot/dts/rockchip/rk3399-ficus.dts 
> b/arch/arm64/boot/dts/rockchip/rk3399-ficus.dts
> index 8978d924eb83..7f6ec37d5a69 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3399-ficus.dts
> +++ b/arch/arm64/boot/dts/rockchip/rk3399-ficus.dts
> @@ -7,8 +7,7 @@
>   */
>  
>  /dts-v1/;
> -#include "rk3399.dtsi"
> -#include "rk3399-opp.dtsi"
> +#include "rk3399-rock960.dtsi"
>  
>  / {
>   model = "96boards RK3399 Ficus";
> @@ -25,31 +24,6 @@
>   #clock-cells = <0>;
>   };
>  
> - vcc1v8_s0: vcc1v8-s0 {
> - compatible = "regulator-fixed";
> - regulator-name = "vcc1v8_s0";
> - regulator-min-microvolt = <180>;
> - regulator-max-microvolt = <180>;
> - regulator-always-on;
> - };
> -
> - vcc_sys: vcc-sys {
> - compatible = "regulator-fixed";
> - regulator-name = "vcc_sys";
> - regulator-min-microvolt = <500>;
> - regulator-max-microvolt = <500>;
> - regulator-always-on;
> - };
> -
> - vcc3v3_sys: vcc3v3-sys {
> - compatible = "regulator-fixed";
> - regulator-name = "vcc3v3_sys";
> - regulator-min-microvolt = <330>;
> - regulator-max-microvolt = <330>;
> - regulator-always-on;
> - vin-supply = <_sys>;
> - };
> -
>   vcc3v3_pcie: vcc3v3-pcie-regulator {
>   compatible = "regulator-fixed";
>   enable-active-high;
> @@ -75,46 +49,6 @@
>   regulator-always-on;
>   vin-supply = <_sys>;
>   };
> -
> - vdd_log: vdd-log {
> - compatible = "pwm-regulator";
> - pwms = < 0 25000 0>;
> - regulator-name = "vdd_log";
> - regulator-min-microvolt = <80>;
> - regulator-max-microvolt = <140>;
> - regulator-always-on;
> - regulator-boot-on;
> - vin-supply = <_sys>;
> - };
> -
> -};
> -
> -_l0 {
> - cpu-supply = <_cpu_l>;
> -};
> -
> -_l1 {
> - cpu-supply = <_cpu_l>;
> -};
> -
> -_l2 {
> - cpu-supply = <_cpu_l>;
> -};
> -
> -_l3 {
> - cpu-supply = <_cpu_l>;
> -};
> -
> -_b0 {
> - cpu-supply = <_cpu_b>;
> -};
> -
> -_b1 {
> - cpu-supply = <_cpu_b>;
> -};
> -
> -_phy {
> - status = "okay";
>  };
>  
>   {
> @@ -133,263 +67,6 @@
>   status = "okay";
>  };
>  
> - {
> - ddc-i2c-bus = <>;
> - pinctrl-names = "default";
> - pinctrl-0 = <_cec>;
> - status = "okay";
> -};
> -
> - {
> - clock-frequency = <40>;
> - i2c-scl-rising-time-ns = <168>;
> - i2c-scl-falling-time-ns = <4>;
> - status = "okay";
> -
> - vdd_cpu_b: regulator@40 {
> - compatible = "silergy,syr827";
> - reg = <0x40>;
> - fcs,suspend-voltage-selector = <1>;
> - regulator-name = "vdd_cpu_b";
> - regulator-min-microvolt = <712500>;
> - regulator-max-microvolt = <150>;
> -  

Re: [PATCH v8 4/8] media: platform: Add Cedrus VPU decoder driver

[Ezequiel Garcia]

On Tue, 2018-08-28 at 09:34 +0200, Paul Kocialkowski wrote:
> This introduces the Cedrus VPU driver that supports the VPU found in
> Allwinner SoCs, also known as Video Engine. It is implemented through
> a v4l2 m2m decoder device and a media device (used for media requests).
> So far, it only supports MPEG2 decoding.
> 
> Since this VPU is stateless, synchronization with media requests is
> required in order to ensure consistency between frame headers that
> contain metadata about the frame to process and the raw slice data that
> is used to generate the frame.
> 
> This driver was made possible thanks to the long-standing effort
> carried out by the linux-sunxi community in the interest of reverse
> engineering, documenting and implementing support for Allwinner VPU.
> 
> Signed-off-by: Paul Kocialkowski 
> ---
>  MAINTAINERS   |   7 +
>  drivers/staging/media/Kconfig |   2 +
>  drivers/staging/media/Makefile|   1 +
>  drivers/staging/media/sunxi/Kconfig   |  15 +
>  drivers/staging/media/sunxi/Makefile  |   1 +
>  drivers/staging/media/sunxi/cedrus/Kconfig|  14 +
>  drivers/staging/media/sunxi/cedrus/Makefile   |   3 +
>  drivers/staging/media/sunxi/cedrus/cedrus.c   | 420 +
>  drivers/staging/media/sunxi/cedrus/cedrus.h   | 167 +
>  .../staging/media/sunxi/cedrus/cedrus_dec.c   | 116 
>  .../staging/media/sunxi/cedrus/cedrus_dec.h   |  28 +
>  .../staging/media/sunxi/cedrus/cedrus_hw.c| 322 ++
>  .../staging/media/sunxi/cedrus/cedrus_hw.h|  30 +
>  .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 235 +++
>  .../staging/media/sunxi/cedrus/cedrus_regs.h  | 233 +++
>  .../staging/media/sunxi/cedrus/cedrus_video.c | 574 ++
>  .../staging/media/sunxi/cedrus/cedrus_video.h |  32 +
>  17 files changed, 2200 insertions(+)
>  create mode 100644 drivers/staging/media/sunxi/Kconfig
>  create mode 100644 drivers/staging/media/sunxi/Makefile
>  create mode 100644 drivers/staging/media/sunxi/cedrus/Kconfig
>  create mode 100644 drivers/staging/media/sunxi/cedrus/Makefile
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.h
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.h
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.h
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_regs.h
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.h
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 435e6c08c694..08065d53c69d 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -656,6 +656,13 @@ L:   linux-cry...@vger.kernel.org
>  S:   Maintained
>  F:   drivers/crypto/sunxi-ss/
>  
> +ALLWINNER VPU DRIVER
> +M:   Maxime Ripard 
> +M:   Paul Kocialkowski 
> +L:   linux-me...@vger.kernel.org
> +S:   Maintained
> +F:   drivers/staging/media/sunxi/cedrus/
> +
>  ALPHA PORT
>  M:   Richard Henderson 
>  M:   Ivan Kokshaysky 
> diff --git a/drivers/staging/media/Kconfig b/drivers/staging/media/Kconfig
> index db5cf67047ad..b3620a8f2d9f 100644
> --- a/drivers/staging/media/Kconfig
> +++ b/drivers/staging/media/Kconfig
> @@ -31,6 +31,8 @@ source "drivers/staging/media/mt9t031/Kconfig"
>  
>  source "drivers/staging/media/omap4iss/Kconfig"
>  
> +source "drivers/staging/media/sunxi/Kconfig"
> +
>  source "drivers/staging/media/tegra-vde/Kconfig"
>  
>  source "drivers/staging/media/zoran/Kconfig"
> diff --git a/drivers/staging/media/Makefile b/drivers/staging/media/Makefile
> index 503fbe47fa58..42948f805548 100644
> --- a/drivers/staging/media/Makefile
> +++ b/drivers/staging/media/Makefile
> @@ -5,5 +5,6 @@ obj-$(CONFIG_SOC_CAMERA_IMX074)   += imx074/
>  obj-$(CONFIG_SOC_CAMERA_MT9T031) += mt9t031/
>  obj-$(CONFIG_VIDEO_DM365_VPFE)   += davinci_vpfe/
>  obj-$(CONFIG_VIDEO_OMAP4)+= omap4iss/
> +obj-$(CONFIG_VIDEO_SUNXI)+= sunxi/
>  obj-$(CONFIG_TEGRA_VDE)  += tegra-vde/
>  obj-$(CONFIG_VIDEO_ZORAN)+= zoran/
> diff --git a/drivers/staging/media/sunxi/Kconfig 
> b/drivers/staging/media/sunxi/Kconfig
> new file mode 100644
> index ..c78d92240ceb
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/Kconfig
> @@ -0,0 +1,15 @@
> +config VIDEO_SUNXI
> + bool "Allwinner sunXi family Video Devices"
> + depends on ARCH_SUNXI || COMPILE_TEST
> + help
> +   If you have an Allwinner SoC based on the sunXi family, say Y.
> +
> +   Note that this option doesn't include new drivers in the
> +   kernel: saying N will just cause Kconfig to skip all the
> +   questions about Allwinner media devices.
> +
> +if 

Re: [PATCH v8 4/8] media: platform: Add Cedrus VPU decoder driver

[Ezequiel Garcia]

On Tue, 2018-08-28 at 09:34 +0200, Paul Kocialkowski wrote:
> This introduces the Cedrus VPU driver that supports the VPU found in
> Allwinner SoCs, also known as Video Engine. It is implemented through
> a v4l2 m2m decoder device and a media device (used for media requests).
> So far, it only supports MPEG2 decoding.
> 
> Since this VPU is stateless, synchronization with media requests is
> required in order to ensure consistency between frame headers that
> contain metadata about the frame to process and the raw slice data that
> is used to generate the frame.
> 
> This driver was made possible thanks to the long-standing effort
> carried out by the linux-sunxi community in the interest of reverse
> engineering, documenting and implementing support for Allwinner VPU.
> 
> Signed-off-by: Paul Kocialkowski 
> ---
>  MAINTAINERS   |   7 +
>  drivers/staging/media/Kconfig |   2 +
>  drivers/staging/media/Makefile|   1 +
>  drivers/staging/media/sunxi/Kconfig   |  15 +
>  drivers/staging/media/sunxi/Makefile  |   1 +
>  drivers/staging/media/sunxi/cedrus/Kconfig|  14 +
>  drivers/staging/media/sunxi/cedrus/Makefile   |   3 +
>  drivers/staging/media/sunxi/cedrus/cedrus.c   | 420 +
>  drivers/staging/media/sunxi/cedrus/cedrus.h   | 167 +
>  .../staging/media/sunxi/cedrus/cedrus_dec.c   | 116 
>  .../staging/media/sunxi/cedrus/cedrus_dec.h   |  28 +
>  .../staging/media/sunxi/cedrus/cedrus_hw.c| 322 ++
>  .../staging/media/sunxi/cedrus/cedrus_hw.h|  30 +
>  .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 235 +++
>  .../staging/media/sunxi/cedrus/cedrus_regs.h  | 233 +++
>  .../staging/media/sunxi/cedrus/cedrus_video.c | 574 ++
>  .../staging/media/sunxi/cedrus/cedrus_video.h |  32 +
>  17 files changed, 2200 insertions(+)
>  create mode 100644 drivers/staging/media/sunxi/Kconfig
>  create mode 100644 drivers/staging/media/sunxi/Makefile
>  create mode 100644 drivers/staging/media/sunxi/cedrus/Kconfig
>  create mode 100644 drivers/staging/media/sunxi/cedrus/Makefile
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.h
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.h
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.h
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_regs.h
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.c
>  create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.h
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 435e6c08c694..08065d53c69d 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -656,6 +656,13 @@ L:   linux-cry...@vger.kernel.org
>  S:   Maintained
>  F:   drivers/crypto/sunxi-ss/
>  
> +ALLWINNER VPU DRIVER
> +M:   Maxime Ripard 
> +M:   Paul Kocialkowski 
> +L:   linux-me...@vger.kernel.org
> +S:   Maintained
> +F:   drivers/staging/media/sunxi/cedrus/
> +
>  ALPHA PORT
>  M:   Richard Henderson 
>  M:   Ivan Kokshaysky 
> diff --git a/drivers/staging/media/Kconfig b/drivers/staging/media/Kconfig
> index db5cf67047ad..b3620a8f2d9f 100644
> --- a/drivers/staging/media/Kconfig
> +++ b/drivers/staging/media/Kconfig
> @@ -31,6 +31,8 @@ source "drivers/staging/media/mt9t031/Kconfig"
>  
>  source "drivers/staging/media/omap4iss/Kconfig"
>  
> +source "drivers/staging/media/sunxi/Kconfig"
> +
>  source "drivers/staging/media/tegra-vde/Kconfig"
>  
>  source "drivers/staging/media/zoran/Kconfig"
> diff --git a/drivers/staging/media/Makefile b/drivers/staging/media/Makefile
> index 503fbe47fa58..42948f805548 100644
> --- a/drivers/staging/media/Makefile
> +++ b/drivers/staging/media/Makefile
> @@ -5,5 +5,6 @@ obj-$(CONFIG_SOC_CAMERA_IMX074)   += imx074/
>  obj-$(CONFIG_SOC_CAMERA_MT9T031) += mt9t031/
>  obj-$(CONFIG_VIDEO_DM365_VPFE)   += davinci_vpfe/
>  obj-$(CONFIG_VIDEO_OMAP4)+= omap4iss/
> +obj-$(CONFIG_VIDEO_SUNXI)+= sunxi/
>  obj-$(CONFIG_TEGRA_VDE)  += tegra-vde/
>  obj-$(CONFIG_VIDEO_ZORAN)+= zoran/
> diff --git a/drivers/staging/media/sunxi/Kconfig 
> b/drivers/staging/media/sunxi/Kconfig
> new file mode 100644
> index ..c78d92240ceb
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/Kconfig
> @@ -0,0 +1,15 @@
> +config VIDEO_SUNXI
> + bool "Allwinner sunXi family Video Devices"
> + depends on ARCH_SUNXI || COMPILE_TEST
> + help
> +   If you have an Allwinner SoC based on the sunXi family, say Y.
> +
> +   Note that this option doesn't include new drivers in the
> +   kernel: saying N will just cause Kconfig to skip all the
> +   questions about Allwinner media devices.
> +
> +if 

Re: [PATCH v1] arm64: dts: rockchip: add support for ROC-RK3399-PC board

[Ezequiel Garcia]

On Thu, 2018-07-26 at 15:13 +0800, d...@t-chip.com.cn wrote:
> From: Levin Du 
> 
> ROC-RK3399-PC is a power efficient 4GB LPDDR4 single board
> computer with USB 3.0 and Gigabit Ethernet in a form factor
> compatible with the Raspberry Pi. It is based on the Rockchip
> RK3399 SoC, powered by the Type-C port.
> 
> The devicetree currently supports peripherals of:
> 
> - Ethernet
> - HDMI
> - SD Card
> - UART2 debug
> - TYPE-C

Hi Levin,

Last time I tried to make the mainline fusb driver work, it really didn't.
Is USB 3.0 Type-C actually working with mainline on this board?

Thanks!
Eze

Re: [PATCH v1] arm64: dts: rockchip: add support for ROC-RK3399-PC board

[Ezequiel Garcia]

On Thu, 2018-07-26 at 15:13 +0800, d...@t-chip.com.cn wrote:
> From: Levin Du 
> 
> ROC-RK3399-PC is a power efficient 4GB LPDDR4 single board
> computer with USB 3.0 and Gigabit Ethernet in a form factor
> compatible with the Raspberry Pi. It is based on the Rockchip
> RK3399 SoC, powered by the Type-C port.
> 
> The devicetree currently supports peripherals of:
> 
> - Ethernet
> - HDMI
> - SD Card
> - UART2 debug
> - TYPE-C

Hi Levin,

Last time I tried to make the mainline fusb driver work, it really didn't.
Is USB 3.0 Type-C actually working with mainline on this board?

Thanks!
Eze

Re: [PATCH v3] PM / devfreq: Fix devfreq_add_device() when drivers are built as modules.

[Ezequiel Garcia]

Hey Akhil,

On Fri, 2018-06-22 at 12:33 +0530, Akhil P Oommen wrote:
> On 6/22/2018 6:41 AM, Ezequiel Garcia wrote:
> > Hey Enric,
> > 
> > On Fri, 2018-06-22 at 00:04 +0200, Enric Balletbo i Serra wrote:
> > > When the devfreq driver and the governor driver are built as
> > > modules,
> > > the call to devfreq_add_device() or governor_store() fails
> > > because
> > > the
> > > governor driver is not loaded at the time the devfreq driver
> > > loads.
> > > The
> > > devfreq driver has a build dependency on the governor but also
> > > should
> > > have a runtime dependency. We need to make sure that the governor
> > > driver
> > > is loaded before the devfreq driver.
> > > 
> > > This patch fixes this bug by adding a try_then_request_governor()
> > > function. First tries to find the governor, and then, if it is
> > > not
> > > found,
> > > it requests the module and tries again.
> > > 
> > > Fixes: 1b5c1be2c88e (PM / devfreq: map devfreq drivers to
> > > governor
> > > using name)
> > > Signed-off-by: Enric Balletbo i Serra  > > om>
> > > ---
> > > 
> > > Changes in v3:
> > > - Remove unneded change in dev_err message.
> > > - Fix err returned value in case to not find the governor.
> > > 
> > > Changes in v2:
> > > - Add a new function to request the module and call that function
> > > from
> > >devfreq_add_device and governor_store.
> > > 
> > >   drivers/devfreq/devfreq.c | 65
> > > -
> > > --
> > 
> > [snip snip]
> > > - governor = find_devfreq_governor(devfreq-
> > > >governor_name);
> > > + governor = try_then_request_governor(devfreq-
> > > > governor_name);
> > > 
> > >   if (IS_ERR(governor)) {
> > >   dev_err(dev, "%s: Unable to find governor for
> > > the
> > > device\n",
> > >   __func__);
> > >   err = PTR_ERR(governor);
> > > - goto err_init;
> > > + goto err_unregister;
> > >   }
> > >   
> > > + mutex_lock(_list_lock);
> > > +
> > 
> > I know it's not something we are introducing in this patch,
> > but still... calling a hook with a mutex held looks
> > fishy to me.
> > 
> > This lock should only protect the list, unless I am missing
> > something.
> > 
> > >   devfreq->governor = governor;
> > >   err = devfreq->governor->event_handler(devfreq,
> > > DEVFREQ_GOV_START,
> > >   NULL);
> > > @@ -663,14 +703,16 @@ struct devfreq *devfreq_add_device(struct
> > > device *dev,
> > >   __func__);
> > >   goto err_init;
> > >   }
> > > +
> > > + list_add(>node, _list);
> > > +
> > >   mutex_unlock(_list_lock);
> > >   
> > >   return devfreq;
> > >   
> > >   err_init:
> > > - list_del(>node);
> > >   mutex_unlock(_list_lock);
> > > -
> > > +err_unregister:
> > >   device_unregister(>dev);
> > >   err_dev:
> > >   if (devfreq)
> > > @@ -988,12 +1030,13 @@ static ssize_t governor_store(struct
> > > device
> > > *dev, struct device_attribute *attr,
> > >   if (ret != 1)
> > >   return -EINVAL;
> > >   
> > > - mutex_lock(_list_lock);
> > > - governor = find_devfreq_governor(str_governor);
> > > + governor = try_then_request_governor(str_governor);
> > >   if (IS_ERR(governor)) {
> > > - ret = PTR_ERR(governor);
> > > - goto out;
> > > + return PTR_ERR(governor);
> > >   }
> > > +
> > > + mutex_lock(_list_lock);
> > > +
> > >   if (df->governor == governor) {
> > >   ret = 0;
> > >   goto out;
> > > -- 
> > > 2.17.1
> > > 
> > > 
> > 
> > Regards,
> > Eze
> 
> Adding to Ezequiel's point, shouldn't we take more granular lock 
> (devfreq->lock) first and then call devfreq_list_lock at the time of 
> adding to the list?
> 

Not sure why we should do that. devfreq->lock should be used to
protect the struct devfreq state, while the devfreq_list_lock
is apparently protecting the two lists (which seem unrelated
lists).

So, the two locks are not correlated.

Regards,
Eze

Re: [PATCH v3] PM / devfreq: Fix devfreq_add_device() when drivers are built as modules.

[Ezequiel Garcia]

Hey Akhil,

On Fri, 2018-06-22 at 12:33 +0530, Akhil P Oommen wrote:
> On 6/22/2018 6:41 AM, Ezequiel Garcia wrote:
> > Hey Enric,
> > 
> > On Fri, 2018-06-22 at 00:04 +0200, Enric Balletbo i Serra wrote:
> > > When the devfreq driver and the governor driver are built as
> > > modules,
> > > the call to devfreq_add_device() or governor_store() fails
> > > because
> > > the
> > > governor driver is not loaded at the time the devfreq driver
> > > loads.
> > > The
> > > devfreq driver has a build dependency on the governor but also
> > > should
> > > have a runtime dependency. We need to make sure that the governor
> > > driver
> > > is loaded before the devfreq driver.
> > > 
> > > This patch fixes this bug by adding a try_then_request_governor()
> > > function. First tries to find the governor, and then, if it is
> > > not
> > > found,
> > > it requests the module and tries again.
> > > 
> > > Fixes: 1b5c1be2c88e (PM / devfreq: map devfreq drivers to
> > > governor
> > > using name)
> > > Signed-off-by: Enric Balletbo i Serra  > > om>
> > > ---
> > > 
> > > Changes in v3:
> > > - Remove unneded change in dev_err message.
> > > - Fix err returned value in case to not find the governor.
> > > 
> > > Changes in v2:
> > > - Add a new function to request the module and call that function
> > > from
> > >devfreq_add_device and governor_store.
> > > 
> > >   drivers/devfreq/devfreq.c | 65
> > > -
> > > --
> > 
> > [snip snip]
> > > - governor = find_devfreq_governor(devfreq-
> > > >governor_name);
> > > + governor = try_then_request_governor(devfreq-
> > > > governor_name);
> > > 
> > >   if (IS_ERR(governor)) {
> > >   dev_err(dev, "%s: Unable to find governor for
> > > the
> > > device\n",
> > >   __func__);
> > >   err = PTR_ERR(governor);
> > > - goto err_init;
> > > + goto err_unregister;
> > >   }
> > >   
> > > + mutex_lock(_list_lock);
> > > +
> > 
> > I know it's not something we are introducing in this patch,
> > but still... calling a hook with a mutex held looks
> > fishy to me.
> > 
> > This lock should only protect the list, unless I am missing
> > something.
> > 
> > >   devfreq->governor = governor;
> > >   err = devfreq->governor->event_handler(devfreq,
> > > DEVFREQ_GOV_START,
> > >   NULL);
> > > @@ -663,14 +703,16 @@ struct devfreq *devfreq_add_device(struct
> > > device *dev,
> > >   __func__);
> > >   goto err_init;
> > >   }
> > > +
> > > + list_add(>node, _list);
> > > +
> > >   mutex_unlock(_list_lock);
> > >   
> > >   return devfreq;
> > >   
> > >   err_init:
> > > - list_del(>node);
> > >   mutex_unlock(_list_lock);
> > > -
> > > +err_unregister:
> > >   device_unregister(>dev);
> > >   err_dev:
> > >   if (devfreq)
> > > @@ -988,12 +1030,13 @@ static ssize_t governor_store(struct
> > > device
> > > *dev, struct device_attribute *attr,
> > >   if (ret != 1)
> > >   return -EINVAL;
> > >   
> > > - mutex_lock(_list_lock);
> > > - governor = find_devfreq_governor(str_governor);
> > > + governor = try_then_request_governor(str_governor);
> > >   if (IS_ERR(governor)) {
> > > - ret = PTR_ERR(governor);
> > > - goto out;
> > > + return PTR_ERR(governor);
> > >   }
> > > +
> > > + mutex_lock(_list_lock);
> > > +
> > >   if (df->governor == governor) {
> > >   ret = 0;
> > >   goto out;
> > > -- 
> > > 2.17.1
> > > 
> > > 
> > 
> > Regards,
> > Eze
> 
> Adding to Ezequiel's point, shouldn't we take more granular lock 
> (devfreq->lock) first and then call devfreq_list_lock at the time of 
> adding to the list?
> 

Not sure why we should do that. devfreq->lock should be used to
protect the struct devfreq state, while the devfreq_list_lock
is apparently protecting the two lists (which seem unrelated
lists).

So, the two locks are not correlated.

Regards,
Eze

Re: [PATCH v3] PM / devfreq: Fix devfreq_add_device() when drivers are built as modules.

[Ezequiel Garcia]

Hey Enric,

On Fri, 2018-06-22 at 00:04 +0200, Enric Balletbo i Serra wrote:
> When the devfreq driver and the governor driver are built as modules,
> the call to devfreq_add_device() or governor_store() fails because
> the
> governor driver is not loaded at the time the devfreq driver loads.
> The
> devfreq driver has a build dependency on the governor but also should
> have a runtime dependency. We need to make sure that the governor
> driver
> is loaded before the devfreq driver.
> 
> This patch fixes this bug by adding a try_then_request_governor()
> function. First tries to find the governor, and then, if it is not
> found,
> it requests the module and tries again.
> 
> Fixes: 1b5c1be2c88e (PM / devfreq: map devfreq drivers to governor
> using name)
> Signed-off-by: Enric Balletbo i Serra 
> ---
> 
> Changes in v3:
> - Remove unneded change in dev_err message.
> - Fix err returned value in case to not find the governor.
> 
> Changes in v2:
> - Add a new function to request the module and call that function
> from
>   devfreq_add_device and governor_store.
> 
>  drivers/devfreq/devfreq.c | 65 -
> --
[snip snip]
> - governor = find_devfreq_governor(devfreq->governor_name);
> + governor = try_then_request_governor(devfreq-
> >governor_name);
>   if (IS_ERR(governor)) {
>   dev_err(dev, "%s: Unable to find governor for the
> device\n",
>   __func__);
>   err = PTR_ERR(governor);
> - goto err_init;
> + goto err_unregister;
>   }
>  
> + mutex_lock(_list_lock);
> +

I know it's not something we are introducing in this patch,
but still... calling a hook with a mutex held looks
fishy to me.

This lock should only protect the list, unless I am missing
something.

>   devfreq->governor = governor;
>   err = devfreq->governor->event_handler(devfreq,
> DEVFREQ_GOV_START,
>   NULL);
> @@ -663,14 +703,16 @@ struct devfreq *devfreq_add_device(struct
> device *dev,
>   __func__);
>   goto err_init;
>   }
> +
> + list_add(>node, _list);
> +
>   mutex_unlock(_list_lock);
>  
>   return devfreq;
>  
>  err_init:
> - list_del(>node);
>   mutex_unlock(_list_lock);
> -
> +err_unregister:
>   device_unregister(>dev);
>  err_dev:
>   if (devfreq)
> @@ -988,12 +1030,13 @@ static ssize_t governor_store(struct device
> *dev, struct device_attribute *attr,
>   if (ret != 1)
>   return -EINVAL;
>  
> - mutex_lock(_list_lock);
> - governor = find_devfreq_governor(str_governor);
> + governor = try_then_request_governor(str_governor);
>   if (IS_ERR(governor)) {
> - ret = PTR_ERR(governor);
> - goto out;
> + return PTR_ERR(governor);
>   }
> +
> + mutex_lock(_list_lock);
> +
>   if (df->governor == governor) {
>   ret = 0;
>   goto out;
> -- 
> 2.17.1
> 
> 


Regards,
Eze

Re: [PATCH v3] PM / devfreq: Fix devfreq_add_device() when drivers are built as modules.

[Ezequiel Garcia]

Hey Enric,

On Fri, 2018-06-22 at 00:04 +0200, Enric Balletbo i Serra wrote:
> When the devfreq driver and the governor driver are built as modules,
> the call to devfreq_add_device() or governor_store() fails because
> the
> governor driver is not loaded at the time the devfreq driver loads.
> The
> devfreq driver has a build dependency on the governor but also should
> have a runtime dependency. We need to make sure that the governor
> driver
> is loaded before the devfreq driver.
> 
> This patch fixes this bug by adding a try_then_request_governor()
> function. First tries to find the governor, and then, if it is not
> found,
> it requests the module and tries again.
> 
> Fixes: 1b5c1be2c88e (PM / devfreq: map devfreq drivers to governor
> using name)
> Signed-off-by: Enric Balletbo i Serra 
> ---
> 
> Changes in v3:
> - Remove unneded change in dev_err message.
> - Fix err returned value in case to not find the governor.
> 
> Changes in v2:
> - Add a new function to request the module and call that function
> from
>   devfreq_add_device and governor_store.
> 
>  drivers/devfreq/devfreq.c | 65 -
> --
[snip snip]
> - governor = find_devfreq_governor(devfreq->governor_name);
> + governor = try_then_request_governor(devfreq-
> >governor_name);
>   if (IS_ERR(governor)) {
>   dev_err(dev, "%s: Unable to find governor for the
> device\n",
>   __func__);
>   err = PTR_ERR(governor);
> - goto err_init;
> + goto err_unregister;
>   }
>  
> + mutex_lock(_list_lock);
> +

I know it's not something we are introducing in this patch,
but still... calling a hook with a mutex held looks
fishy to me.

This lock should only protect the list, unless I am missing
something.

>   devfreq->governor = governor;
>   err = devfreq->governor->event_handler(devfreq,
> DEVFREQ_GOV_START,
>   NULL);
> @@ -663,14 +703,16 @@ struct devfreq *devfreq_add_device(struct
> device *dev,
>   __func__);
>   goto err_init;
>   }
> +
> + list_add(>node, _list);
> +
>   mutex_unlock(_list_lock);
>  
>   return devfreq;
>  
>  err_init:
> - list_del(>node);
>   mutex_unlock(_list_lock);
> -
> +err_unregister:
>   device_unregister(>dev);
>  err_dev:
>   if (devfreq)
> @@ -988,12 +1030,13 @@ static ssize_t governor_store(struct device
> *dev, struct device_attribute *attr,
>   if (ret != 1)
>   return -EINVAL;
>  
> - mutex_lock(_list_lock);
> - governor = find_devfreq_governor(str_governor);
> + governor = try_then_request_governor(str_governor);
>   if (IS_ERR(governor)) {
> - ret = PTR_ERR(governor);
> - goto out;
> + return PTR_ERR(governor);
>   }
> +
> + mutex_lock(_list_lock);
> +
>   if (df->governor == governor) {
>   ret = 0;
>   goto out;
> -- 
> 2.17.1
> 
> 


Regards,
Eze

Re: [PATCH v8 03/14] iommu/rockchip: Request irqs in rk_iommu_probe()

[Ezequiel Garcia]

Hey Jeffy, Robin:

Some odd issues to report here.

On 23 March 2018 at 04:38, Jeffy Chen  wrote:
> Move request_irq to the end of rk_iommu_probe().
>
> Suggested-by: Robin Murphy 
> Signed-off-by: Jeffy Chen 
> Acked-by: Robin Murphy 
> ---
>
> Changes in v8: None
> Changes in v7: None
> Changes in v6: None
> Changes in v5: None
> Changes in v4: None
> Changes in v3:
> Loop platform_get_irq() as Robin suggested.
>
> Changes in v2: None
>
>  drivers/iommu/rockchip-iommu.c | 38 +-
>  1 file changed, 9 insertions(+), 29 deletions(-)
>
> diff --git a/drivers/iommu/rockchip-iommu.c b/drivers/iommu/rockchip-iommu.c
> index 73117dbe839e..ec3ff936aa60 100644
> --- a/drivers/iommu/rockchip-iommu.c
> +++ b/drivers/iommu/rockchip-iommu.c
> @@ -90,8 +90,6 @@ struct rk_iommu {
> struct device *dev;
> void __iomem **bases;
> int num_mmu;
> -   int *irq;
> -   int num_irq;
> bool reset_disabled;
> struct iommu_device iommu;
> struct list_head node; /* entry in rk_iommu_domain.iommus */
> @@ -830,13 +828,6 @@ static int rk_iommu_attach_device(struct iommu_domain 
> *domain,
>
> iommu->domain = domain;
>
> -   for (i = 0; i < iommu->num_irq; i++) {
> -   ret = devm_request_irq(iommu->dev, iommu->irq[i], 
> rk_iommu_irq,
> -  IRQF_SHARED, dev_name(dev), iommu);
> -   if (ret)
> -   return ret;
> -   }
> -
> for (i = 0; i < iommu->num_mmu; i++) {
> rk_iommu_write(iommu->bases[i], RK_MMU_DTE_ADDR,
>rk_domain->dt_dma);
> @@ -885,9 +876,6 @@ static void rk_iommu_detach_device(struct iommu_domain 
> *domain,
> }
> rk_iommu_disable_stall(iommu);
>
> -   for (i = 0; i < iommu->num_irq; i++)
> -   devm_free_irq(iommu->dev, iommu->irq[i], iommu);
> -
> iommu->domain = NULL;
>
> dev_dbg(dev, "Detached from iommu domain\n");
> @@ -1138,7 +1126,7 @@ static int rk_iommu_probe(struct platform_device *pdev)
> struct rk_iommu *iommu;
> struct resource *res;
> int num_res = pdev->num_resources;
> -   int err, i;
> +   int err, i, irq;
>
> iommu = devm_kzalloc(dev, sizeof(*iommu), GFP_KERNEL);
> if (!iommu)
> @@ -1165,23 +1153,15 @@ static int rk_iommu_probe(struct platform_device 
> *pdev)
> if (iommu->num_mmu == 0)
> return PTR_ERR(iommu->bases[0]);
>
> -   iommu->num_irq = platform_irq_count(pdev);
> -   if (iommu->num_irq < 0)
> -   return iommu->num_irq;
> -   if (iommu->num_irq == 0)
> -   return -ENXIO;
> +   i = 0;
> +   while ((irq = platform_get_irq(pdev, i++)) != -ENXIO) {
> +   if (irq < 0)
> +   return irq;
>
> -   iommu->irq = devm_kcalloc(dev, iommu->num_irq, sizeof(*iommu->irq),
> - GFP_KERNEL);
> -   if (!iommu->irq)
> -   return -ENOMEM;
> -
> -   for (i = 0; i < iommu->num_irq; i++) {
> -   iommu->irq[i] = platform_get_irq(pdev, i);
> -   if (iommu->irq[i] < 0) {
> -   dev_err(dev, "Failed to get IRQ, %d\n", 
> iommu->irq[i]);
> -   return -ENXIO;
> -   }
> +   err = devm_request_irq(iommu->dev, irq, rk_iommu_irq,
> +  IRQF_SHARED, dev_name(dev), iommu);
> +   if (err)
> +   return err;
> }
>
> iommu->reset_disabled = device_property_read_bool(dev,
> --
> 2.11.0
>
>

Odd as it may be, this patch is causing problems with DRM,
on any recent kernel, either linux-next or v4.17-rc5 shows
the same issue.

I debugged this issue on a RK3288 Rock2 board connected to
a Samsung TV, but I also saw this warning on a RK3399 board.

The issue is a several-second stall at:

[..]
[2.091953] rockchip-drm display-subsystem: bound ff93.vop (ops 
0xc078ebb4)
[2.100310] rockchip-drm display-subsystem: bound ff94.vop (ops 
0xc078ebb4)
[2.108550] dwhdmi-rockchip ff98.hdmi: Detected HDMI TX controller 
v2.00a with HDCP (DWC MHL PHY)
[2.119307] rockchip-drm display-subsystem: bound ff98.hdmi (ops 
0xc0790860)
[2.127588] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[2.134988] [drm] No driver support for vblank timestamp query.
[boot stalls for several seconds]

followed by this warning:

[2.251400] [ cut here ]
[2.251465] WARNING: CPU: 2 PID: 38 at 
/home/zeta/repos/linux/next/kernel/irq/manage.c:525 enable_irq+0x34/0x6c
[2.251479] Unbalanced enable for IRQ 49
[2.251490] Modules linked in:
[2.251537] CPU: 2 PID: 38 Comm: kworker/2:1 Not tainted 
4.17.0-rc5-1-g5bc6dc2896ec-dirty #31
[

Re: [PATCH v8 03/14] iommu/rockchip: Request irqs in rk_iommu_probe()

[Ezequiel Garcia]

Hey Jeffy, Robin:

Some odd issues to report here.

On 23 March 2018 at 04:38, Jeffy Chen  wrote:
> Move request_irq to the end of rk_iommu_probe().
>
> Suggested-by: Robin Murphy 
> Signed-off-by: Jeffy Chen 
> Acked-by: Robin Murphy 
> ---
>
> Changes in v8: None
> Changes in v7: None
> Changes in v6: None
> Changes in v5: None
> Changes in v4: None
> Changes in v3:
> Loop platform_get_irq() as Robin suggested.
>
> Changes in v2: None
>
>  drivers/iommu/rockchip-iommu.c | 38 +-
>  1 file changed, 9 insertions(+), 29 deletions(-)
>
> diff --git a/drivers/iommu/rockchip-iommu.c b/drivers/iommu/rockchip-iommu.c
> index 73117dbe839e..ec3ff936aa60 100644
> --- a/drivers/iommu/rockchip-iommu.c
> +++ b/drivers/iommu/rockchip-iommu.c
> @@ -90,8 +90,6 @@ struct rk_iommu {
> struct device *dev;
> void __iomem **bases;
> int num_mmu;
> -   int *irq;
> -   int num_irq;
> bool reset_disabled;
> struct iommu_device iommu;
> struct list_head node; /* entry in rk_iommu_domain.iommus */
> @@ -830,13 +828,6 @@ static int rk_iommu_attach_device(struct iommu_domain 
> *domain,
>
> iommu->domain = domain;
>
> -   for (i = 0; i < iommu->num_irq; i++) {
> -   ret = devm_request_irq(iommu->dev, iommu->irq[i], 
> rk_iommu_irq,
> -  IRQF_SHARED, dev_name(dev), iommu);
> -   if (ret)
> -   return ret;
> -   }
> -
> for (i = 0; i < iommu->num_mmu; i++) {
> rk_iommu_write(iommu->bases[i], RK_MMU_DTE_ADDR,
>rk_domain->dt_dma);
> @@ -885,9 +876,6 @@ static void rk_iommu_detach_device(struct iommu_domain 
> *domain,
> }
> rk_iommu_disable_stall(iommu);
>
> -   for (i = 0; i < iommu->num_irq; i++)
> -   devm_free_irq(iommu->dev, iommu->irq[i], iommu);
> -
> iommu->domain = NULL;
>
> dev_dbg(dev, "Detached from iommu domain\n");
> @@ -1138,7 +1126,7 @@ static int rk_iommu_probe(struct platform_device *pdev)
> struct rk_iommu *iommu;
> struct resource *res;
> int num_res = pdev->num_resources;
> -   int err, i;
> +   int err, i, irq;
>
> iommu = devm_kzalloc(dev, sizeof(*iommu), GFP_KERNEL);
> if (!iommu)
> @@ -1165,23 +1153,15 @@ static int rk_iommu_probe(struct platform_device 
> *pdev)
> if (iommu->num_mmu == 0)
> return PTR_ERR(iommu->bases[0]);
>
> -   iommu->num_irq = platform_irq_count(pdev);
> -   if (iommu->num_irq < 0)
> -   return iommu->num_irq;
> -   if (iommu->num_irq == 0)
> -   return -ENXIO;
> +   i = 0;
> +   while ((irq = platform_get_irq(pdev, i++)) != -ENXIO) {
> +   if (irq < 0)
> +   return irq;
>
> -   iommu->irq = devm_kcalloc(dev, iommu->num_irq, sizeof(*iommu->irq),
> - GFP_KERNEL);
> -   if (!iommu->irq)
> -   return -ENOMEM;
> -
> -   for (i = 0; i < iommu->num_irq; i++) {
> -   iommu->irq[i] = platform_get_irq(pdev, i);
> -   if (iommu->irq[i] < 0) {
> -   dev_err(dev, "Failed to get IRQ, %d\n", 
> iommu->irq[i]);
> -   return -ENXIO;
> -   }
> +   err = devm_request_irq(iommu->dev, irq, rk_iommu_irq,
> +  IRQF_SHARED, dev_name(dev), iommu);
> +   if (err)
> +   return err;
> }
>
> iommu->reset_disabled = device_property_read_bool(dev,
> --
> 2.11.0
>
>

Odd as it may be, this patch is causing problems with DRM,
on any recent kernel, either linux-next or v4.17-rc5 shows
the same issue.

I debugged this issue on a RK3288 Rock2 board connected to
a Samsung TV, but I also saw this warning on a RK3399 board.

The issue is a several-second stall at:

[..]
[2.091953] rockchip-drm display-subsystem: bound ff93.vop (ops 
0xc078ebb4)
[2.100310] rockchip-drm display-subsystem: bound ff94.vop (ops 
0xc078ebb4)
[2.108550] dwhdmi-rockchip ff98.hdmi: Detected HDMI TX controller 
v2.00a with HDCP (DWC MHL PHY)
[2.119307] rockchip-drm display-subsystem: bound ff98.hdmi (ops 
0xc0790860)
[2.127588] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[2.134988] [drm] No driver support for vblank timestamp query.
[boot stalls for several seconds]

followed by this warning:

[2.251400] [ cut here ]
[2.251465] WARNING: CPU: 2 PID: 38 at 
/home/zeta/repos/linux/next/kernel/irq/manage.c:525 enable_irq+0x34/0x6c
[2.251479] Unbalanced enable for IRQ 49
[2.251490] Modules linked in:
[2.251537] CPU: 2 PID: 38 Comm: kworker/2:1 Not tainted 
4.17.0-rc5-1-g5bc6dc2896ec-dirty #31
[2.251551] Hardware name: Rockchip (Device Tree)
[2.251595] Workqueue: events deferred_probe_work_func

Re: [PATCH v10 08/16] v4l: mark unordered formats

[Ezequiel Garcia]

On Tue, 2018-05-22 at 13:55 +0200, Hans Verkuil wrote:
> On 21/05/18 18:59, Ezequiel Garcia wrote:
> > From: Gustavo Padovan <gustavo.pado...@collabora.com>
> > 
> > Now that we've introduced the V4L2_FMT_FLAG_UNORDERED flag,
> > mark the appropriate formats.
> > 
> > v2: Set unordered flag before calling the driver callback.
> > 
> > Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
> > Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-ioctl.c | 74 
> > +++-
> >  1 file changed, 57 insertions(+), 17 deletions(-)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index f48c505550e0..2135ac235a96 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1102,6 +1102,27 @@ static int v4l_enumoutput(const struct 
> > v4l2_ioctl_ops *ops,
> > return ops->vidioc_enum_output(file, fh, p);
> >  }
> >  
> > +static void v4l_fill_unordered_fmtdesc(struct v4l2_fmtdesc *fmt)
> > +{
> > +   switch (fmt->pixelformat) {
> > +   case V4L2_PIX_FMT_MPEG:
> > +   case V4L2_PIX_FMT_H264:
> > +   case V4L2_PIX_FMT_H264_NO_SC:
> > +   case V4L2_PIX_FMT_H264_MVC:
> > +   case V4L2_PIX_FMT_H263:
> > +   case V4L2_PIX_FMT_MPEG1:
> > +   case V4L2_PIX_FMT_MPEG2:
> > +   case V4L2_PIX_FMT_MPEG4:
> > +   case V4L2_PIX_FMT_XVID:
> > +   case V4L2_PIX_FMT_VC1_ANNEX_G:
> > +   case V4L2_PIX_FMT_VC1_ANNEX_L:
> > +   case V4L2_PIX_FMT_VP8:
> > +   case V4L2_PIX_FMT_VP9:
> > +   case V4L2_PIX_FMT_HEVC:
> > +   fmt->flags |= V4L2_FMT_FLAG_UNORDERED;
> 
> Please add a break here. This prevents potential future errors if new cases
> are added below this line.
> 

Sure.

> > +   }
> > +}
> > +
> >  static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >  {
> > const unsigned sz = sizeof(fmt->description);
> > @@ -1310,61 +1331,80 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc 
> > *fmt)
> >  
> > if (descr)
> > WARN_ON(strlcpy(fmt->description, descr, sz) >= sz);
> > -   fmt->flags = flags;
> > +   fmt->flags |= flags;
> >  }
> >  
> > -static int v4l_enum_fmt(const struct v4l2_ioctl_ops *ops,
> > -   struct file *file, void *fh, void *arg)
> > -{
> > -   struct v4l2_fmtdesc *p = arg;
> > -   int ret = check_fmt(file, p->type);
> >  
> > -   if (ret)
> > -   return ret;
> > -   ret = -EINVAL;
> > +static int __vidioc_enum_fmt(const struct v4l2_ioctl_ops *ops,
> > +struct v4l2_fmtdesc *p,
> > +struct file *file, void *fh)
> > +{
> > +   int ret = 0;
> >  
> > switch (p->type) {
> > case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_cap))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_cap(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_cap(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_cap_mplane))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_cap_mplane(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_cap_mplane(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_VIDEO_OVERLAY:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_overlay))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_overlay(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_overlay(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_out))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_out(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_out(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_out_mplane))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_out_mplane(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_out_mplane(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_SDR_CAPTURE:

Re: [PATCH v10 08/16] v4l: mark unordered formats

[Ezequiel Garcia]

On Tue, 2018-05-22 at 13:55 +0200, Hans Verkuil wrote:
> On 21/05/18 18:59, Ezequiel Garcia wrote:
> > From: Gustavo Padovan 
> > 
> > Now that we've introduced the V4L2_FMT_FLAG_UNORDERED flag,
> > mark the appropriate formats.
> > 
> > v2: Set unordered flag before calling the driver callback.
> > 
> > Signed-off-by: Gustavo Padovan 
> > Signed-off-by: Ezequiel Garcia 
> > ---
> >  drivers/media/v4l2-core/v4l2-ioctl.c | 74 
> > +++-
> >  1 file changed, 57 insertions(+), 17 deletions(-)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index f48c505550e0..2135ac235a96 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1102,6 +1102,27 @@ static int v4l_enumoutput(const struct 
> > v4l2_ioctl_ops *ops,
> > return ops->vidioc_enum_output(file, fh, p);
> >  }
> >  
> > +static void v4l_fill_unordered_fmtdesc(struct v4l2_fmtdesc *fmt)
> > +{
> > +   switch (fmt->pixelformat) {
> > +   case V4L2_PIX_FMT_MPEG:
> > +   case V4L2_PIX_FMT_H264:
> > +   case V4L2_PIX_FMT_H264_NO_SC:
> > +   case V4L2_PIX_FMT_H264_MVC:
> > +   case V4L2_PIX_FMT_H263:
> > +   case V4L2_PIX_FMT_MPEG1:
> > +   case V4L2_PIX_FMT_MPEG2:
> > +   case V4L2_PIX_FMT_MPEG4:
> > +   case V4L2_PIX_FMT_XVID:
> > +   case V4L2_PIX_FMT_VC1_ANNEX_G:
> > +   case V4L2_PIX_FMT_VC1_ANNEX_L:
> > +   case V4L2_PIX_FMT_VP8:
> > +   case V4L2_PIX_FMT_VP9:
> > +   case V4L2_PIX_FMT_HEVC:
> > +   fmt->flags |= V4L2_FMT_FLAG_UNORDERED;
> 
> Please add a break here. This prevents potential future errors if new cases
> are added below this line.
> 

Sure.

> > +   }
> > +}
> > +
> >  static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >  {
> > const unsigned sz = sizeof(fmt->description);
> > @@ -1310,61 +1331,80 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc 
> > *fmt)
> >  
> > if (descr)
> > WARN_ON(strlcpy(fmt->description, descr, sz) >= sz);
> > -   fmt->flags = flags;
> > +   fmt->flags |= flags;
> >  }
> >  
> > -static int v4l_enum_fmt(const struct v4l2_ioctl_ops *ops,
> > -   struct file *file, void *fh, void *arg)
> > -{
> > -   struct v4l2_fmtdesc *p = arg;
> > -   int ret = check_fmt(file, p->type);
> >  
> > -   if (ret)
> > -   return ret;
> > -   ret = -EINVAL;
> > +static int __vidioc_enum_fmt(const struct v4l2_ioctl_ops *ops,
> > +struct v4l2_fmtdesc *p,
> > +struct file *file, void *fh)
> > +{
> > +   int ret = 0;
> >  
> > switch (p->type) {
> > case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_cap))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_cap(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_cap(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_cap_mplane))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_cap_mplane(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_cap_mplane(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_VIDEO_OVERLAY:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_overlay))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_overlay(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_overlay(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_out))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_out(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_out(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> > if (unlikely(!ops->vidioc_enum_fmt_vid_out_mplane))
> > break;
> > -   ret = ops->vidioc_enum_fmt_vid_out_mplane(file, fh, arg);
> > +   ret = ops->vidioc_enum_fmt_vid_out_mplane(file, fh, p);
> > break;
> > case V4L2_BUF_TYPE_SDR_CAPTURE:
> > if (unlikely(!ops->vidioc_enum_fmt_sdr_cap))
> > break;
&

Re: [PATCH v10 12/16] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On Tue, 2018-05-22 at 18:48 +0200, Hans Verkuil wrote:
> On 22/05/18 18:22, Ezequiel Garcia wrote:
> > > > @@ -1615,7 +1762,12 @@ static void __vb2_dqbuf(struct vb2_buffer *vb)
> > > > return;
> > > >  
> > > > vb->state = VB2_BUF_STATE_DEQUEUED;
> > > > -
> > > > +   if (vb->in_fence) {
> > > > +   if (dma_fence_remove_callback(vb->in_fence, 
> > > > >fence_cb))
> > > > +   __vb2_buffer_put(vb);
> > > > +   dma_fence_put(vb->in_fence);
> > > > +   vb->in_fence = NULL;
> > > > +   }
> > > > /* unmap DMABUF buffer */
> > > > if (q->memory == VB2_MEMORY_DMABUF)
> > > > for (i = 0; i < vb->num_planes; ++i) {
> > > > @@ -1653,7 +1805,7 @@ int vb2_core_dqbuf(struct vb2_queue *q, unsigned 
> > > > int *pindex, void *pb,
> > > > if (pindex)
> > > > *pindex = vb->index;
> > > >  
> > > > -   /* Fill buffer information for the userspace */
> > > > +   /* Fill buffer information for userspace */
> > > > if (pb)
> > > > call_void_bufop(q, fill_user_buffer, vb, pb);
> > > >  
> > > > @@ -1700,8 +1852,8 @@ static void __vb2_queue_cancel(struct vb2_queue 
> > > > *q)
> > > > if (WARN_ON(atomic_read(>owned_by_drv_count))) {
> > > > for (i = 0; i < q->num_buffers; ++i)
> > > > if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
> > > > -   pr_warn("driver bug: stop_streaming 
> > > > operation is leaving buf %p in active
> > > > state\n",
> > > > -   q->bufs[i]);
> > > > +   pr_warn("driver bug: stop_streaming 
> > > > operation is leaving buf[%d] 0x%p in active
> > > > state\n",
> > > > +   q->bufs[i]->index, q->bufs[i]);
> > > > vb2_buffer_done(q->bufs[i], 
> > > > VB2_BUF_STATE_ERROR);
> > > > }
> > > 
> > > Shouldn't any pending fences be canceled here?
> > > 
> > 
> > No, we don't have to flush -- that's the reason of the refcount :)
> > The qbuf_work won't do anything if all the buffers are returned
> > by the driver (with error or done state), and if !streaming.
> > 
> > Also, note that's why qbuf_work checks for the queued state, and not
> > for the error state.
> > 
> > > I feel uncomfortable with the refcounting of buffers, I'd rather that 
> > > when we
> > > cancel the queue all fences for buffers are removed/canceled/whatever.
> > > 
> > > Is there any reason for refcounting if we cancel all pending fences here?
> > > 
> > > Note that besides canceling fences you also need to cancel/flush 
> > > __qbuf_work.
> > > 
> > > 
> > 
> > Like I said above, I'm trying to avoid cancel/flushing the workqueue.
> > Currently, I believe it works fine without any flushing, provided we 
> > refcount
> > the buffers.
> > 
> > The problem with cancelling the workqueue, is that you need to unlock the 
> > queue
> > lock, to avoid a deadlock. It seemed to me that having a refcount is more 
> > natural.
> > 
> > Thoughts?
> > 
> 
> I'll take another look tomorrow morning. Do you have a public git tree 
> containing
> this series that I can browse?
> 
> 

Sure, there you go 
http://git.infradead.org/users/ezequielg/linux/shortlog/refs/heads/fences_v10_v4.17-rc1

Re: [PATCH v10 12/16] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On Tue, 2018-05-22 at 18:48 +0200, Hans Verkuil wrote:
> On 22/05/18 18:22, Ezequiel Garcia wrote:
> > > > @@ -1615,7 +1762,12 @@ static void __vb2_dqbuf(struct vb2_buffer *vb)
> > > > return;
> > > >  
> > > > vb->state = VB2_BUF_STATE_DEQUEUED;
> > > > -
> > > > +   if (vb->in_fence) {
> > > > +   if (dma_fence_remove_callback(vb->in_fence, 
> > > > >fence_cb))
> > > > +   __vb2_buffer_put(vb);
> > > > +   dma_fence_put(vb->in_fence);
> > > > +   vb->in_fence = NULL;
> > > > +   }
> > > > /* unmap DMABUF buffer */
> > > > if (q->memory == VB2_MEMORY_DMABUF)
> > > > for (i = 0; i < vb->num_planes; ++i) {
> > > > @@ -1653,7 +1805,7 @@ int vb2_core_dqbuf(struct vb2_queue *q, unsigned 
> > > > int *pindex, void *pb,
> > > > if (pindex)
> > > > *pindex = vb->index;
> > > >  
> > > > -   /* Fill buffer information for the userspace */
> > > > +   /* Fill buffer information for userspace */
> > > > if (pb)
> > > > call_void_bufop(q, fill_user_buffer, vb, pb);
> > > >  
> > > > @@ -1700,8 +1852,8 @@ static void __vb2_queue_cancel(struct vb2_queue 
> > > > *q)
> > > > if (WARN_ON(atomic_read(>owned_by_drv_count))) {
> > > > for (i = 0; i < q->num_buffers; ++i)
> > > > if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
> > > > -   pr_warn("driver bug: stop_streaming 
> > > > operation is leaving buf %p in active
> > > > state\n",
> > > > -   q->bufs[i]);
> > > > +   pr_warn("driver bug: stop_streaming 
> > > > operation is leaving buf[%d] 0x%p in active
> > > > state\n",
> > > > +   q->bufs[i]->index, q->bufs[i]);
> > > > vb2_buffer_done(q->bufs[i], 
> > > > VB2_BUF_STATE_ERROR);
> > > > }
> > > 
> > > Shouldn't any pending fences be canceled here?
> > > 
> > 
> > No, we don't have to flush -- that's the reason of the refcount :)
> > The qbuf_work won't do anything if all the buffers are returned
> > by the driver (with error or done state), and if !streaming.
> > 
> > Also, note that's why qbuf_work checks for the queued state, and not
> > for the error state.
> > 
> > > I feel uncomfortable with the refcounting of buffers, I'd rather that 
> > > when we
> > > cancel the queue all fences for buffers are removed/canceled/whatever.
> > > 
> > > Is there any reason for refcounting if we cancel all pending fences here?
> > > 
> > > Note that besides canceling fences you also need to cancel/flush 
> > > __qbuf_work.
> > > 
> > > 
> > 
> > Like I said above, I'm trying to avoid cancel/flushing the workqueue.
> > Currently, I believe it works fine without any flushing, provided we 
> > refcount
> > the buffers.
> > 
> > The problem with cancelling the workqueue, is that you need to unlock the 
> > queue
> > lock, to avoid a deadlock. It seemed to me that having a refcount is more 
> > natural.
> > 
> > Thoughts?
> > 
> 
> I'll take another look tomorrow morning. Do you have a public git tree 
> containing
> this series that I can browse?
> 
> 

Sure, there you go 
http://git.infradead.org/users/ezequielg/linux/shortlog/refs/heads/fences_v10_v4.17-rc1

Re: [PATCH v10 12/16] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On Tue, 2018-05-22 at 14:37 +0200, Hans Verkuil wrote:
> On 21/05/18 18:59, Ezequiel Garcia wrote:
> > From: Gustavo Padovan <gustavo.pado...@collabora.com>
> > 
> > Receive in-fence from userspace and add support for waiting on them
> > before queueing the buffer to the driver. Buffers can't be queued to the
> > driver before its fences signal. And a buffer can't be queued to the driver
> > out of the order they were queued from userspace. That means that even if
> > its fence signals it must wait for all other buffers, ahead of it in the 
> > queue,
> > to signal first.
> > 
> > If the fence for some buffer fails we do not queue it to the driver,
> > instead we mark it as error and wait until the previous buffer is done
> > to notify userspace of the error. We wait here to deliver the buffers back
> > to userspace in order.
> > 
> > v13: - cleanup implementation.
> >  - remove wrong Kconfig changes.
> >  - print noisy warning on unexpected enqueue conditioin
> >  - schedule a vb2_start_streaming work from the fence callback
> > 
> > v12: fixed dvb_vb2.c usage of vb2_core_qbuf.
> > 
> > v11: - minor doc/comments fixes (Hans Verkuil)
> >  - reviewed the in-fence path at __fill_v4l2_buffer()
> > 
> > v10: - rename fence to in_fence in many places
> >  - handle fences signalling with error better (Hans Verkuil)
> > 
> > v9: - improve comments and docs (Hans Verkuil)
> > - fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
> > - move in-fences code that was in the out-fences patch here (Alex)
> > 
> > v8: - improve comments about fences with errors
> > 
> > v7: - get rid of the fence array stuff for ordering and just use
> >   get_num_buffers_ready() (Hans)
> > - fix issue of queuing the buffer twice (Hans)
> > - avoid the dma_fence_wait() in core_qbuf() (Alex)
> > - merge preparation commit in
> > 
> > v6: - With fences always keep the order userspace queues the buffers.
> > - Protect in_fence manipulation with a lock (Brian Starkey)
> > - check if fences have the same context before adding a fence array
> > - Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
> > - Clean up fence if __set_in_fence() fails (Brian Starkey)
> > - treat -EINVAL from dma_fence_add_callback() (Brian Starkey)
> > 
> > v5: - use fence_array to keep buffers ordered in vb2 core when
> >   needed (Brian Starkey)
> > - keep backward compat on the reserved2 field (Brian Starkey)
> > - protect fence callback removal with lock (Brian Starkey)
> > 
> > v4: - Add a comment about dma_fence_add_callback() not returning a
> >   error (Hans)
> > - Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
> > - select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
> > - Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
> > - Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
> > - Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
> >   vb2_start_streaming() (Hans)
> > - set IN_FENCE flags on __fill_v4l2_buffer (Hans)
> > - Queue buffers to the driver as soon as they are ready (Hans)
> > - call fill_user_buffer() after queuing the buffer (Hans)
> > - add err: label to clean up fence
> > - add dma_fence_wait() before calling vb2_start_streaming()
> > 
> > v3: - document fence parameter
> > - remove ternary if at vb2_qbuf() return (Mauro)
> > - do not change if conditions behaviour (Mauro)
> > 
> > v2: - fix vb2_queue_or_prepare_buf() ret check
> > - remove check for VB2_MEMORY_DMABUF only (Javier)
> > - check num of ready buffers to start streaming
> > - when queueing, start from the first ready buffer
> > - handle queue cancel
> > 
> > Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
> > Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
> > ---
> >  drivers/media/common/videobuf2/Kconfig  |   1 +
> >  drivers/media/common/videobuf2/videobuf2-core.c | 224 
> > 
> >  drivers/media/common/videobuf2/videobuf2-v4l2.c |  37 +++-
> >  drivers/media/dvb-core/dvb_vb2.c|   2 +-
> >  include/media/videobuf2-core.h  |  19 +-
> >  5 files changed, 242 insertions(+), 41 deletions(-)
> > 
> > diff --git a/drivers/media/common/videobuf2/Kconfig 
> > b/drivers/media/common/videobuf2/Kconfig
> > index 17c32ea58395..27ad9e

Re: [PATCH v10 12/16] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On Tue, 2018-05-22 at 14:37 +0200, Hans Verkuil wrote:
> On 21/05/18 18:59, Ezequiel Garcia wrote:
> > From: Gustavo Padovan 
> > 
> > Receive in-fence from userspace and add support for waiting on them
> > before queueing the buffer to the driver. Buffers can't be queued to the
> > driver before its fences signal. And a buffer can't be queued to the driver
> > out of the order they were queued from userspace. That means that even if
> > its fence signals it must wait for all other buffers, ahead of it in the 
> > queue,
> > to signal first.
> > 
> > If the fence for some buffer fails we do not queue it to the driver,
> > instead we mark it as error and wait until the previous buffer is done
> > to notify userspace of the error. We wait here to deliver the buffers back
> > to userspace in order.
> > 
> > v13: - cleanup implementation.
> >  - remove wrong Kconfig changes.
> >  - print noisy warning on unexpected enqueue conditioin
> >  - schedule a vb2_start_streaming work from the fence callback
> > 
> > v12: fixed dvb_vb2.c usage of vb2_core_qbuf.
> > 
> > v11: - minor doc/comments fixes (Hans Verkuil)
> >  - reviewed the in-fence path at __fill_v4l2_buffer()
> > 
> > v10: - rename fence to in_fence in many places
> >  - handle fences signalling with error better (Hans Verkuil)
> > 
> > v9: - improve comments and docs (Hans Verkuil)
> > - fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
> > - move in-fences code that was in the out-fences patch here (Alex)
> > 
> > v8: - improve comments about fences with errors
> > 
> > v7: - get rid of the fence array stuff for ordering and just use
> >   get_num_buffers_ready() (Hans)
> > - fix issue of queuing the buffer twice (Hans)
> > - avoid the dma_fence_wait() in core_qbuf() (Alex)
> > - merge preparation commit in
> > 
> > v6: - With fences always keep the order userspace queues the buffers.
> > - Protect in_fence manipulation with a lock (Brian Starkey)
> > - check if fences have the same context before adding a fence array
> > - Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
> > - Clean up fence if __set_in_fence() fails (Brian Starkey)
> > - treat -EINVAL from dma_fence_add_callback() (Brian Starkey)
> > 
> > v5: - use fence_array to keep buffers ordered in vb2 core when
> >   needed (Brian Starkey)
> > - keep backward compat on the reserved2 field (Brian Starkey)
> > - protect fence callback removal with lock (Brian Starkey)
> > 
> > v4: - Add a comment about dma_fence_add_callback() not returning a
> >   error (Hans)
> > - Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
> > - select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
> > - Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
> > - Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
> > - Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
> >   vb2_start_streaming() (Hans)
> > - set IN_FENCE flags on __fill_v4l2_buffer (Hans)
> > - Queue buffers to the driver as soon as they are ready (Hans)
> > - call fill_user_buffer() after queuing the buffer (Hans)
> > - add err: label to clean up fence
> > - add dma_fence_wait() before calling vb2_start_streaming()
> > 
> > v3: - document fence parameter
> > - remove ternary if at vb2_qbuf() return (Mauro)
> > - do not change if conditions behaviour (Mauro)
> > 
> > v2: - fix vb2_queue_or_prepare_buf() ret check
> > - remove check for VB2_MEMORY_DMABUF only (Javier)
> > - check num of ready buffers to start streaming
> > - when queueing, start from the first ready buffer
> > - handle queue cancel
> > 
> > Signed-off-by: Gustavo Padovan 
> > Signed-off-by: Ezequiel Garcia 
> > ---
> >  drivers/media/common/videobuf2/Kconfig  |   1 +
> >  drivers/media/common/videobuf2/videobuf2-core.c | 224 
> > 
> >  drivers/media/common/videobuf2/videobuf2-v4l2.c |  37 +++-
> >  drivers/media/dvb-core/dvb_vb2.c|   2 +-
> >  include/media/videobuf2-core.h  |  19 +-
> >  5 files changed, 242 insertions(+), 41 deletions(-)
> > 
> > diff --git a/drivers/media/common/videobuf2/Kconfig 
> > b/drivers/media/common/videobuf2/Kconfig
> > index 17c32ea58395..27ad9e8a268b 100644
> > --- a/drivers/media/common/videobuf2/Kconfig
> > +++ b/drivers/media/com

Re: [PATCH v10 11/16] vb2: add explicit fence user API

[Ezequiel Garcia]

On Tue, 2018-05-22 at 14:05 +0200, Hans Verkuil wrote:
> On 21/05/18 18:59, Ezequiel Garcia wrote:
> > From: Gustavo Padovan <gustavo.pado...@collabora.com>
> > 
> > Turn the reserved2 field into fence_fd that we will use to send
> > an in-fence to the kernel or return an out-fence from the kernel to
> > userspace.
> > 
> > Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> > when sending an in-fence to the kernel to be waited on, and
> > V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
> > 
> > v8: return -1 if new flags are set.
> > v7: minor fixes on the Documentation (Hans Verkuil)
> > 
> > v6: big improvement on doc (Hans Verkuil)
> > 
> > v5: - keep using reserved2 field for cpia2
> > - set fence_fd to 0 for now, for compat with userspace(Mauro)
> > 
> > v4: make it a union with reserved2 and fence_fd (Hans Verkuil)
> > 
> > v3: make the out_fence refer to the current buffer (Hans Verkuil)
> > 
> > v2: add documentation
> > 
> > Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
> > ---
> >  Documentation/media/uapi/v4l/buffer.rst | 48 
> > ++---
> >  drivers/media/common/videobuf2/videobuf2-v4l2.c |  6 +++-
> >  drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
> >  include/uapi/linux/videodev2.h  |  8 -
> >  4 files changed, 58 insertions(+), 8 deletions(-)
> > 
> > diff --git a/Documentation/media/uapi/v4l/buffer.rst 
> > b/Documentation/media/uapi/v4l/buffer.rst
> > index e2c85ddc990b..971b7453040c 100644
> > --- a/Documentation/media/uapi/v4l/buffer.rst
> > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > @@ -300,11 +300,23 @@ struct v4l2_buffer
> > multi-planar API the application sets this to the number of
> > elements in the ``planes`` array. The driver will fill in the
> > actual number of valid elements in that array.
> > -* - __u32
> > -  - ``reserved2``
> > +* - __s32
> > +  - ``fence_fd``
> >-
> > -  - A place holder for future extensions. Drivers and applications
> > -   must set this to 0.
> > +  - Used to communicate a fence file descriptor from userspace to 
> > kernel
> > +   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
> > +   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag 
> > must
> > +   be used and this field set to the fence file descriptor of the in-fence.
> > +   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
> > +
> > +To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
> > +   must be set, the kernel will return the out-fence file descriptor in
> > +   this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
> > +an error.
> > +
> > +   For all other ioctls V4L2 sets this field to -1 if
> > +   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
> > +   otherwise this field is set to 0 for backward compatibility.
> >  * - __u32
> >- ``reserved``
> >-
> > @@ -648,6 +660,34 @@ Buffer Flags
> >- Start Of Exposure. The buffer timestamp has been taken when the
> > exposure of the frame has begun. This is only valid for the
> > ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> > +* .. _`V4L2-BUF-FLAG-IN-FENCE`:
> > +
> > +  - ``V4L2_BUF_FLAG_IN_FENCE``
> > +  - 0x0020
> > +  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
> > +   buffer won't be queued to the driver until the fence signals. The order
> > +   in which buffers are queued is guaranteed to be preserved, so any
> > +   buffers queued after this buffer will also be blocked until this fence
> > +   signals. This flag must be set before calling ``VIDIOC_QBUF``. For
> > +   other ioctls the driver just reports the value of the flag.
> > +
> > +If the fence signals the flag is cleared and not reported anymore.
> > +   If the fence is not valid ``VIDIOC_QBUF`` returns an error.
> > +* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
> > +
> > +  - ``V4L2_BUF_FLAG_OUT_FENCE``
> > +  - 0x0040
> > +  - Request for a fence to be attached to the buffer. The driver will 
> > fill
> > +   in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
> > +   ` returns. This flag must be set before calling
> > +   ``VIDIOC_QBUF``. This flag is only an input, and is not set by the 
> > kernel.
> > +

Re: [PATCH v10 11/16] vb2: add explicit fence user API

[Ezequiel Garcia]

On Tue, 2018-05-22 at 14:05 +0200, Hans Verkuil wrote:
> On 21/05/18 18:59, Ezequiel Garcia wrote:
> > From: Gustavo Padovan 
> > 
> > Turn the reserved2 field into fence_fd that we will use to send
> > an in-fence to the kernel or return an out-fence from the kernel to
> > userspace.
> > 
> > Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> > when sending an in-fence to the kernel to be waited on, and
> > V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
> > 
> > v8: return -1 if new flags are set.
> > v7: minor fixes on the Documentation (Hans Verkuil)
> > 
> > v6: big improvement on doc (Hans Verkuil)
> > 
> > v5: - keep using reserved2 field for cpia2
> > - set fence_fd to 0 for now, for compat with userspace(Mauro)
> > 
> > v4: make it a union with reserved2 and fence_fd (Hans Verkuil)
> > 
> > v3: make the out_fence refer to the current buffer (Hans Verkuil)
> > 
> > v2: add documentation
> > 
> > Signed-off-by: Gustavo Padovan 
> > ---
> >  Documentation/media/uapi/v4l/buffer.rst | 48 
> > ++---
> >  drivers/media/common/videobuf2/videobuf2-v4l2.c |  6 +++-
> >  drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
> >  include/uapi/linux/videodev2.h  |  8 -
> >  4 files changed, 58 insertions(+), 8 deletions(-)
> > 
> > diff --git a/Documentation/media/uapi/v4l/buffer.rst 
> > b/Documentation/media/uapi/v4l/buffer.rst
> > index e2c85ddc990b..971b7453040c 100644
> > --- a/Documentation/media/uapi/v4l/buffer.rst
> > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > @@ -300,11 +300,23 @@ struct v4l2_buffer
> > multi-planar API the application sets this to the number of
> > elements in the ``planes`` array. The driver will fill in the
> > actual number of valid elements in that array.
> > -* - __u32
> > -  - ``reserved2``
> > +* - __s32
> > +  - ``fence_fd``
> >-
> > -  - A place holder for future extensions. Drivers and applications
> > -   must set this to 0.
> > +  - Used to communicate a fence file descriptor from userspace to 
> > kernel
> > +   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
> > +   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag 
> > must
> > +   be used and this field set to the fence file descriptor of the in-fence.
> > +   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
> > +
> > +To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
> > +   must be set, the kernel will return the out-fence file descriptor in
> > +   this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
> > +an error.
> > +
> > +   For all other ioctls V4L2 sets this field to -1 if
> > +   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
> > +   otherwise this field is set to 0 for backward compatibility.
> >  * - __u32
> >- ``reserved``
> >-
> > @@ -648,6 +660,34 @@ Buffer Flags
> >- Start Of Exposure. The buffer timestamp has been taken when the
> > exposure of the frame has begun. This is only valid for the
> > ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> > +* .. _`V4L2-BUF-FLAG-IN-FENCE`:
> > +
> > +  - ``V4L2_BUF_FLAG_IN_FENCE``
> > +  - 0x0020
> > +  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
> > +   buffer won't be queued to the driver until the fence signals. The order
> > +   in which buffers are queued is guaranteed to be preserved, so any
> > +   buffers queued after this buffer will also be blocked until this fence
> > +   signals. This flag must be set before calling ``VIDIOC_QBUF``. For
> > +   other ioctls the driver just reports the value of the flag.
> > +
> > +If the fence signals the flag is cleared and not reported anymore.
> > +   If the fence is not valid ``VIDIOC_QBUF`` returns an error.
> > +* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
> > +
> > +  - ``V4L2_BUF_FLAG_OUT_FENCE``
> > +  - 0x0040
> > +  - Request for a fence to be attached to the buffer. The driver will 
> > fill
> > +   in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
> > +   ` returns. This flag must be set before calling
> > +   ``VIDIOC_QBUF``. This flag is only an input, and is not set by the 
> > kernel.
> > +
> > +If the creation of the out-fence fails ``VIDIOC_QBUF`` 

[PATCH v10 03/16] hackrf: group device capabilities

[Ezequiel Garcia]

From: Gustavo Padovan 

Instead of putting V4L2_CAP_STREAMING and V4L2_CAP_READWRITE
everywhere, set device_caps earlier with these values.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/usb/hackrf/hackrf.c | 11 ---
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/drivers/media/usb/hackrf/hackrf.c 
b/drivers/media/usb/hackrf/hackrf.c
index 7eb53517a82f..6d692fb3e8dd 100644
--- a/drivers/media/usb/hackrf/hackrf.c
+++ b/drivers/media/usb/hackrf/hackrf.c
@@ -909,18 +909,15 @@ static int hackrf_querycap(struct file *file, void *fh,
 
dev_dbg(>dev, "\n");
 
+   cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
if (vdev->vfl_dir == VFL_DIR_RX)
-   cap->device_caps = V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER |
-  V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
-
+   cap->device_caps |= V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER;
else
-   cap->device_caps = V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR |
-  V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
+   cap->device_caps |= V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR;
 
cap->capabilities = V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER |
V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR |
-   V4L2_CAP_STREAMING | V4L2_CAP_READWRITE |
-   V4L2_CAP_DEVICE_CAPS;
+   V4L2_CAP_DEVICE_CAPS | cap->device_caps;
strlcpy(cap->driver, KBUILD_MODNAME, sizeof(cap->driver));
strlcpy(cap->card, dev->rx_vdev.name, sizeof(cap->card));
usb_make_path(dev->udev, cap->bus_info, sizeof(cap->bus_info));
-- 
2.16.3

[PATCH v10 03/16] hackrf: group device capabilities

[Ezequiel Garcia]

From: Gustavo Padovan 

Instead of putting V4L2_CAP_STREAMING and V4L2_CAP_READWRITE
everywhere, set device_caps earlier with these values.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/usb/hackrf/hackrf.c | 11 ---
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/drivers/media/usb/hackrf/hackrf.c 
b/drivers/media/usb/hackrf/hackrf.c
index 7eb53517a82f..6d692fb3e8dd 100644
--- a/drivers/media/usb/hackrf/hackrf.c
+++ b/drivers/media/usb/hackrf/hackrf.c
@@ -909,18 +909,15 @@ static int hackrf_querycap(struct file *file, void *fh,
 
dev_dbg(>dev, "\n");
 
+   cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
if (vdev->vfl_dir == VFL_DIR_RX)
-   cap->device_caps = V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER |
-  V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
-
+   cap->device_caps |= V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER;
else
-   cap->device_caps = V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR |
-  V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
+   cap->device_caps |= V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR;
 
cap->capabilities = V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER |
V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR |
-   V4L2_CAP_STREAMING | V4L2_CAP_READWRITE |
-   V4L2_CAP_DEVICE_CAPS;
+   V4L2_CAP_DEVICE_CAPS | cap->device_caps;
strlcpy(cap->driver, KBUILD_MODNAME, sizeof(cap->driver));
strlcpy(cap->card, dev->rx_vdev.name, sizeof(cap->card));
usb_make_path(dev->udev, cap->bus_info, sizeof(cap->bus_info));
-- 
2.16.3

[PATCH v10 04/16] omap3isp: group device capabilities

[Ezequiel Garcia]

From: Gustavo Padovan 

Instead of putting V4L2_CAP_STREAMING everywhere, set device_caps
earlier with this value.

v2: move cap->capabilities assignment down (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/platform/omap3isp/ispvideo.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/omap3isp/ispvideo.c 
b/drivers/media/platform/omap3isp/ispvideo.c
index a751c89a3ea8..db9aae222134 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -658,13 +658,15 @@ isp_video_querycap(struct file *file, void *fh, struct 
v4l2_capability *cap)
strlcpy(cap->card, video->video.name, sizeof(cap->card));
strlcpy(cap->bus_info, "media", sizeof(cap->bus_info));
 
-   cap->capabilities = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT
-   | V4L2_CAP_STREAMING | V4L2_CAP_DEVICE_CAPS;
+   cap->device_caps = V4L2_CAP_STREAMING;
 
if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-   cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_CAPTURE;
else
-   cap->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_OUTPUT;
+
+   cap->capabilities = cap->device_caps | V4L2_CAP_VIDEO_CAPTURE |
+   V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_DEVICE_CAPS;
 
return 0;
 }
-- 
2.16.3

[PATCH v10 04/16] omap3isp: group device capabilities

[Ezequiel Garcia]

From: Gustavo Padovan 

Instead of putting V4L2_CAP_STREAMING everywhere, set device_caps
earlier with this value.

v2: move cap->capabilities assignment down (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/platform/omap3isp/ispvideo.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/omap3isp/ispvideo.c 
b/drivers/media/platform/omap3isp/ispvideo.c
index a751c89a3ea8..db9aae222134 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -658,13 +658,15 @@ isp_video_querycap(struct file *file, void *fh, struct 
v4l2_capability *cap)
strlcpy(cap->card, video->video.name, sizeof(cap->card));
strlcpy(cap->bus_info, "media", sizeof(cap->bus_info));
 
-   cap->capabilities = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT
-   | V4L2_CAP_STREAMING | V4L2_CAP_DEVICE_CAPS;
+   cap->device_caps = V4L2_CAP_STREAMING;
 
if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-   cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_CAPTURE;
else
-   cap->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_OUTPUT;
+
+   cap->capabilities = cap->device_caps | V4L2_CAP_VIDEO_CAPTURE |
+   V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_DEVICE_CAPS;
 
return 0;
 }
-- 
2.16.3

[PATCH v10 06/16] vb2: add is_unordered callback for drivers

[Ezequiel Garcia]

From: Gustavo Padovan 

Explicit synchronization benefits a lot from ordered queues, they fit
better in a pipeline with DRM for example so create a opt-in way for
drivers notify videobuf2 that the queue is unordered.

Drivers don't need implement it if the queue is ordered.

v5: rename it to vb2_ops_is_unordered() (Hans Verkuil)

v4: go back to a bitfield property for the unordered property.

v3: - make it bool (Hans)
- create vb2_ops_set_unordered() helper

v2: - improve comments for is_unordered flag (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/common/videobuf2/videobuf2-core.c |  6 ++
 include/media/videobuf2-core.h  | 16 
 2 files changed, 22 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 61e7b6407586..a9a0a9d1decb 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -691,6 +691,12 @@ void vb2_ops_wait_finish(struct vb2_queue *vq)
 }
 EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
 
+bool vb2_ops_is_unordered(struct vb2_queue *q)
+{
+   return true;
+}
+EXPORT_SYMBOL_GPL(vb2_ops_is_unordered);
+
 int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
unsigned int *count)
 {
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 137f72702101..71538ae2c255 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -376,6 +376,10 @@ struct vb2_buffer {
  * callback by calling vb2_buffer_done() with either
  * %VB2_BUF_STATE_DONE or %VB2_BUF_STATE_ERROR; may use
  * vb2_wait_for_all_buffers() function
+ * @is_unordered:  tell if the queue is unordered, i.e. buffers can be
+ * dequeued in a different order from how they were queued.
+ * The default is assumed to be ordered and this function
+ * only needs to be implemented for unordered queues.
  * @buf_queue: passes buffer vb to the driver; driver may start
  * hardware operation on this buffer; driver should give
  * the buffer back by calling vb2_buffer_done() function;
@@ -399,6 +403,7 @@ struct vb2_ops {
 
int (*start_streaming)(struct vb2_queue *q, unsigned int count);
void (*stop_streaming)(struct vb2_queue *q);
+   bool (*is_unordered)(struct vb2_queue *q);
 
void (*buf_queue)(struct vb2_buffer *vb);
 };
@@ -421,6 +426,16 @@ void vb2_ops_wait_prepare(struct vb2_queue *vq);
  */
 void vb2_ops_wait_finish(struct vb2_queue *vq);
 
+/**
+ * vb2_ops_is_unordered - helper function to check if queue is unordered
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * This helper just returns true to notify that the driver can't deal with
+ * ordered queues.
+ */
+bool vb2_ops_is_unordered(struct vb2_queue *q);
+
 /**
  * struct vb2_buf_ops - driver-specific callbacks.
  *
@@ -590,6 +605,7 @@ struct vb2_queue {
u32 cnt_wait_finish;
u32 cnt_start_streaming;
u32 cnt_stop_streaming;
+   u32 cnt_is_unordered;
 #endif
 };
 
-- 
2.16.3

[PATCH v10 06/16] vb2: add is_unordered callback for drivers

[Ezequiel Garcia]

From: Gustavo Padovan 

Explicit synchronization benefits a lot from ordered queues, they fit
better in a pipeline with DRM for example so create a opt-in way for
drivers notify videobuf2 that the queue is unordered.

Drivers don't need implement it if the queue is ordered.

v5: rename it to vb2_ops_is_unordered() (Hans Verkuil)

v4: go back to a bitfield property for the unordered property.

v3: - make it bool (Hans)
- create vb2_ops_set_unordered() helper

v2: - improve comments for is_unordered flag (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/common/videobuf2/videobuf2-core.c |  6 ++
 include/media/videobuf2-core.h  | 16 
 2 files changed, 22 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 61e7b6407586..a9a0a9d1decb 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -691,6 +691,12 @@ void vb2_ops_wait_finish(struct vb2_queue *vq)
 }
 EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
 
+bool vb2_ops_is_unordered(struct vb2_queue *q)
+{
+   return true;
+}
+EXPORT_SYMBOL_GPL(vb2_ops_is_unordered);
+
 int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
unsigned int *count)
 {
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 137f72702101..71538ae2c255 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -376,6 +376,10 @@ struct vb2_buffer {
  * callback by calling vb2_buffer_done() with either
  * %VB2_BUF_STATE_DONE or %VB2_BUF_STATE_ERROR; may use
  * vb2_wait_for_all_buffers() function
+ * @is_unordered:  tell if the queue is unordered, i.e. buffers can be
+ * dequeued in a different order from how they were queued.
+ * The default is assumed to be ordered and this function
+ * only needs to be implemented for unordered queues.
  * @buf_queue: passes buffer vb to the driver; driver may start
  * hardware operation on this buffer; driver should give
  * the buffer back by calling vb2_buffer_done() function;
@@ -399,6 +403,7 @@ struct vb2_ops {
 
int (*start_streaming)(struct vb2_queue *q, unsigned int count);
void (*stop_streaming)(struct vb2_queue *q);
+   bool (*is_unordered)(struct vb2_queue *q);
 
void (*buf_queue)(struct vb2_buffer *vb);
 };
@@ -421,6 +426,16 @@ void vb2_ops_wait_prepare(struct vb2_queue *vq);
  */
 void vb2_ops_wait_finish(struct vb2_queue *vq);
 
+/**
+ * vb2_ops_is_unordered - helper function to check if queue is unordered
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * This helper just returns true to notify that the driver can't deal with
+ * ordered queues.
+ */
+bool vb2_ops_is_unordered(struct vb2_queue *q);
+
 /**
  * struct vb2_buf_ops - driver-specific callbacks.
  *
@@ -590,6 +605,7 @@ struct vb2_queue {
u32 cnt_wait_finish;
u32 cnt_start_streaming;
u32 cnt_stop_streaming;
+   u32 cnt_is_unordered;
 #endif
 };
 
-- 
2.16.3

[PATCH v10 11/16] vb2: add explicit fence user API

[Ezequiel Garcia]

From: Gustavo Padovan 

Turn the reserved2 field into fence_fd that we will use to send
an in-fence to the kernel or return an out-fence from the kernel to
userspace.

Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
when sending an in-fence to the kernel to be waited on, and
V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.

v8: return -1 if new flags are set.
v7: minor fixes on the Documentation (Hans Verkuil)

v6: big improvement on doc (Hans Verkuil)

v5: - keep using reserved2 field for cpia2
- set fence_fd to 0 for now, for compat with userspace(Mauro)

v4: make it a union with reserved2 and fence_fd (Hans Verkuil)

v3: make the out_fence refer to the current buffer (Hans Verkuil)

v2: add documentation

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/buffer.rst | 48 ++---
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  6 +++-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
 include/uapi/linux/videodev2.h  |  8 -
 4 files changed, 58 insertions(+), 8 deletions(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index e2c85ddc990b..971b7453040c 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -300,11 +300,23 @@ struct v4l2_buffer
multi-planar API the application sets this to the number of
elements in the ``planes`` array. The driver will fill in the
actual number of valid elements in that array.
-* - __u32
-  - ``reserved2``
+* - __s32
+  - ``fence_fd``
   -
-  - A place holder for future extensions. Drivers and applications
-   must set this to 0.
+  - Used to communicate a fence file descriptor from userspace to kernel
+   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
+   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag 
must
+   be used and this field set to the fence file descriptor of the in-fence.
+   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
+   must be set, the kernel will return the out-fence file descriptor in
+   this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
+an error.
+
+   For all other ioctls V4L2 sets this field to -1 if
+   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
+   otherwise this field is set to 0 for backward compatibility.
 * - __u32
   - ``reserved``
   -
@@ -648,6 +660,34 @@ Buffer Flags
   - Start Of Exposure. The buffer timestamp has been taken when the
exposure of the frame has begun. This is only valid for the
``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
+* .. _`V4L2-BUF-FLAG-IN-FENCE`:
+
+  - ``V4L2_BUF_FLAG_IN_FENCE``
+  - 0x0020
+  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
+   buffer won't be queued to the driver until the fence signals. The order
+   in which buffers are queued is guaranteed to be preserved, so any
+   buffers queued after this buffer will also be blocked until this fence
+   signals. This flag must be set before calling ``VIDIOC_QBUF``. For
+   other ioctls the driver just reports the value of the flag.
+
+If the fence signals the flag is cleared and not reported anymore.
+   If the fence is not valid ``VIDIOC_QBUF`` returns an error.
+* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
+
+  - ``V4L2_BUF_FLAG_OUT_FENCE``
+  - 0x0040
+  - Request for a fence to be attached to the buffer. The driver will fill
+   in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
+   ` returns. This flag must be set before calling
+   ``VIDIOC_QBUF``. This flag is only an input, and is not set by the 
kernel.
+
+If the creation of the out-fence fails ``VIDIOC_QBUF`` returns an
+   error.
+
+Note that it is valid to set both ``V4L2_BUF_FLAG_IN_FENCE`` and
+`V4L2_BUF_FLAG_OUT_FENCE`` flags. In such case, the ``fence_fd``
+field is used to both set the in-fence and return the out-fence.
 
 
 
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c 
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 64503615d00b..8312f61adfa6 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -203,9 +203,13 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void 
*pb)
b->timestamp = ns_to_timeval(vb->timestamp);
b->timecode = vbuf->timecode;
b->sequence = vbuf->sequence;
-   b->reserved2 = 0;
b->reserved = 0;
 
+   if (b->flags & (V4L2_BUF_FLAG_IN_FENCE | V4L2_BUF_FLAG_OUT_FENCE))
+   b->fence_fd = -1;
+   else
+   

[PATCH v10 11/16] vb2: add explicit fence user API

[Ezequiel Garcia]

From: Gustavo Padovan 

Turn the reserved2 field into fence_fd that we will use to send
an in-fence to the kernel or return an out-fence from the kernel to
userspace.

Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
when sending an in-fence to the kernel to be waited on, and
V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.

v8: return -1 if new flags are set.
v7: minor fixes on the Documentation (Hans Verkuil)

v6: big improvement on doc (Hans Verkuil)

v5: - keep using reserved2 field for cpia2
- set fence_fd to 0 for now, for compat with userspace(Mauro)

v4: make it a union with reserved2 and fence_fd (Hans Verkuil)

v3: make the out_fence refer to the current buffer (Hans Verkuil)

v2: add documentation

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/buffer.rst | 48 ++---
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  6 +++-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
 include/uapi/linux/videodev2.h  |  8 -
 4 files changed, 58 insertions(+), 8 deletions(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index e2c85ddc990b..971b7453040c 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -300,11 +300,23 @@ struct v4l2_buffer
multi-planar API the application sets this to the number of
elements in the ``planes`` array. The driver will fill in the
actual number of valid elements in that array.
-* - __u32
-  - ``reserved2``
+* - __s32
+  - ``fence_fd``
   -
-  - A place holder for future extensions. Drivers and applications
-   must set this to 0.
+  - Used to communicate a fence file descriptor from userspace to kernel
+   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
+   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag 
must
+   be used and this field set to the fence file descriptor of the in-fence.
+   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
+   must be set, the kernel will return the out-fence file descriptor in
+   this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
+an error.
+
+   For all other ioctls V4L2 sets this field to -1 if
+   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
+   otherwise this field is set to 0 for backward compatibility.
 * - __u32
   - ``reserved``
   -
@@ -648,6 +660,34 @@ Buffer Flags
   - Start Of Exposure. The buffer timestamp has been taken when the
exposure of the frame has begun. This is only valid for the
``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
+* .. _`V4L2-BUF-FLAG-IN-FENCE`:
+
+  - ``V4L2_BUF_FLAG_IN_FENCE``
+  - 0x0020
+  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
+   buffer won't be queued to the driver until the fence signals. The order
+   in which buffers are queued is guaranteed to be preserved, so any
+   buffers queued after this buffer will also be blocked until this fence
+   signals. This flag must be set before calling ``VIDIOC_QBUF``. For
+   other ioctls the driver just reports the value of the flag.
+
+If the fence signals the flag is cleared and not reported anymore.
+   If the fence is not valid ``VIDIOC_QBUF`` returns an error.
+* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
+
+  - ``V4L2_BUF_FLAG_OUT_FENCE``
+  - 0x0040
+  - Request for a fence to be attached to the buffer. The driver will fill
+   in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
+   ` returns. This flag must be set before calling
+   ``VIDIOC_QBUF``. This flag is only an input, and is not set by the 
kernel.
+
+If the creation of the out-fence fails ``VIDIOC_QBUF`` returns an
+   error.
+
+Note that it is valid to set both ``V4L2_BUF_FLAG_IN_FENCE`` and
+`V4L2_BUF_FLAG_OUT_FENCE`` flags. In such case, the ``fence_fd``
+field is used to both set the in-fence and return the out-fence.
 
 
 
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c 
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 64503615d00b..8312f61adfa6 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -203,9 +203,13 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void 
*pb)
b->timestamp = ns_to_timeval(vb->timestamp);
b->timecode = vbuf->timecode;
b->sequence = vbuf->sequence;
-   b->reserved2 = 0;
b->reserved = 0;
 
+   if (b->flags & (V4L2_BUF_FLAG_IN_FENCE | V4L2_BUF_FLAG_OUT_FENCE))
+   b->fence_fd = -1;
+   else
+   b->fence_fd = 0;
+
if (q->is_multiplanar) {
  

[PATCH v10 10/16] vb2: mark codec drivers as unordered

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

In preparation to have full support to explicit fence we are
marking codec as non-ordered preventively. It is easier and safer from an
uAPI point of view to move from unordered to ordered than the opposite.

v2: mark only codec drivers as unordered (Nicolas and Hans)

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/platform/coda/coda-common.c  | 1 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 1 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1 +
 drivers/media/platform/qcom/venus/vdec.c   | 1 +
 drivers/media/platform/qcom/venus/venc.c   | 1 +
 drivers/media/platform/s5p-mfc/s5p_mfc_dec.c   | 1 +
 drivers/media/platform/s5p-mfc/s5p_mfc_enc.c   | 1 +
 7 files changed, 7 insertions(+)

diff --git a/drivers/media/platform/coda/coda-common.c 
b/drivers/media/platform/coda/coda-common.c
index 04e35d70ce2e..cac36a11efa2 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1649,6 +1649,7 @@ static const struct vb2_ops coda_qops = {
.stop_streaming = coda_stop_streaming,
.wait_prepare   = vb2_ops_wait_prepare,
.wait_finish= vb2_ops_wait_finish,
+   .is_unordered   = vb2_ops_is_unordered,
 };
 
 static int coda_s_ctrl(struct v4l2_ctrl *ctrl)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 86f0a7134365..a4a02f3790fa 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -1445,6 +1445,7 @@ static const struct vb2_ops mtk_vdec_vb2_ops = {
.buf_finish = vb2ops_vdec_buf_finish,
.start_streaming= vb2ops_vdec_start_streaming,
.stop_streaming = vb2ops_vdec_stop_streaming,
+   .is_unordered   = vb2_ops_is_unordered,
 };
 
 const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops = {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 1b1a28abbf1f..d37d670346b9 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -931,6 +931,7 @@ static const struct vb2_ops mtk_venc_vb2_ops = {
.wait_finish= vb2_ops_wait_finish,
.start_streaming= vb2ops_venc_start_streaming,
.stop_streaming = vb2ops_venc_stop_streaming,
+   .is_unordered   = vb2_ops_is_unordered,
 };
 
 static int mtk_venc_encode_header(void *priv)
diff --git a/drivers/media/platform/qcom/venus/vdec.c 
b/drivers/media/platform/qcom/venus/vdec.c
index 49bbd1861d3a..8d7b4fc95880 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -794,6 +794,7 @@ static const struct vb2_ops vdec_vb2_ops = {
.start_streaming = vdec_start_streaming,
.stop_streaming = venus_helper_vb2_stop_streaming,
.buf_queue = venus_helper_vb2_buf_queue,
+   .is_unordered = vb2_ops_is_unordered,
 };
 
 static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
diff --git a/drivers/media/platform/qcom/venus/venc.c 
b/drivers/media/platform/qcom/venus/venc.c
index 6b2ce479584e..713c79ba9639 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -983,6 +983,7 @@ static const struct vb2_ops venc_vb2_ops = {
.start_streaming = venc_start_streaming,
.stop_streaming = venus_helper_vb2_stop_streaming,
.buf_queue = venus_helper_vb2_buf_queue,
+   .is_unordered = vb2_ops_is_unordered,
 };
 
 static void venc_buf_done(struct venus_inst *inst, unsigned int buf_type,
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c 
b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
index 5cf4d9921264..4402a5d621b2 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
@@ -1105,6 +1105,7 @@ static struct vb2_ops s5p_mfc_dec_qops = {
.start_streaming= s5p_mfc_start_streaming,
.stop_streaming = s5p_mfc_stop_streaming,
.buf_queue  = s5p_mfc_buf_queue,
+   .is_unordered   = vb2_ops_is_unordered,
 };
 
 const struct s5p_mfc_codec_ops *get_dec_codec_ops(void)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c 
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index 5c0462ca9993..376bd8eab8d8 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -2613,6 +2613,7 @@ static struct vb2_ops s5p_mfc_enc_qops = {
.start_streaming= s5p_mfc_start_streaming,
.stop_streaming = s5p_mfc_stop_streaming,
.buf_queue  = s5p_mfc_buf_queue,
+  

[PATCH v10 10/16] vb2: mark codec drivers as unordered

[Ezequiel Garcia]

From: Gustavo Padovan 

In preparation to have full support to explicit fence we are
marking codec as non-ordered preventively. It is easier and safer from an
uAPI point of view to move from unordered to ordered than the opposite.

v2: mark only codec drivers as unordered (Nicolas and Hans)

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/platform/coda/coda-common.c  | 1 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 1 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1 +
 drivers/media/platform/qcom/venus/vdec.c   | 1 +
 drivers/media/platform/qcom/venus/venc.c   | 1 +
 drivers/media/platform/s5p-mfc/s5p_mfc_dec.c   | 1 +
 drivers/media/platform/s5p-mfc/s5p_mfc_enc.c   | 1 +
 7 files changed, 7 insertions(+)

diff --git a/drivers/media/platform/coda/coda-common.c 
b/drivers/media/platform/coda/coda-common.c
index 04e35d70ce2e..cac36a11efa2 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1649,6 +1649,7 @@ static const struct vb2_ops coda_qops = {
.stop_streaming = coda_stop_streaming,
.wait_prepare   = vb2_ops_wait_prepare,
.wait_finish= vb2_ops_wait_finish,
+   .is_unordered   = vb2_ops_is_unordered,
 };
 
 static int coda_s_ctrl(struct v4l2_ctrl *ctrl)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 86f0a7134365..a4a02f3790fa 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -1445,6 +1445,7 @@ static const struct vb2_ops mtk_vdec_vb2_ops = {
.buf_finish = vb2ops_vdec_buf_finish,
.start_streaming= vb2ops_vdec_start_streaming,
.stop_streaming = vb2ops_vdec_stop_streaming,
+   .is_unordered   = vb2_ops_is_unordered,
 };
 
 const struct v4l2_ioctl_ops mtk_vdec_ioctl_ops = {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 1b1a28abbf1f..d37d670346b9 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -931,6 +931,7 @@ static const struct vb2_ops mtk_venc_vb2_ops = {
.wait_finish= vb2_ops_wait_finish,
.start_streaming= vb2ops_venc_start_streaming,
.stop_streaming = vb2ops_venc_stop_streaming,
+   .is_unordered   = vb2_ops_is_unordered,
 };
 
 static int mtk_venc_encode_header(void *priv)
diff --git a/drivers/media/platform/qcom/venus/vdec.c 
b/drivers/media/platform/qcom/venus/vdec.c
index 49bbd1861d3a..8d7b4fc95880 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -794,6 +794,7 @@ static const struct vb2_ops vdec_vb2_ops = {
.start_streaming = vdec_start_streaming,
.stop_streaming = venus_helper_vb2_stop_streaming,
.buf_queue = venus_helper_vb2_buf_queue,
+   .is_unordered = vb2_ops_is_unordered,
 };
 
 static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
diff --git a/drivers/media/platform/qcom/venus/venc.c 
b/drivers/media/platform/qcom/venus/venc.c
index 6b2ce479584e..713c79ba9639 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -983,6 +983,7 @@ static const struct vb2_ops venc_vb2_ops = {
.start_streaming = venc_start_streaming,
.stop_streaming = venus_helper_vb2_stop_streaming,
.buf_queue = venus_helper_vb2_buf_queue,
+   .is_unordered = vb2_ops_is_unordered,
 };
 
 static void venc_buf_done(struct venus_inst *inst, unsigned int buf_type,
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c 
b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
index 5cf4d9921264..4402a5d621b2 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
@@ -1105,6 +1105,7 @@ static struct vb2_ops s5p_mfc_dec_qops = {
.start_streaming= s5p_mfc_start_streaming,
.stop_streaming = s5p_mfc_stop_streaming,
.buf_queue  = s5p_mfc_buf_queue,
+   .is_unordered   = vb2_ops_is_unordered,
 };
 
 const struct s5p_mfc_codec_ops *get_dec_codec_ops(void)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c 
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index 5c0462ca9993..376bd8eab8d8 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -2613,6 +2613,7 @@ static struct vb2_ops s5p_mfc_enc_qops = {
.start_streaming= s5p_mfc_start_streaming,
.stop_streaming = s5p_mfc_stop_streaming,
.buf_queue  = s5p_mfc_buf_queue,
+   .is_unordered   = vb2_ops_is_unordered,
 };
 
 const struct s5p_mfc_codec_ops *get_enc_codec_ops(void

[PATCH v10 13/16] vb2: add out-fence support to QBUF

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
an out_fence and send its fd to userspace in the fence_fd field as a
return arg for the QBUF call.

The fence is signaled on buffer_done(), when the job on the buffer is
finished.

v12: - Pass the fence_fd in vb2_qbuf for clarity.
 - Increase fence seqno if fence context if shared
 - Check get_unused_fd return

v11: - Return fence_fd to userpace only in the QBUF ioctl.
 - Rework implementation to avoid storing the sync_file
   as state, which is not really needed.

v10: - use -EIO for fence error (Hans Verkuil)
 - add comment around fence context creation (Hans Verkuil)

v9: - remove in-fences changes from this patch (Alex Courbot)
- improve fence context creation (Hans Verkuil)
- clean up out fences if vb2_core_qbuf() fails (Hans Verkuil)

v8: - return 0 as fence_fd if OUT_FENCE flag not used (Mauro)
- fix crash when checking not using fences in vb2_buffer_done()

v7: - merge patch that add the infrastructure to out-fences into
  this one (Alex Courbot)
- Do not install the fd if there is no fence. (Alex Courbot)
- do not report error on requeueing, just WARN_ON_ONCE() (Hans)

v6: - get rid of the V4L2_EVENT_OUT_FENCE event. We always keep the
  ordering in vb2 for queueing in the driver, so the event is not
  necessary anymore and the out_fence_fd is sent back to userspace
  on QBUF call return arg
- do not allow requeueing with out-fences, instead mark the buffer
  with an error and wake up to userspace.
- send the out_fence_fd back to userspace on the fence_fd field

v5: - delay fd_install to DQ_EVENT (Hans)
- if queue is fully ordered send OUT_FENCE event right away
  (Brian)
- rename 'q->ordered' to 'q->ordered_in_driver'
- merge change to implement OUT_FENCE event here

v4: - return the out_fence_fd in the BUF_QUEUED event(Hans)

v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
- set the OUT_FENCE flag if there is a fence pending (Hans)
- call fd_install() after vb2_core_qbuf() (Hans)
- clean up fence if vb2_core_qbuf() fails (Hans)
- add list to store sync_file and fence for the next queued buffer

v2: check if the queue is ordered.

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/common/videobuf2/videobuf2-core.c | 113 +++-
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  10 ++-
 drivers/media/dvb-core/dvb_vb2.c|   2 +-
 include/media/videobuf2-core.h  |  20 -
 4 files changed, 136 insertions(+), 9 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 86b5ebe25263..edc2fdaf56de 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -25,6 +25,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -380,6 +381,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
}
+   vb->out_fence_fd = -1;
q->bufs[vb->index] = vb;
 
/* Allocate video buffer memory for the MMAP type */
@@ -976,10 +978,22 @@ static void vb2_process_buffer_done(struct vb2_buffer 
*vb, enum vb2_buffer_state
case VB2_BUF_STATE_QUEUED:
return;
case VB2_BUF_STATE_REQUEUEING:
+   /* Requeuing with explicit synchronization, spit warning */
+   WARN_ON_ONCE(vb->out_fence);
+
if (q->start_streaming_called)
__enqueue_in_driver(vb);
return;
default:
+   if (vb->out_fence) {
+   if (state == VB2_BUF_STATE_ERROR)
+   dma_fence_set_error(vb->out_fence, -EIO);
+   dma_fence_signal(vb->out_fence);
+   dma_fence_put(vb->out_fence);
+   vb->out_fence = NULL;
+   vb->out_fence_fd = -1;
+   }
+
/* Inform any processes that may be waiting for buffers */
wake_up(>done_wq);
break;
@@ -1406,6 +1420,76 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned 
int index, void *pb)
 }
 EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
 
+static inline const char *vb2_fence_get_driver_name(struct dma_fence *fence)
+{
+   return "vb2_fence";
+}
+
+static inline const char *vb2_fence_get_timeline_name(struct dma_fence *fence)
+{
+   return "vb2_fence_timeline";
+}
+
+static inline bool vb2_fence_en

[PATCH v10 13/16] vb2: add out-fence support to QBUF

[Ezequiel Garcia]

From: Gustavo Padovan 

If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
an out_fence and send its fd to userspace in the fence_fd field as a
return arg for the QBUF call.

The fence is signaled on buffer_done(), when the job on the buffer is
finished.

v12: - Pass the fence_fd in vb2_qbuf for clarity.
 - Increase fence seqno if fence context if shared
 - Check get_unused_fd return

v11: - Return fence_fd to userpace only in the QBUF ioctl.
 - Rework implementation to avoid storing the sync_file
   as state, which is not really needed.

v10: - use -EIO for fence error (Hans Verkuil)
 - add comment around fence context creation (Hans Verkuil)

v9: - remove in-fences changes from this patch (Alex Courbot)
- improve fence context creation (Hans Verkuil)
- clean up out fences if vb2_core_qbuf() fails (Hans Verkuil)

v8: - return 0 as fence_fd if OUT_FENCE flag not used (Mauro)
- fix crash when checking not using fences in vb2_buffer_done()

v7: - merge patch that add the infrastructure to out-fences into
  this one (Alex Courbot)
- Do not install the fd if there is no fence. (Alex Courbot)
- do not report error on requeueing, just WARN_ON_ONCE() (Hans)

v6: - get rid of the V4L2_EVENT_OUT_FENCE event. We always keep the
  ordering in vb2 for queueing in the driver, so the event is not
  necessary anymore and the out_fence_fd is sent back to userspace
  on QBUF call return arg
- do not allow requeueing with out-fences, instead mark the buffer
  with an error and wake up to userspace.
- send the out_fence_fd back to userspace on the fence_fd field

v5: - delay fd_install to DQ_EVENT (Hans)
- if queue is fully ordered send OUT_FENCE event right away
  (Brian)
- rename 'q->ordered' to 'q->ordered_in_driver'
- merge change to implement OUT_FENCE event here

v4: - return the out_fence_fd in the BUF_QUEUED event(Hans)

v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
- set the OUT_FENCE flag if there is a fence pending (Hans)
- call fd_install() after vb2_core_qbuf() (Hans)
- clean up fence if vb2_core_qbuf() fails (Hans)
- add list to store sync_file and fence for the next queued buffer

v2: check if the queue is ordered.

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/common/videobuf2/videobuf2-core.c | 113 +++-
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  10 ++-
 drivers/media/dvb-core/dvb_vb2.c|   2 +-
 include/media/videobuf2-core.h  |  20 -
 4 files changed, 136 insertions(+), 9 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 86b5ebe25263..edc2fdaf56de 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -25,6 +25,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -380,6 +381,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
}
+   vb->out_fence_fd = -1;
q->bufs[vb->index] = vb;
 
/* Allocate video buffer memory for the MMAP type */
@@ -976,10 +978,22 @@ static void vb2_process_buffer_done(struct vb2_buffer 
*vb, enum vb2_buffer_state
case VB2_BUF_STATE_QUEUED:
return;
case VB2_BUF_STATE_REQUEUEING:
+   /* Requeuing with explicit synchronization, spit warning */
+   WARN_ON_ONCE(vb->out_fence);
+
if (q->start_streaming_called)
__enqueue_in_driver(vb);
return;
default:
+   if (vb->out_fence) {
+   if (state == VB2_BUF_STATE_ERROR)
+   dma_fence_set_error(vb->out_fence, -EIO);
+   dma_fence_signal(vb->out_fence);
+   dma_fence_put(vb->out_fence);
+   vb->out_fence = NULL;
+   vb->out_fence_fd = -1;
+   }
+
/* Inform any processes that may be waiting for buffers */
wake_up(>done_wq);
break;
@@ -1406,6 +1420,76 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned 
int index, void *pb)
 }
 EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
 
+static inline const char *vb2_fence_get_driver_name(struct dma_fence *fence)
+{
+   return "vb2_fence";
+}
+
+static inline const char *vb2_fence_get_timeline_name(struct dma_fence *fence)
+{
+   return "vb2_fence_timeline";
+}
+
+static inline bool vb2_fence_enable_signaling(struct dma_fence *fence)
+{
+   return true;
+}
+
+static const struct dma_fence_ops

[PATCH v10 15/16] v4l: Add V4L2_CAP_FENCES to drivers

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Drivers that use videobuf2 are capable of using fences and
should report that to userspace.

v10: - Add CAPS_FENCES to drivers that don't use fh->m2m_ctx.
 - Keep the ifdef V4L2_MEM2MEM_DEV.
 - Set CAPS_FENCES after vidioc_querycap.
v9: Add in the core.

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/platform/exynos-gsc/gsc-m2m.c|  3 ++-
 drivers/media/platform/m2m-deinterlace.c   |  3 ++-
 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c   |  3 ++-
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |  2 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  3 ++-
 drivers/media/platform/mx2_emmaprp.c   |  3 ++-
 drivers/media/platform/qcom/venus/vdec.c   |  3 ++-
 drivers/media/platform/qcom/venus/venc.c   |  3 ++-
 drivers/media/platform/sh_veu.c|  3 ++-
 drivers/media/v4l2-core/v4l2-ioctl.c   | 24 +-
 10 files changed, 40 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/exynos-gsc/gsc-m2m.c 
b/drivers/media/platform/exynos-gsc/gsc-m2m.c
index e9ff27949a91..fbf072c8aedd 100644
--- a/drivers/media/platform/exynos-gsc/gsc-m2m.c
+++ b/drivers/media/platform/exynos-gsc/gsc-m2m.c
@@ -298,7 +298,8 @@ static int gsc_m2m_querycap(struct file *file, void *fh,
strlcpy(cap->card, GSC_MODULE_NAME " gscaler", sizeof(cap->card));
snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
 dev_name(>pdev->dev));
-   cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE;
+   cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE |
+  V4L2_CAP_FENCES;
cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
return 0;
 }
diff --git a/drivers/media/platform/m2m-deinterlace.c 
b/drivers/media/platform/m2m-deinterlace.c
index 1e4195144f39..b7421de1268f 100644
--- a/drivers/media/platform/m2m-deinterlace.c
+++ b/drivers/media/platform/m2m-deinterlace.c
@@ -461,7 +461,8 @@ static int vidioc_querycap(struct file *file, void *priv,
 * and are scheduled for removal.
 */
cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT |
-  V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING;
+  V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING |
+  V4L2_CAP_FENCES;
cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
 
return 0;
diff --git a/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c 
b/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
index 583d47724ee8..7aba2ba128ba 100644
--- a/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
+++ b/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
@@ -1242,7 +1242,8 @@ int mtk_mdp_register_m2m_device(struct mtk_mdp_dev *mdp)
ret = -ENOMEM;
goto err_video_alloc;
}
-   mdp->vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+   mdp->vdev->device_caps = V4L2_CAP_FENCES | V4L2_CAP_VIDEO_M2M_MPLANE |
+V4L2_CAP_STREAMING;
mdp->vdev->fops = _mdp_m2m_fops;
mdp->vdev->ioctl_ops = _mdp_m2m_ioctl_ops;
mdp->vdev->release = video_device_release;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index 4334b7394861..75318a4129ea 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -321,7 +321,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
vfd_dec->v4l2_dev   = >v4l2_dev;
vfd_dec->vfl_dir= VFL_DIR_M2M;
vfd_dec->device_caps= V4L2_CAP_VIDEO_M2M_MPLANE |
-   V4L2_CAP_STREAMING;
+   V4L2_CAP_STREAMING | V4L2_CAP_FENCES;
 
snprintf(vfd_dec->name, sizeof(vfd_dec->name), "%s",
MTK_VCODEC_DEC_NAME);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 83f859e8509c..1c9d4e7262bb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -339,7 +339,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
vfd_enc->v4l2_dev   = >v4l2_dev;
vfd_enc->vfl_dir= VFL_DIR_M2M;
vfd_enc->device_caps= V4L2_CAP_VIDEO_M2M_MPLANE |
-   V4L2_CAP_STREAMING;
+   V4L2_CAP_STREAMING |
+   V4L2_CAP_FENCES;
 
snprintf(vfd_enc-

[PATCH v10 15/16] v4l: Add V4L2_CAP_FENCES to drivers

[Ezequiel Garcia]

From: Gustavo Padovan 

Drivers that use videobuf2 are capable of using fences and
should report that to userspace.

v10: - Add CAPS_FENCES to drivers that don't use fh->m2m_ctx.
 - Keep the ifdef V4L2_MEM2MEM_DEV.
 - Set CAPS_FENCES after vidioc_querycap.
v9: Add in the core.

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/platform/exynos-gsc/gsc-m2m.c|  3 ++-
 drivers/media/platform/m2m-deinterlace.c   |  3 ++-
 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c   |  3 ++-
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |  2 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |  3 ++-
 drivers/media/platform/mx2_emmaprp.c   |  3 ++-
 drivers/media/platform/qcom/venus/vdec.c   |  3 ++-
 drivers/media/platform/qcom/venus/venc.c   |  3 ++-
 drivers/media/platform/sh_veu.c|  3 ++-
 drivers/media/v4l2-core/v4l2-ioctl.c   | 24 +-
 10 files changed, 40 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/exynos-gsc/gsc-m2m.c 
b/drivers/media/platform/exynos-gsc/gsc-m2m.c
index e9ff27949a91..fbf072c8aedd 100644
--- a/drivers/media/platform/exynos-gsc/gsc-m2m.c
+++ b/drivers/media/platform/exynos-gsc/gsc-m2m.c
@@ -298,7 +298,8 @@ static int gsc_m2m_querycap(struct file *file, void *fh,
strlcpy(cap->card, GSC_MODULE_NAME " gscaler", sizeof(cap->card));
snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
 dev_name(>pdev->dev));
-   cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE;
+   cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE |
+  V4L2_CAP_FENCES;
cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
return 0;
 }
diff --git a/drivers/media/platform/m2m-deinterlace.c 
b/drivers/media/platform/m2m-deinterlace.c
index 1e4195144f39..b7421de1268f 100644
--- a/drivers/media/platform/m2m-deinterlace.c
+++ b/drivers/media/platform/m2m-deinterlace.c
@@ -461,7 +461,8 @@ static int vidioc_querycap(struct file *file, void *priv,
 * and are scheduled for removal.
 */
cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT |
-  V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING;
+  V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING |
+  V4L2_CAP_FENCES;
cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
 
return 0;
diff --git a/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c 
b/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
index 583d47724ee8..7aba2ba128ba 100644
--- a/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
+++ b/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
@@ -1242,7 +1242,8 @@ int mtk_mdp_register_m2m_device(struct mtk_mdp_dev *mdp)
ret = -ENOMEM;
goto err_video_alloc;
}
-   mdp->vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+   mdp->vdev->device_caps = V4L2_CAP_FENCES | V4L2_CAP_VIDEO_M2M_MPLANE |
+V4L2_CAP_STREAMING;
mdp->vdev->fops = _mdp_m2m_fops;
mdp->vdev->ioctl_ops = _mdp_m2m_ioctl_ops;
mdp->vdev->release = video_device_release;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index 4334b7394861..75318a4129ea 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -321,7 +321,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
vfd_dec->v4l2_dev   = >v4l2_dev;
vfd_dec->vfl_dir= VFL_DIR_M2M;
vfd_dec->device_caps= V4L2_CAP_VIDEO_M2M_MPLANE |
-   V4L2_CAP_STREAMING;
+   V4L2_CAP_STREAMING | V4L2_CAP_FENCES;
 
snprintf(vfd_dec->name, sizeof(vfd_dec->name), "%s",
MTK_VCODEC_DEC_NAME);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 83f859e8509c..1c9d4e7262bb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -339,7 +339,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
vfd_enc->v4l2_dev   = >v4l2_dev;
vfd_enc->vfl_dir= VFL_DIR_M2M;
vfd_enc->device_caps= V4L2_CAP_VIDEO_M2M_MPLANE |
-   V4L2_CAP_STREAMING;
+   V4L2_CAP_STREAMING |
+   V4L2_CAP_FENCES;
 
snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
 MTK_VCODEC_ENC_NAME);
diff --git

[PATCH v10 16/16] v4l: Document explicit synchronization behavior

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Add section to VIDIOC_QBUF and VIDIOC_QUERY_BUF about it

v9: assorted improvements.

v8: amend querybuf documentation.

v7: minor issues and English improvements (Hans Verkuil)

v6: Close some gaps in the docs (Hans)

v5: - Remove V4L2_CAP_ORDERED
- Add doc about V4L2_FMT_FLAG_UNORDERED

v4: - Document ordering behavior for in-fences
- Document V4L2_CAP_ORDERED capability
- Remove doc about OUT_FENCE event
- Document immediate return of out-fence in QBUF

v3: - make the out_fence refer to the current buffer (Hans)
- Note what happens when the IN_FENCE is not set (Hans)

v2: - mention that fences are files (Hans)
- rework for the new API

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 Documentation/media/uapi/v4l/vidioc-qbuf.rst | 53 +++-
 Documentation/media/uapi/v4l/vidioc-querybuf.rst | 12 --
 2 files changed, 61 insertions(+), 4 deletions(-)

diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst 
b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
index 9e448a4aa3aa..f09e21247a6c 100644
--- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
+++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
@@ -54,7 +54,7 @@ When the buffer is intended for output (``type`` is
 or ``V4L2_BUF_TYPE_VBI_OUTPUT``) applications must also initialize the
 ``bytesused``, ``field`` and ``timestamp`` fields, see :ref:`buffer`
 for details. Applications must also set ``flags`` to 0. The
-``reserved2`` and ``reserved`` fields must be set to 0. When using the
+``reserved`` field must be set to 0. When using the
 :ref:`multi-planar API `, the ``m.planes`` field must
 contain a userspace pointer to a filled-in array of struct
 :c:type:`v4l2_plane` and the ``length`` field must be set
@@ -118,6 +118,57 @@ immediately with an ``EAGAIN`` error code when no buffer 
is available.
 The struct :c:type:`v4l2_buffer` structure is specified in
 :ref:`buffer`.
 
+Explicit Synchronization
+
+
+Explicit Synchronization allows us to control the synchronization of
+shared buffers from userspace by passing fences to the kernel and/or
+receiving them from it. Fences passed to the kernel are named in-fences and
+the kernel should wait on them to signal before using the buffer. On the other
+side, the kernel can create out-fences for the buffers it queues to the
+drivers. Out-fences signal when the driver is finished with buffer, i.e., the
+buffer is ready. The fences are represented as a file and passed as a file
+descriptor to userspace.
+
+The in-fences are communicated to the kernel at the ``VIDIOC_QBUF`` ioctl
+using the ``V4L2_BUF_FLAG_IN_FENCE`` buffer flag and the `fence_fd` field. If
+an in-fence needs to be passed to the kernel, `fence_fd` should be set to the
+fence file descriptor number and the ``V4L2_BUF_FLAG_IN_FENCE`` should be set
+as well. Setting one but not the other will cause ``VIDIOC_QBUF`` to return
+with an error.
+
+It is guaranteed that all buffers queued with an in-fence will be queued to
+the drivers in the same order. Fences may signal out of order, so this 
guarantee
+is necessary to not change ordering. While waiting for a fence to signal,
+all buffers queued afterwards will also be blocked until that fence signals.
+
+If the in-fence signals with an error the buffer will be marked with
+``V4L2_BUF_FLAG_ERROR`` when returned to userspace at ``VIDIOC_DQBUF``.
+Even with the error the order of dequeueing the buffers is preserved.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE`` flag should
+be set to ask for a fence to be attached to the buffer. The out-fence fd is
+sent to userspace as a ``VIDIOC_QBUF`` return argument in the `fence_fd` field.
+
+Note the same `fence_fd` field is used for both sending the in-fence as
+at input argument and to receive the out-fence as a return argument. A buffer 
can
+have both an in-fence and an out-fence.
+
+At streamoff the out-fences will either signal normally if the driver waits
+for the operations on the buffers to finish or signal with an error if the
+driver cancels the pending operations. Buffers with in-fences won't be queued
+to the driver if their fences signal. They will be cleaned up.
+
+The ``V4L2_FMT_FLAG_UNORDERED`` flag in ``VIDIOC_ENUM_FMT`` tells userspace
+that when using this format, the order in which buffers are dequeued can
+be different from the order in which they were queued.
+
+Ordering is important to fences because it can optimize the pipeline with
+other drivers like a DRM/KMS display driver. For example, if a capture from the
+camera is happening in an ordered manner one can send the capture buffer
+out-fence to the DRM/KMS driver and rest assured that the buffers will be 
shown on
+the screen in the correct order. If the queue is not ordered, then such
+arrangements with other drivers may not be possibl

[PATCH v10 16/16] v4l: Document explicit synchronization behavior

[Ezequiel Garcia]

From: Gustavo Padovan 

Add section to VIDIOC_QBUF and VIDIOC_QUERY_BUF about it

v9: assorted improvements.

v8: amend querybuf documentation.

v7: minor issues and English improvements (Hans Verkuil)

v6: Close some gaps in the docs (Hans)

v5: - Remove V4L2_CAP_ORDERED
- Add doc about V4L2_FMT_FLAG_UNORDERED

v4: - Document ordering behavior for in-fences
- Document V4L2_CAP_ORDERED capability
- Remove doc about OUT_FENCE event
- Document immediate return of out-fence in QBUF

v3: - make the out_fence refer to the current buffer (Hans)
- Note what happens when the IN_FENCE is not set (Hans)

v2: - mention that fences are files (Hans)
- rework for the new API

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 Documentation/media/uapi/v4l/vidioc-qbuf.rst | 53 +++-
 Documentation/media/uapi/v4l/vidioc-querybuf.rst | 12 --
 2 files changed, 61 insertions(+), 4 deletions(-)

diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst 
b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
index 9e448a4aa3aa..f09e21247a6c 100644
--- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
+++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
@@ -54,7 +54,7 @@ When the buffer is intended for output (``type`` is
 or ``V4L2_BUF_TYPE_VBI_OUTPUT``) applications must also initialize the
 ``bytesused``, ``field`` and ``timestamp`` fields, see :ref:`buffer`
 for details. Applications must also set ``flags`` to 0. The
-``reserved2`` and ``reserved`` fields must be set to 0. When using the
+``reserved`` field must be set to 0. When using the
 :ref:`multi-planar API `, the ``m.planes`` field must
 contain a userspace pointer to a filled-in array of struct
 :c:type:`v4l2_plane` and the ``length`` field must be set
@@ -118,6 +118,57 @@ immediately with an ``EAGAIN`` error code when no buffer 
is available.
 The struct :c:type:`v4l2_buffer` structure is specified in
 :ref:`buffer`.
 
+Explicit Synchronization
+
+
+Explicit Synchronization allows us to control the synchronization of
+shared buffers from userspace by passing fences to the kernel and/or
+receiving them from it. Fences passed to the kernel are named in-fences and
+the kernel should wait on them to signal before using the buffer. On the other
+side, the kernel can create out-fences for the buffers it queues to the
+drivers. Out-fences signal when the driver is finished with buffer, i.e., the
+buffer is ready. The fences are represented as a file and passed as a file
+descriptor to userspace.
+
+The in-fences are communicated to the kernel at the ``VIDIOC_QBUF`` ioctl
+using the ``V4L2_BUF_FLAG_IN_FENCE`` buffer flag and the `fence_fd` field. If
+an in-fence needs to be passed to the kernel, `fence_fd` should be set to the
+fence file descriptor number and the ``V4L2_BUF_FLAG_IN_FENCE`` should be set
+as well. Setting one but not the other will cause ``VIDIOC_QBUF`` to return
+with an error.
+
+It is guaranteed that all buffers queued with an in-fence will be queued to
+the drivers in the same order. Fences may signal out of order, so this 
guarantee
+is necessary to not change ordering. While waiting for a fence to signal,
+all buffers queued afterwards will also be blocked until that fence signals.
+
+If the in-fence signals with an error the buffer will be marked with
+``V4L2_BUF_FLAG_ERROR`` when returned to userspace at ``VIDIOC_DQBUF``.
+Even with the error the order of dequeueing the buffers is preserved.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE`` flag should
+be set to ask for a fence to be attached to the buffer. The out-fence fd is
+sent to userspace as a ``VIDIOC_QBUF`` return argument in the `fence_fd` field.
+
+Note the same `fence_fd` field is used for both sending the in-fence as
+at input argument and to receive the out-fence as a return argument. A buffer 
can
+have both an in-fence and an out-fence.
+
+At streamoff the out-fences will either signal normally if the driver waits
+for the operations on the buffers to finish or signal with an error if the
+driver cancels the pending operations. Buffers with in-fences won't be queued
+to the driver if their fences signal. They will be cleaned up.
+
+The ``V4L2_FMT_FLAG_UNORDERED`` flag in ``VIDIOC_ENUM_FMT`` tells userspace
+that when using this format, the order in which buffers are dequeued can
+be different from the order in which they were queued.
+
+Ordering is important to fences because it can optimize the pipeline with
+other drivers like a DRM/KMS display driver. For example, if a capture from the
+camera is happening in an ordered manner one can send the capture buffer
+out-fence to the DRM/KMS driver and rest assured that the buffers will be 
shown on
+the screen in the correct order. If the queue is not ordered, then such
+arrangements with other drivers may not be possible.
 
 Return Value
 
diff --git a/Documentation/media/uapi/v4l/vidioc-querybuf.rst 
b/Documentation/media

[PATCH v10 14/16] v4l: introduce the fences capability

[Ezequiel Garcia]

From: Gustavo Padovan 

Drivers capable of using fences (vb2 drivers) should report the
V4L2_CAP_FENCES to userspace, so add this flag to the uapi.

v2: minor doc/english fix (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/vidioc-querycap.rst | 3 +++
 include/uapi/linux/videodev2.h   | 1 +
 2 files changed, 4 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-querycap.rst 
b/Documentation/media/uapi/v4l/vidioc-querycap.rst
index 66fb1b3d6e6e..df3ad57f07a3 100644
--- a/Documentation/media/uapi/v4l/vidioc-querycap.rst
+++ b/Documentation/media/uapi/v4l/vidioc-querycap.rst
@@ -254,6 +254,9 @@ specification the ioctl returns an ``EINVAL`` error code.
 * - ``V4L2_CAP_TOUCH``
   - 0x1000
   - This is a touch device.
+* - ``V4L2_CAP_FENCES``
+  - 0x2000
+  - The device supports explicit synchronization.
 * - ``V4L2_CAP_DEVICE_CAPS``
   - 0x8000
   - The driver fills the ``device_caps`` field. This capability can
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1f18dc68ecab..cab35fca7c7f 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -460,6 +460,7 @@ struct v4l2_capability {
 #define V4L2_CAP_STREAMING  0x0400  /* streaming I/O ioctls */
 
 #define V4L2_CAP_TOUCH  0x1000  /* Is a touch device */
+#define V4L2_CAP_FENCES 0x2000  /* Supports explicit 
synchronization */
 
 #define V4L2_CAP_DEVICE_CAPS0x8000  /* sets device 
capabilities field */
 
-- 
2.16.3

[PATCH v10 14/16] v4l: introduce the fences capability

[Ezequiel Garcia]

From: Gustavo Padovan 

Drivers capable of using fences (vb2 drivers) should report the
V4L2_CAP_FENCES to userspace, so add this flag to the uapi.

v2: minor doc/english fix (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/vidioc-querycap.rst | 3 +++
 include/uapi/linux/videodev2.h   | 1 +
 2 files changed, 4 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-querycap.rst 
b/Documentation/media/uapi/v4l/vidioc-querycap.rst
index 66fb1b3d6e6e..df3ad57f07a3 100644
--- a/Documentation/media/uapi/v4l/vidioc-querycap.rst
+++ b/Documentation/media/uapi/v4l/vidioc-querycap.rst
@@ -254,6 +254,9 @@ specification the ioctl returns an ``EINVAL`` error code.
 * - ``V4L2_CAP_TOUCH``
   - 0x1000
   - This is a touch device.
+* - ``V4L2_CAP_FENCES``
+  - 0x2000
+  - The device supports explicit synchronization.
 * - ``V4L2_CAP_DEVICE_CAPS``
   - 0x8000
   - The driver fills the ``device_caps`` field. This capability can
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1f18dc68ecab..cab35fca7c7f 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -460,6 +460,7 @@ struct v4l2_capability {
 #define V4L2_CAP_STREAMING  0x0400  /* streaming I/O ioctls */
 
 #define V4L2_CAP_TOUCH  0x1000  /* Is a touch device */
+#define V4L2_CAP_FENCES 0x2000  /* Supports explicit 
synchronization */
 
 #define V4L2_CAP_DEVICE_CAPS0x8000  /* sets device 
capabilities field */
 
-- 
2.16.3

[PATCH v10 12/16] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Receive in-fence from userspace and add support for waiting on them
before queueing the buffer to the driver. Buffers can't be queued to the
driver before its fences signal. And a buffer can't be queued to the driver
out of the order they were queued from userspace. That means that even if
its fence signals it must wait for all other buffers, ahead of it in the queue,
to signal first.

If the fence for some buffer fails we do not queue it to the driver,
instead we mark it as error and wait until the previous buffer is done
to notify userspace of the error. We wait here to deliver the buffers back
to userspace in order.

v13: - cleanup implementation.
 - remove wrong Kconfig changes.
 - print noisy warning on unexpected enqueue conditioin
 - schedule a vb2_start_streaming work from the fence callback

v12: fixed dvb_vb2.c usage of vb2_core_qbuf.

v11: - minor doc/comments fixes (Hans Verkuil)
 - reviewed the in-fence path at __fill_v4l2_buffer()

v10: - rename fence to in_fence in many places
 - handle fences signalling with error better (Hans Verkuil)

v9: - improve comments and docs (Hans Verkuil)
- fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
- move in-fences code that was in the out-fences patch here (Alex)

v8: - improve comments about fences with errors

v7: - get rid of the fence array stuff for ordering and just use
  get_num_buffers_ready() (Hans)
- fix issue of queuing the buffer twice (Hans)
- avoid the dma_fence_wait() in core_qbuf() (Alex)
- merge preparation commit in

v6: - With fences always keep the order userspace queues the buffers.
- Protect in_fence manipulation with a lock (Brian Starkey)
- check if fences have the same context before adding a fence array
- Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
- Clean up fence if __set_in_fence() fails (Brian Starkey)
- treat -EINVAL from dma_fence_add_callback() (Brian Starkey)

v5: - use fence_array to keep buffers ordered in vb2 core when
  needed (Brian Starkey)
- keep backward compat on the reserved2 field (Brian Starkey)
- protect fence callback removal with lock (Brian Starkey)

v4: - Add a comment about dma_fence_add_callback() not returning a
  error (Hans)
- Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
- select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
- Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
- Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
- Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
  vb2_start_streaming() (Hans)
- set IN_FENCE flags on __fill_v4l2_buffer (Hans)
- Queue buffers to the driver as soon as they are ready (Hans)
- call fill_user_buffer() after queuing the buffer (Hans)
- add err: label to clean up fence
- add dma_fence_wait() before calling vb2_start_streaming()

v3: - document fence parameter
- remove ternary if at vb2_qbuf() return (Mauro)
- do not change if conditions behaviour (Mauro)

v2: - fix vb2_queue_or_prepare_buf() ret check
- remove check for VB2_MEMORY_DMABUF only (Javier)
- check num of ready buffers to start streaming
- when queueing, start from the first ready buffer
- handle queue cancel

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/common/videobuf2/Kconfig  |   1 +
 drivers/media/common/videobuf2/videobuf2-core.c | 224 
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  37 +++-
 drivers/media/dvb-core/dvb_vb2.c|   2 +-
 include/media/videobuf2-core.h  |  19 +-
 5 files changed, 242 insertions(+), 41 deletions(-)

diff --git a/drivers/media/common/videobuf2/Kconfig 
b/drivers/media/common/videobuf2/Kconfig
index 17c32ea58395..27ad9e8a268b 100644
--- a/drivers/media/common/videobuf2/Kconfig
+++ b/drivers/media/common/videobuf2/Kconfig
@@ -1,6 +1,7 @@
 # Used by drivers that need Videobuf2 modules
 config VIDEOBUF2_CORE
select DMA_SHARED_BUFFER
+   select SYNC_FILE
tristate
 
 config VIDEOBUF2_V4L2
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index a9a0a9d1decb..86b5ebe25263 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -27,6 +27,7 @@
 #include 
 
 #include 
+#include 
 #include 
 
 #include 
@@ -189,6 +190,7 @@ module_param(debug, int, 0644);
 
 static void __vb2_queue_cancel(struct vb2_queue *q);
 static void __enqueue_in_driver(struct vb2_buffer *vb);
+static void __qbuf_work(struct work_struct *work);
 
 static void __vb2_buffer_free(struct kref *kref)
 {
@@ -373,6 +375,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
  

[PATCH v10 12/16] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

From: Gustavo Padovan 

Receive in-fence from userspace and add support for waiting on them
before queueing the buffer to the driver. Buffers can't be queued to the
driver before its fences signal. And a buffer can't be queued to the driver
out of the order they were queued from userspace. That means that even if
its fence signals it must wait for all other buffers, ahead of it in the queue,
to signal first.

If the fence for some buffer fails we do not queue it to the driver,
instead we mark it as error and wait until the previous buffer is done
to notify userspace of the error. We wait here to deliver the buffers back
to userspace in order.

v13: - cleanup implementation.
 - remove wrong Kconfig changes.
 - print noisy warning on unexpected enqueue conditioin
 - schedule a vb2_start_streaming work from the fence callback

v12: fixed dvb_vb2.c usage of vb2_core_qbuf.

v11: - minor doc/comments fixes (Hans Verkuil)
 - reviewed the in-fence path at __fill_v4l2_buffer()

v10: - rename fence to in_fence in many places
 - handle fences signalling with error better (Hans Verkuil)

v9: - improve comments and docs (Hans Verkuil)
- fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
- move in-fences code that was in the out-fences patch here (Alex)

v8: - improve comments about fences with errors

v7: - get rid of the fence array stuff for ordering and just use
  get_num_buffers_ready() (Hans)
- fix issue of queuing the buffer twice (Hans)
- avoid the dma_fence_wait() in core_qbuf() (Alex)
- merge preparation commit in

v6: - With fences always keep the order userspace queues the buffers.
- Protect in_fence manipulation with a lock (Brian Starkey)
- check if fences have the same context before adding a fence array
- Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
- Clean up fence if __set_in_fence() fails (Brian Starkey)
- treat -EINVAL from dma_fence_add_callback() (Brian Starkey)

v5: - use fence_array to keep buffers ordered in vb2 core when
  needed (Brian Starkey)
- keep backward compat on the reserved2 field (Brian Starkey)
- protect fence callback removal with lock (Brian Starkey)

v4: - Add a comment about dma_fence_add_callback() not returning a
  error (Hans)
- Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
- select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
- Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
- Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
- Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
  vb2_start_streaming() (Hans)
- set IN_FENCE flags on __fill_v4l2_buffer (Hans)
- Queue buffers to the driver as soon as they are ready (Hans)
- call fill_user_buffer() after queuing the buffer (Hans)
- add err: label to clean up fence
- add dma_fence_wait() before calling vb2_start_streaming()

v3: - document fence parameter
- remove ternary if at vb2_qbuf() return (Mauro)
- do not change if conditions behaviour (Mauro)

v2: - fix vb2_queue_or_prepare_buf() ret check
- remove check for VB2_MEMORY_DMABUF only (Javier)
- check num of ready buffers to start streaming
- when queueing, start from the first ready buffer
- handle queue cancel

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/common/videobuf2/Kconfig  |   1 +
 drivers/media/common/videobuf2/videobuf2-core.c | 224 
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  37 +++-
 drivers/media/dvb-core/dvb_vb2.c|   2 +-
 include/media/videobuf2-core.h  |  19 +-
 5 files changed, 242 insertions(+), 41 deletions(-)

diff --git a/drivers/media/common/videobuf2/Kconfig 
b/drivers/media/common/videobuf2/Kconfig
index 17c32ea58395..27ad9e8a268b 100644
--- a/drivers/media/common/videobuf2/Kconfig
+++ b/drivers/media/common/videobuf2/Kconfig
@@ -1,6 +1,7 @@
 # Used by drivers that need Videobuf2 modules
 config VIDEOBUF2_CORE
select DMA_SHARED_BUFFER
+   select SYNC_FILE
tristate
 
 config VIDEOBUF2_V4L2
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index a9a0a9d1decb..86b5ebe25263 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -27,6 +27,7 @@
 #include 
 
 #include 
+#include 
 #include 
 
 #include 
@@ -189,6 +190,7 @@ module_param(debug, int, 0644);
 
 static void __vb2_queue_cancel(struct vb2_queue *q);
 static void __enqueue_in_driver(struct vb2_buffer *vb);
+static void __qbuf_work(struct work_struct *work);
 
 static void __vb2_buffer_free(struct kref *kref)
 {
@@ -373,6 +375,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->index = q->num_buffers + buffer;
vb->type = q->type;
  

[PATCH v10 09/16] cobalt: add .is_unordered() for cobalt

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

The cobalt driver may reorder the capture buffers so we need to report
it as such.

v3: set formats as unordered
v2: use vb2_ops_set_unordered() helper

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/pci/cobalt/cobalt-v4l2.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/drivers/media/pci/cobalt/cobalt-v4l2.c 
b/drivers/media/pci/cobalt/cobalt-v4l2.c
index e2a4c705d353..ccca1a96df90 100644
--- a/drivers/media/pci/cobalt/cobalt-v4l2.c
+++ b/drivers/media/pci/cobalt/cobalt-v4l2.c
@@ -430,6 +430,7 @@ static const struct vb2_ops cobalt_qops = {
.stop_streaming = cobalt_stop_streaming,
.wait_prepare = vb2_ops_wait_prepare,
.wait_finish = vb2_ops_wait_finish,
+   .is_unordered = vb2_ops_is_unordered,
 };
 
 /* V4L2 ioctls */
@@ -695,14 +696,17 @@ static int cobalt_enum_fmt_vid_cap(struct file *file, 
void *priv_fh,
case 0:
strlcpy(f->description, "YUV 4:2:2", sizeof(f->description));
f->pixelformat = V4L2_PIX_FMT_YUYV;
+   f->flags |= V4L2_FMT_FLAG_UNORDERED;
break;
case 1:
strlcpy(f->description, "RGB24", sizeof(f->description));
f->pixelformat = V4L2_PIX_FMT_RGB24;
+   f->flags |= V4L2_FMT_FLAG_UNORDERED;
break;
case 2:
strlcpy(f->description, "RGB32", sizeof(f->description));
f->pixelformat = V4L2_PIX_FMT_BGR32;
+   f->flags |= V4L2_FMT_FLAG_UNORDERED;
break;
default:
return -EINVAL;
-- 
2.16.3

[PATCH v10 09/16] cobalt: add .is_unordered() for cobalt

[Ezequiel Garcia]

From: Gustavo Padovan 

The cobalt driver may reorder the capture buffers so we need to report
it as such.

v3: set formats as unordered
v2: use vb2_ops_set_unordered() helper

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/pci/cobalt/cobalt-v4l2.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/drivers/media/pci/cobalt/cobalt-v4l2.c 
b/drivers/media/pci/cobalt/cobalt-v4l2.c
index e2a4c705d353..ccca1a96df90 100644
--- a/drivers/media/pci/cobalt/cobalt-v4l2.c
+++ b/drivers/media/pci/cobalt/cobalt-v4l2.c
@@ -430,6 +430,7 @@ static const struct vb2_ops cobalt_qops = {
.stop_streaming = cobalt_stop_streaming,
.wait_prepare = vb2_ops_wait_prepare,
.wait_finish = vb2_ops_wait_finish,
+   .is_unordered = vb2_ops_is_unordered,
 };
 
 /* V4L2 ioctls */
@@ -695,14 +696,17 @@ static int cobalt_enum_fmt_vid_cap(struct file *file, 
void *priv_fh,
case 0:
strlcpy(f->description, "YUV 4:2:2", sizeof(f->description));
f->pixelformat = V4L2_PIX_FMT_YUYV;
+   f->flags |= V4L2_FMT_FLAG_UNORDERED;
break;
case 1:
strlcpy(f->description, "RGB24", sizeof(f->description));
f->pixelformat = V4L2_PIX_FMT_RGB24;
+   f->flags |= V4L2_FMT_FLAG_UNORDERED;
break;
case 2:
strlcpy(f->description, "RGB32", sizeof(f->description));
f->pixelformat = V4L2_PIX_FMT_BGR32;
+   f->flags |= V4L2_FMT_FLAG_UNORDERED;
break;
default:
return -EINVAL;
-- 
2.16.3

[PATCH v10 08/16] v4l: mark unordered formats

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Now that we've introduced the V4L2_FMT_FLAG_UNORDERED flag,
mark the appropriate formats.

v2: Set unordered flag before calling the driver callback.

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 74 +++-
 1 file changed, 57 insertions(+), 17 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
b/drivers/media/v4l2-core/v4l2-ioctl.c
index f48c505550e0..2135ac235a96 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1102,6 +1102,27 @@ static int v4l_enumoutput(const struct v4l2_ioctl_ops 
*ops,
return ops->vidioc_enum_output(file, fh, p);
 }
 
+static void v4l_fill_unordered_fmtdesc(struct v4l2_fmtdesc *fmt)
+{
+   switch (fmt->pixelformat) {
+   case V4L2_PIX_FMT_MPEG:
+   case V4L2_PIX_FMT_H264:
+   case V4L2_PIX_FMT_H264_NO_SC:
+   case V4L2_PIX_FMT_H264_MVC:
+   case V4L2_PIX_FMT_H263:
+   case V4L2_PIX_FMT_MPEG1:
+   case V4L2_PIX_FMT_MPEG2:
+   case V4L2_PIX_FMT_MPEG4:
+   case V4L2_PIX_FMT_XVID:
+   case V4L2_PIX_FMT_VC1_ANNEX_G:
+   case V4L2_PIX_FMT_VC1_ANNEX_L:
+   case V4L2_PIX_FMT_VP8:
+   case V4L2_PIX_FMT_VP9:
+   case V4L2_PIX_FMT_HEVC:
+   fmt->flags |= V4L2_FMT_FLAG_UNORDERED;
+   }
+}
+
 static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 {
const unsigned sz = sizeof(fmt->description);
@@ -1310,61 +1331,80 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 
if (descr)
WARN_ON(strlcpy(fmt->description, descr, sz) >= sz);
-   fmt->flags = flags;
+   fmt->flags |= flags;
 }
 
-static int v4l_enum_fmt(const struct v4l2_ioctl_ops *ops,
-   struct file *file, void *fh, void *arg)
-{
-   struct v4l2_fmtdesc *p = arg;
-   int ret = check_fmt(file, p->type);
 
-   if (ret)
-   return ret;
-   ret = -EINVAL;
+static int __vidioc_enum_fmt(const struct v4l2_ioctl_ops *ops,
+struct v4l2_fmtdesc *p,
+struct file *file, void *fh)
+{
+   int ret = 0;
 
switch (p->type) {
case V4L2_BUF_TYPE_VIDEO_CAPTURE:
if (unlikely(!ops->vidioc_enum_fmt_vid_cap))
break;
-   ret = ops->vidioc_enum_fmt_vid_cap(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_cap(file, fh, p);
break;
case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
if (unlikely(!ops->vidioc_enum_fmt_vid_cap_mplane))
break;
-   ret = ops->vidioc_enum_fmt_vid_cap_mplane(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_cap_mplane(file, fh, p);
break;
case V4L2_BUF_TYPE_VIDEO_OVERLAY:
if (unlikely(!ops->vidioc_enum_fmt_vid_overlay))
break;
-   ret = ops->vidioc_enum_fmt_vid_overlay(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_overlay(file, fh, p);
break;
case V4L2_BUF_TYPE_VIDEO_OUTPUT:
if (unlikely(!ops->vidioc_enum_fmt_vid_out))
break;
-   ret = ops->vidioc_enum_fmt_vid_out(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_out(file, fh, p);
break;
case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
if (unlikely(!ops->vidioc_enum_fmt_vid_out_mplane))
break;
-   ret = ops->vidioc_enum_fmt_vid_out_mplane(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_out_mplane(file, fh, p);
break;
case V4L2_BUF_TYPE_SDR_CAPTURE:
if (unlikely(!ops->vidioc_enum_fmt_sdr_cap))
break;
-   ret = ops->vidioc_enum_fmt_sdr_cap(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_sdr_cap(file, fh, p);
break;
case V4L2_BUF_TYPE_SDR_OUTPUT:
if (unlikely(!ops->vidioc_enum_fmt_sdr_out))
break;
-   ret = ops->vidioc_enum_fmt_sdr_out(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_sdr_out(file, fh, p);
break;
case V4L2_BUF_TYPE_META_CAPTURE:
if (unlikely(!ops->vidioc_enum_fmt_meta_cap))
break;
-   ret = ops->vidioc_enum_fmt_meta_cap(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_meta_cap(file, fh, p);
break;
}
+   return ret;
+}
+
+static int v4l_enum_fmt(const struct v4l2_ioctl_ops *ops,
+   struct file *file, void *fh, void *arg)
+{
+  

[PATCH v10 08/16] v4l: mark unordered formats

[Ezequiel Garcia]

From: Gustavo Padovan 

Now that we've introduced the V4L2_FMT_FLAG_UNORDERED flag,
mark the appropriate formats.

v2: Set unordered flag before calling the driver callback.

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 74 +++-
 1 file changed, 57 insertions(+), 17 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
b/drivers/media/v4l2-core/v4l2-ioctl.c
index f48c505550e0..2135ac235a96 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1102,6 +1102,27 @@ static int v4l_enumoutput(const struct v4l2_ioctl_ops 
*ops,
return ops->vidioc_enum_output(file, fh, p);
 }
 
+static void v4l_fill_unordered_fmtdesc(struct v4l2_fmtdesc *fmt)
+{
+   switch (fmt->pixelformat) {
+   case V4L2_PIX_FMT_MPEG:
+   case V4L2_PIX_FMT_H264:
+   case V4L2_PIX_FMT_H264_NO_SC:
+   case V4L2_PIX_FMT_H264_MVC:
+   case V4L2_PIX_FMT_H263:
+   case V4L2_PIX_FMT_MPEG1:
+   case V4L2_PIX_FMT_MPEG2:
+   case V4L2_PIX_FMT_MPEG4:
+   case V4L2_PIX_FMT_XVID:
+   case V4L2_PIX_FMT_VC1_ANNEX_G:
+   case V4L2_PIX_FMT_VC1_ANNEX_L:
+   case V4L2_PIX_FMT_VP8:
+   case V4L2_PIX_FMT_VP9:
+   case V4L2_PIX_FMT_HEVC:
+   fmt->flags |= V4L2_FMT_FLAG_UNORDERED;
+   }
+}
+
 static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 {
const unsigned sz = sizeof(fmt->description);
@@ -1310,61 +1331,80 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 
if (descr)
WARN_ON(strlcpy(fmt->description, descr, sz) >= sz);
-   fmt->flags = flags;
+   fmt->flags |= flags;
 }
 
-static int v4l_enum_fmt(const struct v4l2_ioctl_ops *ops,
-   struct file *file, void *fh, void *arg)
-{
-   struct v4l2_fmtdesc *p = arg;
-   int ret = check_fmt(file, p->type);
 
-   if (ret)
-   return ret;
-   ret = -EINVAL;
+static int __vidioc_enum_fmt(const struct v4l2_ioctl_ops *ops,
+struct v4l2_fmtdesc *p,
+struct file *file, void *fh)
+{
+   int ret = 0;
 
switch (p->type) {
case V4L2_BUF_TYPE_VIDEO_CAPTURE:
if (unlikely(!ops->vidioc_enum_fmt_vid_cap))
break;
-   ret = ops->vidioc_enum_fmt_vid_cap(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_cap(file, fh, p);
break;
case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
if (unlikely(!ops->vidioc_enum_fmt_vid_cap_mplane))
break;
-   ret = ops->vidioc_enum_fmt_vid_cap_mplane(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_cap_mplane(file, fh, p);
break;
case V4L2_BUF_TYPE_VIDEO_OVERLAY:
if (unlikely(!ops->vidioc_enum_fmt_vid_overlay))
break;
-   ret = ops->vidioc_enum_fmt_vid_overlay(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_overlay(file, fh, p);
break;
case V4L2_BUF_TYPE_VIDEO_OUTPUT:
if (unlikely(!ops->vidioc_enum_fmt_vid_out))
break;
-   ret = ops->vidioc_enum_fmt_vid_out(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_out(file, fh, p);
break;
case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
if (unlikely(!ops->vidioc_enum_fmt_vid_out_mplane))
break;
-   ret = ops->vidioc_enum_fmt_vid_out_mplane(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_vid_out_mplane(file, fh, p);
break;
case V4L2_BUF_TYPE_SDR_CAPTURE:
if (unlikely(!ops->vidioc_enum_fmt_sdr_cap))
break;
-   ret = ops->vidioc_enum_fmt_sdr_cap(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_sdr_cap(file, fh, p);
break;
case V4L2_BUF_TYPE_SDR_OUTPUT:
if (unlikely(!ops->vidioc_enum_fmt_sdr_out))
break;
-   ret = ops->vidioc_enum_fmt_sdr_out(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_sdr_out(file, fh, p);
break;
case V4L2_BUF_TYPE_META_CAPTURE:
if (unlikely(!ops->vidioc_enum_fmt_meta_cap))
break;
-   ret = ops->vidioc_enum_fmt_meta_cap(file, fh, arg);
+   ret = ops->vidioc_enum_fmt_meta_cap(file, fh, p);
break;
}
+   return ret;
+}
+
+static int v4l_enum_fmt(const struct v4l2_ioctl_ops *ops,
+   struct file *file, void *fh, void *arg)
+{
+   struct v4l2_fmtdesc *p = arg;
+   int ret = check_fmt(file, p->type);
+
+   if (r

[PATCH v10 05/16] vb2: move vb2_ops functions to videobuf2-core.[ch]

[Ezequiel Garcia]

From: Gustavo Padovan 

vb2_ops_wait_prepare() and vb2_ops_wait_finish() were in the
wrong file.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/common/videobuf2/videobuf2-core.c | 14 ++
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 14 --
 include/media/videobuf2-core.h  | 18 ++
 include/media/videobuf2-v4l2.h  | 18 --
 4 files changed, 32 insertions(+), 32 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index f1feb45c1e37..61e7b6407586 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -677,6 +677,20 @@ int vb2_verify_memory_type(struct vb2_queue *q,
 }
 EXPORT_SYMBOL(vb2_verify_memory_type);
 
+/* vb2_ops helpers. Only use if vq->lock is non-NULL. */
+
+void vb2_ops_wait_prepare(struct vb2_queue *vq)
+{
+   mutex_unlock(vq->lock);
+}
+EXPORT_SYMBOL_GPL(vb2_ops_wait_prepare);
+
+void vb2_ops_wait_finish(struct vb2_queue *vq)
+{
+   mutex_lock(vq->lock);
+}
+EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
+
 int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
unsigned int *count)
 {
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c 
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 886a2d8d5c6c..64503615d00b 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -947,20 +947,6 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, 
unsigned long addr,
 EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
 #endif
 
-/* vb2_ops helpers. Only use if vq->lock is non-NULL. */
-
-void vb2_ops_wait_prepare(struct vb2_queue *vq)
-{
-   mutex_unlock(vq->lock);
-}
-EXPORT_SYMBOL_GPL(vb2_ops_wait_prepare);
-
-void vb2_ops_wait_finish(struct vb2_queue *vq)
-{
-   mutex_lock(vq->lock);
-}
-EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
-
 MODULE_DESCRIPTION("Driver helper framework for Video for Linux 2");
 MODULE_AUTHOR("Pawel Osciak , Marek Szyprowski");
 MODULE_LICENSE("GPL");
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index baa4632c7e59..137f72702101 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -403,6 +403,24 @@ struct vb2_ops {
void (*buf_queue)(struct vb2_buffer *vb);
 };
 
+/**
+ * vb2_ops_wait_prepare - helper function to lock a struct _queue
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * ..note:: only use if vq->lock is non-NULL.
+ */
+void vb2_ops_wait_prepare(struct vb2_queue *vq);
+
+/**
+ * vb2_ops_wait_finish - helper function to unlock a struct _queue
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * ..note:: only use if vq->lock is non-NULL.
+ */
+void vb2_ops_wait_finish(struct vb2_queue *vq);
+
 /**
  * struct vb2_buf_ops - driver-specific callbacks.
  *
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 3d5e2d739f05..cf83b01dc44e 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -273,22 +273,4 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, 
unsigned long addr,
unsigned long len, unsigned long pgoff, unsigned long flags);
 #endif
 
-/**
- * vb2_ops_wait_prepare - helper function to lock a struct _queue
- *
- * @vq: pointer to  vb2_queue
- *
- * ..note:: only use if vq->lock is non-NULL.
- */
-void vb2_ops_wait_prepare(struct vb2_queue *vq);
-
-/**
- * vb2_ops_wait_finish - helper function to unlock a struct _queue
- *
- * @vq: pointer to  vb2_queue
- *
- * ..note:: only use if vq->lock is non-NULL.
- */
-void vb2_ops_wait_finish(struct vb2_queue *vq);
-
 #endif /* _MEDIA_VIDEOBUF2_V4L2_H */
-- 
2.16.3

[PATCH v10 05/16] vb2: move vb2_ops functions to videobuf2-core.[ch]

[Ezequiel Garcia]

From: Gustavo Padovan 

vb2_ops_wait_prepare() and vb2_ops_wait_finish() were in the
wrong file.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/common/videobuf2/videobuf2-core.c | 14 ++
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 14 --
 include/media/videobuf2-core.h  | 18 ++
 include/media/videobuf2-v4l2.h  | 18 --
 4 files changed, 32 insertions(+), 32 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index f1feb45c1e37..61e7b6407586 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -677,6 +677,20 @@ int vb2_verify_memory_type(struct vb2_queue *q,
 }
 EXPORT_SYMBOL(vb2_verify_memory_type);
 
+/* vb2_ops helpers. Only use if vq->lock is non-NULL. */
+
+void vb2_ops_wait_prepare(struct vb2_queue *vq)
+{
+   mutex_unlock(vq->lock);
+}
+EXPORT_SYMBOL_GPL(vb2_ops_wait_prepare);
+
+void vb2_ops_wait_finish(struct vb2_queue *vq)
+{
+   mutex_lock(vq->lock);
+}
+EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
+
 int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
unsigned int *count)
 {
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c 
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 886a2d8d5c6c..64503615d00b 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -947,20 +947,6 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, 
unsigned long addr,
 EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
 #endif
 
-/* vb2_ops helpers. Only use if vq->lock is non-NULL. */
-
-void vb2_ops_wait_prepare(struct vb2_queue *vq)
-{
-   mutex_unlock(vq->lock);
-}
-EXPORT_SYMBOL_GPL(vb2_ops_wait_prepare);
-
-void vb2_ops_wait_finish(struct vb2_queue *vq)
-{
-   mutex_lock(vq->lock);
-}
-EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
-
 MODULE_DESCRIPTION("Driver helper framework for Video for Linux 2");
 MODULE_AUTHOR("Pawel Osciak , Marek Szyprowski");
 MODULE_LICENSE("GPL");
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index baa4632c7e59..137f72702101 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -403,6 +403,24 @@ struct vb2_ops {
void (*buf_queue)(struct vb2_buffer *vb);
 };
 
+/**
+ * vb2_ops_wait_prepare - helper function to lock a struct _queue
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * ..note:: only use if vq->lock is non-NULL.
+ */
+void vb2_ops_wait_prepare(struct vb2_queue *vq);
+
+/**
+ * vb2_ops_wait_finish - helper function to unlock a struct _queue
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * ..note:: only use if vq->lock is non-NULL.
+ */
+void vb2_ops_wait_finish(struct vb2_queue *vq);
+
 /**
  * struct vb2_buf_ops - driver-specific callbacks.
  *
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 3d5e2d739f05..cf83b01dc44e 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -273,22 +273,4 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, 
unsigned long addr,
unsigned long len, unsigned long pgoff, unsigned long flags);
 #endif
 
-/**
- * vb2_ops_wait_prepare - helper function to lock a struct _queue
- *
- * @vq: pointer to  vb2_queue
- *
- * ..note:: only use if vq->lock is non-NULL.
- */
-void vb2_ops_wait_prepare(struct vb2_queue *vq);
-
-/**
- * vb2_ops_wait_finish - helper function to unlock a struct _queue
- *
- * @vq: pointer to  vb2_queue
- *
- * ..note:: only use if vq->lock is non-NULL.
- */
-void vb2_ops_wait_finish(struct vb2_queue *vq);
-
 #endif /* _MEDIA_VIDEOBUF2_V4L2_H */
-- 
2.16.3

[PATCH v10 07/16] v4l: add unordered flag to format description ioctl

[Ezequiel Garcia]

From: Gustavo Padovan 

For explicit synchronization it important for userspace to know if the
format being used by the driver can deliver the buffers back to userspace
in the same order they were queued with QBUF.

Ordered streams fits nicely in a pipeline with DRM for example, where
ordered buffer are expected.

v2: Improve documentation (Hans)

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/vidioc-enum-fmt.rst | 7 +++
 include/uapi/linux/videodev2.h   | 1 +
 2 files changed, 8 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst 
b/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
index 019c513df217..df8e039b9ac2 100644
--- a/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
+++ b/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
@@ -116,6 +116,13 @@ one until ``EINVAL`` is returned.
   - This format is not native to the device but emulated through
software (usually libv4l2), where possible try to use a native
format instead for better performance.
+* - ``V4L2_FMT_FLAG_UNORDERED``
+  - 0x0004
+  - This format doesn't guarantee ordered buffer handling. I.e. the order
+   in which buffers are dequeued with
+   :ref:`VIDIOC_DQBUF ` may be different
+   from the order in which they were queued with
+   :ref:`VIDIOC_QBUF `.
 
 
 Return Value
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 600877be5c22..a8842a5ca636 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -717,6 +717,7 @@ struct v4l2_fmtdesc {
 
 #define V4L2_FMT_FLAG_COMPRESSED 0x0001
 #define V4L2_FMT_FLAG_EMULATED   0x0002
+#define V4L2_FMT_FLAG_UNORDERED  0x0004
 
/* Frame Size and frame rate enumeration */
 /*
-- 
2.16.3

[PATCH v10 07/16] v4l: add unordered flag to format description ioctl

[Ezequiel Garcia]

From: Gustavo Padovan 

For explicit synchronization it important for userspace to know if the
format being used by the driver can deliver the buffers back to userspace
in the same order they were queued with QBUF.

Ordered streams fits nicely in a pipeline with DRM for example, where
ordered buffer are expected.

v2: Improve documentation (Hans)

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/vidioc-enum-fmt.rst | 7 +++
 include/uapi/linux/videodev2.h   | 1 +
 2 files changed, 8 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst 
b/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
index 019c513df217..df8e039b9ac2 100644
--- a/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
+++ b/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
@@ -116,6 +116,13 @@ one until ``EINVAL`` is returned.
   - This format is not native to the device but emulated through
software (usually libv4l2), where possible try to use a native
format instead for better performance.
+* - ``V4L2_FMT_FLAG_UNORDERED``
+  - 0x0004
+  - This format doesn't guarantee ordered buffer handling. I.e. the order
+   in which buffers are dequeued with
+   :ref:`VIDIOC_DQBUF ` may be different
+   from the order in which they were queued with
+   :ref:`VIDIOC_QBUF `.
 
 
 Return Value
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 600877be5c22..a8842a5ca636 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -717,6 +717,7 @@ struct v4l2_fmtdesc {
 
 #define V4L2_FMT_FLAG_COMPRESSED 0x0001
 #define V4L2_FMT_FLAG_EMULATED   0x0002
+#define V4L2_FMT_FLAG_UNORDERED  0x0004
 
/* Frame Size and frame rate enumeration */
 /*
-- 
2.16.3

[PATCH v10 02/16] xilinx: regroup caps on querycap

[Ezequiel Garcia]

From: Gustavo Padovan 

To better organize the code we concentrate the setting of
V4L2_CAP_STREAMING in one place.

v2: move cap->capabilities assignment down (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/platform/xilinx/xilinx-dma.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c 
b/drivers/media/platform/xilinx/xilinx-dma.c
index 522cdfdd3345..d041f94be832 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -494,13 +494,15 @@ xvip_dma_querycap(struct file *file, void *fh, struct 
v4l2_capability *cap)
struct v4l2_fh *vfh = file->private_data;
struct xvip_dma *dma = to_xvip_dma(vfh->vdev);
 
-   cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
- | dma->xdev->v4l2_caps;
+   cap->device_caps = V4L2_CAP_STREAMING;
 
if (dma->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-   cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_CAPTURE;
else
-   cap->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_OUTPUT;
+
+   cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS
+ | dma->xdev->v4l2_caps;
 
strlcpy(cap->driver, "xilinx-vipp", sizeof(cap->driver));
strlcpy(cap->card, dma->video.name, sizeof(cap->card));
-- 
2.16.3

[PATCH v10 02/16] xilinx: regroup caps on querycap

[Ezequiel Garcia]

From: Gustavo Padovan 

To better organize the code we concentrate the setting of
V4L2_CAP_STREAMING in one place.

v2: move cap->capabilities assignment down (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/platform/xilinx/xilinx-dma.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c 
b/drivers/media/platform/xilinx/xilinx-dma.c
index 522cdfdd3345..d041f94be832 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -494,13 +494,15 @@ xvip_dma_querycap(struct file *file, void *fh, struct 
v4l2_capability *cap)
struct v4l2_fh *vfh = file->private_data;
struct xvip_dma *dma = to_xvip_dma(vfh->vdev);
 
-   cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
- | dma->xdev->v4l2_caps;
+   cap->device_caps = V4L2_CAP_STREAMING;
 
if (dma->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-   cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_CAPTURE;
else
-   cap->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_OUTPUT;
+
+   cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS
+ | dma->xdev->v4l2_caps;
 
strlcpy(cap->driver, "xilinx-vipp", sizeof(cap->driver));
strlcpy(cap->card, dma->video.name, sizeof(cap->card));
-- 
2.16.3

[PATCH v10 01/16] videobuf2: Make struct vb2_buffer refcounted

[Ezequiel Garcia]

The in-fence implementation involves having a per-buffer fence callback,
that triggers on the fence signal. The fence callback is called asynchronously
and needs a valid reference to the associated ideobuf2 buffer.

Allow this by making the vb2_buffer refcounted, so it can be passed
to other contexts.

Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/common/videobuf2/videobuf2-core.c | 27 ++---
 include/media/videobuf2-core.h  |  7 +--
 2 files changed, 29 insertions(+), 5 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index d3f7bb33a54d..f1feb45c1e37 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -190,6 +190,26 @@ module_param(debug, int, 0644);
 static void __vb2_queue_cancel(struct vb2_queue *q);
 static void __enqueue_in_driver(struct vb2_buffer *vb);
 
+static void __vb2_buffer_free(struct kref *kref)
+{
+   struct vb2_buffer *vb =
+   container_of(kref, struct vb2_buffer, refcount);
+   kfree(vb);
+}
+
+static void __vb2_buffer_put(struct vb2_buffer *vb)
+{
+   if (vb)
+   kref_put(>refcount, __vb2_buffer_free);
+}
+
+static struct vb2_buffer *__vb2_buffer_get(struct vb2_buffer *vb)
+{
+   if (vb)
+   kref_get(>refcount);
+   return vb;
+}
+
 /*
  * __vb2_buf_mem_alloc() - allocate video memory for the given buffer
  */
@@ -346,6 +366,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
break;
}
 
+   kref_init(>refcount);
vb->state = VB2_BUF_STATE_DEQUEUED;
vb->vb2_queue = q;
vb->num_planes = num_planes;
@@ -365,7 +386,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
dprintk(1, "failed allocating memory for buffer 
%d\n",
buffer);
q->bufs[vb->index] = NULL;
-   kfree(vb);
+   __vb2_buffer_put(vb);
break;
}
__setup_offsets(vb);
@@ -380,7 +401,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
buffer, vb);
__vb2_buf_mem_free(vb);
q->bufs[vb->index] = NULL;
-   kfree(vb);
+   __vb2_buffer_put(vb);
break;
}
}
@@ -520,7 +541,7 @@ static int __vb2_queue_free(struct vb2_queue *q, unsigned 
int buffers)
/* Free videobuf buffers */
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
 ++buffer) {
-   kfree(q->bufs[buffer]);
+   __vb2_buffer_put(q->bufs[buffer]);
q->bufs[buffer] = NULL;
}
 
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index f6818f732f34..baa4632c7e59 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -12,11 +12,12 @@
 #ifndef _MEDIA_VIDEOBUF2_CORE_H
 #define _MEDIA_VIDEOBUF2_CORE_H
 
+#include 
+#include 
+#include 
 #include 
 #include 
 #include 
-#include 
-#include 
 
 #define VB2_MAX_FRAME  (32)
 #define VB2_MAX_PLANES (8)
@@ -249,6 +250,7 @@ struct vb2_buffer {
 
/* private: internal use only
 *
+* refcount:refcount for this buffer
 * state:   current buffer state; do not change
 * queued_entry:entry on the queued buffers list, which holds
 *  all buffers queued from userspace
@@ -256,6 +258,7 @@ struct vb2_buffer {
 *  to be dequeued to userspace
 * vb2_plane:   per-plane information; do not change
 */
+   struct kref refcount;
enum vb2_buffer_state   state;
 
struct vb2_planeplanes[VB2_MAX_PLANES];
-- 
2.16.3

[PATCH v10 01/16] videobuf2: Make struct vb2_buffer refcounted

[Ezequiel Garcia]

The in-fence implementation involves having a per-buffer fence callback,
that triggers on the fence signal. The fence callback is called asynchronously
and needs a valid reference to the associated ideobuf2 buffer.

Allow this by making the vb2_buffer refcounted, so it can be passed
to other contexts.

Signed-off-by: Ezequiel Garcia 
---
 drivers/media/common/videobuf2/videobuf2-core.c | 27 ++---
 include/media/videobuf2-core.h  |  7 +--
 2 files changed, 29 insertions(+), 5 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index d3f7bb33a54d..f1feb45c1e37 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -190,6 +190,26 @@ module_param(debug, int, 0644);
 static void __vb2_queue_cancel(struct vb2_queue *q);
 static void __enqueue_in_driver(struct vb2_buffer *vb);
 
+static void __vb2_buffer_free(struct kref *kref)
+{
+   struct vb2_buffer *vb =
+   container_of(kref, struct vb2_buffer, refcount);
+   kfree(vb);
+}
+
+static void __vb2_buffer_put(struct vb2_buffer *vb)
+{
+   if (vb)
+   kref_put(>refcount, __vb2_buffer_free);
+}
+
+static struct vb2_buffer *__vb2_buffer_get(struct vb2_buffer *vb)
+{
+   if (vb)
+   kref_get(>refcount);
+   return vb;
+}
+
 /*
  * __vb2_buf_mem_alloc() - allocate video memory for the given buffer
  */
@@ -346,6 +366,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
break;
}
 
+   kref_init(>refcount);
vb->state = VB2_BUF_STATE_DEQUEUED;
vb->vb2_queue = q;
vb->num_planes = num_planes;
@@ -365,7 +386,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
dprintk(1, "failed allocating memory for buffer 
%d\n",
buffer);
q->bufs[vb->index] = NULL;
-   kfree(vb);
+   __vb2_buffer_put(vb);
break;
}
__setup_offsets(vb);
@@ -380,7 +401,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
buffer, vb);
__vb2_buf_mem_free(vb);
q->bufs[vb->index] = NULL;
-   kfree(vb);
+   __vb2_buffer_put(vb);
break;
}
}
@@ -520,7 +541,7 @@ static int __vb2_queue_free(struct vb2_queue *q, unsigned 
int buffers)
/* Free videobuf buffers */
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
 ++buffer) {
-   kfree(q->bufs[buffer]);
+   __vb2_buffer_put(q->bufs[buffer]);
q->bufs[buffer] = NULL;
}
 
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index f6818f732f34..baa4632c7e59 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -12,11 +12,12 @@
 #ifndef _MEDIA_VIDEOBUF2_CORE_H
 #define _MEDIA_VIDEOBUF2_CORE_H
 
+#include 
+#include 
+#include 
 #include 
 #include 
 #include 
-#include 
-#include 
 
 #define VB2_MAX_FRAME  (32)
 #define VB2_MAX_PLANES (8)
@@ -249,6 +250,7 @@ struct vb2_buffer {
 
/* private: internal use only
 *
+* refcount:refcount for this buffer
 * state:   current buffer state; do not change
 * queued_entry:entry on the queued buffers list, which holds
 *  all buffers queued from userspace
@@ -256,6 +258,7 @@ struct vb2_buffer {
 *  to be dequeued to userspace
 * vb2_plane:   per-plane information; do not change
 */
+   struct kref refcount;
enum vb2_buffer_state   state;
 
struct vb2_planeplanes[VB2_MAX_PLANES];
-- 
2.16.3

[PATCH v10 00/16] V4L2 Explicit Synchronization

[Ezequiel Garcia]

Hi all,

The most relevant change for this round is that all the work
done in the fence callback is now moved to a workqueue,
that runs with the queue lock held.
 
Although this introduces some latency, it is however needed
to take the vb2_queue mutex and safely call vb2 ops.

Given the fence callback can be called fully asynchronously,
and given it needs to be associated with a vb2_buffer,
we now need to refcount vb2_buffer. This allows to safely "attach"
the vb2_buffer to the fence callback.

To prevent annoying deadlocks, and because the fence callback
is called with the fence spinlock, it's best to avoid calling
dma_fence_put in the fence callback itself. So the fence is now
put in the DQBUF operation (or in cancel paths).

Hopefully, I took care of all the feedback provided by
Hans and Brian on v9. Please let me know if you guys catch
anything else.

Thanks!

Changes from v9:

 * Move fence callback to workqueue, and call vb2_start_streaming if needed.
 * Refcount vb2_buffer.
 * Check return of get_unused_fd.
 * Increase seqno for out-fences that reuse the context.
 * Go back to is_unordered callback.
 * Mark unordered formats in cobalt driver.
 * Improve CAP_FENCES check.
 * Minor documentation changes.

Ezequiel Garcia (1):
  videobuf2: Make struct vb2_buffer refcounted

Gustavo Padovan (15):
  xilinx: regroup caps on querycap
  hackrf: group device capabilities
  omap3isp: group device capabilities
  vb2: move vb2_ops functions to videobuf2-core.[ch]
  vb2: add is_unordered callback for drivers
  v4l: add unordered flag to format description ioctl
  v4l: mark unordered formats
  cobalt: add .is_unordered() for cobalt
  vb2: mark codec drivers as unordered
  vb2: add explicit fence user API
  vb2: add in-fence support to QBUF
  vb2: add out-fence support to QBUF
  v4l: introduce the fences capability
  v4l: Add V4L2_CAP_FENCES to drivers
  v4l: Document explicit synchronization behavior

 Documentation/media/uapi/v4l/buffer.rst|  48 ++-
 Documentation/media/uapi/v4l/vidioc-enum-fmt.rst   |   7 +
 Documentation/media/uapi/v4l/vidioc-qbuf.rst   |  53 ++-
 Documentation/media/uapi/v4l/vidioc-querybuf.rst   |  12 +-
 Documentation/media/uapi/v4l/vidioc-querycap.rst   |   3 +
 drivers/media/common/videobuf2/Kconfig |   1 +
 drivers/media/common/videobuf2/videobuf2-core.c| 376 ++---
 drivers/media/common/videobuf2/videobuf2-v4l2.c|  65 +++-
 drivers/media/dvb-core/dvb_vb2.c   |   2 +-
 drivers/media/pci/cobalt/cobalt-v4l2.c |   4 +
 drivers/media/platform/coda/coda-common.c  |   1 +
 drivers/media/platform/exynos-gsc/gsc-m2m.c|   3 +-
 drivers/media/platform/m2m-deinterlace.c   |   3 +-
 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c   |   3 +-
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c |   1 +
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |   2 +-
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c |   1 +
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |   3 +-
 drivers/media/platform/mx2_emmaprp.c   |   3 +-
 drivers/media/platform/omap3isp/ispvideo.c |  10 +-
 drivers/media/platform/qcom/venus/vdec.c   |   4 +-
 drivers/media/platform/qcom/venus/venc.c   |   4 +-
 drivers/media/platform/s5p-mfc/s5p_mfc_dec.c   |   1 +
 drivers/media/platform/s5p-mfc/s5p_mfc_enc.c   |   1 +
 drivers/media/platform/sh_veu.c|   3 +-
 drivers/media/platform/xilinx/xilinx-dma.c |  10 +-
 drivers/media/usb/hackrf/hackrf.c  |  11 +-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c  |   4 +-
 drivers/media/v4l2-core/v4l2-ioctl.c   |  98 +-
 include/media/videobuf2-core.h |  76 -
 include/media/videobuf2-v4l2.h |  18 -
 include/uapi/linux/videodev2.h |  10 +-
 32 files changed, 708 insertions(+), 133 deletions(-)

-- 
2.16.3

[PATCH v10 00/16] V4L2 Explicit Synchronization

[Ezequiel Garcia]

Hi all,

The most relevant change for this round is that all the work
done in the fence callback is now moved to a workqueue,
that runs with the queue lock held.
 
Although this introduces some latency, it is however needed
to take the vb2_queue mutex and safely call vb2 ops.

Given the fence callback can be called fully asynchronously,
and given it needs to be associated with a vb2_buffer,
we now need to refcount vb2_buffer. This allows to safely "attach"
the vb2_buffer to the fence callback.

To prevent annoying deadlocks, and because the fence callback
is called with the fence spinlock, it's best to avoid calling
dma_fence_put in the fence callback itself. So the fence is now
put in the DQBUF operation (or in cancel paths).

Hopefully, I took care of all the feedback provided by
Hans and Brian on v9. Please let me know if you guys catch
anything else.

Thanks!

Changes from v9:

 * Move fence callback to workqueue, and call vb2_start_streaming if needed.
 * Refcount vb2_buffer.
 * Check return of get_unused_fd.
 * Increase seqno for out-fences that reuse the context.
 * Go back to is_unordered callback.
 * Mark unordered formats in cobalt driver.
 * Improve CAP_FENCES check.
 * Minor documentation changes.

Ezequiel Garcia (1):
  videobuf2: Make struct vb2_buffer refcounted

Gustavo Padovan (15):
  xilinx: regroup caps on querycap
  hackrf: group device capabilities
  omap3isp: group device capabilities
  vb2: move vb2_ops functions to videobuf2-core.[ch]
  vb2: add is_unordered callback for drivers
  v4l: add unordered flag to format description ioctl
  v4l: mark unordered formats
  cobalt: add .is_unordered() for cobalt
  vb2: mark codec drivers as unordered
  vb2: add explicit fence user API
  vb2: add in-fence support to QBUF
  vb2: add out-fence support to QBUF
  v4l: introduce the fences capability
  v4l: Add V4L2_CAP_FENCES to drivers
  v4l: Document explicit synchronization behavior

 Documentation/media/uapi/v4l/buffer.rst|  48 ++-
 Documentation/media/uapi/v4l/vidioc-enum-fmt.rst   |   7 +
 Documentation/media/uapi/v4l/vidioc-qbuf.rst   |  53 ++-
 Documentation/media/uapi/v4l/vidioc-querybuf.rst   |  12 +-
 Documentation/media/uapi/v4l/vidioc-querycap.rst   |   3 +
 drivers/media/common/videobuf2/Kconfig |   1 +
 drivers/media/common/videobuf2/videobuf2-core.c| 376 ++---
 drivers/media/common/videobuf2/videobuf2-v4l2.c|  65 +++-
 drivers/media/dvb-core/dvb_vb2.c   |   2 +-
 drivers/media/pci/cobalt/cobalt-v4l2.c |   4 +
 drivers/media/platform/coda/coda-common.c  |   1 +
 drivers/media/platform/exynos-gsc/gsc-m2m.c|   3 +-
 drivers/media/platform/m2m-deinterlace.c   |   3 +-
 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c   |   3 +-
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c |   1 +
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c |   2 +-
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c |   1 +
 .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c |   3 +-
 drivers/media/platform/mx2_emmaprp.c   |   3 +-
 drivers/media/platform/omap3isp/ispvideo.c |  10 +-
 drivers/media/platform/qcom/venus/vdec.c   |   4 +-
 drivers/media/platform/qcom/venus/venc.c   |   4 +-
 drivers/media/platform/s5p-mfc/s5p_mfc_dec.c   |   1 +
 drivers/media/platform/s5p-mfc/s5p_mfc_enc.c   |   1 +
 drivers/media/platform/sh_veu.c|   3 +-
 drivers/media/platform/xilinx/xilinx-dma.c |  10 +-
 drivers/media/usb/hackrf/hackrf.c  |  11 +-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c  |   4 +-
 drivers/media/v4l2-core/v4l2-ioctl.c   |  98 +-
 include/media/videobuf2-core.h |  76 -
 include/media/videobuf2-v4l2.h |  18 -
 include/uapi/linux/videodev2.h |  10 +-
 32 files changed, 708 insertions(+), 133 deletions(-)

-- 
2.16.3

Re: [PATCH v9 10/15] vb2: add explicit fence user API

[Ezequiel Garcia]

On Wed, 2018-05-09 at 17:33 +0100, Brian Starkey wrote:
> On Wed, May 09, 2018 at 12:52:26PM -0300, Ezequiel Garcia wrote:
> > On Wed, 2018-05-09 at 11:33 +0100, Brian Starkey wrote:
> > > Hi Ezequiel,
> > > 
> > > On Fri, May 04, 2018 at 05:06:07PM -0300, Ezequiel Garcia wrote:
> > > > From: Gustavo Padovan <gustavo.pado...@collabora.com>
> > > > 
> > > > Turn the reserved2 field into fence_fd that we will use to send
> > > > an in-fence to the kernel or return an out-fence from the kernel to
> > > > userspace.
> > > > 
> > > > Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> > > > when sending an in-fence to the kernel to be waited on, and
> > > > V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
> > > > 
> > > > v7: minor fixes on the Documentation (Hans Verkuil)
> > > > 
> > > > v6: big improvement on doc (Hans Verkuil)
> > > > 
> > > > v5: - keep using reserved2 field for cpia2
> > > >- set fence_fd to 0 for now, for compat with userspace(Mauro)
> > > > 
> > > > v4: make it a union with reserved2 and fence_fd (Hans Verkuil)
> > > > 
> > > > v3: make the out_fence refer to the current buffer (Hans Verkuil)
> > > > 
> > > > v2: add documentation
> > > > 
> > > > Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
> > > > ---
> > > > Documentation/media/uapi/v4l/buffer.rst | 45 
> > > > +++--
> > > > drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
> > > > drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
> > > > include/uapi/linux/videodev2.h  |  8 -
> > > > 4 files changed, 52 insertions(+), 7 deletions(-)
> > > > 
> > > > diff --git a/Documentation/media/uapi/v4l/buffer.rst 
> > > > b/Documentation/media/uapi/v4l/buffer.rst
> > > > index e2c85ddc990b..be9719cf5745 100644
> > > > --- a/Documentation/media/uapi/v4l/buffer.rst
> > > > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > > > @@ -301,10 +301,22 @@ struct v4l2_buffer
> > > > elements in the ``planes`` array. The driver will fill in the
> > > > actual number of valid elements in that array.
> > > > * - __u32
> > > > -  - ``reserved2``
> > > > +  - ``fence_fd``
> > > >   -
> > > > -  - A place holder for future extensions. Drivers and applications
> > > > -   must set this to 0.
> > > > +  - Used to communicate a fence file descriptors from userspace to 
> > > > kernel
> > > > +   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
> > > > +   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` 
> > > > flag must
> > > > +   be used and this field set to the fence file descriptor of the 
> > > > in-fence.
> > > > +   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
> > > > +
> > > > +To get an out-fence back from V4L2 the 
> > > > ``V4L2_BUF_FLAG_OUT_FENCE``
> > > > +   must be set, the kernel will return the out-fence file 
> > > > descriptor in
> > > > +   this field. If it fails to create the out-fence ``VIDIOC_QBUF` 
> > > > returns
> > > > +an error.
> > > > +
> > > > +   For all other ioctls V4L2 sets this field to -1 if
> > > > +   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` 
> > > > are set,
> > > > +   otherwise this field is set to 0 for backward compatibility.
> > > > * - __u32
> > > >   - ``reserved``
> > > >   -
> > > > @@ -648,6 +660,33 @@ Buffer Flags
> > > >   - Start Of Exposure. The buffer timestamp has been taken when the
> > > > exposure of the frame has begun. This is only valid for the
> > > > ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> > > > +* .. _`V4L2-BUF-FLAG-IN-FENCE`:
> > > > +
> > > > +  - ``V4L2_BUF_FLAG_IN_FENCE``
> > > > +  - 0x0020
> > > > +  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` 
> > > > field. The
> > > > +   buffer won't be queued to the driver until the fence signals. 
&

Re: [PATCH v9 10/15] vb2: add explicit fence user API

[Ezequiel Garcia]

On Wed, 2018-05-09 at 17:33 +0100, Brian Starkey wrote:
> On Wed, May 09, 2018 at 12:52:26PM -0300, Ezequiel Garcia wrote:
> > On Wed, 2018-05-09 at 11:33 +0100, Brian Starkey wrote:
> > > Hi Ezequiel,
> > > 
> > > On Fri, May 04, 2018 at 05:06:07PM -0300, Ezequiel Garcia wrote:
> > > > From: Gustavo Padovan 
> > > > 
> > > > Turn the reserved2 field into fence_fd that we will use to send
> > > > an in-fence to the kernel or return an out-fence from the kernel to
> > > > userspace.
> > > > 
> > > > Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> > > > when sending an in-fence to the kernel to be waited on, and
> > > > V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
> > > > 
> > > > v7: minor fixes on the Documentation (Hans Verkuil)
> > > > 
> > > > v6: big improvement on doc (Hans Verkuil)
> > > > 
> > > > v5: - keep using reserved2 field for cpia2
> > > >- set fence_fd to 0 for now, for compat with userspace(Mauro)
> > > > 
> > > > v4: make it a union with reserved2 and fence_fd (Hans Verkuil)
> > > > 
> > > > v3: make the out_fence refer to the current buffer (Hans Verkuil)
> > > > 
> > > > v2: add documentation
> > > > 
> > > > Signed-off-by: Gustavo Padovan 
> > > > ---
> > > > Documentation/media/uapi/v4l/buffer.rst | 45 
> > > > +++--
> > > > drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
> > > > drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
> > > > include/uapi/linux/videodev2.h  |  8 -
> > > > 4 files changed, 52 insertions(+), 7 deletions(-)
> > > > 
> > > > diff --git a/Documentation/media/uapi/v4l/buffer.rst 
> > > > b/Documentation/media/uapi/v4l/buffer.rst
> > > > index e2c85ddc990b..be9719cf5745 100644
> > > > --- a/Documentation/media/uapi/v4l/buffer.rst
> > > > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > > > @@ -301,10 +301,22 @@ struct v4l2_buffer
> > > > elements in the ``planes`` array. The driver will fill in the
> > > > actual number of valid elements in that array.
> > > > * - __u32
> > > > -  - ``reserved2``
> > > > +  - ``fence_fd``
> > > >   -
> > > > -  - A place holder for future extensions. Drivers and applications
> > > > -   must set this to 0.
> > > > +  - Used to communicate a fence file descriptors from userspace to 
> > > > kernel
> > > > +   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
> > > > +   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` 
> > > > flag must
> > > > +   be used and this field set to the fence file descriptor of the 
> > > > in-fence.
> > > > +   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
> > > > +
> > > > +To get an out-fence back from V4L2 the 
> > > > ``V4L2_BUF_FLAG_OUT_FENCE``
> > > > +   must be set, the kernel will return the out-fence file 
> > > > descriptor in
> > > > +   this field. If it fails to create the out-fence ``VIDIOC_QBUF` 
> > > > returns
> > > > +an error.
> > > > +
> > > > +   For all other ioctls V4L2 sets this field to -1 if
> > > > +   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` 
> > > > are set,
> > > > +   otherwise this field is set to 0 for backward compatibility.
> > > > * - __u32
> > > >   - ``reserved``
> > > >   -
> > > > @@ -648,6 +660,33 @@ Buffer Flags
> > > >   - Start Of Exposure. The buffer timestamp has been taken when the
> > > > exposure of the frame has begun. This is only valid for the
> > > > ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> > > > +* .. _`V4L2-BUF-FLAG-IN-FENCE`:
> > > > +
> > > > +  - ``V4L2_BUF_FLAG_IN_FENCE``
> > > > +  - 0x0020
> > > > +  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` 
> > > > field. The
> > > > +   buffer won't be queued to the driver until the fence signals. 
> > > > The order
> > > > +   in which b

Re: [PATCH v9 10/15] vb2: add explicit fence user API

[Ezequiel Garcia]

On Wed, 2018-05-09 at 17:33 +0100, Brian Starkey wrote:
> On Wed, May 09, 2018 at 12:52:26PM -0300, Ezequiel Garcia wrote:
> > On Wed, 2018-05-09 at 11:33 +0100, Brian Starkey wrote:
> > > Hi Ezequiel,
> > > 
> > > On Fri, May 04, 2018 at 05:06:07PM -0300, Ezequiel Garcia wrote:
> > > > From: Gustavo Padovan <gustavo.pado...@collabora.com>
> > > > 
> > > > Turn the reserved2 field into fence_fd that we will use to send
> > > > an in-fence to the kernel or return an out-fence from the kernel to
> > > > userspace.
> > > > 
> > > > Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> > > > when sending an in-fence to the kernel to be waited on, and
> > > > V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
> > > > 
> > > > v7: minor fixes on the Documentation (Hans Verkuil)
> > > > 
> > > > v6: big improvement on doc (Hans Verkuil)
> > > > 
> > > > v5: - keep using reserved2 field for cpia2
> > > >- set fence_fd to 0 for now, for compat with userspace(Mauro)
> > > > 
> > > > v4: make it a union with reserved2 and fence_fd (Hans Verkuil)
> > > > 
> > > > v3: make the out_fence refer to the current buffer (Hans Verkuil)
> > > > 
> > > > v2: add documentation
> > > > 
> > > > Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
> > > > ---
> > > > Documentation/media/uapi/v4l/buffer.rst | 45 
> > > > +++--
> > > > drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
> > > > drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
> > > > include/uapi/linux/videodev2.h  |  8 -
> > > > 4 files changed, 52 insertions(+), 7 deletions(-)
> > > > 
> > > > diff --git a/Documentation/media/uapi/v4l/buffer.rst 
> > > > b/Documentation/media/uapi/v4l/buffer.rst
> > > > index e2c85ddc990b..be9719cf5745 100644
> > > > --- a/Documentation/media/uapi/v4l/buffer.rst
> > > > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > > > @@ -301,10 +301,22 @@ struct v4l2_buffer
> > > > elements in the ``planes`` array. The driver will fill in the
> > > > actual number of valid elements in that array.
> > > > * - __u32
> > > > -  - ``reserved2``
> > > > +  - ``fence_fd``
> > > >   -
> > > > -  - A place holder for future extensions. Drivers and applications
> > > > -   must set this to 0.
> > > > +  - Used to communicate a fence file descriptors from userspace to 
> > > > kernel
> > > > +   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
> > > > +   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` 
> > > > flag must
> > > > +   be used and this field set to the fence file descriptor of the 
> > > > in-fence.
> > > > +   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
> > > > +
> > > > +To get an out-fence back from V4L2 the 
> > > > ``V4L2_BUF_FLAG_OUT_FENCE``
> > > > +   must be set, the kernel will return the out-fence file 
> > > > descriptor in
> > > > +   this field. If it fails to create the out-fence ``VIDIOC_QBUF` 
> > > > returns
> > > > +an error.
> > > > +
> > > > +   For all other ioctls V4L2 sets this field to -1 if
> > > > +   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` 
> > > > are set,
> > > > +   otherwise this field is set to 0 for backward compatibility.
> > > > * - __u32
> > > >   - ``reserved``
> > > >   -
> > > > @@ -648,6 +660,33 @@ Buffer Flags
> > > >   - Start Of Exposure. The buffer timestamp has been taken when the
> > > > exposure of the frame has begun. This is only valid for the
> > > > ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> > > > +* .. _`V4L2-BUF-FLAG-IN-FENCE`:
> > > > +
> > > > +  - ``V4L2_BUF_FLAG_IN_FENCE``
> > > > +  - 0x0020
> > > > +  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` 
> > > > field. The
> > > > +   buffer won't be queued to the driver until the fence signals. 
&

Re: [PATCH v9 10/15] vb2: add explicit fence user API

[Ezequiel Garcia]

On Wed, 2018-05-09 at 17:33 +0100, Brian Starkey wrote:
> On Wed, May 09, 2018 at 12:52:26PM -0300, Ezequiel Garcia wrote:
> > On Wed, 2018-05-09 at 11:33 +0100, Brian Starkey wrote:
> > > Hi Ezequiel,
> > > 
> > > On Fri, May 04, 2018 at 05:06:07PM -0300, Ezequiel Garcia wrote:
> > > > From: Gustavo Padovan 
> > > > 
> > > > Turn the reserved2 field into fence_fd that we will use to send
> > > > an in-fence to the kernel or return an out-fence from the kernel to
> > > > userspace.
> > > > 
> > > > Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> > > > when sending an in-fence to the kernel to be waited on, and
> > > > V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
> > > > 
> > > > v7: minor fixes on the Documentation (Hans Verkuil)
> > > > 
> > > > v6: big improvement on doc (Hans Verkuil)
> > > > 
> > > > v5: - keep using reserved2 field for cpia2
> > > >- set fence_fd to 0 for now, for compat with userspace(Mauro)
> > > > 
> > > > v4: make it a union with reserved2 and fence_fd (Hans Verkuil)
> > > > 
> > > > v3: make the out_fence refer to the current buffer (Hans Verkuil)
> > > > 
> > > > v2: add documentation
> > > > 
> > > > Signed-off-by: Gustavo Padovan 
> > > > ---
> > > > Documentation/media/uapi/v4l/buffer.rst | 45 
> > > > +++--
> > > > drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
> > > > drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
> > > > include/uapi/linux/videodev2.h  |  8 -
> > > > 4 files changed, 52 insertions(+), 7 deletions(-)
> > > > 
> > > > diff --git a/Documentation/media/uapi/v4l/buffer.rst 
> > > > b/Documentation/media/uapi/v4l/buffer.rst
> > > > index e2c85ddc990b..be9719cf5745 100644
> > > > --- a/Documentation/media/uapi/v4l/buffer.rst
> > > > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > > > @@ -301,10 +301,22 @@ struct v4l2_buffer
> > > > elements in the ``planes`` array. The driver will fill in the
> > > > actual number of valid elements in that array.
> > > > * - __u32
> > > > -  - ``reserved2``
> > > > +  - ``fence_fd``
> > > >   -
> > > > -  - A place holder for future extensions. Drivers and applications
> > > > -   must set this to 0.
> > > > +  - Used to communicate a fence file descriptors from userspace to 
> > > > kernel
> > > > +   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
> > > > +   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` 
> > > > flag must
> > > > +   be used and this field set to the fence file descriptor of the 
> > > > in-fence.
> > > > +   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
> > > > +
> > > > +To get an out-fence back from V4L2 the 
> > > > ``V4L2_BUF_FLAG_OUT_FENCE``
> > > > +   must be set, the kernel will return the out-fence file 
> > > > descriptor in
> > > > +   this field. If it fails to create the out-fence ``VIDIOC_QBUF` 
> > > > returns
> > > > +an error.
> > > > +
> > > > +   For all other ioctls V4L2 sets this field to -1 if
> > > > +   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` 
> > > > are set,
> > > > +   otherwise this field is set to 0 for backward compatibility.
> > > > * - __u32
> > > >   - ``reserved``
> > > >   -
> > > > @@ -648,6 +660,33 @@ Buffer Flags
> > > >   - Start Of Exposure. The buffer timestamp has been taken when the
> > > > exposure of the frame has begun. This is only valid for the
> > > > ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> > > > +* .. _`V4L2-BUF-FLAG-IN-FENCE`:
> > > > +
> > > > +  - ``V4L2_BUF_FLAG_IN_FENCE``
> > > > +  - 0x0020
> > > > +  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` 
> > > > field. The
> > > > +   buffer won't be queued to the driver until the fence signals. 
> > > > The order
> > > > +   in which b

Re: [PATCH v9 11/15] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On Wed, 2018-05-09 at 11:36 +0100, Brian Starkey wrote:

[..]
> > @@ -203,9 +215,14 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, 
> > void *pb)
> > b->timestamp = ns_to_timeval(vb->timestamp);
> > b->timecode = vbuf->timecode;
> > b->sequence = vbuf->sequence;
> > -   b->fence_fd = 0;
> > b->reserved = 0;
> > 
> > +   b->fence_fd = 0;
> 
> I didn't understand why we're returning 0 instead of -1. Actually the
> doc in patch 10 seems to say it will be -1 or 0 depending on whether
> we set one of the fence flags? I'm not sure:
> 
> For all other ioctls V4L2 sets this field to -1 if
> ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
> otherwise this field is set to 0 for backward compatibility.
> 

Well, I think that for backwards compatibility (userspace not knowing
about fence_fd field), we should return 0, unless the flags are explicitly
set.

That is what the doc says and it sounds sane.

The bits are implemented in patch 12, but as I mentioned in my reply to
patch 10, I will move it to patch 10, for consistency.

Thanks,
Eze

Re: [PATCH v9 11/15] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On Wed, 2018-05-09 at 11:36 +0100, Brian Starkey wrote:

[..]
> > @@ -203,9 +215,14 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, 
> > void *pb)
> > b->timestamp = ns_to_timeval(vb->timestamp);
> > b->timecode = vbuf->timecode;
> > b->sequence = vbuf->sequence;
> > -   b->fence_fd = 0;
> > b->reserved = 0;
> > 
> > +   b->fence_fd = 0;
> 
> I didn't understand why we're returning 0 instead of -1. Actually the
> doc in patch 10 seems to say it will be -1 or 0 depending on whether
> we set one of the fence flags? I'm not sure:
> 
> For all other ioctls V4L2 sets this field to -1 if
> ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
> otherwise this field is set to 0 for backward compatibility.
> 

Well, I think that for backwards compatibility (userspace not knowing
about fence_fd field), we should return 0, unless the flags are explicitly
set.

That is what the doc says and it sounds sane.

The bits are implemented in patch 12, but as I mentioned in my reply to
patch 10, I will move it to patch 10, for consistency.

Thanks,
Eze

Re: [PATCH v9 10/15] vb2: add explicit fence user API

[Ezequiel Garcia]

On Wed, 2018-05-09 at 11:33 +0100, Brian Starkey wrote:
> Hi Ezequiel,
> 
> On Fri, May 04, 2018 at 05:06:07PM -0300, Ezequiel Garcia wrote:
> > From: Gustavo Padovan <gustavo.pado...@collabora.com>
> > 
> > Turn the reserved2 field into fence_fd that we will use to send
> > an in-fence to the kernel or return an out-fence from the kernel to
> > userspace.
> > 
> > Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> > when sending an in-fence to the kernel to be waited on, and
> > V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
> > 
> > v7: minor fixes on the Documentation (Hans Verkuil)
> > 
> > v6: big improvement on doc (Hans Verkuil)
> > 
> > v5: - keep using reserved2 field for cpia2
> >- set fence_fd to 0 for now, for compat with userspace(Mauro)
> > 
> > v4: make it a union with reserved2 and fence_fd (Hans Verkuil)
> > 
> > v3: make the out_fence refer to the current buffer (Hans Verkuil)
> > 
> > v2: add documentation
> > 
> > Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
> > ---
> > Documentation/media/uapi/v4l/buffer.rst | 45 
> > +++--
> > drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
> > drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
> > include/uapi/linux/videodev2.h  |  8 -
> > 4 files changed, 52 insertions(+), 7 deletions(-)
> > 
> > diff --git a/Documentation/media/uapi/v4l/buffer.rst 
> > b/Documentation/media/uapi/v4l/buffer.rst
> > index e2c85ddc990b..be9719cf5745 100644
> > --- a/Documentation/media/uapi/v4l/buffer.rst
> > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > @@ -301,10 +301,22 @@ struct v4l2_buffer
> > elements in the ``planes`` array. The driver will fill in the
> > actual number of valid elements in that array.
> > * - __u32
> > -  - ``reserved2``
> > +  - ``fence_fd``
> >   -
> > -  - A place holder for future extensions. Drivers and applications
> > -   must set this to 0.
> > +  - Used to communicate a fence file descriptors from userspace to 
> > kernel
> > +   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
> > +   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag 
> > must
> > +   be used and this field set to the fence file descriptor of the in-fence.
> > +   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
> > +
> > +To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
> > +   must be set, the kernel will return the out-fence file descriptor in
> > +   this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
> > +an error.
> > +
> > +   For all other ioctls V4L2 sets this field to -1 if
> > +   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
> > +   otherwise this field is set to 0 for backward compatibility.
> > * - __u32
> >   - ``reserved``
> >   -
> > @@ -648,6 +660,33 @@ Buffer Flags
> >   - Start Of Exposure. The buffer timestamp has been taken when the
> > exposure of the frame has begun. This is only valid for the
> > ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> > +* .. _`V4L2-BUF-FLAG-IN-FENCE`:
> > +
> > +  - ``V4L2_BUF_FLAG_IN_FENCE``
> > +  - 0x0020
> > +  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
> > +   buffer won't be queued to the driver until the fence signals. The order
> > +   in which buffers are queued is guaranteed to be preserved, so any
> > +   buffers queued after this buffer will also be blocked until this fence
> > +   signals. This flag must be set before calling ``VIDIOC_QBUF``. For
> > +   other ioctls the driver just reports the value of the flag.
> > +
> > +If the fence signals the flag is cleared and not reported anymore.
> > +   If the fence is not valid ``VIDIOC_QBUF`` returns an error.
> > +
> > +
> > +* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
> > +
> > +  - ``V4L2_BUF_FLAG_OUT_FENCE``
> > +  - 0x0040
> > +  - Request for a fence to be attached to the buffer. The driver will 
> > fill
> > +   in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
> > +   ` returns. This flag must be set before calling
> > +   ``VIDIOC_QBUF``. For other ioctls the driver just reports the value of
> > +   the flag.
> > +
> > +If the creation of the out-fence fails ``VIDIOC_QBU

Re: [PATCH v9 10/15] vb2: add explicit fence user API

[Ezequiel Garcia]

On Wed, 2018-05-09 at 11:33 +0100, Brian Starkey wrote:
> Hi Ezequiel,
> 
> On Fri, May 04, 2018 at 05:06:07PM -0300, Ezequiel Garcia wrote:
> > From: Gustavo Padovan 
> > 
> > Turn the reserved2 field into fence_fd that we will use to send
> > an in-fence to the kernel or return an out-fence from the kernel to
> > userspace.
> > 
> > Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> > when sending an in-fence to the kernel to be waited on, and
> > V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
> > 
> > v7: minor fixes on the Documentation (Hans Verkuil)
> > 
> > v6: big improvement on doc (Hans Verkuil)
> > 
> > v5: - keep using reserved2 field for cpia2
> >- set fence_fd to 0 for now, for compat with userspace(Mauro)
> > 
> > v4: make it a union with reserved2 and fence_fd (Hans Verkuil)
> > 
> > v3: make the out_fence refer to the current buffer (Hans Verkuil)
> > 
> > v2: add documentation
> > 
> > Signed-off-by: Gustavo Padovan 
> > ---
> > Documentation/media/uapi/v4l/buffer.rst | 45 
> > +++--
> > drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
> > drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
> > include/uapi/linux/videodev2.h  |  8 -
> > 4 files changed, 52 insertions(+), 7 deletions(-)
> > 
> > diff --git a/Documentation/media/uapi/v4l/buffer.rst 
> > b/Documentation/media/uapi/v4l/buffer.rst
> > index e2c85ddc990b..be9719cf5745 100644
> > --- a/Documentation/media/uapi/v4l/buffer.rst
> > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > @@ -301,10 +301,22 @@ struct v4l2_buffer
> > elements in the ``planes`` array. The driver will fill in the
> > actual number of valid elements in that array.
> > * - __u32
> > -  - ``reserved2``
> > +  - ``fence_fd``
> >   -
> > -  - A place holder for future extensions. Drivers and applications
> > -   must set this to 0.
> > +  - Used to communicate a fence file descriptors from userspace to 
> > kernel
> > +   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
> > +   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag 
> > must
> > +   be used and this field set to the fence file descriptor of the in-fence.
> > +   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
> > +
> > +To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
> > +   must be set, the kernel will return the out-fence file descriptor in
> > +   this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
> > +an error.
> > +
> > +   For all other ioctls V4L2 sets this field to -1 if
> > +   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
> > +   otherwise this field is set to 0 for backward compatibility.
> > * - __u32
> >   - ``reserved``
> >   -
> > @@ -648,6 +660,33 @@ Buffer Flags
> >   - Start Of Exposure. The buffer timestamp has been taken when the
> > exposure of the frame has begun. This is only valid for the
> > ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> > +* .. _`V4L2-BUF-FLAG-IN-FENCE`:
> > +
> > +  - ``V4L2_BUF_FLAG_IN_FENCE``
> > +  - 0x0020
> > +  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
> > +   buffer won't be queued to the driver until the fence signals. The order
> > +   in which buffers are queued is guaranteed to be preserved, so any
> > +   buffers queued after this buffer will also be blocked until this fence
> > +   signals. This flag must be set before calling ``VIDIOC_QBUF``. For
> > +   other ioctls the driver just reports the value of the flag.
> > +
> > +If the fence signals the flag is cleared and not reported anymore.
> > +   If the fence is not valid ``VIDIOC_QBUF`` returns an error.
> > +
> > +
> > +* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
> > +
> > +  - ``V4L2_BUF_FLAG_OUT_FENCE``
> > +  - 0x0040
> > +  - Request for a fence to be attached to the buffer. The driver will 
> > fill
> > +   in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
> > +   ` returns. This flag must be set before calling
> > +   ``VIDIOC_QBUF``. For other ioctls the driver just reports the value of
> > +   the flag.
> > +
> > +If the creation of the out-fence fails ``VIDIOC_QBUF`` returns an
> > +   error.
> > 
> 
> I commented similarly 

Re: [PATCH v9 11/15] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On Mon, 2018-05-07 at 14:07 +0200, Hans Verkuil wrote:
> On 04/05/18 22:06, Ezequiel Garcia wrote:
> > From: Gustavo Padovan <gustavo.pado...@collabora.com>
> > 
> > Receive in-fence from userspace and add support for waiting on them
> > before queueing the buffer to the driver. Buffers can't be queued to the
> > driver before its fences signal. And a buffer can't be queued to the driver
> > out of the order they were queued from userspace. That means that even if
> > its fence signals it must wait for all other buffers, ahead of it in the 
> > queue,
> > to signal first.
> > 
> > If the fence for some buffer fails we do not queue it to the driver,
> > instead we mark it as error and wait until the previous buffer is done
> > to notify userspace of the error. We wait here to deliver the buffers back
> > to userspace in order.
> > 
> > v12: fixed dvb_vb2.c usage of vb2_core_qbuf.
> > 
> > v11: - minor doc/comments fixes (Hans Verkuil)
> >  - reviewed the in-fence path at __fill_v4l2_buffer()
> > 
> > v10: - rename fence to in_fence in many places
> >  - handle fences signalling with error better (Hans Verkuil)
> > 
> > v9: - improve comments and docs (Hans Verkuil)
> > - fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
> > - move in-fences code that was in the out-fences patch here (Alex)
> > 
> > v8: - improve comments about fences with errors
> > 
> > v7: - get rid of the fence array stuff for ordering and just use
> >   get_num_buffers_ready() (Hans)
> > - fix issue of queuing the buffer twice (Hans)
> > - avoid the dma_fence_wait() in core_qbuf() (Alex)
> > - merge preparation commit in
> > 
> > v6: - With fences always keep the order userspace queues the buffers.
> > - Protect in_fence manipulation with a lock (Brian Starkey)
> > - check if fences have the same context before adding a fence array
> > - Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
> > - Clean up fence if __set_in_fence() fails (Brian Starkey)
> > - treat -EINVAL from dma_fence_add_callback() (Brian Starkey)
> > 
> > v5: - use fence_array to keep buffers ordered in vb2 core when
> >   needed (Brian Starkey)
> > - keep backward compat on the reserved2 field (Brian Starkey)
> > - protect fence callback removal with lock (Brian Starkey)
> > 
> > v4: - Add a comment about dma_fence_add_callback() not returning a
> >   error (Hans)
> > - Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
> > - select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
> > - Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
> > - Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
> > - Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
> >   vb2_start_streaming() (Hans)
> > - set IN_FENCE flags on __fill_v4l2_buffer (Hans)
> > - Queue buffers to the driver as soon as they are ready (Hans)
> > - call fill_user_buffer() after queuing the buffer (Hans)
> > - add err: label to clean up fence
> > - add dma_fence_wait() before calling vb2_start_streaming()
> > 
> > v3: - document fence parameter
> > - remove ternary if at vb2_qbuf() return (Mauro)
> >     - do not change if conditions behaviour (Mauro)
> > 
> > v2: - fix vb2_queue_or_prepare_buf() ret check
> > - remove check for VB2_MEMORY_DMABUF only (Javier)
> > - check num of ready buffers to start streaming
> > - when queueing, start from the first ready buffer
> > - handle queue cancel
> > 
> > Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
> > Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
> > ---
> >  drivers/media/common/videobuf2/videobuf2-core.c | 197 
> > 
> >  drivers/media/common/videobuf2/videobuf2-v4l2.c |  34 +++-
> >  drivers/media/dvb-core/dvb_vb2.c|   2 +-
> >  drivers/media/v4l2-core/Kconfig |  33 
> >  include/media/videobuf2-core.h  |  14 +-
> >  5 files changed, 249 insertions(+), 31 deletions(-)
> > 
> > diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
> > b/drivers/media/common/videobuf2/videobuf2-core.c
> > index 6b8e083893ad..996b99497a98 100644
> > --- a/drivers/media/common/videobuf2/videobuf2-core.c
> > +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> > @@ -352,6 +352,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, 

Re: [PATCH v9 11/15] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On Mon, 2018-05-07 at 14:07 +0200, Hans Verkuil wrote:
> On 04/05/18 22:06, Ezequiel Garcia wrote:
> > From: Gustavo Padovan 
> > 
> > Receive in-fence from userspace and add support for waiting on them
> > before queueing the buffer to the driver. Buffers can't be queued to the
> > driver before its fences signal. And a buffer can't be queued to the driver
> > out of the order they were queued from userspace. That means that even if
> > its fence signals it must wait for all other buffers, ahead of it in the 
> > queue,
> > to signal first.
> > 
> > If the fence for some buffer fails we do not queue it to the driver,
> > instead we mark it as error and wait until the previous buffer is done
> > to notify userspace of the error. We wait here to deliver the buffers back
> > to userspace in order.
> > 
> > v12: fixed dvb_vb2.c usage of vb2_core_qbuf.
> > 
> > v11: - minor doc/comments fixes (Hans Verkuil)
> >  - reviewed the in-fence path at __fill_v4l2_buffer()
> > 
> > v10: - rename fence to in_fence in many places
> >  - handle fences signalling with error better (Hans Verkuil)
> > 
> > v9: - improve comments and docs (Hans Verkuil)
> > - fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
> > - move in-fences code that was in the out-fences patch here (Alex)
> > 
> > v8: - improve comments about fences with errors
> > 
> > v7: - get rid of the fence array stuff for ordering and just use
> >   get_num_buffers_ready() (Hans)
> > - fix issue of queuing the buffer twice (Hans)
> > - avoid the dma_fence_wait() in core_qbuf() (Alex)
> > - merge preparation commit in
> > 
> > v6: - With fences always keep the order userspace queues the buffers.
> > - Protect in_fence manipulation with a lock (Brian Starkey)
> > - check if fences have the same context before adding a fence array
> > - Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
> > - Clean up fence if __set_in_fence() fails (Brian Starkey)
> > - treat -EINVAL from dma_fence_add_callback() (Brian Starkey)
> > 
> > v5: - use fence_array to keep buffers ordered in vb2 core when
> >   needed (Brian Starkey)
> > - keep backward compat on the reserved2 field (Brian Starkey)
> > - protect fence callback removal with lock (Brian Starkey)
> > 
> > v4: - Add a comment about dma_fence_add_callback() not returning a
> >   error (Hans)
> > - Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
> > - select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
> > - Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
> > - Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
> > - Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
> >   vb2_start_streaming() (Hans)
> > - set IN_FENCE flags on __fill_v4l2_buffer (Hans)
> > - Queue buffers to the driver as soon as they are ready (Hans)
> > - call fill_user_buffer() after queuing the buffer (Hans)
> > - add err: label to clean up fence
> > - add dma_fence_wait() before calling vb2_start_streaming()
> > 
> > v3: - document fence parameter
> > - remove ternary if at vb2_qbuf() return (Mauro)
> > - do not change if conditions behaviour (Mauro)
> > 
> > v2: - fix vb2_queue_or_prepare_buf() ret check
> > - remove check for VB2_MEMORY_DMABUF only (Javier)
> > - check num of ready buffers to start streaming
> > - when queueing, start from the first ready buffer
> > - handle queue cancel
> > 
> > Signed-off-by: Gustavo Padovan 
> > Signed-off-by: Ezequiel Garcia 
> > ---
> >  drivers/media/common/videobuf2/videobuf2-core.c | 197 
> > 
> >  drivers/media/common/videobuf2/videobuf2-v4l2.c |  34 +++-
> >  drivers/media/dvb-core/dvb_vb2.c|   2 +-
> >  drivers/media/v4l2-core/Kconfig |  33 
> >  include/media/videobuf2-core.h  |  14 +-
> >  5 files changed, 249 insertions(+), 31 deletions(-)
> > 
> > diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
> > b/drivers/media/common/videobuf2/videobuf2-core.c
> > index 6b8e083893ad..996b99497a98 100644
> > --- a/drivers/media/common/videobuf2/videobuf2-core.c
> > +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> > @@ -352,6 +352,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
> > vb2_memory memory,
> > vb->index = q->num_buffers + buffer;
>

[PATCH v9 04/15] vb2: move vb2_ops functions to videobuf2-core.[ch]

[Ezequiel Garcia]

From: Gustavo Padovan 

vb2_ops_wait_prepare() and vb2_ops_wait_finish() were in the
wrong file.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/common/videobuf2/videobuf2-core.c | 14 ++
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 14 --
 include/media/videobuf2-core.h  | 18 ++
 include/media/videobuf2-v4l2.h  | 18 --
 4 files changed, 32 insertions(+), 32 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index d3f7bb33a54d..6b8e083893ad 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -656,6 +656,20 @@ int vb2_verify_memory_type(struct vb2_queue *q,
 }
 EXPORT_SYMBOL(vb2_verify_memory_type);
 
+/* vb2_ops helpers. Only use if vq->lock is non-NULL. */
+
+void vb2_ops_wait_prepare(struct vb2_queue *vq)
+{
+   mutex_unlock(vq->lock);
+}
+EXPORT_SYMBOL_GPL(vb2_ops_wait_prepare);
+
+void vb2_ops_wait_finish(struct vb2_queue *vq)
+{
+   mutex_lock(vq->lock);
+}
+EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
+
 int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
unsigned int *count)
 {
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c 
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 886a2d8d5c6c..64503615d00b 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -947,20 +947,6 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, 
unsigned long addr,
 EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
 #endif
 
-/* vb2_ops helpers. Only use if vq->lock is non-NULL. */
-
-void vb2_ops_wait_prepare(struct vb2_queue *vq)
-{
-   mutex_unlock(vq->lock);
-}
-EXPORT_SYMBOL_GPL(vb2_ops_wait_prepare);
-
-void vb2_ops_wait_finish(struct vb2_queue *vq)
-{
-   mutex_lock(vq->lock);
-}
-EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
-
 MODULE_DESCRIPTION("Driver helper framework for Video for Linux 2");
 MODULE_AUTHOR("Pawel Osciak , Marek Szyprowski");
 MODULE_LICENSE("GPL");
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index f6818f732f34..f9633de0386c 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -400,6 +400,24 @@ struct vb2_ops {
void (*buf_queue)(struct vb2_buffer *vb);
 };
 
+/**
+ * vb2_ops_wait_prepare - helper function to lock a struct _queue
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * ..note:: only use if vq->lock is non-NULL.
+ */
+void vb2_ops_wait_prepare(struct vb2_queue *vq);
+
+/**
+ * vb2_ops_wait_finish - helper function to unlock a struct _queue
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * ..note:: only use if vq->lock is non-NULL.
+ */
+void vb2_ops_wait_finish(struct vb2_queue *vq);
+
 /**
  * struct vb2_buf_ops - driver-specific callbacks.
  *
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 3d5e2d739f05..cf83b01dc44e 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -273,22 +273,4 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, 
unsigned long addr,
unsigned long len, unsigned long pgoff, unsigned long flags);
 #endif
 
-/**
- * vb2_ops_wait_prepare - helper function to lock a struct _queue
- *
- * @vq: pointer to  vb2_queue
- *
- * ..note:: only use if vq->lock is non-NULL.
- */
-void vb2_ops_wait_prepare(struct vb2_queue *vq);
-
-/**
- * vb2_ops_wait_finish - helper function to unlock a struct _queue
- *
- * @vq: pointer to  vb2_queue
- *
- * ..note:: only use if vq->lock is non-NULL.
- */
-void vb2_ops_wait_finish(struct vb2_queue *vq);
-
 #endif /* _MEDIA_VIDEOBUF2_V4L2_H */
-- 
2.16.3

[PATCH v9 04/15] vb2: move vb2_ops functions to videobuf2-core.[ch]

[Ezequiel Garcia]

From: Gustavo Padovan 

vb2_ops_wait_prepare() and vb2_ops_wait_finish() were in the
wrong file.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/common/videobuf2/videobuf2-core.c | 14 ++
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 14 --
 include/media/videobuf2-core.h  | 18 ++
 include/media/videobuf2-v4l2.h  | 18 --
 4 files changed, 32 insertions(+), 32 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index d3f7bb33a54d..6b8e083893ad 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -656,6 +656,20 @@ int vb2_verify_memory_type(struct vb2_queue *q,
 }
 EXPORT_SYMBOL(vb2_verify_memory_type);
 
+/* vb2_ops helpers. Only use if vq->lock is non-NULL. */
+
+void vb2_ops_wait_prepare(struct vb2_queue *vq)
+{
+   mutex_unlock(vq->lock);
+}
+EXPORT_SYMBOL_GPL(vb2_ops_wait_prepare);
+
+void vb2_ops_wait_finish(struct vb2_queue *vq)
+{
+   mutex_lock(vq->lock);
+}
+EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
+
 int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
unsigned int *count)
 {
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c 
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 886a2d8d5c6c..64503615d00b 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -947,20 +947,6 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, 
unsigned long addr,
 EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
 #endif
 
-/* vb2_ops helpers. Only use if vq->lock is non-NULL. */
-
-void vb2_ops_wait_prepare(struct vb2_queue *vq)
-{
-   mutex_unlock(vq->lock);
-}
-EXPORT_SYMBOL_GPL(vb2_ops_wait_prepare);
-
-void vb2_ops_wait_finish(struct vb2_queue *vq)
-{
-   mutex_lock(vq->lock);
-}
-EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
-
 MODULE_DESCRIPTION("Driver helper framework for Video for Linux 2");
 MODULE_AUTHOR("Pawel Osciak , Marek Szyprowski");
 MODULE_LICENSE("GPL");
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index f6818f732f34..f9633de0386c 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -400,6 +400,24 @@ struct vb2_ops {
void (*buf_queue)(struct vb2_buffer *vb);
 };
 
+/**
+ * vb2_ops_wait_prepare - helper function to lock a struct _queue
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * ..note:: only use if vq->lock is non-NULL.
+ */
+void vb2_ops_wait_prepare(struct vb2_queue *vq);
+
+/**
+ * vb2_ops_wait_finish - helper function to unlock a struct _queue
+ *
+ * @vq: pointer to  vb2_queue
+ *
+ * ..note:: only use if vq->lock is non-NULL.
+ */
+void vb2_ops_wait_finish(struct vb2_queue *vq);
+
 /**
  * struct vb2_buf_ops - driver-specific callbacks.
  *
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 3d5e2d739f05..cf83b01dc44e 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -273,22 +273,4 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, 
unsigned long addr,
unsigned long len, unsigned long pgoff, unsigned long flags);
 #endif
 
-/**
- * vb2_ops_wait_prepare - helper function to lock a struct _queue
- *
- * @vq: pointer to  vb2_queue
- *
- * ..note:: only use if vq->lock is non-NULL.
- */
-void vb2_ops_wait_prepare(struct vb2_queue *vq);
-
-/**
- * vb2_ops_wait_finish - helper function to unlock a struct _queue
- *
- * @vq: pointer to  vb2_queue
- *
- * ..note:: only use if vq->lock is non-NULL.
- */
-void vb2_ops_wait_finish(struct vb2_queue *vq);
-
 #endif /* _MEDIA_VIDEOBUF2_V4L2_H */
-- 
2.16.3

[PATCH v9 07/15] v4l: mark unordered formats

[Ezequiel Garcia]

From: Gustavo Padovan 

Now that we've introduced the V4L2_FMT_FLAG_UNORDERED flag,
mark the appropriate formats.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 55 
 1 file changed, 30 insertions(+), 25 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
b/drivers/media/v4l2-core/v4l2-ioctl.c
index f48c505550e0..f75ad954a6f2 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1260,20 +1260,6 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_MJPEG:descr = "Motion-JPEG"; break;
case V4L2_PIX_FMT_JPEG: descr = "JFIF JPEG"; break;
case V4L2_PIX_FMT_DV:   descr = "1394"; break;
-   case V4L2_PIX_FMT_MPEG: descr = "MPEG-1/2/4"; break;
-   case V4L2_PIX_FMT_H264: descr = "H.264"; break;
-   case V4L2_PIX_FMT_H264_NO_SC:   descr = "H.264 (No Start 
Codes)"; break;
-   case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; break;
-   case V4L2_PIX_FMT_H263: descr = "H.263"; break;
-   case V4L2_PIX_FMT_MPEG1:descr = "MPEG-1 ES"; break;
-   case V4L2_PIX_FMT_MPEG2:descr = "MPEG-2 ES"; break;
-   case V4L2_PIX_FMT_MPEG4:descr = "MPEG-4 part 2 ES"; 
break;
-   case V4L2_PIX_FMT_XVID: descr = "Xvid"; break;
-   case V4L2_PIX_FMT_VC1_ANNEX_G:  descr = "VC-1 (SMPTE 412M Annex 
G)"; break;
-   case V4L2_PIX_FMT_VC1_ANNEX_L:  descr = "VC-1 (SMPTE 412M Annex 
L)"; break;
-   case V4L2_PIX_FMT_VP8:  descr = "VP8"; break;
-   case V4L2_PIX_FMT_VP9:  descr = "VP9"; break;
-   case V4L2_PIX_FMT_HEVC: descr = "HEVC"; break; /* aka 
H.265 */
case V4L2_PIX_FMT_CPIA1:descr = "GSPCA CPiA YUV"; break;
case V4L2_PIX_FMT_WNVA: descr = "WNVA"; break;
case V4L2_PIX_FMT_SN9C10X:  descr = "GSPCA SN9C10X"; break;
@@ -1294,17 +1280,36 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved 
UYVY/JPEG"; break;
case V4L2_PIX_FMT_MT21C:descr = "Mediatek Compressed 
Format"; break;
default:
-   WARN(1, "Unknown pixelformat 0x%08x\n", 
fmt->pixelformat);
-   if (fmt->description[0])
-   return;
-   flags = 0;
-   snprintf(fmt->description, sz, "%c%c%c%c%s",
-   (char)(fmt->pixelformat & 0x7f),
-   (char)((fmt->pixelformat >> 8) & 0x7f),
-   (char)((fmt->pixelformat >> 16) & 0x7f),
-   (char)((fmt->pixelformat >> 24) & 0x7f),
-   (fmt->pixelformat & (1 << 31)) ? "-BE" 
: "");
-   break;
+   /* Unordered formats */
+   flags = V4L2_FMT_FLAG_UNORDERED;
+   switch (fmt->pixelformat) {
+   case V4L2_PIX_FMT_MPEG: descr = "MPEG-1/2/4"; 
break;
+   case V4L2_PIX_FMT_H264: descr = "H.264"; break;
+   case V4L2_PIX_FMT_H264_NO_SC:   descr = "H.264 (No 
Start Codes)"; break;
+   case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; 
break;
+   case V4L2_PIX_FMT_H263: descr = "H.263"; break;
+   case V4L2_PIX_FMT_MPEG1:descr = "MPEG-1 ES"; 
break;
+   case V4L2_PIX_FMT_MPEG2:descr = "MPEG-2 ES"; 
break;
+   case V4L2_PIX_FMT_MPEG4:descr = "MPEG-4 part 2 
ES"; break;
+   case V4L2_PIX_FMT_XVID: descr = "Xvid"; break;
+   case V4L2_PIX_FMT_VC1_ANNEX_G:  descr = "VC-1 (SMPTE 
412M Annex G)"; break;
+   case V4L2_PIX_FMT_VC1_ANNEX_L:  descr = "VC-1 (SMPTE 
412M Annex L)"; break;
+   case V4L2_PIX_FMT_VP8:  descr = "VP8"; break;
+   case V4L2_PIX_FMT_VP9:  descr = "VP9"; break;
+   case V4L2_PIX_FMT_HEVC: descr = "HEVC"; break; 
/* aka H.265 */
+   default:
+   WARN(1, "Unknown pixelformat 0x%08x\n", 
fmt->pixelformat);
+   if (fmt->description[0])
+   return;
+   flags = 0;
+   snprintf(fmt->description, sz, "%c%c%c%c%s",
+   

[PATCH v9 07/15] v4l: mark unordered formats

[Ezequiel Garcia]

From: Gustavo Padovan 

Now that we've introduced the V4L2_FMT_FLAG_UNORDERED flag,
mark the appropriate formats.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 55 
 1 file changed, 30 insertions(+), 25 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
b/drivers/media/v4l2-core/v4l2-ioctl.c
index f48c505550e0..f75ad954a6f2 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1260,20 +1260,6 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_MJPEG:descr = "Motion-JPEG"; break;
case V4L2_PIX_FMT_JPEG: descr = "JFIF JPEG"; break;
case V4L2_PIX_FMT_DV:   descr = "1394"; break;
-   case V4L2_PIX_FMT_MPEG: descr = "MPEG-1/2/4"; break;
-   case V4L2_PIX_FMT_H264: descr = "H.264"; break;
-   case V4L2_PIX_FMT_H264_NO_SC:   descr = "H.264 (No Start 
Codes)"; break;
-   case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; break;
-   case V4L2_PIX_FMT_H263: descr = "H.263"; break;
-   case V4L2_PIX_FMT_MPEG1:descr = "MPEG-1 ES"; break;
-   case V4L2_PIX_FMT_MPEG2:descr = "MPEG-2 ES"; break;
-   case V4L2_PIX_FMT_MPEG4:descr = "MPEG-4 part 2 ES"; 
break;
-   case V4L2_PIX_FMT_XVID: descr = "Xvid"; break;
-   case V4L2_PIX_FMT_VC1_ANNEX_G:  descr = "VC-1 (SMPTE 412M Annex 
G)"; break;
-   case V4L2_PIX_FMT_VC1_ANNEX_L:  descr = "VC-1 (SMPTE 412M Annex 
L)"; break;
-   case V4L2_PIX_FMT_VP8:  descr = "VP8"; break;
-   case V4L2_PIX_FMT_VP9:  descr = "VP9"; break;
-   case V4L2_PIX_FMT_HEVC: descr = "HEVC"; break; /* aka 
H.265 */
case V4L2_PIX_FMT_CPIA1:descr = "GSPCA CPiA YUV"; break;
case V4L2_PIX_FMT_WNVA: descr = "WNVA"; break;
case V4L2_PIX_FMT_SN9C10X:  descr = "GSPCA SN9C10X"; break;
@@ -1294,17 +1280,36 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved 
UYVY/JPEG"; break;
case V4L2_PIX_FMT_MT21C:descr = "Mediatek Compressed 
Format"; break;
default:
-   WARN(1, "Unknown pixelformat 0x%08x\n", 
fmt->pixelformat);
-   if (fmt->description[0])
-   return;
-   flags = 0;
-   snprintf(fmt->description, sz, "%c%c%c%c%s",
-   (char)(fmt->pixelformat & 0x7f),
-   (char)((fmt->pixelformat >> 8) & 0x7f),
-   (char)((fmt->pixelformat >> 16) & 0x7f),
-   (char)((fmt->pixelformat >> 24) & 0x7f),
-   (fmt->pixelformat & (1 << 31)) ? "-BE" 
: "");
-   break;
+   /* Unordered formats */
+   flags = V4L2_FMT_FLAG_UNORDERED;
+   switch (fmt->pixelformat) {
+   case V4L2_PIX_FMT_MPEG: descr = "MPEG-1/2/4"; 
break;
+   case V4L2_PIX_FMT_H264: descr = "H.264"; break;
+   case V4L2_PIX_FMT_H264_NO_SC:   descr = "H.264 (No 
Start Codes)"; break;
+   case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; 
break;
+   case V4L2_PIX_FMT_H263: descr = "H.263"; break;
+   case V4L2_PIX_FMT_MPEG1:descr = "MPEG-1 ES"; 
break;
+   case V4L2_PIX_FMT_MPEG2:descr = "MPEG-2 ES"; 
break;
+   case V4L2_PIX_FMT_MPEG4:descr = "MPEG-4 part 2 
ES"; break;
+   case V4L2_PIX_FMT_XVID: descr = "Xvid"; break;
+   case V4L2_PIX_FMT_VC1_ANNEX_G:  descr = "VC-1 (SMPTE 
412M Annex G)"; break;
+   case V4L2_PIX_FMT_VC1_ANNEX_L:  descr = "VC-1 (SMPTE 
412M Annex L)"; break;
+   case V4L2_PIX_FMT_VP8:  descr = "VP8"; break;
+   case V4L2_PIX_FMT_VP9:  descr = "VP9"; break;
+   case V4L2_PIX_FMT_HEVC: descr = "HEVC"; break; 
/* aka H.265 */
+   default:
+   WARN(1, "Unknown pixelformat 0x%08x\n", 
fmt->pixelformat);
+   if (fmt->description[0])
+   return;
+   flags = 0;
+   snprintf(fmt->description, sz, "%c%c%c%c%s",
+   (char)(fmt->pixelformat & 0x7f),
+   

[PATCH v9 01/15] xilinx: regroup caps on querycap

[Ezequiel Garcia]

From: Gustavo Padovan 

To better organize the code we concentrate the setting of
V4L2_CAP_STREAMING in one place.

v2: move cap->capabilities assignment down (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/platform/xilinx/xilinx-dma.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c 
b/drivers/media/platform/xilinx/xilinx-dma.c
index 522cdfdd3345..d041f94be832 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -494,13 +494,15 @@ xvip_dma_querycap(struct file *file, void *fh, struct 
v4l2_capability *cap)
struct v4l2_fh *vfh = file->private_data;
struct xvip_dma *dma = to_xvip_dma(vfh->vdev);
 
-   cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
- | dma->xdev->v4l2_caps;
+   cap->device_caps = V4L2_CAP_STREAMING;
 
if (dma->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-   cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_CAPTURE;
else
-   cap->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_OUTPUT;
+
+   cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS
+ | dma->xdev->v4l2_caps;
 
strlcpy(cap->driver, "xilinx-vipp", sizeof(cap->driver));
strlcpy(cap->card, dma->video.name, sizeof(cap->card));
-- 
2.16.3

[PATCH v9 01/15] xilinx: regroup caps on querycap

[Ezequiel Garcia]

From: Gustavo Padovan 

To better organize the code we concentrate the setting of
V4L2_CAP_STREAMING in one place.

v2: move cap->capabilities assignment down (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/platform/xilinx/xilinx-dma.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c 
b/drivers/media/platform/xilinx/xilinx-dma.c
index 522cdfdd3345..d041f94be832 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -494,13 +494,15 @@ xvip_dma_querycap(struct file *file, void *fh, struct 
v4l2_capability *cap)
struct v4l2_fh *vfh = file->private_data;
struct xvip_dma *dma = to_xvip_dma(vfh->vdev);
 
-   cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
- | dma->xdev->v4l2_caps;
+   cap->device_caps = V4L2_CAP_STREAMING;
 
if (dma->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-   cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_CAPTURE;
else
-   cap->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_OUTPUT;
+
+   cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS
+ | dma->xdev->v4l2_caps;
 
strlcpy(cap->driver, "xilinx-vipp", sizeof(cap->driver));
strlcpy(cap->card, dma->video.name, sizeof(cap->card));
-- 
2.16.3

[PATCH v9 02/15] hackrf: group device capabilities

[Ezequiel Garcia]

From: Gustavo Padovan 

Instead of putting V4L2_CAP_STREAMING and V4L2_CAP_READWRITE
everywhere, set device_caps earlier with these values.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/usb/hackrf/hackrf.c | 11 ---
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/drivers/media/usb/hackrf/hackrf.c 
b/drivers/media/usb/hackrf/hackrf.c
index 7eb53517a82f..6d692fb3e8dd 100644
--- a/drivers/media/usb/hackrf/hackrf.c
+++ b/drivers/media/usb/hackrf/hackrf.c
@@ -909,18 +909,15 @@ static int hackrf_querycap(struct file *file, void *fh,
 
dev_dbg(>dev, "\n");
 
+   cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
if (vdev->vfl_dir == VFL_DIR_RX)
-   cap->device_caps = V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER |
-  V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
-
+   cap->device_caps |= V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER;
else
-   cap->device_caps = V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR |
-  V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
+   cap->device_caps |= V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR;
 
cap->capabilities = V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER |
V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR |
-   V4L2_CAP_STREAMING | V4L2_CAP_READWRITE |
-   V4L2_CAP_DEVICE_CAPS;
+   V4L2_CAP_DEVICE_CAPS | cap->device_caps;
strlcpy(cap->driver, KBUILD_MODNAME, sizeof(cap->driver));
strlcpy(cap->card, dev->rx_vdev.name, sizeof(cap->card));
usb_make_path(dev->udev, cap->bus_info, sizeof(cap->bus_info));
-- 
2.16.3

[PATCH v9 02/15] hackrf: group device capabilities

[Ezequiel Garcia]

From: Gustavo Padovan 

Instead of putting V4L2_CAP_STREAMING and V4L2_CAP_READWRITE
everywhere, set device_caps earlier with these values.

Signed-off-by: Gustavo Padovan 
---
 drivers/media/usb/hackrf/hackrf.c | 11 ---
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/drivers/media/usb/hackrf/hackrf.c 
b/drivers/media/usb/hackrf/hackrf.c
index 7eb53517a82f..6d692fb3e8dd 100644
--- a/drivers/media/usb/hackrf/hackrf.c
+++ b/drivers/media/usb/hackrf/hackrf.c
@@ -909,18 +909,15 @@ static int hackrf_querycap(struct file *file, void *fh,
 
dev_dbg(>dev, "\n");
 
+   cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
if (vdev->vfl_dir == VFL_DIR_RX)
-   cap->device_caps = V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER |
-  V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
-
+   cap->device_caps |= V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER;
else
-   cap->device_caps = V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR |
-  V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
+   cap->device_caps |= V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR;
 
cap->capabilities = V4L2_CAP_SDR_CAPTURE | V4L2_CAP_TUNER |
V4L2_CAP_SDR_OUTPUT | V4L2_CAP_MODULATOR |
-   V4L2_CAP_STREAMING | V4L2_CAP_READWRITE |
-   V4L2_CAP_DEVICE_CAPS;
+   V4L2_CAP_DEVICE_CAPS | cap->device_caps;
strlcpy(cap->driver, KBUILD_MODNAME, sizeof(cap->driver));
strlcpy(cap->card, dev->rx_vdev.name, sizeof(cap->card));
usb_make_path(dev->udev, cap->bus_info, sizeof(cap->bus_info));
-- 
2.16.3

[PATCH v9 08/15] cobalt: set queue as unordered

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

The cobalt driver may reorder the capture buffers so we need to report
it as such.

v3: set unordered as a property
v2: use vb2_ops_set_unordered() helper

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/pci/cobalt/cobalt-v4l2.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/pci/cobalt/cobalt-v4l2.c 
b/drivers/media/pci/cobalt/cobalt-v4l2.c
index e2a4c705d353..8f06cc7f1c81 100644
--- a/drivers/media/pci/cobalt/cobalt-v4l2.c
+++ b/drivers/media/pci/cobalt/cobalt-v4l2.c
@@ -1236,6 +1236,7 @@ static int cobalt_node_register(struct cobalt *cobalt, 
int node)
q->min_buffers_needed = 2;
q->lock = >lock;
q->dev = >pci_dev->dev;
+   q->unordered = 1;
vdev->queue = q;
 
video_set_drvdata(vdev, s);
-- 
2.16.3

[PATCH v9 08/15] cobalt: set queue as unordered

[Ezequiel Garcia]

From: Gustavo Padovan 

The cobalt driver may reorder the capture buffers so we need to report
it as such.

v3: set unordered as a property
v2: use vb2_ops_set_unordered() helper

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/pci/cobalt/cobalt-v4l2.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/pci/cobalt/cobalt-v4l2.c 
b/drivers/media/pci/cobalt/cobalt-v4l2.c
index e2a4c705d353..8f06cc7f1c81 100644
--- a/drivers/media/pci/cobalt/cobalt-v4l2.c
+++ b/drivers/media/pci/cobalt/cobalt-v4l2.c
@@ -1236,6 +1236,7 @@ static int cobalt_node_register(struct cobalt *cobalt, 
int node)
q->min_buffers_needed = 2;
q->lock = >lock;
q->dev = >pci_dev->dev;
+   q->unordered = 1;
vdev->queue = q;
 
video_set_drvdata(vdev, s);
-- 
2.16.3

[PATCH v9 09/15] vb2: mark codec drivers as unordered

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

In preparation to have full support to explicit fence we are
marking codec as non-ordered preventively. It is easier and safer from an
uAPI point of view to move from unordered to ordered than the opposite.

v3: set property instead of callback
v2: mark only codec drivers as unordered (Nicolas and Hans)

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 2 ++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1 +
 drivers/media/platform/qcom/venus/venc.c   | 2 ++
 drivers/media/platform/s5p-mfc/s5p_mfc.c   | 2 ++
 4 files changed, 7 insertions(+)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 86f0a7134365..c03cefde0c28 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -1498,6 +1498,7 @@ int mtk_vcodec_dec_queue_init(void *priv, struct 
vb2_queue *src_vq,
src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
src_vq->lock= >dev->dev_mutex;
src_vq->dev = >dev->plat_dev->dev;
+   src_vq->unordered   = 1;
 
ret = vb2_queue_init(src_vq);
if (ret) {
@@ -1513,6 +1514,7 @@ int mtk_vcodec_dec_queue_init(void *priv, struct 
vb2_queue *src_vq,
dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
dst_vq->lock= >dev->dev_mutex;
dst_vq->dev = >dev->plat_dev->dev;
+   src_vq->unordered   = 1;
 
ret = vb2_queue_init(dst_vq);
if (ret) {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 1b1a28abbf1f..81751c9aa19d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1340,6 +1340,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct 
vb2_queue *src_vq,
dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
dst_vq->lock= >dev->dev_mutex;
dst_vq->dev = >dev->plat_dev->dev;
+   dst_vq->unordered   = 1;
 
return vb2_queue_init(dst_vq);
 }
diff --git a/drivers/media/platform/qcom/venus/venc.c 
b/drivers/media/platform/qcom/venus/venc.c
index 6b2ce479584e..17ec2d218aa5 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -1053,6 +1053,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue 
*src_vq,
src_vq->buf_struct_size = sizeof(struct venus_buffer);
src_vq->allow_zero_bytesused = 1;
src_vq->min_buffers_needed = 1;
+   src_vq->unordered = 1;
src_vq->dev = inst->core->dev;
if (inst->core->res->hfi_version == HFI_VERSION_1XX)
src_vq->bidirectional = 1;
@@ -1069,6 +1070,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue 
*src_vq,
dst_vq->buf_struct_size = sizeof(struct venus_buffer);
dst_vq->allow_zero_bytesused = 1;
dst_vq->min_buffers_needed = 1;
+   src_vq->unordered = 1;
dst_vq->dev = inst->core->dev;
ret = vb2_queue_init(dst_vq);
if (ret) {
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc.c 
b/drivers/media/platform/s5p-mfc/s5p_mfc.c
index a80251ed3143..6a4251f988ab 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc.c
@@ -863,6 +863,7 @@ static int s5p_mfc_open(struct file *file)
q->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES;
q->mem_ops = _dma_contig_memops;
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+   q->unordered = 1;
ret = vb2_queue_init(q);
if (ret) {
mfc_err("Failed to initialize videobuf2 queue(capture)\n");
@@ -898,6 +899,7 @@ static int s5p_mfc_open(struct file *file)
q->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES;
q->mem_ops = _dma_contig_memops;
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+   q->unordered = 1;
ret = vb2_queue_init(q);
if (ret) {
mfc_err("Failed to initialize videobuf2 queue(output)\n");
-- 
2.16.3

[PATCH v9 09/15] vb2: mark codec drivers as unordered

[Ezequiel Garcia]

From: Gustavo Padovan 

In preparation to have full support to explicit fence we are
marking codec as non-ordered preventively. It is easier and safer from an
uAPI point of view to move from unordered to ordered than the opposite.

v3: set property instead of callback
v2: mark only codec drivers as unordered (Nicolas and Hans)

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 2 ++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1 +
 drivers/media/platform/qcom/venus/venc.c   | 2 ++
 drivers/media/platform/s5p-mfc/s5p_mfc.c   | 2 ++
 4 files changed, 7 insertions(+)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 86f0a7134365..c03cefde0c28 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -1498,6 +1498,7 @@ int mtk_vcodec_dec_queue_init(void *priv, struct 
vb2_queue *src_vq,
src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
src_vq->lock= >dev->dev_mutex;
src_vq->dev = >dev->plat_dev->dev;
+   src_vq->unordered   = 1;
 
ret = vb2_queue_init(src_vq);
if (ret) {
@@ -1513,6 +1514,7 @@ int mtk_vcodec_dec_queue_init(void *priv, struct 
vb2_queue *src_vq,
dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
dst_vq->lock= >dev->dev_mutex;
dst_vq->dev = >dev->plat_dev->dev;
+   src_vq->unordered   = 1;
 
ret = vb2_queue_init(dst_vq);
if (ret) {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c 
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 1b1a28abbf1f..81751c9aa19d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1340,6 +1340,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct 
vb2_queue *src_vq,
dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
dst_vq->lock= >dev->dev_mutex;
dst_vq->dev = >dev->plat_dev->dev;
+   dst_vq->unordered   = 1;
 
return vb2_queue_init(dst_vq);
 }
diff --git a/drivers/media/platform/qcom/venus/venc.c 
b/drivers/media/platform/qcom/venus/venc.c
index 6b2ce479584e..17ec2d218aa5 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -1053,6 +1053,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue 
*src_vq,
src_vq->buf_struct_size = sizeof(struct venus_buffer);
src_vq->allow_zero_bytesused = 1;
src_vq->min_buffers_needed = 1;
+   src_vq->unordered = 1;
src_vq->dev = inst->core->dev;
if (inst->core->res->hfi_version == HFI_VERSION_1XX)
src_vq->bidirectional = 1;
@@ -1069,6 +1070,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue 
*src_vq,
dst_vq->buf_struct_size = sizeof(struct venus_buffer);
dst_vq->allow_zero_bytesused = 1;
dst_vq->min_buffers_needed = 1;
+   src_vq->unordered = 1;
dst_vq->dev = inst->core->dev;
ret = vb2_queue_init(dst_vq);
if (ret) {
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc.c 
b/drivers/media/platform/s5p-mfc/s5p_mfc.c
index a80251ed3143..6a4251f988ab 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc.c
@@ -863,6 +863,7 @@ static int s5p_mfc_open(struct file *file)
q->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES;
q->mem_ops = _dma_contig_memops;
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+   q->unordered = 1;
ret = vb2_queue_init(q);
if (ret) {
mfc_err("Failed to initialize videobuf2 queue(capture)\n");
@@ -898,6 +899,7 @@ static int s5p_mfc_open(struct file *file)
q->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES;
q->mem_ops = _dma_contig_memops;
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+   q->unordered = 1;
ret = vb2_queue_init(q);
if (ret) {
mfc_err("Failed to initialize videobuf2 queue(output)\n");
-- 
2.16.3

[PATCH v9 13/15] v4l: introduce the fences capability

[Ezequiel Garcia]

From: Gustavo Padovan 

Drivers capable of using fences (vb2 drivers) should report the
V4L2_CAP_FENCES to userspace, so add this flag to the uapi.

v2: minor doc/english fix (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/vidioc-querycap.rst | 3 +++
 include/uapi/linux/videodev2.h   | 1 +
 2 files changed, 4 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-querycap.rst 
b/Documentation/media/uapi/v4l/vidioc-querycap.rst
index 66fb1b3d6e6e..df3ad57f07a3 100644
--- a/Documentation/media/uapi/v4l/vidioc-querycap.rst
+++ b/Documentation/media/uapi/v4l/vidioc-querycap.rst
@@ -254,6 +254,9 @@ specification the ioctl returns an ``EINVAL`` error code.
 * - ``V4L2_CAP_TOUCH``
   - 0x1000
   - This is a touch device.
+* - ``V4L2_CAP_FENCES``
+  - 0x2000
+  - The device supports explicit synchronization.
 * - ``V4L2_CAP_DEVICE_CAPS``
   - 0x8000
   - The driver fills the ``device_caps`` field. This capability can
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1f18dc68ecab..cab35fca7c7f 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -460,6 +460,7 @@ struct v4l2_capability {
 #define V4L2_CAP_STREAMING  0x0400  /* streaming I/O ioctls */
 
 #define V4L2_CAP_TOUCH  0x1000  /* Is a touch device */
+#define V4L2_CAP_FENCES 0x2000  /* Supports explicit 
synchronization */
 
 #define V4L2_CAP_DEVICE_CAPS0x8000  /* sets device 
capabilities field */
 
-- 
2.16.3

[PATCH v9 13/15] v4l: introduce the fences capability

[Ezequiel Garcia]

From: Gustavo Padovan 

Drivers capable of using fences (vb2 drivers) should report the
V4L2_CAP_FENCES to userspace, so add this flag to the uapi.

v2: minor doc/english fix (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/vidioc-querycap.rst | 3 +++
 include/uapi/linux/videodev2.h   | 1 +
 2 files changed, 4 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-querycap.rst 
b/Documentation/media/uapi/v4l/vidioc-querycap.rst
index 66fb1b3d6e6e..df3ad57f07a3 100644
--- a/Documentation/media/uapi/v4l/vidioc-querycap.rst
+++ b/Documentation/media/uapi/v4l/vidioc-querycap.rst
@@ -254,6 +254,9 @@ specification the ioctl returns an ``EINVAL`` error code.
 * - ``V4L2_CAP_TOUCH``
   - 0x1000
   - This is a touch device.
+* - ``V4L2_CAP_FENCES``
+  - 0x2000
+  - The device supports explicit synchronization.
 * - ``V4L2_CAP_DEVICE_CAPS``
   - 0x8000
   - The driver fills the ``device_caps`` field. This capability can
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1f18dc68ecab..cab35fca7c7f 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -460,6 +460,7 @@ struct v4l2_capability {
 #define V4L2_CAP_STREAMING  0x0400  /* streaming I/O ioctls */
 
 #define V4L2_CAP_TOUCH  0x1000  /* Is a touch device */
+#define V4L2_CAP_FENCES 0x2000  /* Supports explicit 
synchronization */
 
 #define V4L2_CAP_DEVICE_CAPS0x8000  /* sets device 
capabilities field */
 
-- 
2.16.3

[PATCH v9 15/15] v4l: Document explicit synchronization behavior

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Add section to VIDIOC_QBUF and VIDIOC_QUERY_BUF about it

v8: amend querybuf documentation.

v7: minor issues and English improvements (Hans Verkuil)

v6: Close some gaps in the docs (Hans)

v5: - Remove V4L2_CAP_ORDERED
- Add doc about V4L2_FMT_FLAG_UNORDERED

v4: - Document ordering behavior for in-fences
- Document V4L2_CAP_ORDERED capability
- Remove doc about OUT_FENCE event
- Document immediate return of out-fence in QBUF

v3: - make the out_fence refer to the current buffer (Hans)
- Note what happens when the IN_FENCE is not set (Hans)

v2: - mention that fences are files (Hans)
- rework for the new API

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 Documentation/media/uapi/v4l/vidioc-qbuf.rst | 54 +++-
 Documentation/media/uapi/v4l/vidioc-querybuf.rst | 12 --
 2 files changed, 62 insertions(+), 4 deletions(-)

diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst 
b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
index 9e448a4aa3aa..88a51b8bec7e 100644
--- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
+++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
@@ -54,7 +54,7 @@ When the buffer is intended for output (``type`` is
 or ``V4L2_BUF_TYPE_VBI_OUTPUT``) applications must also initialize the
 ``bytesused``, ``field`` and ``timestamp`` fields, see :ref:`buffer`
 for details. Applications must also set ``flags`` to 0. The
-``reserved2`` and ``reserved`` fields must be set to 0. When using the
+``reserved`` field must be set to 0. When using the
 :ref:`multi-planar API `, the ``m.planes`` field must
 contain a userspace pointer to a filled-in array of struct
 :c:type:`v4l2_plane` and the ``length`` field must be set
@@ -118,6 +118,58 @@ immediately with an ``EAGAIN`` error code when no buffer 
is available.
 The struct :c:type:`v4l2_buffer` structure is specified in
 :ref:`buffer`.
 
+Explicit Synchronization
+
+
+Explicit Synchronization allows us to control the synchronization of
+shared buffers from userspace by passing fences to the kernel and/or
+receiving them from it. Fences passed to the kernel are named in-fences and
+the kernel should wait on them to signal before using the buffer. On the other
+side, the kernel can create out-fences for the buffers it queues to the
+drivers. Out-fences signal when the driver is finished with buffer, i.e., the
+buffer is ready. The fences are represented as a file and passed as a file
+descriptor to userspace.
+
+The in-fences are communicated to the kernel at the ``VIDIOC_QBUF`` ioctl
+using the ``V4L2_BUF_FLAG_IN_FENCE`` buffer flag and the `fence_fd` field. If
+an in-fence needs to be passed to the kernel, `fence_fd` should be set to the
+fence file descriptor number and the ``V4L2_BUF_FLAG_IN_FENCE`` should be set
+as well. Setting one but not the othe will cause ``VIDIOC_QBUF`` to return
+with an error.
+
+The videobuf2-core will guarantee that all buffers queued with an in-fence will
+be queued to the drivers in the same order. Fences may signal out of order, so
+this guarantee at videobuf2 is necessary to not change ordering. So when
+waiting on a fence to signal all buffers queued afterwards will also be blocked
+until that fence signals.
+
+If the in-fence signals with an error the buffer will be marked with
+``V4L2_BUF_FLAG_ERROR`` when returned to userspace at ``VIDIOC_DQBUF``.
+Even with the error the order of dequeueing the buffers is preserved.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE`` flag should
+be set to ask for a fence to be attached to the buffer. The out-fence fd is
+sent to userspace as a ``VIDIOC_QBUF`` return argument in the `fence_fd` field.
+
+Note the same `fence_fd` field is used for both sending the in-fence as
+at input argument and to receive the out-fence as a return argument. A buffer 
can
+have both an in-fence and an out-fence.
+
+At streamoff the out-fences will either signal normally if the driver waits
+for the operations on the buffers to finish or signal with an error if the
+driver cancels the pending operations. Buffers with in-fences won't be queued
+to the driver if their fences signal. They will be cleaned up.
+
+The ``V4L2_FMT_FLAG_UNORDERED`` flag in ``VIDIOC_ENUM_FMT`` tells userspace
+that the  when using this format the order in which buffers are dequeued can
+be different from the order in which they were queued.
+
+Ordering is important to fences because it can optimize the pipeline with
+other drivers like a DRM/KMS display driver. For example, if a capture from the
+camera is happening in an orderly manner one can send the capture buffer
+out-fence to the DRM/KMS driver and rest sure that the buffers will be shown on
+the screen at the correct order. If an ordered queue can not be set then such
+arrangements with other drivers may not be possibl

[PATCH v9 15/15] v4l: Document explicit synchronization behavior

[Ezequiel Garcia]

From: Gustavo Padovan 

Add section to VIDIOC_QBUF and VIDIOC_QUERY_BUF about it

v8: amend querybuf documentation.

v7: minor issues and English improvements (Hans Verkuil)

v6: Close some gaps in the docs (Hans)

v5: - Remove V4L2_CAP_ORDERED
- Add doc about V4L2_FMT_FLAG_UNORDERED

v4: - Document ordering behavior for in-fences
- Document V4L2_CAP_ORDERED capability
- Remove doc about OUT_FENCE event
- Document immediate return of out-fence in QBUF

v3: - make the out_fence refer to the current buffer (Hans)
- Note what happens when the IN_FENCE is not set (Hans)

v2: - mention that fences are files (Hans)
- rework for the new API

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 Documentation/media/uapi/v4l/vidioc-qbuf.rst | 54 +++-
 Documentation/media/uapi/v4l/vidioc-querybuf.rst | 12 --
 2 files changed, 62 insertions(+), 4 deletions(-)

diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst 
b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
index 9e448a4aa3aa..88a51b8bec7e 100644
--- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
+++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
@@ -54,7 +54,7 @@ When the buffer is intended for output (``type`` is
 or ``V4L2_BUF_TYPE_VBI_OUTPUT``) applications must also initialize the
 ``bytesused``, ``field`` and ``timestamp`` fields, see :ref:`buffer`
 for details. Applications must also set ``flags`` to 0. The
-``reserved2`` and ``reserved`` fields must be set to 0. When using the
+``reserved`` field must be set to 0. When using the
 :ref:`multi-planar API `, the ``m.planes`` field must
 contain a userspace pointer to a filled-in array of struct
 :c:type:`v4l2_plane` and the ``length`` field must be set
@@ -118,6 +118,58 @@ immediately with an ``EAGAIN`` error code when no buffer 
is available.
 The struct :c:type:`v4l2_buffer` structure is specified in
 :ref:`buffer`.
 
+Explicit Synchronization
+
+
+Explicit Synchronization allows us to control the synchronization of
+shared buffers from userspace by passing fences to the kernel and/or
+receiving them from it. Fences passed to the kernel are named in-fences and
+the kernel should wait on them to signal before using the buffer. On the other
+side, the kernel can create out-fences for the buffers it queues to the
+drivers. Out-fences signal when the driver is finished with buffer, i.e., the
+buffer is ready. The fences are represented as a file and passed as a file
+descriptor to userspace.
+
+The in-fences are communicated to the kernel at the ``VIDIOC_QBUF`` ioctl
+using the ``V4L2_BUF_FLAG_IN_FENCE`` buffer flag and the `fence_fd` field. If
+an in-fence needs to be passed to the kernel, `fence_fd` should be set to the
+fence file descriptor number and the ``V4L2_BUF_FLAG_IN_FENCE`` should be set
+as well. Setting one but not the othe will cause ``VIDIOC_QBUF`` to return
+with an error.
+
+The videobuf2-core will guarantee that all buffers queued with an in-fence will
+be queued to the drivers in the same order. Fences may signal out of order, so
+this guarantee at videobuf2 is necessary to not change ordering. So when
+waiting on a fence to signal all buffers queued afterwards will also be blocked
+until that fence signals.
+
+If the in-fence signals with an error the buffer will be marked with
+``V4L2_BUF_FLAG_ERROR`` when returned to userspace at ``VIDIOC_DQBUF``.
+Even with the error the order of dequeueing the buffers is preserved.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE`` flag should
+be set to ask for a fence to be attached to the buffer. The out-fence fd is
+sent to userspace as a ``VIDIOC_QBUF`` return argument in the `fence_fd` field.
+
+Note the same `fence_fd` field is used for both sending the in-fence as
+at input argument and to receive the out-fence as a return argument. A buffer 
can
+have both an in-fence and an out-fence.
+
+At streamoff the out-fences will either signal normally if the driver waits
+for the operations on the buffers to finish or signal with an error if the
+driver cancels the pending operations. Buffers with in-fences won't be queued
+to the driver if their fences signal. They will be cleaned up.
+
+The ``V4L2_FMT_FLAG_UNORDERED`` flag in ``VIDIOC_ENUM_FMT`` tells userspace
+that the  when using this format the order in which buffers are dequeued can
+be different from the order in which they were queued.
+
+Ordering is important to fences because it can optimize the pipeline with
+other drivers like a DRM/KMS display driver. For example, if a capture from the
+camera is happening in an orderly manner one can send the capture buffer
+out-fence to the DRM/KMS driver and rest sure that the buffers will be shown on
+the screen at the correct order. If an ordered queue can not be set then such
+arrangements with other drivers may not be possible.
 
 Return Value
 
diff --git a/Documentation/media/uapi/v4l/vidioc-querybuf.rst 
b/Documentation

[PATCH v9 14/15] v4l: Add V4L2_CAP_FENCES to drivers

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Drivers that use videobuf2 are capable of using fences and
should report that to userspace.

v9: Add in the core.

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 8 
 include/media/v4l2-fh.h  | 2 --
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
b/drivers/media/v4l2-core/v4l2-ioctl.c
index f75ad954a6f2..2ae527ef0bc7 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1002,12 +1002,20 @@ static int v4l_querycap(const struct v4l2_ioctl_ops 
*ops,
 {
struct v4l2_capability *cap = (struct v4l2_capability *)arg;
struct video_device *vfd = video_devdata(file);
+   struct v4l2_fh *vfh =
+   test_bit(V4L2_FL_USES_V4L2_FH, >flags) ? fh : NULL;
int ret;
 
cap->version = LINUX_VERSION_CODE;
cap->device_caps = vfd->device_caps;
cap->capabilities = vfd->device_caps | V4L2_CAP_DEVICE_CAPS;
 
+   /* If it has a queue or a m2m context, then the
+* device supports fence synchronization.
+*/
+   if (vfd->queue || (vfh && vfh->m2m_ctx))
+   cap->device_caps |= V4L2_CAP_FENCES;
+
ret = ops->vidioc_querycap(file, fh, cap);
 
cap->capabilities |= V4L2_CAP_EXT_PIX_FORMAT;
diff --git a/include/media/v4l2-fh.h b/include/media/v4l2-fh.h
index ea73fef8bdc0..e993ddc06991 100644
--- a/include/media/v4l2-fh.h
+++ b/include/media/v4l2-fh.h
@@ -57,9 +57,7 @@ struct v4l2_fh {
unsigned intnavailable;
u32 sequence;
 
-#if IS_ENABLED(CONFIG_V4L2_MEM2MEM_DEV)
struct v4l2_m2m_ctx *m2m_ctx;
-#endif
 };
 
 /**
-- 
2.16.3

[PATCH v9 14/15] v4l: Add V4L2_CAP_FENCES to drivers

[Ezequiel Garcia]

From: Gustavo Padovan 

Drivers that use videobuf2 are capable of using fences and
should report that to userspace.

v9: Add in the core.

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 8 
 include/media/v4l2-fh.h  | 2 --
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
b/drivers/media/v4l2-core/v4l2-ioctl.c
index f75ad954a6f2..2ae527ef0bc7 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1002,12 +1002,20 @@ static int v4l_querycap(const struct v4l2_ioctl_ops 
*ops,
 {
struct v4l2_capability *cap = (struct v4l2_capability *)arg;
struct video_device *vfd = video_devdata(file);
+   struct v4l2_fh *vfh =
+   test_bit(V4L2_FL_USES_V4L2_FH, >flags) ? fh : NULL;
int ret;
 
cap->version = LINUX_VERSION_CODE;
cap->device_caps = vfd->device_caps;
cap->capabilities = vfd->device_caps | V4L2_CAP_DEVICE_CAPS;
 
+   /* If it has a queue or a m2m context, then the
+* device supports fence synchronization.
+*/
+   if (vfd->queue || (vfh && vfh->m2m_ctx))
+   cap->device_caps |= V4L2_CAP_FENCES;
+
ret = ops->vidioc_querycap(file, fh, cap);
 
cap->capabilities |= V4L2_CAP_EXT_PIX_FORMAT;
diff --git a/include/media/v4l2-fh.h b/include/media/v4l2-fh.h
index ea73fef8bdc0..e993ddc06991 100644
--- a/include/media/v4l2-fh.h
+++ b/include/media/v4l2-fh.h
@@ -57,9 +57,7 @@ struct v4l2_fh {
unsigned intnavailable;
u32 sequence;
 
-#if IS_ENABLED(CONFIG_V4L2_MEM2MEM_DEV)
struct v4l2_m2m_ctx *m2m_ctx;
-#endif
 };
 
 /**
-- 
2.16.3

[PATCH v9 12/15] vb2: add out-fence support to QBUF

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
an out_fence and send its fd to userspace in the fence_fd field as a
return arg for the QBUF call.

The fence is signaled on buffer_done(), when the job on the buffer is
finished.

v11: - Return fence_fd to userpace only in the QBUF ioctl.
 - Rework implementation to avoid storing the sync_file
   as state, which is not really needed.

v10: - use -EIO for fence error (Hans Verkuil)
 - add comment around fence context creation (Hans Verkuil)

v9: - remove in-fences changes from this patch (Alex Courbot)
- improve fence context creation (Hans Verkuil)
- clean up out fences if vb2_core_qbuf() fails (Hans Verkuil)

v8: - return 0 as fence_fd if OUT_FENCE flag not used (Mauro)
- fix crash when checking not using fences in vb2_buffer_done()

v7: - merge patch that add the infrastructure to out-fences into
  this one (Alex Courbot)
- Do not install the fd if there is no fence. (Alex Courbot)
- do not report error on requeueing, just WARN_ON_ONCE() (Hans)

v6: - get rid of the V4L2_EVENT_OUT_FENCE event. We always keep the
  ordering in vb2 for queueing in the driver, so the event is not
  necessary anymore and the out_fence_fd is sent back to userspace
  on QBUF call return arg
- do not allow requeueing with out-fences, instead mark the buffer
  with an error and wake up to userspace.
- send the out_fence_fd back to userspace on the fence_fd field

v5: - delay fd_install to DQ_EVENT (Hans)
- if queue is fully ordered send OUT_FENCE event right away
  (Brian)
- rename 'q->ordered' to 'q->ordered_in_driver'
- merge change to implement OUT_FENCE event here

v4: - return the out_fence_fd in the BUF_QUEUED event(Hans)

v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
- set the OUT_FENCE flag if there is a fence pending (Hans)
- call fd_install() after vb2_core_qbuf() (Hans)
- clean up fence if vb2_core_qbuf() fails (Hans)
- add list to store sync_file and fence for the next queued buffer

v2: check if the queue is ordered.

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/common/videobuf2/videobuf2-core.c | 95 +++--
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 22 +-
 drivers/media/dvb-core/dvb_vb2.c|  2 +-
 include/media/videobuf2-core.h  | 16 -
 4 files changed, 127 insertions(+), 8 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 996b99497a98..0f7306a04db7 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -25,6 +25,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -357,6 +358,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
}
+   vb->out_fence_fd = -1;
q->bufs[vb->index] = vb;
 
/* Allocate video buffer memory for the MMAP type */
@@ -948,10 +950,22 @@ static void vb2_process_buffer_done(struct vb2_buffer 
*vb, enum vb2_buffer_state
case VB2_BUF_STATE_QUEUED:
return;
case VB2_BUF_STATE_REQUEUEING:
+   /* Requeuing with explicit synchronization, spit warning */
+   WARN_ON_ONCE(vb->out_fence);
+
if (q->start_streaming_called)
__enqueue_in_driver(vb);
return;
default:
+   if (vb->out_fence) {
+   if (state == VB2_BUF_STATE_ERROR)
+   dma_fence_set_error(vb->out_fence, -EIO);
+   dma_fence_signal(vb->out_fence);
+   dma_fence_put(vb->out_fence);
+   vb->out_fence = NULL;
+   vb->out_fence_fd = -1;
+   }
+
/* Inform any processes that may be waiting for buffers */
wake_up(>done_wq);
break;
@@ -1367,6 +1381,68 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned 
int index, void *pb)
 }
 EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
 
+static inline const char *vb2_fence_get_driver_name(struct dma_fence *fence)
+{
+   return "vb2_fence";
+}
+
+static inline const char *vb2_fence_get_timeline_name(struct dma_fence *fence)
+{
+   return "vb2_fence_timeline";
+}
+
+static inline bool vb2_fence_enable_signaling(struct dma_fence *fence)
+{
+   return true;
+}
+
+static const struct dma_fence_ops vb2_fence_ops = {
+   .get_driver_n

[PATCH v9 11/15] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Receive in-fence from userspace and add support for waiting on them
before queueing the buffer to the driver. Buffers can't be queued to the
driver before its fences signal. And a buffer can't be queued to the driver
out of the order they were queued from userspace. That means that even if
its fence signals it must wait for all other buffers, ahead of it in the queue,
to signal first.

If the fence for some buffer fails we do not queue it to the driver,
instead we mark it as error and wait until the previous buffer is done
to notify userspace of the error. We wait here to deliver the buffers back
to userspace in order.

v12: fixed dvb_vb2.c usage of vb2_core_qbuf.

v11: - minor doc/comments fixes (Hans Verkuil)
 - reviewed the in-fence path at __fill_v4l2_buffer()

v10: - rename fence to in_fence in many places
 - handle fences signalling with error better (Hans Verkuil)

v9: - improve comments and docs (Hans Verkuil)
- fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
- move in-fences code that was in the out-fences patch here (Alex)

v8: - improve comments about fences with errors

v7: - get rid of the fence array stuff for ordering and just use
  get_num_buffers_ready() (Hans)
- fix issue of queuing the buffer twice (Hans)
- avoid the dma_fence_wait() in core_qbuf() (Alex)
- merge preparation commit in

v6: - With fences always keep the order userspace queues the buffers.
- Protect in_fence manipulation with a lock (Brian Starkey)
- check if fences have the same context before adding a fence array
- Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
- Clean up fence if __set_in_fence() fails (Brian Starkey)
- treat -EINVAL from dma_fence_add_callback() (Brian Starkey)

v5: - use fence_array to keep buffers ordered in vb2 core when
  needed (Brian Starkey)
- keep backward compat on the reserved2 field (Brian Starkey)
- protect fence callback removal with lock (Brian Starkey)

v4: - Add a comment about dma_fence_add_callback() not returning a
  error (Hans)
- Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
- select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
- Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
- Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
- Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
  vb2_start_streaming() (Hans)
- set IN_FENCE flags on __fill_v4l2_buffer (Hans)
- Queue buffers to the driver as soon as they are ready (Hans)
- call fill_user_buffer() after queuing the buffer (Hans)
- add err: label to clean up fence
- add dma_fence_wait() before calling vb2_start_streaming()

v3: - document fence parameter
- remove ternary if at vb2_qbuf() return (Mauro)
- do not change if conditions behaviour (Mauro)

v2: - fix vb2_queue_or_prepare_buf() ret check
- remove check for VB2_MEMORY_DMABUF only (Javier)
- check num of ready buffers to start streaming
- when queueing, start from the first ready buffer
- handle queue cancel

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 drivers/media/common/videobuf2/videobuf2-core.c | 197 
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  34 +++-
 drivers/media/dvb-core/dvb_vb2.c|   2 +-
 drivers/media/v4l2-core/Kconfig |  33 
 include/media/videobuf2-core.h  |  14 +-
 5 files changed, 249 insertions(+), 31 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 6b8e083893ad..996b99497a98 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -352,6 +352,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->index = q->num_buffers + buffer;
vb->type = q->type;
vb->memory = memory;
+   spin_lock_init(>fence_cb_lock);
for (plane = 0; plane < num_planes; ++plane) {
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
@@ -905,20 +906,12 @@ void *vb2_plane_cookie(struct vb2_buffer *vb, unsigned 
int plane_no)
 }
 EXPORT_SYMBOL_GPL(vb2_plane_cookie);
 
-void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
+static void vb2_process_buffer_done(struct vb2_buffer *vb, enum 
vb2_buffer_state state)
 {
struct vb2_queue *q = vb->vb2_queue;
unsigned long flags;
unsigned int plane;
 
-   if (WARN_ON(vb->state != VB2_BUF_STATE_ACTIVE))
-   return;
-
-   if (WARN_ON(state != VB2_BUF_ST

[PATCH v9 06/15] v4l: add unordered flag to format description ioctl

[Ezequiel Garcia]

From: Gustavo Padovan 

For explicit synchronization it important for userspace to know if the
format being used by the driver can deliver the buffers back to userspace
in the same order they were queued with QBUF.

Ordered streams fits nicely in a pipeline with DRM for example, where
ordered buffer are expected.

v2: Improve documentation (Hans)

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/vidioc-enum-fmt.rst | 7 +++
 include/uapi/linux/videodev2.h   | 1 +
 2 files changed, 8 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst 
b/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
index 019c513df217..df8e039b9ac2 100644
--- a/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
+++ b/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
@@ -116,6 +116,13 @@ one until ``EINVAL`` is returned.
   - This format is not native to the device but emulated through
software (usually libv4l2), where possible try to use a native
format instead for better performance.
+* - ``V4L2_FMT_FLAG_UNORDERED``
+  - 0x0004
+  - This format doesn't guarantee ordered buffer handling. I.e. the order
+   in which buffers are dequeued with
+   :ref:`VIDIOC_DQBUF ` may be different
+   from the order in which they were queued with
+   :ref:`VIDIOC_QBUF `.
 
 
 Return Value
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 600877be5c22..a8842a5ca636 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -717,6 +717,7 @@ struct v4l2_fmtdesc {
 
 #define V4L2_FMT_FLAG_COMPRESSED 0x0001
 #define V4L2_FMT_FLAG_EMULATED   0x0002
+#define V4L2_FMT_FLAG_UNORDERED  0x0004
 
/* Frame Size and frame rate enumeration */
 /*
-- 
2.16.3

[PATCH v9 12/15] vb2: add out-fence support to QBUF

[Ezequiel Garcia]

From: Gustavo Padovan 

If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
an out_fence and send its fd to userspace in the fence_fd field as a
return arg for the QBUF call.

The fence is signaled on buffer_done(), when the job on the buffer is
finished.

v11: - Return fence_fd to userpace only in the QBUF ioctl.
 - Rework implementation to avoid storing the sync_file
   as state, which is not really needed.

v10: - use -EIO for fence error (Hans Verkuil)
 - add comment around fence context creation (Hans Verkuil)

v9: - remove in-fences changes from this patch (Alex Courbot)
- improve fence context creation (Hans Verkuil)
- clean up out fences if vb2_core_qbuf() fails (Hans Verkuil)

v8: - return 0 as fence_fd if OUT_FENCE flag not used (Mauro)
- fix crash when checking not using fences in vb2_buffer_done()

v7: - merge patch that add the infrastructure to out-fences into
  this one (Alex Courbot)
- Do not install the fd if there is no fence. (Alex Courbot)
- do not report error on requeueing, just WARN_ON_ONCE() (Hans)

v6: - get rid of the V4L2_EVENT_OUT_FENCE event. We always keep the
  ordering in vb2 for queueing in the driver, so the event is not
  necessary anymore and the out_fence_fd is sent back to userspace
  on QBUF call return arg
- do not allow requeueing with out-fences, instead mark the buffer
  with an error and wake up to userspace.
- send the out_fence_fd back to userspace on the fence_fd field

v5: - delay fd_install to DQ_EVENT (Hans)
- if queue is fully ordered send OUT_FENCE event right away
  (Brian)
- rename 'q->ordered' to 'q->ordered_in_driver'
- merge change to implement OUT_FENCE event here

v4: - return the out_fence_fd in the BUF_QUEUED event(Hans)

v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
- set the OUT_FENCE flag if there is a fence pending (Hans)
- call fd_install() after vb2_core_qbuf() (Hans)
- clean up fence if vb2_core_qbuf() fails (Hans)
- add list to store sync_file and fence for the next queued buffer

v2: check if the queue is ordered.

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/common/videobuf2/videobuf2-core.c | 95 +++--
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 22 +-
 drivers/media/dvb-core/dvb_vb2.c|  2 +-
 include/media/videobuf2-core.h  | 16 -
 4 files changed, 127 insertions(+), 8 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 996b99497a98..0f7306a04db7 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -25,6 +25,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -357,6 +358,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
}
+   vb->out_fence_fd = -1;
q->bufs[vb->index] = vb;
 
/* Allocate video buffer memory for the MMAP type */
@@ -948,10 +950,22 @@ static void vb2_process_buffer_done(struct vb2_buffer 
*vb, enum vb2_buffer_state
case VB2_BUF_STATE_QUEUED:
return;
case VB2_BUF_STATE_REQUEUEING:
+   /* Requeuing with explicit synchronization, spit warning */
+   WARN_ON_ONCE(vb->out_fence);
+
if (q->start_streaming_called)
__enqueue_in_driver(vb);
return;
default:
+   if (vb->out_fence) {
+   if (state == VB2_BUF_STATE_ERROR)
+   dma_fence_set_error(vb->out_fence, -EIO);
+   dma_fence_signal(vb->out_fence);
+   dma_fence_put(vb->out_fence);
+   vb->out_fence = NULL;
+   vb->out_fence_fd = -1;
+   }
+
/* Inform any processes that may be waiting for buffers */
wake_up(>done_wq);
break;
@@ -1367,6 +1381,68 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned 
int index, void *pb)
 }
 EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
 
+static inline const char *vb2_fence_get_driver_name(struct dma_fence *fence)
+{
+   return "vb2_fence";
+}
+
+static inline const char *vb2_fence_get_timeline_name(struct dma_fence *fence)
+{
+   return "vb2_fence_timeline";
+}
+
+static inline bool vb2_fence_enable_signaling(struct dma_fence *fence)
+{
+   return true;
+}
+
+static const struct dma_fence_ops vb2_fence_ops = {
+   .get_driver_name = vb2_fence_get_driver_name,
+   .get_timeline_name = vb2_fence_get_timeline_name

[PATCH v9 11/15] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

From: Gustavo Padovan 

Receive in-fence from userspace and add support for waiting on them
before queueing the buffer to the driver. Buffers can't be queued to the
driver before its fences signal. And a buffer can't be queued to the driver
out of the order they were queued from userspace. That means that even if
its fence signals it must wait for all other buffers, ahead of it in the queue,
to signal first.

If the fence for some buffer fails we do not queue it to the driver,
instead we mark it as error and wait until the previous buffer is done
to notify userspace of the error. We wait here to deliver the buffers back
to userspace in order.

v12: fixed dvb_vb2.c usage of vb2_core_qbuf.

v11: - minor doc/comments fixes (Hans Verkuil)
 - reviewed the in-fence path at __fill_v4l2_buffer()

v10: - rename fence to in_fence in many places
 - handle fences signalling with error better (Hans Verkuil)

v9: - improve comments and docs (Hans Verkuil)
- fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
- move in-fences code that was in the out-fences patch here (Alex)

v8: - improve comments about fences with errors

v7: - get rid of the fence array stuff for ordering and just use
  get_num_buffers_ready() (Hans)
- fix issue of queuing the buffer twice (Hans)
- avoid the dma_fence_wait() in core_qbuf() (Alex)
- merge preparation commit in

v6: - With fences always keep the order userspace queues the buffers.
- Protect in_fence manipulation with a lock (Brian Starkey)
- check if fences have the same context before adding a fence array
- Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
- Clean up fence if __set_in_fence() fails (Brian Starkey)
- treat -EINVAL from dma_fence_add_callback() (Brian Starkey)

v5: - use fence_array to keep buffers ordered in vb2 core when
  needed (Brian Starkey)
- keep backward compat on the reserved2 field (Brian Starkey)
- protect fence callback removal with lock (Brian Starkey)

v4: - Add a comment about dma_fence_add_callback() not returning a
  error (Hans)
- Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
- select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
- Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
- Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
- Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
  vb2_start_streaming() (Hans)
- set IN_FENCE flags on __fill_v4l2_buffer (Hans)
- Queue buffers to the driver as soon as they are ready (Hans)
- call fill_user_buffer() after queuing the buffer (Hans)
- add err: label to clean up fence
- add dma_fence_wait() before calling vb2_start_streaming()

v3: - document fence parameter
- remove ternary if at vb2_qbuf() return (Mauro)
- do not change if conditions behaviour (Mauro)

v2: - fix vb2_queue_or_prepare_buf() ret check
- remove check for VB2_MEMORY_DMABUF only (Javier)
- check num of ready buffers to start streaming
- when queueing, start from the first ready buffer
- handle queue cancel

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 drivers/media/common/videobuf2/videobuf2-core.c | 197 
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  34 +++-
 drivers/media/dvb-core/dvb_vb2.c|   2 +-
 drivers/media/v4l2-core/Kconfig |  33 
 include/media/videobuf2-core.h  |  14 +-
 5 files changed, 249 insertions(+), 31 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 6b8e083893ad..996b99497a98 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -352,6 +352,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->index = q->num_buffers + buffer;
vb->type = q->type;
vb->memory = memory;
+   spin_lock_init(>fence_cb_lock);
for (plane = 0; plane < num_planes; ++plane) {
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
@@ -905,20 +906,12 @@ void *vb2_plane_cookie(struct vb2_buffer *vb, unsigned 
int plane_no)
 }
 EXPORT_SYMBOL_GPL(vb2_plane_cookie);
 
-void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
+static void vb2_process_buffer_done(struct vb2_buffer *vb, enum 
vb2_buffer_state state)
 {
struct vb2_queue *q = vb->vb2_queue;
unsigned long flags;
unsigned int plane;
 
-   if (WARN_ON(vb->state != VB2_BUF_STATE_ACTIVE))
-   return;
-
-   if (WARN_ON(state != VB2_BUF_STATE_DONE &&
-   state != VB2_BUF_STATE_ERROR &&
-   

[PATCH v9 06/15] v4l: add unordered flag to format description ioctl

[Ezequiel Garcia]

From: Gustavo Padovan 

For explicit synchronization it important for userspace to know if the
format being used by the driver can deliver the buffers back to userspace
in the same order they were queued with QBUF.

Ordered streams fits nicely in a pipeline with DRM for example, where
ordered buffer are expected.

v2: Improve documentation (Hans)

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/vidioc-enum-fmt.rst | 7 +++
 include/uapi/linux/videodev2.h   | 1 +
 2 files changed, 8 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst 
b/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
index 019c513df217..df8e039b9ac2 100644
--- a/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
+++ b/Documentation/media/uapi/v4l/vidioc-enum-fmt.rst
@@ -116,6 +116,13 @@ one until ``EINVAL`` is returned.
   - This format is not native to the device but emulated through
software (usually libv4l2), where possible try to use a native
format instead for better performance.
+* - ``V4L2_FMT_FLAG_UNORDERED``
+  - 0x0004
+  - This format doesn't guarantee ordered buffer handling. I.e. the order
+   in which buffers are dequeued with
+   :ref:`VIDIOC_DQBUF ` may be different
+   from the order in which they were queued with
+   :ref:`VIDIOC_QBUF `.
 
 
 Return Value
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 600877be5c22..a8842a5ca636 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -717,6 +717,7 @@ struct v4l2_fmtdesc {
 
 #define V4L2_FMT_FLAG_COMPRESSED 0x0001
 #define V4L2_FMT_FLAG_EMULATED   0x0002
+#define V4L2_FMT_FLAG_UNORDERED  0x0004
 
/* Frame Size and frame rate enumeration */
 /*
-- 
2.16.3

[PATCH v9 05/15] vb2: add unordered vb2_queue property for drivers

[Ezequiel Garcia]

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Explicit synchronization benefits a lot from ordered queues, they fit
better in a pipeline with DRM for example so create a opt-in way for
drivers notify videobuf2 that the queue is unordered.

v5: go back to a bitfield property for the unordered property.

v4: rename it to vb2_ops_is_unordered() (Hans Verkuil)

v3: - make it bool (Hans)
- create vb2_ops_set_unordered() helper

v2: improve comments for is_unordered flag (Hans Verkuil)

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
 include/media/videobuf2-core.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index f9633de0386c..364e4cb41b10 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -467,6 +467,8 @@ struct vb2_buf_ops {
  * @quirk_poll_must_check_waiting_for_buffers: Return %EPOLLERR at poll when 
QBUF
  *  has not been called. This is a vb1 idiom that has been adopted
  *  also by vb2.
+ * @unordered: tell if the queue is unordered, i.e. buffers can be
+ * dequeued in a different order from how they were queued.
  * @lock:  pointer to a mutex that protects the  vb2_queue. The
  * driver can set this to a mutex to let the v4l2 core serialize
  * the queuing ioctls. If the driver wants to handle locking
@@ -533,6 +535,7 @@ struct vb2_queue {
unsignedfileio_read_once:1;
unsignedfileio_write_immediately:1;
unsignedallow_zero_bytesused:1;
+   unsignedunordered:1;
unsigned   quirk_poll_must_check_waiting_for_buffers:1;
 
struct mutex*lock;
-- 
2.16.3

[PATCH v9 10/15] vb2: add explicit fence user API

[Ezequiel Garcia]

From: Gustavo Padovan 

Turn the reserved2 field into fence_fd that we will use to send
an in-fence to the kernel or return an out-fence from the kernel to
userspace.

Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
when sending an in-fence to the kernel to be waited on, and
V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.

v7: minor fixes on the Documentation (Hans Verkuil)

v6: big improvement on doc (Hans Verkuil)

v5: - keep using reserved2 field for cpia2
- set fence_fd to 0 for now, for compat with userspace(Mauro)

v4: make it a union with reserved2 and fence_fd (Hans Verkuil)

v3: make the out_fence refer to the current buffer (Hans Verkuil)

v2: add documentation

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/buffer.rst | 45 +++--
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
 include/uapi/linux/videodev2.h  |  8 -
 4 files changed, 52 insertions(+), 7 deletions(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index e2c85ddc990b..be9719cf5745 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -301,10 +301,22 @@ struct v4l2_buffer
elements in the ``planes`` array. The driver will fill in the
actual number of valid elements in that array.
 * - __u32
-  - ``reserved2``
+  - ``fence_fd``
   -
-  - A place holder for future extensions. Drivers and applications
-   must set this to 0.
+  - Used to communicate a fence file descriptors from userspace to kernel
+   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
+   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag 
must
+   be used and this field set to the fence file descriptor of the in-fence.
+   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
+   must be set, the kernel will return the out-fence file descriptor in
+   this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
+an error.
+
+   For all other ioctls V4L2 sets this field to -1 if
+   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
+   otherwise this field is set to 0 for backward compatibility.
 * - __u32
   - ``reserved``
   -
@@ -648,6 +660,33 @@ Buffer Flags
   - Start Of Exposure. The buffer timestamp has been taken when the
exposure of the frame has begun. This is only valid for the
``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
+* .. _`V4L2-BUF-FLAG-IN-FENCE`:
+
+  - ``V4L2_BUF_FLAG_IN_FENCE``
+  - 0x0020
+  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
+   buffer won't be queued to the driver until the fence signals. The order
+   in which buffers are queued is guaranteed to be preserved, so any
+   buffers queued after this buffer will also be blocked until this fence
+   signals. This flag must be set before calling ``VIDIOC_QBUF``. For
+   other ioctls the driver just reports the value of the flag.
+
+If the fence signals the flag is cleared and not reported anymore.
+   If the fence is not valid ``VIDIOC_QBUF`` returns an error.
+
+
+* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
+
+  - ``V4L2_BUF_FLAG_OUT_FENCE``
+  - 0x0040
+  - Request for a fence to be attached to the buffer. The driver will fill
+   in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
+   ` returns. This flag must be set before calling
+   ``VIDIOC_QBUF``. For other ioctls the driver just reports the value of
+   the flag.
+
+If the creation of the out-fence fails ``VIDIOC_QBUF`` returns an
+   error.
 
 
 
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c 
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 64503615d00b..b1c0fa2b0b88 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -203,7 +203,7 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void 
*pb)
b->timestamp = ns_to_timeval(vb->timestamp);
b->timecode = vbuf->timecode;
b->sequence = vbuf->sequence;
-   b->reserved2 = 0;
+   b->fence_fd = 0;
b->reserved = 0;
 
if (q->is_multiplanar) {
diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c 
b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
index 4312935f1dfc..93c752459aec 100644
--- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
+++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
@@ -388,7 +388,7 @@ struct v4l2_buffer32 {
__s32   fd;
} m;
__u32   length;
-   __u32   

[PATCH v9 05/15] vb2: add unordered vb2_queue property for drivers

[Ezequiel Garcia]

From: Gustavo Padovan 

Explicit synchronization benefits a lot from ordered queues, they fit
better in a pipeline with DRM for example so create a opt-in way for
drivers notify videobuf2 that the queue is unordered.

v5: go back to a bitfield property for the unordered property.

v4: rename it to vb2_ops_is_unordered() (Hans Verkuil)

v3: - make it bool (Hans)
- create vb2_ops_set_unordered() helper

v2: improve comments for is_unordered flag (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
 include/media/videobuf2-core.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index f9633de0386c..364e4cb41b10 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -467,6 +467,8 @@ struct vb2_buf_ops {
  * @quirk_poll_must_check_waiting_for_buffers: Return %EPOLLERR at poll when 
QBUF
  *  has not been called. This is a vb1 idiom that has been adopted
  *  also by vb2.
+ * @unordered: tell if the queue is unordered, i.e. buffers can be
+ * dequeued in a different order from how they were queued.
  * @lock:  pointer to a mutex that protects the  vb2_queue. The
  * driver can set this to a mutex to let the v4l2 core serialize
  * the queuing ioctls. If the driver wants to handle locking
@@ -533,6 +535,7 @@ struct vb2_queue {
unsignedfileio_read_once:1;
unsignedfileio_write_immediately:1;
unsignedallow_zero_bytesused:1;
+   unsignedunordered:1;
unsigned   quirk_poll_must_check_waiting_for_buffers:1;
 
struct mutex*lock;
-- 
2.16.3

[PATCH v9 10/15] vb2: add explicit fence user API

[Ezequiel Garcia]

From: Gustavo Padovan 

Turn the reserved2 field into fence_fd that we will use to send
an in-fence to the kernel or return an out-fence from the kernel to
userspace.

Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
when sending an in-fence to the kernel to be waited on, and
V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.

v7: minor fixes on the Documentation (Hans Verkuil)

v6: big improvement on doc (Hans Verkuil)

v5: - keep using reserved2 field for cpia2
- set fence_fd to 0 for now, for compat with userspace(Mauro)

v4: make it a union with reserved2 and fence_fd (Hans Verkuil)

v3: make the out_fence refer to the current buffer (Hans Verkuil)

v2: add documentation

Signed-off-by: Gustavo Padovan 
---
 Documentation/media/uapi/v4l/buffer.rst | 45 +++--
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
 include/uapi/linux/videodev2.h  |  8 -
 4 files changed, 52 insertions(+), 7 deletions(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index e2c85ddc990b..be9719cf5745 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -301,10 +301,22 @@ struct v4l2_buffer
elements in the ``planes`` array. The driver will fill in the
actual number of valid elements in that array.
 * - __u32
-  - ``reserved2``
+  - ``fence_fd``
   -
-  - A place holder for future extensions. Drivers and applications
-   must set this to 0.
+  - Used to communicate a fence file descriptors from userspace to kernel
+   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
+   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag 
must
+   be used and this field set to the fence file descriptor of the in-fence.
+   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
+   must be set, the kernel will return the out-fence file descriptor in
+   this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
+an error.
+
+   For all other ioctls V4L2 sets this field to -1 if
+   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
+   otherwise this field is set to 0 for backward compatibility.
 * - __u32
   - ``reserved``
   -
@@ -648,6 +660,33 @@ Buffer Flags
   - Start Of Exposure. The buffer timestamp has been taken when the
exposure of the frame has begun. This is only valid for the
``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
+* .. _`V4L2-BUF-FLAG-IN-FENCE`:
+
+  - ``V4L2_BUF_FLAG_IN_FENCE``
+  - 0x0020
+  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
+   buffer won't be queued to the driver until the fence signals. The order
+   in which buffers are queued is guaranteed to be preserved, so any
+   buffers queued after this buffer will also be blocked until this fence
+   signals. This flag must be set before calling ``VIDIOC_QBUF``. For
+   other ioctls the driver just reports the value of the flag.
+
+If the fence signals the flag is cleared and not reported anymore.
+   If the fence is not valid ``VIDIOC_QBUF`` returns an error.
+
+
+* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
+
+  - ``V4L2_BUF_FLAG_OUT_FENCE``
+  - 0x0040
+  - Request for a fence to be attached to the buffer. The driver will fill
+   in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
+   ` returns. This flag must be set before calling
+   ``VIDIOC_QBUF``. For other ioctls the driver just reports the value of
+   the flag.
+
+If the creation of the out-fence fails ``VIDIOC_QBUF`` returns an
+   error.
 
 
 
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c 
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 64503615d00b..b1c0fa2b0b88 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -203,7 +203,7 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void 
*pb)
b->timestamp = ns_to_timeval(vb->timestamp);
b->timecode = vbuf->timecode;
b->sequence = vbuf->sequence;
-   b->reserved2 = 0;
+   b->fence_fd = 0;
b->reserved = 0;
 
if (q->is_multiplanar) {
diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c 
b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
index 4312935f1dfc..93c752459aec 100644
--- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
+++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
@@ -388,7 +388,7 @@ struct v4l2_buffer32 {
__s32   fd;
} m;
__u32   length;
-   __u32   reserved2;
+   __s32   

[PATCH v9 03/15] omap3isp: group device capabilities

[Ezequiel Garcia]

From: Gustavo Padovan 

Instead of putting V4L2_CAP_STREAMING everywhere, set device_caps
earlier with this value.

v2: move cap->capabilities assignment down (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/platform/omap3isp/ispvideo.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/omap3isp/ispvideo.c 
b/drivers/media/platform/omap3isp/ispvideo.c
index a751c89a3ea8..db9aae222134 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -658,13 +658,15 @@ isp_video_querycap(struct file *file, void *fh, struct 
v4l2_capability *cap)
strlcpy(cap->card, video->video.name, sizeof(cap->card));
strlcpy(cap->bus_info, "media", sizeof(cap->bus_info));
 
-   cap->capabilities = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT
-   | V4L2_CAP_STREAMING | V4L2_CAP_DEVICE_CAPS;
+   cap->device_caps = V4L2_CAP_STREAMING;
 
if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-   cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_CAPTURE;
else
-   cap->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_OUTPUT;
+
+   cap->capabilities = cap->device_caps | V4L2_CAP_VIDEO_CAPTURE |
+   V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_DEVICE_CAPS;
 
return 0;
 }
-- 
2.16.3

[PATCH v9 03/15] omap3isp: group device capabilities

[Ezequiel Garcia]

From: Gustavo Padovan 

Instead of putting V4L2_CAP_STREAMING everywhere, set device_caps
earlier with this value.

v2: move cap->capabilities assignment down (Hans Verkuil)

Signed-off-by: Gustavo Padovan 
---
 drivers/media/platform/omap3isp/ispvideo.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/omap3isp/ispvideo.c 
b/drivers/media/platform/omap3isp/ispvideo.c
index a751c89a3ea8..db9aae222134 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -658,13 +658,15 @@ isp_video_querycap(struct file *file, void *fh, struct 
v4l2_capability *cap)
strlcpy(cap->card, video->video.name, sizeof(cap->card));
strlcpy(cap->bus_info, "media", sizeof(cap->bus_info));
 
-   cap->capabilities = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT
-   | V4L2_CAP_STREAMING | V4L2_CAP_DEVICE_CAPS;
+   cap->device_caps = V4L2_CAP_STREAMING;
 
if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-   cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_CAPTURE;
else
-   cap->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+   cap->device_caps |= V4L2_CAP_VIDEO_OUTPUT;
+
+   cap->capabilities = cap->device_caps | V4L2_CAP_VIDEO_CAPTURE |
+   V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_DEVICE_CAPS;
 
return 0;
 }
-- 
2.16.3

[PATCH v9 00/15] V4L2 Explicit Synchronization

[Ezequiel Garcia]

Hi all,

Gustavo has asked to me to take care of the final
issues with this series.

I'm working on adding some fences tests to v4l2-compliance,
which I'll be posting shortly.

So, here's a new version of the "video4linux meet fences"
series. This new round hopefully addresses all the feedback
received in v8.

There are some additional changes:

 * Removed the vb2_ops_is_unordered callback,
   and instead use a simpler unordered bitfield.
   After inspecting the code, I really saw no reason
   for having a callback. Please correct me if I missed
   anything.

 * Reworked the out-fence setup in vb2_core_qbuf,
   which results in getting rid of the sync_file
   state. After inspecting the code, it became apparent
   that the sync_file wasn't meant to be part of the
   buffer state. The resulting code is simpler.

 * Avoid returning fence file descriptors anywhere
   but in the QBUF result. Also, fixed the documentation
   to be clear about this.

Worth mentioning, I've decided to drop the is-signaled
flag suggested by Hans. It'll be easier to review and
merge if we keep this simple. We can always add stuff
later.

Gustavo Padovan (15):
  xilinx: regroup caps on querycap
  hackrf: group device capabilities
  omap3isp: group device capabilities
  vb2: move vb2_ops functions to videobuf2-core.[ch]
  vb2: add unordered vb2_queue property for drivers
  v4l: add unordered flag to format description ioctl
  v4l: mark unordered formats
  cobalt: set queue as unordered
  vb2: mark codec drivers as unordered
  vb2: add explicit fence user API
  vb2: add in-fence support to QBUF
  vb2: add out-fence support to QBUF
  v4l: introduce the fences capability
  v4l: Add V4L2_CAP_FENCES to drivers
  v4l: Document explicit synchronization behavior

 Documentation/media/uapi/v4l/buffer.rst|  45 +++-
 Documentation/media/uapi/v4l/vidioc-enum-fmt.rst   |   7 +
 Documentation/media/uapi/v4l/vidioc-qbuf.rst   |  54 +++-
 Documentation/media/uapi/v4l/vidioc-querybuf.rst   |  12 +-
 Documentation/media/uapi/v4l/vidioc-querycap.rst   |   3 +
 drivers/media/common/videobuf2/videobuf2-core.c| 298 +++--
 drivers/media/common/videobuf2/videobuf2-v4l2.c|  66 +++--
 drivers/media/dvb-core/dvb_vb2.c   |   2 +-
 drivers/media/pci/cobalt/cobalt-v4l2.c |   1 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c |   2 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c |   1 +
 drivers/media/platform/omap3isp/ispvideo.c |  10 +-
 drivers/media/platform/qcom/venus/venc.c   |   2 +
 drivers/media/platform/s5p-mfc/s5p_mfc.c   |   2 +
 drivers/media/platform/xilinx/xilinx-dma.c |  10 +-
 drivers/media/usb/hackrf/hackrf.c  |  11 +-
 drivers/media/v4l2-core/Kconfig|  33 +++
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c  |   4 +-
 drivers/media/v4l2-core/v4l2-ioctl.c   |  63 +++--
 include/media/v4l2-fh.h|   2 -
 include/media/videobuf2-core.h |  49 +++-
 include/media/videobuf2-v4l2.h |  18 --
 include/uapi/linux/videodev2.h |  10 +-
 23 files changed, 590 insertions(+), 115 deletions(-)

-- 
2.16.3

[PATCH v9 00/15] V4L2 Explicit Synchronization

[Ezequiel Garcia]

Hi all,

Gustavo has asked to me to take care of the final
issues with this series.

I'm working on adding some fences tests to v4l2-compliance,
which I'll be posting shortly.

So, here's a new version of the "video4linux meet fences"
series. This new round hopefully addresses all the feedback
received in v8.

There are some additional changes:

 * Removed the vb2_ops_is_unordered callback,
   and instead use a simpler unordered bitfield.
   After inspecting the code, I really saw no reason
   for having a callback. Please correct me if I missed
   anything.

 * Reworked the out-fence setup in vb2_core_qbuf,
   which results in getting rid of the sync_file
   state. After inspecting the code, it became apparent
   that the sync_file wasn't meant to be part of the
   buffer state. The resulting code is simpler.

 * Avoid returning fence file descriptors anywhere
   but in the QBUF result. Also, fixed the documentation
   to be clear about this.

Worth mentioning, I've decided to drop the is-signaled
flag suggested by Hans. It'll be easier to review and
merge if we keep this simple. We can always add stuff
later.

Gustavo Padovan (15):
  xilinx: regroup caps on querycap
  hackrf: group device capabilities
  omap3isp: group device capabilities
  vb2: move vb2_ops functions to videobuf2-core.[ch]
  vb2: add unordered vb2_queue property for drivers
  v4l: add unordered flag to format description ioctl
  v4l: mark unordered formats
  cobalt: set queue as unordered
  vb2: mark codec drivers as unordered
  vb2: add explicit fence user API
  vb2: add in-fence support to QBUF
  vb2: add out-fence support to QBUF
  v4l: introduce the fences capability
  v4l: Add V4L2_CAP_FENCES to drivers
  v4l: Document explicit synchronization behavior

 Documentation/media/uapi/v4l/buffer.rst|  45 +++-
 Documentation/media/uapi/v4l/vidioc-enum-fmt.rst   |   7 +
 Documentation/media/uapi/v4l/vidioc-qbuf.rst   |  54 +++-
 Documentation/media/uapi/v4l/vidioc-querybuf.rst   |  12 +-
 Documentation/media/uapi/v4l/vidioc-querycap.rst   |   3 +
 drivers/media/common/videobuf2/videobuf2-core.c| 298 +++--
 drivers/media/common/videobuf2/videobuf2-v4l2.c|  66 +++--
 drivers/media/dvb-core/dvb_vb2.c   |   2 +-
 drivers/media/pci/cobalt/cobalt-v4l2.c |   1 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c |   2 +
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c |   1 +
 drivers/media/platform/omap3isp/ispvideo.c |  10 +-
 drivers/media/platform/qcom/venus/venc.c   |   2 +
 drivers/media/platform/s5p-mfc/s5p_mfc.c   |   2 +
 drivers/media/platform/xilinx/xilinx-dma.c |  10 +-
 drivers/media/usb/hackrf/hackrf.c  |  11 +-
 drivers/media/v4l2-core/Kconfig|  33 +++
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c  |   4 +-
 drivers/media/v4l2-core/v4l2-ioctl.c   |  63 +++--
 include/media/v4l2-fh.h|   2 -
 include/media/videobuf2-core.h |  49 +++-
 include/media/videobuf2-v4l2.h |  18 --
 include/uapi/linux/videodev2.h |  10 +-
 23 files changed, 590 insertions(+), 115 deletions(-)

-- 
2.16.3

Re: [PATCH v8 10/13] [media] vb2: add out-fence support to QBUF

[Ezequiel Garcia]

Hi guys,

I've a couple questions.

On 9 March 2018 at 14:49, Gustavo Padovan  wrote:
> From: Gustavo Padovan 
>
> If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
> an out_fence and send its fd to userspace on the fence_fd field as a
> return arg for the QBUF call.
>
> The fence is signaled on buffer_done(), when the job on the buffer is
> finished.
>
> v9: - remove in-fences changes from this patch (Alex Courbot)
> - improve fence context creation (Hans Verkuil)
> - clean up out fences if vb2_core_qbuf() fails (Hans Verkuil)
>
> v8:
> - return 0 as fence_fd if OUT_FENCE flag not used (Mauro)
> - fix crash when checking not using fences in vb2_buffer_done()
>
> v7:
> - merge patch that add the infrastructure to out-fences into
> this one (Alex Courbot)
> - Do not install the fd if there is no fence. (Alex Courbot)
> - do not report error on requeueing, just WARN_ON_ONCE() (Hans)
>
> v6
> - get rid of the V4L2_EVENT_OUT_FENCE event. We always keep the
> ordering in vb2 for queueing in the driver, so the event is not
> necessary anymore and the out_fence_fd is sent back to userspace
> on QBUF call return arg
> - do not allow requeueing with out-fences, instead mark the buffer
> with an error and wake up to userspace.
> - send the out_fence_fd back to userspace on the fence_fd field
>
> v5:
> - delay fd_install to DQ_EVENT (Hans)
> - if queue is fully ordered send OUT_FENCE event right away
> (Brian)
> - rename 'q->ordered' to 'q->ordered_in_driver'
> - merge change to implement OUT_FENCE event here
>
> v4:
> - return the out_fence_fd in the BUF_QUEUED event(Hans)
>
> v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
> - set the OUT_FENCE flag if there is a fence pending (Hans)
> - call fd_install() after vb2_core_qbuf() (Hans)
> - clean up fence if vb2_core_qbuf() fails (Hans)
> - add list to store sync_file and fence for the next queued buffer
>
> v2: check if the queue is ordered.
>
> Signed-off-by: Gustavo Padovan 
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 88 
> +
>  drivers/media/common/videobuf2/videobuf2-v4l2.c | 20 +-
>  include/media/videobuf2-core.h  | 25 +++
>  3 files changed, 132 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
> b/drivers/media/common/videobuf2/videobuf2-core.c
> index 5de5e35cfc40..dd18a9f345c7 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -25,6 +25,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>
>  #include 
>  #include 
> @@ -357,6 +358,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
> vb2_memory memory,
> vb->planes[plane].length = plane_sizes[plane];
> vb->planes[plane].min_length = plane_sizes[plane];
> }
> +   vb->out_fence_fd = -1;
> q->bufs[vb->index] = vb;
>
> /* Allocate video buffer memory for the MMAP type */
> @@ -934,10 +936,22 @@ static void vb2_process_buffer_done(struct vb2_buffer 
> *vb, enum vb2_buffer_state
> case VB2_BUF_STATE_QUEUED:
> return;
> case VB2_BUF_STATE_REQUEUEING:
> +   /* Requeuing with explicit synchronization, spit warning */
> +   WARN_ON_ONCE(vb->out_fence);
> +
> if (q->start_streaming_called)
> __enqueue_in_driver(vb);
> return;
> default:
> +   if (vb->out_fence) {
> +   if (state == VB2_BUF_STATE_ERROR)
> +   dma_fence_set_error(vb->out_fence, -EFAULT);
> +   dma_fence_signal(vb->out_fence);
> +   dma_fence_put(vb->out_fence);
> +   vb->out_fence = NULL;
> +   vb->out_fence_fd = -1;
> +   }
> +
> /* Inform any processes that may be waiting for buffers */
> wake_up(>done_wq);
> break;
> @@ -1353,6 +1367,62 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned 
> int index, void *pb)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
> +static inline const char *vb2_fence_get_driver_name(struct dma_fence *fence)
> +{
> +   return "vb2_fence";
> +}
> +
> +static inline const char *vb2_fence_get_timeline_name(struct dma_fence 
> *fence)
> +{
> +   return "vb2_fence_timeline";
> +}
> +
> +static inline bool vb2_fence_enable_signaling(struct dma_fence *fence)
> +{
> +   return true;
> +}
> +
> +static const struct dma_fence_ops vb2_fence_ops = {
> +   .get_driver_name = 

Re: [PATCH v8 10/13] [media] vb2: add out-fence support to QBUF

[Ezequiel Garcia]

Hi guys,

I've a couple questions.

On 9 March 2018 at 14:49, Gustavo Padovan  wrote:
> From: Gustavo Padovan 
>
> If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
> an out_fence and send its fd to userspace on the fence_fd field as a
> return arg for the QBUF call.
>
> The fence is signaled on buffer_done(), when the job on the buffer is
> finished.
>
> v9: - remove in-fences changes from this patch (Alex Courbot)
> - improve fence context creation (Hans Verkuil)
> - clean up out fences if vb2_core_qbuf() fails (Hans Verkuil)
>
> v8:
> - return 0 as fence_fd if OUT_FENCE flag not used (Mauro)
> - fix crash when checking not using fences in vb2_buffer_done()
>
> v7:
> - merge patch that add the infrastructure to out-fences into
> this one (Alex Courbot)
> - Do not install the fd if there is no fence. (Alex Courbot)
> - do not report error on requeueing, just WARN_ON_ONCE() (Hans)
>
> v6
> - get rid of the V4L2_EVENT_OUT_FENCE event. We always keep the
> ordering in vb2 for queueing in the driver, so the event is not
> necessary anymore and the out_fence_fd is sent back to userspace
> on QBUF call return arg
> - do not allow requeueing with out-fences, instead mark the buffer
> with an error and wake up to userspace.
> - send the out_fence_fd back to userspace on the fence_fd field
>
> v5:
> - delay fd_install to DQ_EVENT (Hans)
> - if queue is fully ordered send OUT_FENCE event right away
> (Brian)
> - rename 'q->ordered' to 'q->ordered_in_driver'
> - merge change to implement OUT_FENCE event here
>
> v4:
> - return the out_fence_fd in the BUF_QUEUED event(Hans)
>
> v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
> - set the OUT_FENCE flag if there is a fence pending (Hans)
> - call fd_install() after vb2_core_qbuf() (Hans)
> - clean up fence if vb2_core_qbuf() fails (Hans)
> - add list to store sync_file and fence for the next queued buffer
>
> v2: check if the queue is ordered.
>
> Signed-off-by: Gustavo Padovan 
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 88 
> +
>  drivers/media/common/videobuf2/videobuf2-v4l2.c | 20 +-
>  include/media/videobuf2-core.h  | 25 +++
>  3 files changed, 132 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
> b/drivers/media/common/videobuf2/videobuf2-core.c
> index 5de5e35cfc40..dd18a9f345c7 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -25,6 +25,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>
>  #include 
>  #include 
> @@ -357,6 +358,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
> vb2_memory memory,
> vb->planes[plane].length = plane_sizes[plane];
> vb->planes[plane].min_length = plane_sizes[plane];
> }
> +   vb->out_fence_fd = -1;
> q->bufs[vb->index] = vb;
>
> /* Allocate video buffer memory for the MMAP type */
> @@ -934,10 +936,22 @@ static void vb2_process_buffer_done(struct vb2_buffer 
> *vb, enum vb2_buffer_state
> case VB2_BUF_STATE_QUEUED:
> return;
> case VB2_BUF_STATE_REQUEUEING:
> +   /* Requeuing with explicit synchronization, spit warning */
> +   WARN_ON_ONCE(vb->out_fence);
> +
> if (q->start_streaming_called)
> __enqueue_in_driver(vb);
> return;
> default:
> +   if (vb->out_fence) {
> +   if (state == VB2_BUF_STATE_ERROR)
> +   dma_fence_set_error(vb->out_fence, -EFAULT);
> +   dma_fence_signal(vb->out_fence);
> +   dma_fence_put(vb->out_fence);
> +   vb->out_fence = NULL;
> +   vb->out_fence_fd = -1;
> +   }
> +
> /* Inform any processes that may be waiting for buffers */
> wake_up(>done_wq);
> break;
> @@ -1353,6 +1367,62 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned 
> int index, void *pb)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
> +static inline const char *vb2_fence_get_driver_name(struct dma_fence *fence)
> +{
> +   return "vb2_fence";
> +}
> +
> +static inline const char *vb2_fence_get_timeline_name(struct dma_fence 
> *fence)
> +{
> +   return "vb2_fence_timeline";
> +}
> +
> +static inline bool vb2_fence_enable_signaling(struct dma_fence *fence)
> +{
> +   return true;
> +}
> +
> +static const struct dma_fence_ops vb2_fence_ops = {
> +   .get_driver_name = vb2_fence_get_driver_name,
> +   .get_timeline_name = vb2_fence_get_timeline_name,
> +   

Re: [PATCH v8 09/13] [media] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On 14 March 2018 at 12:55, Hans Verkuil  wrote:
> On 03/09/2018 09:49 AM, Gustavo Padovan wrote:
>> From: Gustavo Padovan 
>>
>> Receive in-fence from userspace and add support for waiting on them
>> before queueing the buffer to the driver. Buffers can't be queued to the
>> driver before its fences signal. And a buffer can't be queue to the driver
>
> queue -> queued
>
>> out of the order they were queued from userspace. That means that even if
>> it fence signal it must wait all other buffers, ahead of it in the queue,
>
> it fence signal -> its fence signals
> wait all -> wait for all
>
>> to signal first.
>>
>> If the fence for some buffer fails we do not queue it to the driver,
>> instead we mark it as error and wait until the previous buffer is done
>> to notify userspace of the error. We wait here to deliver the buffers back
>> to userspace in order.
>>
>> v9:   - rename fence to in_fence in many places
>>   - handle fences signalling with error better (Hans Verkuil)
>>
>> v8:   - improve comments and docs (Hans Verkuil)
>>   - fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
>>   - move in-fences code that was in the out-fences patch here (Alex)
>>
>> v8:   - improve comments about fences with errors
>
> v9? Two v8 entries?
>
>>
>> v7:
>>   - get rid of the fence array stuff for ordering and just use
>>   get_num_buffers_ready() (Hans)
>>   - fix issue of queuing the buffer twice (Hans)
>>   - avoid the dma_fence_wait() in core_qbuf() (Alex)
>>   - merge preparation commit in
>>
>> v6:
>>   - With fences always keep the order userspace queues the buffers.
>>   - Protect in_fence manipulation with a lock (Brian Starkey)
>>   - check if fences have the same context before adding a fence array
>>   - Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
>>   - Clean up fence if __set_in_fence() fails (Brian Starkey)
>>   - treat -EINVAL from dma_fence_add_callback() (Brian Starkey)
>>
>> v5:   - use fence_array to keep buffers ordered in vb2 core when
>>   needed (Brian Starkey)
>>   - keep backward compat on the reserved2 field (Brian Starkey)
>>   - protect fence callback removal with lock (Brian Starkey)
>>
>> v4:
>>   - Add a comment about dma_fence_add_callback() not returning a
>>   error (Hans)
>>   - Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
>>   - select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
>>   - Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
>>   - Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
>>   -  Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
>>   vb2_start_streaming() (Hans)
>>   - set IN_FENCE flags on __fill_v4l2_buffer (Hans)
>>   - Queue buffers to the driver as soon as they are ready (Hans)
>>   - call fill_user_buffer() after queuing the buffer (Hans)
>>   - add err: label to clean up fence
>>   - add dma_fence_wait() before calling vb2_start_streaming()
>>
>> v3:   - document fence parameter
>>   - remove ternary if at vb2_qbuf() return (Mauro)
>>   - do not change if conditions behaviour (Mauro)
>>
>> v2:
>>   - fix vb2_queue_or_prepare_buf() ret check
>>   - remove check for VB2_MEMORY_DMABUF only (Javier)
>>   - check num of ready buffers to start streaming
>>   - when queueing, start from the first ready buffer
>>   - handle queue cancel
>>
>> Signed-off-by: Gustavo Padovan 
>> ---
>>  drivers/media/common/videobuf2/videobuf2-core.c | 197 
>> 
>>  drivers/media/common/videobuf2/videobuf2-v4l2.c |  34 +++-
>>  drivers/media/v4l2-core/Kconfig |  33 
>>  include/media/videobuf2-core.h  |  14 +-
>>  4 files changed, 248 insertions(+), 30 deletions(-)
>>
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index d3f7bb33a54d..5de5e35cfc40 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -352,6 +352,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
>> vb2_memory memory,
>>   vb->index = q->num_buffers + buffer;
>>   vb->type = q->type;
>>   vb->memory = memory;
>> + spin_lock_init(>fence_cb_lock);
>>   for (plane = 0; plane < num_planes; ++plane) {
>>   vb->planes[plane].length = plane_sizes[plane];
>>   vb->planes[plane].min_length = plane_sizes[plane];
>> @@ -891,20 +892,12 @@ void *vb2_plane_cookie(struct vb2_buffer *vb, unsigned 
>> int plane_no)
>>  }
>>  EXPORT_SYMBOL_GPL(vb2_plane_cookie);
>>
>> -void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
>> +static void vb2_process_buffer_done(struct vb2_buffer *vb, enum 
>> 

Re: [PATCH v8 09/13] [media] vb2: add in-fence support to QBUF

[Ezequiel Garcia]

On 14 March 2018 at 12:55, Hans Verkuil  wrote:
> On 03/09/2018 09:49 AM, Gustavo Padovan wrote:
>> From: Gustavo Padovan 
>>
>> Receive in-fence from userspace and add support for waiting on them
>> before queueing the buffer to the driver. Buffers can't be queued to the
>> driver before its fences signal. And a buffer can't be queue to the driver
>
> queue -> queued
>
>> out of the order they were queued from userspace. That means that even if
>> it fence signal it must wait all other buffers, ahead of it in the queue,
>
> it fence signal -> its fence signals
> wait all -> wait for all
>
>> to signal first.
>>
>> If the fence for some buffer fails we do not queue it to the driver,
>> instead we mark it as error and wait until the previous buffer is done
>> to notify userspace of the error. We wait here to deliver the buffers back
>> to userspace in order.
>>
>> v9:   - rename fence to in_fence in many places
>>   - handle fences signalling with error better (Hans Verkuil)
>>
>> v8:   - improve comments and docs (Hans Verkuil)
>>   - fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
>>   - move in-fences code that was in the out-fences patch here (Alex)
>>
>> v8:   - improve comments about fences with errors
>
> v9? Two v8 entries?
>
>>
>> v7:
>>   - get rid of the fence array stuff for ordering and just use
>>   get_num_buffers_ready() (Hans)
>>   - fix issue of queuing the buffer twice (Hans)
>>   - avoid the dma_fence_wait() in core_qbuf() (Alex)
>>   - merge preparation commit in
>>
>> v6:
>>   - With fences always keep the order userspace queues the buffers.
>>   - Protect in_fence manipulation with a lock (Brian Starkey)
>>   - check if fences have the same context before adding a fence array
>>   - Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
>>   - Clean up fence if __set_in_fence() fails (Brian Starkey)
>>   - treat -EINVAL from dma_fence_add_callback() (Brian Starkey)
>>
>> v5:   - use fence_array to keep buffers ordered in vb2 core when
>>   needed (Brian Starkey)
>>   - keep backward compat on the reserved2 field (Brian Starkey)
>>   - protect fence callback removal with lock (Brian Starkey)
>>
>> v4:
>>   - Add a comment about dma_fence_add_callback() not returning a
>>   error (Hans)
>>   - Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
>>   - select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
>>   - Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
>>   - Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
>>   -  Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
>>   vb2_start_streaming() (Hans)
>>   - set IN_FENCE flags on __fill_v4l2_buffer (Hans)
>>   - Queue buffers to the driver as soon as they are ready (Hans)
>>   - call fill_user_buffer() after queuing the buffer (Hans)
>>   - add err: label to clean up fence
>>   - add dma_fence_wait() before calling vb2_start_streaming()
>>
>> v3:   - document fence parameter
>>   - remove ternary if at vb2_qbuf() return (Mauro)
>>   - do not change if conditions behaviour (Mauro)
>>
>> v2:
>>   - fix vb2_queue_or_prepare_buf() ret check
>>   - remove check for VB2_MEMORY_DMABUF only (Javier)
>>   - check num of ready buffers to start streaming
>>   - when queueing, start from the first ready buffer
>>   - handle queue cancel
>>
>> Signed-off-by: Gustavo Padovan 
>> ---
>>  drivers/media/common/videobuf2/videobuf2-core.c | 197 
>> 
>>  drivers/media/common/videobuf2/videobuf2-v4l2.c |  34 +++-
>>  drivers/media/v4l2-core/Kconfig |  33 
>>  include/media/videobuf2-core.h  |  14 +-
>>  4 files changed, 248 insertions(+), 30 deletions(-)
>>
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index d3f7bb33a54d..5de5e35cfc40 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -352,6 +352,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
>> vb2_memory memory,
>>   vb->index = q->num_buffers + buffer;
>>   vb->type = q->type;
>>   vb->memory = memory;
>> + spin_lock_init(>fence_cb_lock);
>>   for (plane = 0; plane < num_planes; ++plane) {
>>   vb->planes[plane].length = plane_sizes[plane];
>>   vb->planes[plane].min_length = plane_sizes[plane];
>> @@ -891,20 +892,12 @@ void *vb2_plane_cookie(struct vb2_buffer *vb, unsigned 
>> int plane_no)
>>  }
>>  EXPORT_SYMBOL_GPL(vb2_plane_cookie);
>>
>> -void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
>> +static void vb2_process_buffer_done(struct vb2_buffer *vb, enum 
>> vb2_buffer_state state)
>>  {
>>   struct vb2_queue *q = vb->vb2_queue;
>>