Re: [RFC RESEND 2/3] media: uapi: Add VP9 stateless decoder controls
Hi Hans, just a partial reply, I'll let Andrzej extend. Le jeudi 29 avril 2021 à 12:20 +0200, Hans Verkuil a écrit : > > + - ``frame_width_minus_1`` > > + - Add 1 to get the frame width expressed in pixels. > > + * - __u16 > > + - ``frame_height_minus_1`` > > + - Add 1 to get the frame height expressed in pixels. > > These two fields are weird. Isn't this defined by setting the output format? > And why the 'minus_1'? > > > + * - __u16 > > + - ``render_width_minus_1`` > > + - Add 1 to get the expected render width expressed in pixels. This is > > + not used during the decoding process but might be used by HW > > scalers to > > + prepare a frame that's ready for scanout. > > + * - __u16 > > + - render_height_minus_1 > > + - Add 1 to get the expected render height expressed in pixels. This > > is > > + not used during the decoding process but might be used by HW > > scalers to > > + prepare a frame that's ready for scanout. > > No idea what these fields are about. I suspect this can be defined by setting > the capture format, but I'm not sure. We have the same for other codecs. Each codec bitstream include the coded and the display size in some form. For H264/H265 that was passed as as number of macroblock and a crop rectangle. For VP9 value - 1 is as defined in the spec. As 0 is not valid, the boolean coder can save some bits when storing the value. Though, for parameters, we usually start with the way it's encoded first, and change it only if we think it make sense. We'll take note and consider this whenever we have a second driver to look at. Now, why do we include both here. Well in fact, the HW may have some extra constraints. Which means there will be up to 3 frame sizes that matters. We can't also determine if the display/render or coded size will be used for minim CAPTURE format, as this will in fact depends if a post processor will be used or not. So to recap, we limit the CAPTURE format base on this information, and not the opposite. The width/height on OUTPUT FMT has been define as meaningless in the spec (to align with stateful I suppose ?). This way, the driver got all the information aligned with how it works inside the codec, without having to do a translation dance, and then properly implement CAPTURE TRY_FMT base on that. To make an analogy with stateful codec, this replaces the queuing of a frame that contains codec headers. We skip the SRC_CH events, since this is no longer asynchronous. Nicolas ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [RFC RESEND 2/3] media: uapi: Add VP9 stateless decoder controls
Hi Hans,
Thank you for the review. I'm working on a v2 to address your comments.
Also please see inline.
Andrzej
W dniu 29.04.2021 o 12:20, Hans Verkuil pisze:
On 21/04/2021 12:00, Andrzej Pietrasiewicz wrote:
Add the VP9 stateless decoder controls plus the documentation that goes
with it.
Signed-off-by: Boris Brezillon
Signed-off-by: Ezequiel Garcia
Signed-off-by: Adrian Ratiu
Signed-off-by: Andrzej Pietrasiewicz
---
.../userspace-api/media/v4l/biblio.rst| 10 +
.../media/v4l/ext-ctrls-codec-stateless.rst | 523 ++
.../media/v4l/pixfmt-compressed.rst | 15 +
.../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
.../media/v4l/vidioc-queryctrl.rst| 12 +
.../media/videodev2.h.rst.exceptions | 2 +
drivers/media/v4l2-core/v4l2-ctrls.c | 244
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/media/v4l2-ctrls.h| 4 +
include/uapi/linux/v4l2-controls.h| 455 +++
include/uapi/linux/videodev2.h| 6 +
11 files changed, 1280 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/biblio.rst
b/Documentation/userspace-api/media/v4l/biblio.rst
index 64d241daf63c..051982896375 100644
--- a/Documentation/userspace-api/media/v4l/biblio.rst
+++ b/Documentation/userspace-api/media/v4l/biblio.rst
@@ -417,3 +417,13 @@ VP8
:title: RFC 6386: "VP8 Data Format and Decoding Guide"
:author:J. Bankoski et al.
+
+.. _vp9:
+
+VP9
+===
+
+
+:title: VP9 Bitstream & Decoding Process Specification
+
+:author:Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan
Hunt (Argon Design)
diff --git
a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 3fc04daa9ffb..ab40c878c8ad 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -1244,3 +1244,526 @@ FWHT Flags
* - __u8
- ``padding[3]``
- Applications and drivers must set this to zero.
+
+.. _v4l2-codec-stateless-vp9:
+
+``V4L2_CID_STATELESS_VP9_COMPRESSED_HDR_PROBS (struct)``
+Stores VP9 probabilities updates as parsed from the current compressed
frame
+header. A value of zero in a struct member means no update of the relevant
+probability. Motion vector-related updates contain a new value or zero. All
+other updates contain values translated with inv_map_table[] (see 6.3.5 in
+:ref:`vp9`).
+
+.. c:type:: v4l2_ctrl_vp9_compressed_hdr_probs
+
+.. cssclass:: longtable
+
+.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
+
+.. flat-table:: struct v4l2_ctrl_vp9_compressed_hdr_probs
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u8
+ - ``tx8[2][1]``
+ - TX 8x8 probabilities delta.
+* - __u8
+ - ``tx16[2][2]``
+ - TX 16x16 probabilities delta.
+* - __u8
+ - ``tx32[2][3]``
+ - TX 32x32 probabilities delta.
+* - __u8
+ - ``coef[4][2][2][6][6][3]``
+ - Coefficient probabilities delta.
+* - __u8
+ - ``skip[3]``
+ - Skip probabilities delta.
+* - __u8
+ - ``inter_mode[7][3]``
+ - Inter prediction mode probabilities delta.
+* - __u8
+ - ``interp_filter[4][2]``
+ - Interpolation filter probabilities delta.
+* - __u8
+ - ``is_inter[4]``
+ - Is inter-block probabilities delta.
+* - __u8
+ - ``comp_mode[5]``
+ - Compound prediction mode probabilities delta.
+* - __u8
+ - ``single_ref[5][2]``
+ - Single reference probabilities delta.
+* - __u8
+ - ``comp_mode[5]``
+ - Compound reference probabilities delta.
+* - __u8
+ - ``y_mode[4][9]``
+ - Y prediction mode probabilities delta.
+* - __u8
+ - ``uv_mode[10][9]``
+ - UV prediction mode probabilities delta.
+* - __u8
+ - ``partition[16][3]``
+ - Partition probabilities delta.
+* - __u8
+ - ``partition[16][3]``
+ - Partition probabilities delta.
+* - __u8
+ - ``mv.joint[3]``
+ - Motion vector joint probabilities delta.
+* - __u8
+ - ``mv.sign[2]``
+ - Motion vector sign probabilities delta.
+* - __u8
+ - ``mv.class[2][10]``
+ - Motion vector class probabilities delta.
+* - __u8
+ - ``mv.class0_bit[2]``
+ - Motion vector class0 bit probabilities delta.
+* - __u8
+ - ``mv.bits[2][10]``
+ - Motion vector bits probabilities delta.
+* - __u8
+ - ``mv.class0_fr[2][2][3]``
+ - Motion vector class0 fractional bit probabilities delta.
+* - __u8
+ - ``mv.fr[2][3]``
+ - Motion vector fractional bit probabilities delta.
+* - __u8
+ - ``mv.class0_hp[2]``
+ - Motion vector class0 high precision fractional bit probabilities delta.
+* - __u8
+ - ``mv.hp[2]``
+
Re: [RFC RESEND 2/3] media: uapi: Add VP9 stateless decoder controls
On 21/04/2021 12:00, Andrzej Pietrasiewicz wrote:
> Add the VP9 stateless decoder controls plus the documentation that goes
> with it.
>
> Signed-off-by: Boris Brezillon
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> Signed-off-by: Andrzej Pietrasiewicz
> ---
> .../userspace-api/media/v4l/biblio.rst| 10 +
> .../media/v4l/ext-ctrls-codec-stateless.rst | 523 ++
> .../media/v4l/pixfmt-compressed.rst | 15 +
> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
> .../media/v4l/vidioc-queryctrl.rst| 12 +
> .../media/videodev2.h.rst.exceptions | 2 +
> drivers/media/v4l2-core/v4l2-ctrls.c | 244
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> include/media/v4l2-ctrls.h| 4 +
> include/uapi/linux/v4l2-controls.h| 455 +++
> include/uapi/linux/videodev2.h| 6 +
> 11 files changed, 1280 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/biblio.rst
> b/Documentation/userspace-api/media/v4l/biblio.rst
> index 64d241daf63c..051982896375 100644
> --- a/Documentation/userspace-api/media/v4l/biblio.rst
> +++ b/Documentation/userspace-api/media/v4l/biblio.rst
> @@ -417,3 +417,13 @@ VP8
> :title: RFC 6386: "VP8 Data Format and Decoding Guide"
>
> :author:J. Bankoski et al.
> +
> +.. _vp9:
> +
> +VP9
> +===
> +
> +
> +:title: VP9 Bitstream & Decoding Process Specification
> +
> +:author:Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan
> Hunt (Argon Design)
> diff --git
> a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
> b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
> index 3fc04daa9ffb..ab40c878c8ad 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
> @@ -1244,3 +1244,526 @@ FWHT Flags
> * - __u8
>- ``padding[3]``
>- Applications and drivers must set this to zero.
> +
> +.. _v4l2-codec-stateless-vp9:
> +
> +``V4L2_CID_STATELESS_VP9_COMPRESSED_HDR_PROBS (struct)``
> +Stores VP9 probabilities updates as parsed from the current compressed
> frame
> +header. A value of zero in a struct member means no update of the
> relevant
> +probability. Motion vector-related updates contain a new value or zero.
> All
> +other updates contain values translated with inv_map_table[] (see 6.3.5
> in
> +:ref:`vp9`).
> +
> +.. c:type:: v4l2_ctrl_vp9_compressed_hdr_probs
> +
> +.. cssclass:: longtable
> +
> +.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
> +
> +.. flat-table:: struct v4l2_ctrl_vp9_compressed_hdr_probs
> +:header-rows: 0
> +:stub-columns: 0
> +:widths: 1 1 2
> +
> +* - __u8
> + - ``tx8[2][1]``
> + - TX 8x8 probabilities delta.
> +* - __u8
> + - ``tx16[2][2]``
> + - TX 16x16 probabilities delta.
> +* - __u8
> + - ``tx32[2][3]``
> + - TX 32x32 probabilities delta.
> +* - __u8
> + - ``coef[4][2][2][6][6][3]``
> + - Coefficient probabilities delta.
> +* - __u8
> + - ``skip[3]``
> + - Skip probabilities delta.
> +* - __u8
> + - ``inter_mode[7][3]``
> + - Inter prediction mode probabilities delta.
> +* - __u8
> + - ``interp_filter[4][2]``
> + - Interpolation filter probabilities delta.
> +* - __u8
> + - ``is_inter[4]``
> + - Is inter-block probabilities delta.
> +* - __u8
> + - ``comp_mode[5]``
> + - Compound prediction mode probabilities delta.
> +* - __u8
> + - ``single_ref[5][2]``
> + - Single reference probabilities delta.
> +* - __u8
> + - ``comp_mode[5]``
> + - Compound reference probabilities delta.
> +* - __u8
> + - ``y_mode[4][9]``
> + - Y prediction mode probabilities delta.
> +* - __u8
> + - ``uv_mode[10][9]``
> + - UV prediction mode probabilities delta.
> +* - __u8
> + - ``partition[16][3]``
> + - Partition probabilities delta.
> +* - __u8
> + - ``partition[16][3]``
> + - Partition probabilities delta.
> +* - __u8
> + - ``mv.joint[3]``
> + - Motion vector joint probabilities delta.
> +* - __u8
> + - ``mv.sign[2]``
> + - Motion vector sign probabilities delta.
> +* - __u8
> + - ``mv.class[2][10]``
> + - Motion vector class probabilities delta.
> +* - __u8
> + - ``mv.class0_bit[2]``
> + - Motion vector class0 bit probabilities delta.
> +* - __u8
> + - ``mv.bits[2][10]``
> + - Motion vector bits probabilities delta.
> +* - __u8
> + - ``mv.class0_fr[2][2][3]``
> + - Motion vector class0 fractional bit probabilities delta.
> +* - __u8
> + - ``mv.fr[2][3]``
> + - Motion vector fractional bit probabilities delta.
> +* - __u8
> + - ``mv.class0_hp[2]``
> + -
[RFC RESEND 2/3] media: uapi: Add VP9 stateless decoder controls
Add the VP9 stateless decoder controls plus the documentation that goes
with it.
Signed-off-by: Boris Brezillon
Signed-off-by: Ezequiel Garcia
Signed-off-by: Adrian Ratiu
Signed-off-by: Andrzej Pietrasiewicz
---
.../userspace-api/media/v4l/biblio.rst| 10 +
.../media/v4l/ext-ctrls-codec-stateless.rst | 523 ++
.../media/v4l/pixfmt-compressed.rst | 15 +
.../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
.../media/v4l/vidioc-queryctrl.rst| 12 +
.../media/videodev2.h.rst.exceptions | 2 +
drivers/media/v4l2-core/v4l2-ctrls.c | 244
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/media/v4l2-ctrls.h| 4 +
include/uapi/linux/v4l2-controls.h| 455 +++
include/uapi/linux/videodev2.h| 6 +
11 files changed, 1280 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/biblio.rst
b/Documentation/userspace-api/media/v4l/biblio.rst
index 64d241daf63c..051982896375 100644
--- a/Documentation/userspace-api/media/v4l/biblio.rst
+++ b/Documentation/userspace-api/media/v4l/biblio.rst
@@ -417,3 +417,13 @@ VP8
:title: RFC 6386: "VP8 Data Format and Decoding Guide"
:author:J. Bankoski et al.
+
+.. _vp9:
+
+VP9
+===
+
+
+:title: VP9 Bitstream & Decoding Process Specification
+
+:author:Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan
Hunt (Argon Design)
diff --git
a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 3fc04daa9ffb..ab40c878c8ad 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -1244,3 +1244,526 @@ FWHT Flags
* - __u8
- ``padding[3]``
- Applications and drivers must set this to zero.
+
+.. _v4l2-codec-stateless-vp9:
+
+``V4L2_CID_STATELESS_VP9_COMPRESSED_HDR_PROBS (struct)``
+Stores VP9 probabilities updates as parsed from the current compressed
frame
+header. A value of zero in a struct member means no update of the relevant
+probability. Motion vector-related updates contain a new value or zero. All
+other updates contain values translated with inv_map_table[] (see 6.3.5 in
+:ref:`vp9`).
+
+.. c:type:: v4l2_ctrl_vp9_compressed_hdr_probs
+
+.. cssclass:: longtable
+
+.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
+
+.. flat-table:: struct v4l2_ctrl_vp9_compressed_hdr_probs
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u8
+ - ``tx8[2][1]``
+ - TX 8x8 probabilities delta.
+* - __u8
+ - ``tx16[2][2]``
+ - TX 16x16 probabilities delta.
+* - __u8
+ - ``tx32[2][3]``
+ - TX 32x32 probabilities delta.
+* - __u8
+ - ``coef[4][2][2][6][6][3]``
+ - Coefficient probabilities delta.
+* - __u8
+ - ``skip[3]``
+ - Skip probabilities delta.
+* - __u8
+ - ``inter_mode[7][3]``
+ - Inter prediction mode probabilities delta.
+* - __u8
+ - ``interp_filter[4][2]``
+ - Interpolation filter probabilities delta.
+* - __u8
+ - ``is_inter[4]``
+ - Is inter-block probabilities delta.
+* - __u8
+ - ``comp_mode[5]``
+ - Compound prediction mode probabilities delta.
+* - __u8
+ - ``single_ref[5][2]``
+ - Single reference probabilities delta.
+* - __u8
+ - ``comp_mode[5]``
+ - Compound reference probabilities delta.
+* - __u8
+ - ``y_mode[4][9]``
+ - Y prediction mode probabilities delta.
+* - __u8
+ - ``uv_mode[10][9]``
+ - UV prediction mode probabilities delta.
+* - __u8
+ - ``partition[16][3]``
+ - Partition probabilities delta.
+* - __u8
+ - ``partition[16][3]``
+ - Partition probabilities delta.
+* - __u8
+ - ``mv.joint[3]``
+ - Motion vector joint probabilities delta.
+* - __u8
+ - ``mv.sign[2]``
+ - Motion vector sign probabilities delta.
+* - __u8
+ - ``mv.class[2][10]``
+ - Motion vector class probabilities delta.
+* - __u8
+ - ``mv.class0_bit[2]``
+ - Motion vector class0 bit probabilities delta.
+* - __u8
+ - ``mv.bits[2][10]``
+ - Motion vector bits probabilities delta.
+* - __u8
+ - ``mv.class0_fr[2][2][3]``
+ - Motion vector class0 fractional bit probabilities delta.
+* - __u8
+ - ``mv.fr[2][3]``
+ - Motion vector fractional bit probabilities delta.
+* - __u8
+ - ``mv.class0_hp[2]``
+ - Motion vector class0 high precision fractional bit probabilities delta.
+* - __u8
+ - ``mv.hp[2]``
+ - Motion vector high precision fractional bit probabilities delta.
+
+``V4L2_CID_STATELESS_VP9_FRAME_DECODE_PARAMS (struct)``
+Specifies the frame parameters for the associated VP9 frame decode request.
+This includes the
