Re: [RFC PATCH v4 00/42] Color Pipeline API w/ VKMS

2024-06-11 Thread Harry Wentland



On 2024-06-05 06:04, Melissa Wen wrote:
> On 02/26, Harry Wentland wrote:
>> This is an RFC set for a color pipeline API, along with a sample
>> implementation in VKMS. All the key API bits are here. VKMS now
>> supports two named transfer function colorops and two matrix
>> colorops. We have IGT tests that check all four of these colorops
>> with a pixel-by-pixel comparison that checks that these colorops
>> do what we expect them to do with a +/- 1 8 bpc code point margin.
>>
>> The big new change with v4 is the addition of an amdgpu color
>> pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support
>> the following:
>>
>> 1. 1D Curve EOTF
>> 2. 3x4 CTM
>> 3. Multiplier
>> 4. 1D Curve Inverse EOTF
>> 5. 1D LUT
>> 6. 1D Curve EOTF
>> 7. 1D LUT
>>
>> The supported curves for the 1D Curve type are:
>> - sRGB EOTF and its inverse
>> - PQ EOTF, scaled to [0.0, 125.0] and its inverse
>> - BT.2020/BT.709 OETF and its inverse
> 
> So, as we talked in the 2024 Linux Display Next Hackfest, I hacked
> `drm_info` to show the KMS color pipeline properties. You can find the
> experimental-and-ugly code here:
> - https://gitlab.freedesktop.org/mwen/drm_info/-/merge_requests/1
> It depends on updating libdrm [1] and you will only see something if you
> use a custom kernel with this series applied.
> 

Thanks. That's very useful.

> After checking the output, I missed a kind of Default or "Identity"
> curve for the `CURVE_1D_TYPE` enum. I understand that if the color
> operation is set bypass, we can ignore all property values, but I didn't
> find a similar approach on plane properties, so it looks weird to me:
> 
> └───"CURVE_1D_TYPE" (atomic): enum {sRGB Inverse EOTF, BT.2020 OETF, PQ 125 
> Inverse EOTF} = invalid (0)
> 

Good catch. That's where drm_info is useful. We shouldn't be reporting
"invalid (0)" but default to any other value. I'll fix that.

While your suggestion would work for AMD I'm a bit reluctant to add an
Identity element. It doesn't add value over "BYPASS". But more importantly
we discussed that there might be some HW that doesn't support HW on
colorops. It would be better if we avoid assumptions about the availability
of identity or bypass.

> Another thing that caught my attention was the size property for 1D
> Curve Custom LUT, that I expected a similar setup to CRTC 1D LUTs:
> 
> └───"GAMMA_LUT_SIZE" (immutable): range [0, UINT32_MAX] = 4096
> 
> But I got:
> 
> ├───"SIZE" (atomic): range [4096, 4096] = 4096
> 
> Any thoughts?

We should stick with same behavior as GAMMA_LUT_SIZE. I'll change
that.

> 
> Anyway, see below an example of `drm_info` output on AMD DCN301 for a
> given Overlay/Primary plane without userspace usage of the properties:
> 
> │   └───"COLOR_PIPELINE" (atomic): enum {Bypass, Color Pipeline 265} 
> = Bypass
> │   ├───Bypass
> │   └───Color Pipeline 265
> │   ├───Color Operation 265
> │   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve 
> Custom LUT, 3x4 Matrix, Multiplier} = 1D Curve
> │   │   ├───"BYPASS" (atomic): range [0, 1] = 1
> │   │   └───"CURVE_1D_TYPE" (atomic): enum {sRGB EOTF, 
> BT.2020 Inverse OETF, PQ 125 EOTF} = sRGB EOTF
> │   ├───Color Operation 270
> │   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve 
> Custom LUT, 3x4 Matrix, Multiplier} = 3x4 Matrix
> │   │   ├───"BYPASS" (atomic): range [0, 1] = 1
> │   │   └───"DATA" (atomic): blob = 0
> │   ├───Color Operation 275
> │   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve 
> Custom LUT, 3x4 Matrix, Multiplier} = Multiplier
> │   │   ├───"BYPASS" (atomic): range [0, 1] = 1
> │   │   └───"MULTIPLIER" (atomic): range [0, UINT64_MAX] = 0
> │   ├───Color Operation 280
> │   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve 
> Custom LUT, 3x4 Matrix, Multiplier} = 1D Curve
> │   │   ├───"BYPASS" (atomic): range [0, 1] = 1
> │   │   └───"CURVE_1D_TYPE" (atomic): enum {sRGB Inverse 
> EOTF, BT.2020 OETF, PQ 125 Inverse EOTF} = invalid (0)
> │   ├───Color Operation 285
> │   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve 
> Custom LUT, 3x4 Matrix, Multiplier} = 1D Curve Custom LUT
> │   │   ├───"BYPASS" (atomic): range [0, 1] = 1
> │   │   ├───"SIZE" (atomic): range [4096, 4096] = 4096
> │   │   └───"DATA" (atomic): blob = 0
> │   ├───Color Operation 291
> │   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve 
> Custom LUT, 3x4 Matrix, Multiplier} = 1D Curve
> │   │   ├───"BYPASS" (atomic): range [0, 1] = 1
> │   │   └───"CURVE_1D_TYPE" (atomic): enum {sRGB EOTF, 
> BT.2020 Inverse OETF, PQ 125 EOTF} = sRGB EOTF
> │   └───Color 

Re: [RFC PATCH v4 00/42] Color Pipeline API w/ VKMS

2024-06-05 Thread Melissa Wen
On 02/26, Harry Wentland wrote:
> This is an RFC set for a color pipeline API, along with a sample
> implementation in VKMS. All the key API bits are here. VKMS now
> supports two named transfer function colorops and two matrix
> colorops. We have IGT tests that check all four of these colorops
> with a pixel-by-pixel comparison that checks that these colorops
> do what we expect them to do with a +/- 1 8 bpc code point margin.
> 
> The big new change with v4 is the addition of an amdgpu color
> pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support
> the following:
> 
> 1. 1D Curve EOTF
> 2. 3x4 CTM
> 3. Multiplier
> 4. 1D Curve Inverse EOTF
> 5. 1D LUT
> 6. 1D Curve EOTF
> 7. 1D LUT
> 
> The supported curves for the 1D Curve type are:
> - sRGB EOTF and its inverse
> - PQ EOTF, scaled to [0.0, 125.0] and its inverse
> - BT.2020/BT.709 OETF and its inverse

So, as we talked in the 2024 Linux Display Next Hackfest, I hacked
`drm_info` to show the KMS color pipeline properties. You can find the
experimental-and-ugly code here:
- https://gitlab.freedesktop.org/mwen/drm_info/-/merge_requests/1
It depends on updating libdrm [1] and you will only see something if you
use a custom kernel with this series applied.

After checking the output, I missed a kind of Default or "Identity"
curve for the `CURVE_1D_TYPE` enum. I understand that if the color
operation is set bypass, we can ignore all property values, but I didn't
find a similar approach on plane properties, so it looks weird to me:

└───"CURVE_1D_TYPE" (atomic): enum {sRGB Inverse EOTF, BT.2020 OETF, PQ 125 
Inverse EOTF} = invalid (0)

Another thing that caught my attention was the size property for 1D
Curve Custom LUT, that I expected a similar setup to CRTC 1D LUTs:

└───"GAMMA_LUT_SIZE" (immutable): range [0, UINT32_MAX] = 4096

But I got:

├───"SIZE" (atomic): range [4096, 4096] = 4096

Any thoughts?

Anyway, see below an example of `drm_info` output on AMD DCN301 for a
given Overlay/Primary plane without userspace usage of the properties:

│   └───"COLOR_PIPELINE" (atomic): enum {Bypass, Color Pipeline 265} = 
Bypass
│   ├───Bypass
│   └───Color Pipeline 265
│   ├───Color Operation 265
│   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve Custom 
LUT, 3x4 Matrix, Multiplier} = 1D Curve
│   │   ├───"BYPASS" (atomic): range [0, 1] = 1
│   │   └───"CURVE_1D_TYPE" (atomic): enum {sRGB EOTF, BT.2020 
Inverse OETF, PQ 125 EOTF} = sRGB EOTF
│   ├───Color Operation 270
│   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve Custom 
LUT, 3x4 Matrix, Multiplier} = 3x4 Matrix
│   │   ├───"BYPASS" (atomic): range [0, 1] = 1
│   │   └───"DATA" (atomic): blob = 0
│   ├───Color Operation 275
│   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve Custom 
LUT, 3x4 Matrix, Multiplier} = Multiplier
│   │   ├───"BYPASS" (atomic): range [0, 1] = 1
│   │   └───"MULTIPLIER" (atomic): range [0, UINT64_MAX] = 0
│   ├───Color Operation 280
│   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve Custom 
LUT, 3x4 Matrix, Multiplier} = 1D Curve
│   │   ├───"BYPASS" (atomic): range [0, 1] = 1
│   │   └───"CURVE_1D_TYPE" (atomic): enum {sRGB Inverse EOTF, 
BT.2020 OETF, PQ 125 Inverse EOTF} = invalid (0)
│   ├───Color Operation 285
│   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve Custom 
LUT, 3x4 Matrix, Multiplier} = 1D Curve Custom LUT
│   │   ├───"BYPASS" (atomic): range [0, 1] = 1
│   │   ├───"SIZE" (atomic): range [4096, 4096] = 4096
│   │   └───"DATA" (atomic): blob = 0
│   ├───Color Operation 291
│   │   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve Custom 
LUT, 3x4 Matrix, Multiplier} = 1D Curve
│   │   ├───"BYPASS" (atomic): range [0, 1] = 1
│   │   └───"CURVE_1D_TYPE" (atomic): enum {sRGB EOTF, BT.2020 
Inverse OETF, PQ 125 EOTF} = sRGB EOTF
│   └───Color Operation 296
│   ├───"TYPE" (immutable): enum {1D Curve, 1D Curve Custom 
LUT, 3x4 Matrix, Multiplier} = 1D Curve Custom LUT
│   ├───"BYPASS" (atomic): range [0, 1] = 1
│   ├───"SIZE" (atomic): range [4096, 4096] = 4096
│   └───"DATA" (atomic): blob = 0

It's a WIP and the output still needs to be reviewed. So feel free to
point out any mistake.

BR,

Melissa

> 
> Note that the 1st and 5th colorops take the EOTF or Inverse
> OETF while the 3rd colorop takes the Inverse EOTF or OETF.
> 
> We are working on two more ops for amdgpu, the HDR multiplier
> and the 3DLUT, which will give us this:
> 
> 1. 1D Curve EOTF
> 2. 3x4 CTM
> 3. HDR Multiplier
> 4. 1D Curve Inverse EOTF

Re: [RFC PATCH v4 00/42] Color Pipeline API w/ VKMS

2024-05-21 Thread Melissa Wen
On 02/26, Harry Wentland wrote:
> This is an RFC set for a color pipeline API, along with a sample
> implementation in VKMS. All the key API bits are here. VKMS now
> supports two named transfer function colorops and two matrix
> colorops. We have IGT tests that check all four of these colorops
> with a pixel-by-pixel comparison that checks that these colorops
> do what we expect them to do with a +/- 1 8 bpc code point margin.
> 
> The big new change with v4 is the addition of an amdgpu color
> pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support
> the following:
> 
> 1. 1D Curve EOTF
> 2. 3x4 CTM
> 3. Multiplier
> 4. 1D Curve Inverse EOTF
> 5. 1D LUT
> 6. 1D Curve EOTF
> 7. 1D LUT
> 
> The supported curves for the 1D Curve type are:
> - sRGB EOTF and its inverse
> - PQ EOTF, scaled to [0.0, 125.0] and its inverse
> - BT.2020/BT.709 OETF and its inverse
> 
> Note that the 1st and 5th colorops take the EOTF or Inverse
> OETF while the 3rd colorop takes the Inverse EOTF or OETF.
> 
> We are working on two more ops for amdgpu, the HDR multiplier
> and the 3DLUT, which will give us this:
> 
> 1. 1D Curve EOTF
> 2. 3x4 CTM
> 3. HDR Multiplier
> 4. 1D Curve Inverse EOTF
> 5. 1D LUT
> 6. 3D LUT
> 7. 1D Curve EOTF
> 8. 1D LUT
> 
> This, essentially mirrors the color pipeline used by gamescope
> and presented by Melissa Wen, with the exception of the DEGAM
> LUT, which is not currently used. See
> [1] 
> https://indico.freedesktop.org/event/4/contributions/186/attachments/138/218/xdc2023-TheRainbowTreasureMap-MelissaWen.pdf
> 
> After this we'd like to also add the following ops:
> - Scaler (Informational only)
> - Color Encoding, to replace drm_plane's COLOR_ENCODING
> - Color Range, to replace drm_plane's COLOR_RANGE
> 
> This patchset is grouped as follows:
>  - Patches 1-3: couple general patches/fixes
>  - Patches 4-7: introduce kunit to VKMS
>  - Patch 7: description of motivation and details behind the
> Color Pipeline API. If you're reading nothing else
> but are interested in the topic I highly recommend
> you take a look at this.
>  - Patches 7-27: DRM core and VKMS changes for color pipeline API
>  - Patches 28-40: DRM core and amdgpu changes for color pipeline API
> 
> VKMS patches could still be improved in a few ways, though the
> payoff might be limited and I would rather focus on other work
> at the moment. The most obvious thing to improve would be to
> eliminate the hard-coded LUTs for identity, and sRGB, and replace
> them with fixed-point math instead.
> 
> There are plenty of things that I would like to see here but
> haven't had a chance to look at. These will (hopefully) be
> addressed in future iterations, either in VKMS or amdgpu:
>  - Clear documentation for each drm_colorop_type
>  - Add custom LUT colorops to VKMS
>  - Add pre-blending 3DLUT
>  - How to support HW which can't bypass entire pipeline?
>  - Add ability to create colorops that don't have BYPASS
>  - Can we do a LOAD / COMMIT model for LUTs (and other properties)?
>  - read-only scaling colorop which defines scaling taps and position
>  - read-only color format colorop to define supported color formats
>for a pipeline
>  - named matrices, for things like converting YUV to RGB
> 
> IGT tests can be found at
> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1
> 
> IGT patches are also being sent to the igt-dev mailing list.
> 
> If you prefer a gitlab MR for review you can find it at
> https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5
> 
> v4:

Hey,

I'm not sure if this is the latest version, but this is the one I'm
using now. I just pointed out some minor issues that I found when
applying this series for testing, so I won't forget to report. But I'm
still performing tests and validations on my side.

Melissa

>  - Add amdgpu color pipeline (WIP)
>  - Don't block setting of deprecated properties, instead pass client cap
>to atomic check so drivers can ignore these props
>  - Drop IOCTL definitions (Pekka)
>  - Use enum property for colorop TYPE (Pekka)
>  - A few cleanups to the docs (Pekka)
>  - Rework the TYPE enum to name relation to avoid code duplication (Pekka)
>  - Add missing function declarations (Chaitanya Kumar Borah)
>  - Allow setting of NEXT property to NULL in _set_ function (Chaitanya Kumar 
> Borah)
>  - Add helper for creation of pipeline drm_plane property (Pekka)
>  - Always create Bypass pipeline (Pekka)
>  - A bunch of changes to VKMS kunit tests (Pekka)
>  - Fix index in CTM doc (Pekka)
> 
> v3:
>  - Abandon IOCTLs and discover colorops as clients iterate the pipeline
>  - Remove need for libdrm
>  - Add color_pipeline client cap and make mutually exclusive with
>COLOR_RANGE and COLOR_ENCODING properties
>  - add CTM colorop to VKMS
>  - Use include way for kunit testing static functions (Arthur)
>  - Make TYPE a range property
>  - Move enum drm_colorop_type to uapi header
>  - and a bunch of smaller 

Re: [RFC PATCH v4 00/42] Color Pipeline API w/ VKMS

2024-02-29 Thread Joshua Ashton




On 2/29/24 10:43, Daniel Vetter wrote:

On Mon, Feb 26, 2024 at 04:10:14PM -0500, Harry Wentland wrote:

This is an RFC set for a color pipeline API, along with a sample
implementation in VKMS. All the key API bits are here. VKMS now
supports two named transfer function colorops and two matrix
colorops. We have IGT tests that check all four of these colorops
with a pixel-by-pixel comparison that checks that these colorops
do what we expect them to do with a +/- 1 8 bpc code point margin.


So vkms is definitely great to make sure the igts are generic enough and
somewhat useful, but ... does steam run on vkms too? I think that would be
a really good test to show that the api we have here is actually useful
for compositors in a cross-driver way, and not just a neat idea that
doesn't survive harsh reality.

And yes I realize that's probably going to be a bunch of work, but I feel
like the color pipeline discussion has dragged around enough in
hypotheticals and concerns that I think it would really help a lot.


I don't think we have ever tested Steam/Gamescope on vkms.

The last time I tried stuff there, there was all the problems with the 
ttm page table tail thing for virtio stuff that made using Steam games + 
Gamescope unfeasable because Vulkan + bindless, but I have heard those 
are solved now?


I will have to try it again at the weekend and see where it's at.
I am willing to place my bets that some part of the stack will fall over 
relating to modifiers somehow... =P


But yes, testing there would be good too, as we have the full Steam Deck 
OLED HDR color pipeline implemented in shader-based composition 
validated as being 1:1 on a suite of HDR and SDR test images.
(That *will* definitely rely on 3D LUTs being tetrahedrally interpolated 
though)


I'll have a look at this at the weekend and also see about getting a 
Gamescope branch that uses the new wip colorop stuff.


- Joshie ✨



Thoughts?
-Sima



The big new change with v4 is the addition of an amdgpu color
pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support
the following:

1. 1D Curve EOTF
2. 3x4 CTM
3. Multiplier
4. 1D Curve Inverse EOTF
5. 1D LUT
6. 1D Curve EOTF
7. 1D LUT

The supported curves for the 1D Curve type are:
- sRGB EOTF and its inverse
- PQ EOTF, scaled to [0.0, 125.0] and its inverse
- BT.2020/BT.709 OETF and its inverse

Note that the 1st and 5th colorops take the EOTF or Inverse
OETF while the 3rd colorop takes the Inverse EOTF or OETF.

We are working on two more ops for amdgpu, the HDR multiplier
and the 3DLUT, which will give us this:

1. 1D Curve EOTF
2. 3x4 CTM
3. HDR Multiplier
4. 1D Curve Inverse EOTF
5. 1D LUT
6. 3D LUT
7. 1D Curve EOTF
8. 1D LUT

This, essentially mirrors the color pipeline used by gamescope
and presented by Melissa Wen, with the exception of the DEGAM
LUT, which is not currently used. See
[1] 
https://indico.freedesktop.org/event/4/contributions/186/attachments/138/218/xdc2023-TheRainbowTreasureMap-MelissaWen.pdf

After this we'd like to also add the following ops:
- Scaler (Informational only)
- Color Encoding, to replace drm_plane's COLOR_ENCODING
- Color Range, to replace drm_plane's COLOR_RANGE

This patchset is grouped as follows:
  - Patches 1-3: couple general patches/fixes
  - Patches 4-7: introduce kunit to VKMS
  - Patch 7: description of motivation and details behind the
 Color Pipeline API. If you're reading nothing else
 but are interested in the topic I highly recommend
 you take a look at this.
  - Patches 7-27: DRM core and VKMS changes for color pipeline API
  - Patches 28-40: DRM core and amdgpu changes for color pipeline API

VKMS patches could still be improved in a few ways, though the
payoff might be limited and I would rather focus on other work
at the moment. The most obvious thing to improve would be to
eliminate the hard-coded LUTs for identity, and sRGB, and replace
them with fixed-point math instead.

There are plenty of things that I would like to see here but
haven't had a chance to look at. These will (hopefully) be
addressed in future iterations, either in VKMS or amdgpu:
  - Clear documentation for each drm_colorop_type
  - Add custom LUT colorops to VKMS
  - Add pre-blending 3DLUT
  - How to support HW which can't bypass entire pipeline?
  - Add ability to create colorops that don't have BYPASS
  - Can we do a LOAD / COMMIT model for LUTs (and other properties)?
  - read-only scaling colorop which defines scaling taps and position
  - read-only color format colorop to define supported color formats
for a pipeline
  - named matrices, for things like converting YUV to RGB

IGT tests can be found at
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1

IGT patches are also being sent to the igt-dev mailing list.

If you prefer a gitlab MR for review you can find it at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5

v4:
  - Add amdgpu color pipeline (WIP)
  - Don't 

Re: [RFC PATCH v4 00/42] Color Pipeline API w/ VKMS

2024-02-29 Thread Daniel Vetter
On Mon, Feb 26, 2024 at 04:10:14PM -0500, Harry Wentland wrote:
> This is an RFC set for a color pipeline API, along with a sample
> implementation in VKMS. All the key API bits are here. VKMS now
> supports two named transfer function colorops and two matrix
> colorops. We have IGT tests that check all four of these colorops
> with a pixel-by-pixel comparison that checks that these colorops
> do what we expect them to do with a +/- 1 8 bpc code point margin.

So vkms is definitely great to make sure the igts are generic enough and
somewhat useful, but ... does steam run on vkms too? I think that would be
a really good test to show that the api we have here is actually useful
for compositors in a cross-driver way, and not just a neat idea that
doesn't survive harsh reality.

And yes I realize that's probably going to be a bunch of work, but I feel
like the color pipeline discussion has dragged around enough in
hypotheticals and concerns that I think it would really help a lot.

Thoughts?
-Sima

> 
> The big new change with v4 is the addition of an amdgpu color
> pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support
> the following:
> 
> 1. 1D Curve EOTF
> 2. 3x4 CTM
> 3. Multiplier
> 4. 1D Curve Inverse EOTF
> 5. 1D LUT
> 6. 1D Curve EOTF
> 7. 1D LUT
> 
> The supported curves for the 1D Curve type are:
> - sRGB EOTF and its inverse
> - PQ EOTF, scaled to [0.0, 125.0] and its inverse
> - BT.2020/BT.709 OETF and its inverse
> 
> Note that the 1st and 5th colorops take the EOTF or Inverse
> OETF while the 3rd colorop takes the Inverse EOTF or OETF.
> 
> We are working on two more ops for amdgpu, the HDR multiplier
> and the 3DLUT, which will give us this:
> 
> 1. 1D Curve EOTF
> 2. 3x4 CTM
> 3. HDR Multiplier
> 4. 1D Curve Inverse EOTF
> 5. 1D LUT
> 6. 3D LUT
> 7. 1D Curve EOTF
> 8. 1D LUT
> 
> This, essentially mirrors the color pipeline used by gamescope
> and presented by Melissa Wen, with the exception of the DEGAM
> LUT, which is not currently used. See
> [1] 
> https://indico.freedesktop.org/event/4/contributions/186/attachments/138/218/xdc2023-TheRainbowTreasureMap-MelissaWen.pdf
> 
> After this we'd like to also add the following ops:
> - Scaler (Informational only)
> - Color Encoding, to replace drm_plane's COLOR_ENCODING
> - Color Range, to replace drm_plane's COLOR_RANGE
> 
> This patchset is grouped as follows:
>  - Patches 1-3: couple general patches/fixes
>  - Patches 4-7: introduce kunit to VKMS
>  - Patch 7: description of motivation and details behind the
> Color Pipeline API. If you're reading nothing else
> but are interested in the topic I highly recommend
> you take a look at this.
>  - Patches 7-27: DRM core and VKMS changes for color pipeline API
>  - Patches 28-40: DRM core and amdgpu changes for color pipeline API
> 
> VKMS patches could still be improved in a few ways, though the
> payoff might be limited and I would rather focus on other work
> at the moment. The most obvious thing to improve would be to
> eliminate the hard-coded LUTs for identity, and sRGB, and replace
> them with fixed-point math instead.
> 
> There are plenty of things that I would like to see here but
> haven't had a chance to look at. These will (hopefully) be
> addressed in future iterations, either in VKMS or amdgpu:
>  - Clear documentation for each drm_colorop_type
>  - Add custom LUT colorops to VKMS
>  - Add pre-blending 3DLUT
>  - How to support HW which can't bypass entire pipeline?
>  - Add ability to create colorops that don't have BYPASS
>  - Can we do a LOAD / COMMIT model for LUTs (and other properties)?
>  - read-only scaling colorop which defines scaling taps and position
>  - read-only color format colorop to define supported color formats
>for a pipeline
>  - named matrices, for things like converting YUV to RGB
> 
> IGT tests can be found at
> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1
> 
> IGT patches are also being sent to the igt-dev mailing list.
> 
> If you prefer a gitlab MR for review you can find it at
> https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5
> 
> v4:
>  - Add amdgpu color pipeline (WIP)
>  - Don't block setting of deprecated properties, instead pass client cap
>to atomic check so drivers can ignore these props
>  - Drop IOCTL definitions (Pekka)
>  - Use enum property for colorop TYPE (Pekka)
>  - A few cleanups to the docs (Pekka)
>  - Rework the TYPE enum to name relation to avoid code duplication (Pekka)
>  - Add missing function declarations (Chaitanya Kumar Borah)
>  - Allow setting of NEXT property to NULL in _set_ function (Chaitanya Kumar 
> Borah)
>  - Add helper for creation of pipeline drm_plane property (Pekka)
>  - Always create Bypass pipeline (Pekka)
>  - A bunch of changes to VKMS kunit tests (Pekka)
>  - Fix index in CTM doc (Pekka)
> 
> v3:
>  - Abandon IOCTLs and discover colorops as clients iterate the pipeline
>  - Remove need for libdrm
>  - Add 

Re: [RFC PATCH v4 00/42] Color Pipeline API w/ VKMS

2024-02-27 Thread Harry Wentland



On 2024-02-27 05:26, Joshua Ashton wrote:
> 
> 
> On 2/26/24 21:10, Harry Wentland wrote:
>> This is an RFC set for a color pipeline API, along with a sample
>> implementation in VKMS. All the key API bits are here. VKMS now
>> supports two named transfer function colorops and two matrix
>> colorops. We have IGT tests that check all four of these colorops
>> with a pixel-by-pixel comparison that checks that these colorops
>> do what we expect them to do with a +/- 1 8 bpc code point margin.
>>
>> The big new change with v4 is the addition of an amdgpu color
>> pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support
>> the following:
>>
>> 1. 1D Curve EOTF
>> 2. 3x4 CTM
>> 3. Multiplier
>> 4. 1D Curve Inverse EOTF
>> 5. 1D LUT
>> 6. 1D Curve EOTF
>> 7. 1D LUT
>>
>> The supported curves for the 1D Curve type are:
>> - sRGB EOTF and its inverse
>> - PQ EOTF, scaled to [0.0, 125.0] and its inverse
>> - BT.2020/BT.709 OETF and its inverse
>>
>> Note that the 1st and 5th colorops take the EOTF or Inverse
>> OETF while the 3rd colorop takes the Inverse EOTF or OETF.
>>
>> We are working on two more ops for amdgpu, the HDR multiplier
>> and the 3DLUT, which will give us this:
>>
>> 1. 1D Curve EOTF
>> 2. 3x4 CTM
>> 3. HDR Multiplier
>> 4. 1D Curve Inverse EOTF
>> 5. 1D LUT
>> 6. 3D LUT
>> 7. 1D Curve EOTF
>> 8. 1D LUT
>>
>> This, essentially mirrors the color pipeline used by gamescope
>> and presented by Melissa Wen, with the exception of the DEGAM
>> LUT, which is not currently used. See
>> [1] 
>> https://indico.freedesktop.org/event/4/contributions/186/attachments/138/218/xdc2023-TheRainbowTreasureMap-MelissaWen.pdf
>>
>> After this we'd like to also add the following ops:
>> - Scaler (Informational only)
> 
> Why informational only? Having NEAREST and in general custom taps should be 
> possible on AMDGPU right?
> 
> We don't have to solve this now, but I just want to make sure that we aren't 
> locking this to info only.
> 

No, this isn't locking it to information only. We could allow for NEAREST or
even custom taps in the future. Just don't want to open that debate now if
we don't have a good reason to.

Harry

> Thanks
> 
> - Joshie ✨
> 
>> - Color Encoding, to replace drm_plane's COLOR_ENCODING
>> - Color Range, to replace drm_plane's COLOR_RANGE
>>
>> This patchset is grouped as follows:
>>   - Patches 1-3: couple general patches/fixes
>>   - Patches 4-7: introduce kunit to VKMS
>>   - Patch 7: description of motivation and details behind the
>>  Color Pipeline API. If you're reading nothing else
>>  but are interested in the topic I highly recommend
>>  you take a look at this.
>>   - Patches 7-27: DRM core and VKMS changes for color pipeline API
>>   - Patches 28-40: DRM core and amdgpu changes for color pipeline API
>>
>> VKMS patches could still be improved in a few ways, though the
>> payoff might be limited and I would rather focus on other work
>> at the moment. The most obvious thing to improve would be to
>> eliminate the hard-coded LUTs for identity, and sRGB, and replace
>> them with fixed-point math instead.
>>
>> There are plenty of things that I would like to see here but
>> haven't had a chance to look at. These will (hopefully) be
>> addressed in future iterations, either in VKMS or amdgpu:
>>   - Clear documentation for each drm_colorop_type
>>   - Add custom LUT colorops to VKMS
>>   - Add pre-blending 3DLUT
>>   - How to support HW which can't bypass entire pipeline?
>>   - Add ability to create colorops that don't have BYPASS
>>   - Can we do a LOAD / COMMIT model for LUTs (and other properties)?
>>   - read-only scaling colorop which defines scaling taps and position
>>   - read-only color format colorop to define supported color formats
>>     for a pipeline
>>   - named matrices, for things like converting YUV to RGB
>>
>> IGT tests can be found at
>> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1
>>
>> IGT patches are also being sent to the igt-dev mailing list.
>>
>> If you prefer a gitlab MR for review you can find it at
>> https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5
>>
>> v4:
>>   - Add amdgpu color pipeline (WIP)
>>   - Don't block setting of deprecated properties, instead pass client cap
>>     to atomic check so drivers can ignore these props
>>   - Drop IOCTL definitions (Pekka)
>>   - Use enum property for colorop TYPE (Pekka)
>>   - A few cleanups to the docs (Pekka)
>>   - Rework the TYPE enum to name relation to avoid code duplication (Pekka)
>>   - Add missing function declarations (Chaitanya Kumar Borah)
>>   - Allow setting of NEXT property to NULL in _set_ function (Chaitanya 
>> Kumar Borah)
>>   - Add helper for creation of pipeline drm_plane property (Pekka)
>>   - Always create Bypass pipeline (Pekka)
>>   - A bunch of changes to VKMS kunit tests (Pekka)
>>   - Fix index in CTM doc (Pekka)
>>
>> v3:
>>   - Abandon IOCTLs and discover colorops as clients iterate 

Re: [RFC PATCH v4 00/42] Color Pipeline API w/ VKMS

2024-02-27 Thread Joshua Ashton




On 2/26/24 21:10, Harry Wentland wrote:

This is an RFC set for a color pipeline API, along with a sample
implementation in VKMS. All the key API bits are here. VKMS now
supports two named transfer function colorops and two matrix
colorops. We have IGT tests that check all four of these colorops
with a pixel-by-pixel comparison that checks that these colorops
do what we expect them to do with a +/- 1 8 bpc code point margin.

The big new change with v4 is the addition of an amdgpu color
pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support
the following:

1. 1D Curve EOTF
2. 3x4 CTM
3. Multiplier
4. 1D Curve Inverse EOTF
5. 1D LUT
6. 1D Curve EOTF
7. 1D LUT

The supported curves for the 1D Curve type are:
- sRGB EOTF and its inverse
- PQ EOTF, scaled to [0.0, 125.0] and its inverse
- BT.2020/BT.709 OETF and its inverse

Note that the 1st and 5th colorops take the EOTF or Inverse
OETF while the 3rd colorop takes the Inverse EOTF or OETF.

We are working on two more ops for amdgpu, the HDR multiplier
and the 3DLUT, which will give us this:

1. 1D Curve EOTF
2. 3x4 CTM
3. HDR Multiplier
4. 1D Curve Inverse EOTF
5. 1D LUT
6. 3D LUT
7. 1D Curve EOTF
8. 1D LUT

This, essentially mirrors the color pipeline used by gamescope
and presented by Melissa Wen, with the exception of the DEGAM
LUT, which is not currently used. See
[1] 
https://indico.freedesktop.org/event/4/contributions/186/attachments/138/218/xdc2023-TheRainbowTreasureMap-MelissaWen.pdf

After this we'd like to also add the following ops:
- Scaler (Informational only)


Why informational only? Having NEAREST and in general custom taps should 
be possible on AMDGPU right?


We don't have to solve this now, but I just want to make sure that we 
aren't locking this to info only.


Thanks

- Joshie ✨


- Color Encoding, to replace drm_plane's COLOR_ENCODING
- Color Range, to replace drm_plane's COLOR_RANGE

This patchset is grouped as follows:
  - Patches 1-3: couple general patches/fixes
  - Patches 4-7: introduce kunit to VKMS
  - Patch 7: description of motivation and details behind the
 Color Pipeline API. If you're reading nothing else
 but are interested in the topic I highly recommend
 you take a look at this.
  - Patches 7-27: DRM core and VKMS changes for color pipeline API
  - Patches 28-40: DRM core and amdgpu changes for color pipeline API

VKMS patches could still be improved in a few ways, though the
payoff might be limited and I would rather focus on other work
at the moment. The most obvious thing to improve would be to
eliminate the hard-coded LUTs for identity, and sRGB, and replace
them with fixed-point math instead.

There are plenty of things that I would like to see here but
haven't had a chance to look at. These will (hopefully) be
addressed in future iterations, either in VKMS or amdgpu:
  - Clear documentation for each drm_colorop_type
  - Add custom LUT colorops to VKMS
  - Add pre-blending 3DLUT
  - How to support HW which can't bypass entire pipeline?
  - Add ability to create colorops that don't have BYPASS
  - Can we do a LOAD / COMMIT model for LUTs (and other properties)?
  - read-only scaling colorop which defines scaling taps and position
  - read-only color format colorop to define supported color formats
for a pipeline
  - named matrices, for things like converting YUV to RGB

IGT tests can be found at
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1

IGT patches are also being sent to the igt-dev mailing list.

If you prefer a gitlab MR for review you can find it at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5

v4:
  - Add amdgpu color pipeline (WIP)
  - Don't block setting of deprecated properties, instead pass client cap
to atomic check so drivers can ignore these props
  - Drop IOCTL definitions (Pekka)
  - Use enum property for colorop TYPE (Pekka)
  - A few cleanups to the docs (Pekka)
  - Rework the TYPE enum to name relation to avoid code duplication (Pekka)
  - Add missing function declarations (Chaitanya Kumar Borah)
  - Allow setting of NEXT property to NULL in _set_ function (Chaitanya Kumar 
Borah)
  - Add helper for creation of pipeline drm_plane property (Pekka)
  - Always create Bypass pipeline (Pekka)
  - A bunch of changes to VKMS kunit tests (Pekka)
  - Fix index in CTM doc (Pekka)

v3:
  - Abandon IOCTLs and discover colorops as clients iterate the pipeline
  - Remove need for libdrm
  - Add color_pipeline client cap and make mutually exclusive with
COLOR_RANGE and COLOR_ENCODING properties
  - add CTM colorop to VKMS
  - Use include way for kunit testing static functions (Arthur)
  - Make TYPE a range property
  - Move enum drm_colorop_type to uapi header
  - and a bunch of smaller bits that are highlighted in the relevant commit
description

v2:
  - Rebased on drm-misc-next
  - Introduce a VKMS Kunit so we can test LUT functionality in vkms_composer
  - Incorporate feedback in 

[RFC PATCH v4 00/42] Color Pipeline API w/ VKMS

2024-02-26 Thread Harry Wentland
This is an RFC set for a color pipeline API, along with a sample
implementation in VKMS. All the key API bits are here. VKMS now
supports two named transfer function colorops and two matrix
colorops. We have IGT tests that check all four of these colorops
with a pixel-by-pixel comparison that checks that these colorops
do what we expect them to do with a +/- 1 8 bpc code point margin.

The big new change with v4 is the addition of an amdgpu color
pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support
the following:

1. 1D Curve EOTF
2. 3x4 CTM
3. Multiplier
4. 1D Curve Inverse EOTF
5. 1D LUT
6. 1D Curve EOTF
7. 1D LUT

The supported curves for the 1D Curve type are:
- sRGB EOTF and its inverse
- PQ EOTF, scaled to [0.0, 125.0] and its inverse
- BT.2020/BT.709 OETF and its inverse

Note that the 1st and 5th colorops take the EOTF or Inverse
OETF while the 3rd colorop takes the Inverse EOTF or OETF.

We are working on two more ops for amdgpu, the HDR multiplier
and the 3DLUT, which will give us this:

1. 1D Curve EOTF
2. 3x4 CTM
3. HDR Multiplier
4. 1D Curve Inverse EOTF
5. 1D LUT
6. 3D LUT
7. 1D Curve EOTF
8. 1D LUT

This, essentially mirrors the color pipeline used by gamescope
and presented by Melissa Wen, with the exception of the DEGAM
LUT, which is not currently used. See
[1] 
https://indico.freedesktop.org/event/4/contributions/186/attachments/138/218/xdc2023-TheRainbowTreasureMap-MelissaWen.pdf

After this we'd like to also add the following ops:
- Scaler (Informational only)
- Color Encoding, to replace drm_plane's COLOR_ENCODING
- Color Range, to replace drm_plane's COLOR_RANGE

This patchset is grouped as follows:
 - Patches 1-3: couple general patches/fixes
 - Patches 4-7: introduce kunit to VKMS
 - Patch 7: description of motivation and details behind the
Color Pipeline API. If you're reading nothing else
but are interested in the topic I highly recommend
you take a look at this.
 - Patches 7-27: DRM core and VKMS changes for color pipeline API
 - Patches 28-40: DRM core and amdgpu changes for color pipeline API

VKMS patches could still be improved in a few ways, though the
payoff might be limited and I would rather focus on other work
at the moment. The most obvious thing to improve would be to
eliminate the hard-coded LUTs for identity, and sRGB, and replace
them with fixed-point math instead.

There are plenty of things that I would like to see here but
haven't had a chance to look at. These will (hopefully) be
addressed in future iterations, either in VKMS or amdgpu:
 - Clear documentation for each drm_colorop_type
 - Add custom LUT colorops to VKMS
 - Add pre-blending 3DLUT
 - How to support HW which can't bypass entire pipeline?
 - Add ability to create colorops that don't have BYPASS
 - Can we do a LOAD / COMMIT model for LUTs (and other properties)?
 - read-only scaling colorop which defines scaling taps and position
 - read-only color format colorop to define supported color formats
   for a pipeline
 - named matrices, for things like converting YUV to RGB

IGT tests can be found at
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1

IGT patches are also being sent to the igt-dev mailing list.

If you prefer a gitlab MR for review you can find it at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5

v4:
 - Add amdgpu color pipeline (WIP)
 - Don't block setting of deprecated properties, instead pass client cap
   to atomic check so drivers can ignore these props
 - Drop IOCTL definitions (Pekka)
 - Use enum property for colorop TYPE (Pekka)
 - A few cleanups to the docs (Pekka)
 - Rework the TYPE enum to name relation to avoid code duplication (Pekka)
 - Add missing function declarations (Chaitanya Kumar Borah)
 - Allow setting of NEXT property to NULL in _set_ function (Chaitanya Kumar 
Borah)
 - Add helper for creation of pipeline drm_plane property (Pekka)
 - Always create Bypass pipeline (Pekka)
 - A bunch of changes to VKMS kunit tests (Pekka)
 - Fix index in CTM doc (Pekka)

v3:
 - Abandon IOCTLs and discover colorops as clients iterate the pipeline
 - Remove need for libdrm
 - Add color_pipeline client cap and make mutually exclusive with
   COLOR_RANGE and COLOR_ENCODING properties
 - add CTM colorop to VKMS
 - Use include way for kunit testing static functions (Arthur)
 - Make TYPE a range property
 - Move enum drm_colorop_type to uapi header
 - and a bunch of smaller bits that are highlighted in the relevant commit
   description

v2:
 - Rebased on drm-misc-next
 - Introduce a VKMS Kunit so we can test LUT functionality in vkms_composer
 - Incorporate feedback in color_pipeline.rst doc
 - Add support for sRGB inverse EOTF
 - Add 2nd enumerated TF colorop to VKMS
 - Fix LUTs and some issues with applying LUTs in VKMS

Cc: Ville Syrjala 
Cc: Pekka Paalanen 
Cc: Simon Ser 
Cc: Harry Wentland 
Cc: Melissa Wen 
Cc: Jonas Ådahl 
Cc: Sebastian Wick 
Cc: Shashank Sharma 
Cc: Alexander Goins 
Cc: