## Introduction

We are looking to enable HDR support for a couple of single-plane and 
multi-plane scenarios. To do this effectively we recommend new interfaces to 
drm_plane. Below I'll give a bit of background on HDR and why we propose these 
interfaces.


## Defining a pixel's luminance

Currently the luminance space of pixels in a framebuffer/plane presented to the 
display is not well defined. It's usually assumed to be in a 2.2 or 2.4 gamma 
space and has no mapping to an absolute luminance value but is interpreted in 
relative terms.

Luminance can be measured and described in absolute terms as candela per meter 
squared, or cd/m2, or nits. Even though a pixel value can be mapped to 
luminance in a linear fashion to do so without losing a lot of detail requires 
16-bpc color depth. The reason for this is that human perception can 
distinguish roughly between a 0.5-1% luminance delta. A linear representation 
is suboptimal, wasting precision in the highlights and losing precision in the 
shadows.

A gamma curve is a decent approximation to a human's perception of luminance, 
but the PQ (perceptual quantizer) function [1] improves on it. It also defines 
the luminance values in absolute terms, with the highest value being 10,000 
nits and the lowest 0.0005 nits.

Using a content that's defined in PQ space we can approximate the real world in 
a much better way.

Here are some examples of real-life objects and their approximate luminance 
values:

| Object            | Luminance in nits |
| ----------------- | ----------------- |
| Sun               | 1.6 million       |
| Fluorescent light | 10,000            |
| Highlights        | 1,000 - sunlight  |
| White Objects     | 250 - 1,000       |
| Typical objects   | 1 - 250           |
| Shadows           | 0.01 - 1          |
| Ultra Blacks      | 0 - 0.0005        |


## Describing the luminance space

**We propose a new drm_plane property to describe the Eletro-Optical Transfer 
Function (EOTF) with which its framebuffer was composed.** Examples of EOTF are:

| EOTF      | Description                                                       
        |
| --------- 
|:------------------------------------------------------------------------- |
| Gamma 2.2 | a simple 2.2 gamma                                                
        |
| sRGB      | 2.4 gamma with small initial linear section                       
        |
| PQ 2084   | SMPTE ST 2084; used for HDR video and allows for up to 10,000 nit 
support |
| Linear    | Linear relationship between pixel value and luminance value       
        |


## Mastering Luminances

Now we are able to use the PQ 2084 EOTF to define the luminance of pixels in 
absolute terms. Unfortunately we're again presented with physical limitations 
of the display technologies on the market today. Here are a few examples of 
luminance ranges of displays.

| Display                  | Luminance range in nits |
| ------------------------ | ----------------------- |
| Typical PC display       | 0.3 - 200               |
| Excellent LCD HDTV       | 0.3 - 400               |
| HDR LCD w/ local dimming | 0.05 - 1,500            |

Since no display can currently show the full 0.0005 to 10,000 nits luminance 
range the display will need to tonemap the HDR content, i.e to fit the content 
within a display's capabilities. To assist with tonemapping HDR content is 
usually accompanied with a metadata that describes (among other things) the 
minimum and maximum mastering luminance, i.e. the maximum and minimum luminance 
of the display that was used to master the HDR content.

The HDR metadata is currently defined on the drm_connector via the 
hdr_output_metadata blob property.

It might be useful to define per-plane hdr metadata, as different planes might 
have been mastered differently.


## SDR Luminance

Since SDR covers a smaller luminance range than HDR, an SDR plane might look 
dark when blended with HDR content. Since the max HDR luminance can be quite 
variable (200-1,500 nits on actual displays) it is best to make the SDR maximum 
luminance value configurable.

**We propose a drm_plane property to specfy the desired maximum luminance of 
the SDR plane in nits.** This allows us to map the SDR content predictably into 
HDR's absolute luminance space.


## Let There Be Color

So far we've only talked about luminance, ignoring colors altogether. Just like 
in the luminance space, traditionally the color space of display outputs has 
not been well defined. Similar to how an EOTF defines a mapping of pixel data 
to an absolute luminance value, the color space maps color information for each 
pixel onto the CIE 1931 chromaticity space. This can be thought of as a mapping 
to an absolute, real-life, color value.

A color space is defined by its primaries and white point. The primaries and 
white point are expressed as coordinates in the CIE 1931 color space. Think of 
the red primary as the reddest red that can be displayed within the color 
space. Same for green and blue.

Examples of color spaces are:

| Color Space | Description                                |
| ----------- | ------------------------------------------ |
| BT 601      | similar to BT 709                          |
| BT 709      | used by sRGB content; ~53% of BT 2020      |
| DCI-P3      | used by most HDR displays; ~72% of BT 2020 |
| BT 2020     | standard for most HDR content              |

The color space is defined in DRM for YCbCr planes via the color_encoding 
property of the drm_plane. 

**We propose to add definitions for the RGB variants of the BT color spaces.**


## Color Primaries and White Point

Just like displays can currently not represent the entire 0.0005 - 10,000 nits 
HDR range of the PQ 2084 EOTF, they are currently not capable of representing 
the entire BT.2020 color Gamut. For this reason video content will often 
specify the color primaries and white point used to master the video, in order 
to allow displays to be able to map the image as best as possible onto the 
display's gamut.


## Displays and Tonemapping

External displays are able to do their own tone and color mapping, based on the 
mastering luminance, color primaries, and white space defined in the HDR 
metadata.

Internal panels (which are currently few and far between) usually don't include 
the complex HW to do tone and color mapping on their own and will require the 
display driver to perform appropriate mapping.


## Pixel Formats

The pixel formats, such as ARGB8888, ARGB2101010, P010, or FP16 are unrelated 
to color space and EOTF definitions. HDR pixels can be formatted in different 
ways but in order to not lose precision HDR content requires at least 10 bpc 
precision. For this reason ARGB2101010, P010, and FP16 are the obvious 
candidates for HDR. ARGB2101010 and P010 have the advantage of requiring only 
half the bandwidth as FP16, while FP16 has the advantage of enough precision to 
operate in a linear space, i.e. without EOTF.


## Proposed use-cases

Although the userspace side of this work is still in the early stages it is 
clear that we will want to support the following two use-cases:

**One XRGB2101010 HDR Plane:** A single, composited plane of HDR content. The 
use-case is a video player on a desktop with the compositor owning the 
composition of SDR and HDR content. The content shall be PQ BT.2020 formatted. 
The drm_connector's hdr_output_metadata shall be set.

**One ARGB8888 SDR Plane + One P010 HDR Plane:** A normal 8bpc desktop plane, 
with a P010 HDR video plane underlayed. The HDR plane shall be PQ BT.2020 
formatted. The desktop plane shall specify an SDR boost value. The 
drm_connector's hdr_output_metadata shall be set.

**One XRGB8888 SDR Plane - HDR output:** In order to support a smooth 
transition we recommend an OS that supports HDR output to provide the 
hdr_output_metadata on the drm_connector to configure the output for HDR, even 
when the content is only SDR. This will allow for a smooth transition between 
SDR-only and HDR content. In this use-case the SDR max luminance value should 
be provided on the drm_plane.

In DCN we will de-PQ or de-Gamma all input in order to blend in linear space. 
For SDR content we will also apply any desired boost before blending. After 
blending we will then re-apply the PQ EOTF and do RGB to YCbCr conversion if 
needed.


## Summary of proposed interface changes

per drm_plane:
- new RGB color space definitions, mirroring the existing YUV color space 
definitions
- new transfer function property
- new SDR maximum white level property


## References

[1] https://en.wikipedia.org/wiki/High-dynamic-range_video#Perceptual_Quantizer


## Further Reading

https://gitlab.freedesktop.org/swick/wayland-protocols/-/blob/color/unstable/color-management/color.rst
http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP309.pdf
https://app.spectracal.com/Documents/White%20Papers/HDR_Demystified.pdf


Bhawanpreet Lakha (3):
  drm/color: Add RGB Color encodings
  drm/color: Add Color transfer functions for HDR/SDR
  drm/color: Add sdr boost property

 .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c |  4 +-
 .../gpu/drm/arm/display/komeda/komeda_plane.c |  4 +-
 drivers/gpu/drm/arm/malidp_planes.c           |  4 +-
 drivers/gpu/drm/armada/armada_overlay.c       |  4 +-
 drivers/gpu/drm/drm_atomic_uapi.c             |  8 ++
 drivers/gpu/drm/drm_color_mgmt.c              | 84 +++++++++++++++++--
 drivers/gpu/drm/i915/display/intel_sprite.c   |  4 +-
 .../drm/i915/display/skl_universal_plane.c    |  4 +-
 drivers/gpu/drm/nouveau/dispnv04/overlay.c    |  4 +-
 drivers/gpu/drm/omapdrm/omap_plane.c          |  4 +-
 drivers/gpu/drm/sun4i/sun8i_vi_layer.c        |  4 +-
 drivers/gpu/drm/tidss/tidss_plane.c           |  6 +-
 include/drm/drm_color_mgmt.h                  | 25 +++++-
 include/drm/drm_plane.h                       | 30 +++++++
 14 files changed, 173 insertions(+), 16 deletions(-)

-- 
2.31.0

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Reply via email to