RE: [RFC 0/5] Introduce drm sharpening property
> -Original Message- > From: Pekka Paalanen > Sent: Thursday, March 28, 2024 3:35 PM > To: Garg, Nemesa > Cc: Simon Ser ; intel-...@lists.freedesktop.org; dri- > de...@lists.freedesktop.org; G M, Adarsh > Subject: Re: [RFC 0/5] Introduce drm sharpening property > > On Wed, 27 Mar 2024 13:29:16 +0200 > Pekka Paalanen wrote: > > > On Wed, 27 Mar 2024 07:11:48 + > > "Garg, Nemesa" wrote: > > > > > > -Original Message- > > > > From: Pekka Paalanen > > > > Sent: Wednesday, March 13, 2024 3:07 PM > > > > To: Garg, Nemesa > > > > Cc: Simon Ser ; > > > > intel-...@lists.freedesktop.org; dri- de...@lists.freedesktop.org; > > > > G M, Adarsh > > > > Subject: Re: [RFC 0/5] Introduce drm sharpening property > > > > > > > > On Tue, 12 Mar 2024 16:26:00 +0200 Pekka Paalanen > > > > wrote: > > > > > > > > > On Tue, 12 Mar 2024 08:30:34 + "Garg, Nemesa" > > > > > wrote: > > > > > > > > > > > This KMS property is not implementing any formula > > > > > > > > > > Sure it is. Maybe Intel just does not want to tell what the > > > > > algorithm is, or maybe it's even patented. > > > > > > > > > > > and the values > > > > > > that are being used are based on empirical analysis and > > > > > > certain experiments done on the hardware. These values are > > > > > > fixed and is not expected to change and this can change from > > > > > > vendor to vendor. The client can choose any sharpness value on > > > > > > the scale and on the basis of it the sharpness will be set. > > > > > > The sharpness effect can be changed from content to content > > > > > > and from display to display so user needs to adjust the > > > > > > optimum intensity value so as to get good experience on the screen. > > > > > > > > > > > > > > > > IOW, it's an opaque box operation, and there is no way to > > > > > reproduce its results without the specific Intel hardware. > > > > > Definitely no way to reproduce its results in free open source > > > > > software > alone. > > > > > > > > > > Such opaque box operations can only occur after KMS blending, at > > > > > the CRTC or later stage. They cannot appear before blending, not > > > > > in the new KMS color pipeline design at least. The reason is > > > > > that the modern way to use KMS planes is opportunistic composition > > > > > off- > loading. > > > > > Opportunistic means that userspace decides from time to time > > > > > whether it composes the final picture using KMS or some other > > > > > rendering method (usually GPU and shaders). Since userspace will > > > > > arbitrarily switch between KMS and render composition, both must > > > > > result in the exact same image, or end users will observe unwanted > > > > > flicker. > > > > > > > > > > Such opaque box operations are fine after blending, because there they > > > > > can be configured once and remain on forever. No switching, no > > > > > flicker. > > > > > > > > If you want to see how sharpness property would apply in Wayland > > > > design, it would be in step 5 "Adjust (settings UI)" of > > > > https://gitlab.freedesktop.org/pq/color-and-hdr/-/blob/main/doc/co > > > > lor- management-model.md#compositor-color-management-model > > > > > > > > To relate that diagram to KMS color processing, you can identify step 3 > "Compose" > > > > as the KMS blending step. Everything before step 3 happens in KMS > > > > plane color processing, and steps 4-5 happen in KMS CRTC color > > > > processing. > > > > > > > > Sharpening would essentially be a "compositor color effect", it > > > > just happens to be implementable only by specific Intel hardware. > > > > > > > > If a color effect is dynamic or content-dependant, it will > > > > preclude colorimetric monitor calibration. > > > > > > > > > > > > Thanks, > > > > pq > > > > > > > > > > > > > Where does "sharpeness" operation oc
RE: [RFC 0/5] Introduce drm sharpening property
> -Original Message- > From: Pekka Paalanen > Sent: Wednesday, March 13, 2024 3:07 PM > To: Garg, Nemesa > Cc: Simon Ser ; intel-...@lists.freedesktop.org; dri- > de...@lists.freedesktop.org; G M, Adarsh > Subject: Re: [RFC 0/5] Introduce drm sharpening property > > On Tue, 12 Mar 2024 16:26:00 +0200 > Pekka Paalanen wrote: > > > On Tue, 12 Mar 2024 08:30:34 + > > "Garg, Nemesa" wrote: > > > > > This KMS property is not implementing any formula > > > > Sure it is. Maybe Intel just does not want to tell what the algorithm > > is, or maybe it's even patented. > > > > > and the values > > > that are being used are based on empirical analysis and certain > > > experiments done on the hardware. These values are fixed and is not > > > expected to change and this can change from vendor to vendor. The > > > client can choose any sharpness value on the scale and on the basis > > > of it the sharpness will be set. The sharpness effect can be changed > > > from content to content and from display to display so user needs to > > > adjust the optimum intensity value so as to get good experience on > > > the screen. > > > > > > > IOW, it's an opaque box operation, and there is no way to reproduce > > its results without the specific Intel hardware. Definitely no way to > > reproduce its results in free open source software alone. > > > > Such opaque box operations can only occur after KMS blending, at the > > CRTC or later stage. They cannot appear before blending, not in the > > new KMS color pipeline design at least. The reason is that the modern > > way to use KMS planes is opportunistic composition off-loading. > > Opportunistic means that userspace decides from time to time whether > > it composes the final picture using KMS or some other rendering method > > (usually GPU and shaders). Since userspace will arbitrarily switch > > between KMS and render composition, both must result in the exact same > > image, or end users will observe unwanted flicker. > > > > Such opaque box operations are fine after blending, because there they > > can be configured once and remain on forever. No switching, no flicker. > > If you want to see how sharpness property would apply in Wayland design, it > would be in step 5 "Adjust (settings UI)" of > https://gitlab.freedesktop.org/pq/color-and-hdr/-/blob/main/doc/color- > management-model.md#compositor-color-management-model > > To relate that diagram to KMS color processing, you can identify step 3 > "Compose" > as the KMS blending step. Everything before step 3 happens in KMS plane color > processing, and steps 4-5 happen in KMS CRTC color processing. > > Sharpening would essentially be a "compositor color effect", it just happens > to be > implementable only by specific Intel hardware. > > If a color effect is dynamic or content-dependant, it will preclude > colorimetric > monitor calibration. > > > Thanks, > pq > > > > Where does "sharpeness" operation occur in the Intel color processing > > chain? Is it before or after blending? > > Thank you for detail explanation and link. Sharpness operation occur post blending in CRTC ie on the final composed output after blending . Yes Pekka you are right as per the diagram it is done at step 5 "Adjust (settings UI)"). I will also document this thing along with documentation change. > > What kind of transfer characteristics does it expect from the image, > > and can those be realized with KMS CRTC properties if KMS is > > configured such that the blending happens using some other characteristics > (e.g. > > blending in optical space)? > > The filter values are not dependent/calculated on the inputs of image but depending on the blending space and other inputs the blended output gets changed and the sharpness is applied post blending so according to the content user needs to adjust the strength value to get the better visual effect. So tuning of sharpness strength may be needed by user based on the input contents and blending policy to get the desired experience. > > What about SDR vs. HDR imagery? > > The interface can be used for both HDR and SDR. The effect is more prominent for SDR use cases. For HDR filter values and tap value may change. Thanks, Nemesa > > > > Thanks, > > pq > > > > > > -Original Message- > > > > From: dri-devel On > > > > Behalf Of Simon Ser > > > > Sent: Monday, March 4, 2024 7:46 PM > > > > To: Garg, Nem
RE: [PATCH 1/5] drm: Introduce sharpness mode property
> -Original Message-
> From: dri-devel On Behalf Of Pekka
> Paalanen
> Sent: Friday, March 8, 2024 2:18 PM
> To: Garg, Nemesa
> Cc: intel-...@lists.freedesktop.org; dri-devel@lists.freedesktop.org
> Subject: Re: [PATCH 1/5] drm: Introduce sharpness mode property
>
> On Thu, 7 Mar 2024 14:02:33 +0530
> Nemesa Garg wrote:
>
> > This allows the user to set the intensity so as to get the sharpness
> > effect.
> >
> > It is useful in scenario when the output is blurry and user want to
> > sharpen the pixels.
> >
> > Signed-off-by: Nemesa Garg
> > ---
> > drivers/gpu/drm/drm_atomic_uapi.c | 4
> > drivers/gpu/drm/drm_crtc.c| 17 +
> > include/drm/drm_crtc.h| 17 +
> > 3 files changed, 38 insertions(+)
>
> Hi,
>
> the UAPI documentation is completely missing. This cannot be discussed until
> the
> UAPI contract is drafted. It needs to end up in the appropriate "Properties"
> section under https://dri.freedesktop.org/docs/drm/gpu/drm-kms.html#kms-
> properties
> when documentation is built.
>
> I also do not see any of the previous review comments being addressed that I
> recall.
>
> The typo "sharpeness" is very common in these patches, and it is even in the
> UAPI
> as the property name. Also doc comments use different spelling than actual
> code.
> And sometimes you use even "sharpening"
> instead of "sharpness". Which one is it?
Thank you for pointing it out. I'll change it to sharpness to make it
consistent everywhere and will add document in next revision. It seems I have
missed some queries I will try to address it.
Thanks,
Nemesa
>
>
> Thanks,
> pq
>
> >
> > diff --git a/drivers/gpu/drm/drm_atomic_uapi.c
> > b/drivers/gpu/drm/drm_atomic_uapi.c
> > index 29d4940188d4..773873726b87 100644
> > --- a/drivers/gpu/drm/drm_atomic_uapi.c
> > +++ b/drivers/gpu/drm/drm_atomic_uapi.c
> > @@ -417,6 +417,8 @@ static int drm_atomic_crtc_set_property(struct
> drm_crtc *crtc,
> > set_out_fence_for_crtc(state->state, crtc, fence_ptr);
> > } else if (property == crtc->scaling_filter_property) {
> > state->scaling_filter = val;
> > + } else if (property == crtc->sharpening_strength_prop) {
> > + state->sharpeness_strength = val;
> > } else if (crtc->funcs->atomic_set_property) {
> > return crtc->funcs->atomic_set_property(crtc, state, property,
> val);
> > } else {
> > @@ -454,6 +456,8 @@ drm_atomic_crtc_get_property(struct drm_crtc *crtc,
> > *val = 0;
> > else if (property == crtc->scaling_filter_property)
> > *val = state->scaling_filter;
> > + else if (property == crtc->sharpening_strength_prop)
> > + *val = state->sharpeness_strength;
> > else if (crtc->funcs->atomic_get_property)
> > return crtc->funcs->atomic_get_property(crtc, state, property,
> val);
> > else {
> > diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
> > index cb90e70d85e8..d01ab76a719f 100644
> > --- a/drivers/gpu/drm/drm_crtc.c
> > +++ b/drivers/gpu/drm/drm_crtc.c
> > @@ -955,3 +955,20 @@ int drm_crtc_create_scaling_filter_property(struct
> drm_crtc *crtc,
> > return 0;
> > }
> > EXPORT_SYMBOL(drm_crtc_create_scaling_filter_property);
> > +
> > +int drm_crtc_create_sharpening_strength_property(struct drm_crtc
> > +*crtc) {
> > + struct drm_device *dev = crtc->dev;
> > +
> > + struct drm_property *prop =
> > + drm_property_create_range(dev, 0, "SHARPENESS_STRENGTH",
> 0, 255);
> > +
> > + if (!prop)
> > + return -ENOMEM;
> > +
> > + crtc->sharpening_strength_prop = prop;
> > + drm_object_attach_property(>base, prop, 0);
> > +
> > + return 0;
> > +}
> > +EXPORT_SYMBOL(drm_crtc_create_sharpening_strength_property);
> > diff --git a/include/drm/drm_crtc.h b/include/drm/drm_crtc.h index
> > 8b48a1974da3..241514fc3eea 100644
> > --- a/include/drm/drm_crtc.h
> > +++ b/include/drm/drm_crtc.h
> > @@ -317,6 +317,16 @@ struct drm_crtc_state {
> > */
> > enum drm_scaling_filter scaling_filter;
> >
> > + /**
> > +* @sharpness_strength
> > +*
> > +* Used by the user to set the sharpness intensity.
> > +* The value ranges from 0-255.
> > +* Any value greater than 0 means enabling
RE: [RFC 0/5] Introduce drm sharpening property
This KMS property is not implementing any formula and the values that are being used are based on empirical analysis and certain experiments done on the hardware. These values are fixed and is not expected to change and this can change from vendor to vendor. The client can choose any sharpness value on the scale and on the basis of it the sharpness will be set. The sharpness effect can be changed from content to content and from display to display so user needs to adjust the optimum intensity value so as to get good experience on the screen. > -Original Message- > From: dri-devel On Behalf Of Simon > Ser > Sent: Monday, March 4, 2024 7:46 PM > To: Garg, Nemesa > Cc: Pekka Paalanen ; intel- > g...@lists.freedesktop.org; dri-devel@lists.freedesktop.org; G M, Adarsh > > Subject: RE: [RFC 0/5] Introduce drm sharpening property > > On Monday, March 4th, 2024 at 15:04, Garg, Nemesa > wrote: > > > This is generic as sharpness effect is applied post blending. > > Depending on the color gamut, pixel format and other inputs the image > > gets blended and once we get blended output it can be sharpened based > > on strength value provided by the user. > > It would really help if you could provide the exact mathematical formula > applied > by this KMS property.
RE: [RFC 0/5] Introduce drm sharpening property
This is generic as sharpness effect is applied post blending. Depending on the color gamut, pixel format and other inputs the image gets blended and once we get blended output it can be sharpened based on strength value provided by the user. On intel platform it is implemented through pipe scaler. Pipe scaler can be used as either scaler or sharpness filter. As mentioned earlier the client can provide any strength value between 0-255 or any other scale based on discussions. Perhaps userspace can have provide options like low-med-high sharpness or in percentage form or steps which is mapped to 0-255. I will add the documentation regarding property. > -Original Message- > From: dri-devel On Behalf Of Pekka > Paalanen > Sent: Friday, February 16, 2024 2:06 PM > To: Garg, Nemesa > Cc: Simon Ser ; intel-...@lists.freedesktop.org; dri- > de...@lists.freedesktop.org; G M, Adarsh > Subject: Re: [RFC 0/5] Introduce drm sharpening property > > On Fri, 16 Feb 2024 04:28:41 + > "Garg, Nemesa" wrote: > > > It is not intel specific and the goal is to have a generic API for > > configuring Sharpness, accessible to various vendors. Intel currently > > offers sharpness support through the Display Engine, while other > > vendors seem to support Sharpness through the GPU using GL shaders > > (Vulcan/Open GL based). > > Do you mean that all vendors use the exact same mathematical algorithm (with > only differences in operation precision at most)? > > If yes, good. > > If not, then we need to know where exactly in the virtual KMS color pipeline > the > operation happens, whether this can be generic or not. > > Does this also work the same regardless of pixel formats, dynamic range, color > gamut, transfer functions etc. on both plane input and connector output > configurations? > > > In terms of sharpness intensity adjustment, the plan is to provide > > users with the ability to customize and regulate sharpness levels. We > > suggest setting a minimum and maximum strength range from 1 to 255, > > where a value of 0 signifies that the sharpness feature is disabled, > > indicated by a u8 data type. For now we have mapped the strength value > > 0.0 to 14.9375 to 0-239 but as the datatype is u8 user can give value > > upto 255 which is gets clamped to 239. > > Naturally you will need to document all that, so that all drivers and vendors > do the > exact same thing. > > I did not see any actual documentation in the patch series yet, e.g. a > reference to > a specific algorithm. > > As Ville pointed out, there was also no specification at which point of the > virtual > color pipeline this operation will apply. Before/after > DEGAMMA/CTM/GAMMA/scaling in plane/blending/CRTC? > > Is the property being added to the list in > https://dri.freedesktop.org/docs/drm/gpu/drm-kms.html#standard-crtc- > properties > or where-ever it belongs? > > > Thanks, > pq > > > We are also open to have alternative scales, such as 1-100 or 1-10, as > > long as a general consensus is reached within the community comprising > > OEMs and vendors. > > > > > -Original Message- > > > From: Simon Ser > > > Sent: Thursday, February 15, 2024 2:03 PM > > > To: Garg, Nemesa > > > Cc: intel-...@lists.freedesktop.org; dri-devel@lists.freedesktop.org > > > Subject: Re: [RFC 0/5] Introduce drm sharpening property > > > > > > How much of this is Intel-specific? Are there any hardware vendors > > > with a similar feature? How much is the "sharpness" knob tied to > > > Intel hardware?
RE: [RFC 0/5] Introduce drm sharpening property
It is not intel specific and the goal is to have a generic API for configuring Sharpness, accessible to various vendors. Intel currently offers sharpness support through the Display Engine, while other vendors seem to support Sharpness through the GPU using GL shaders (Vulcan/Open GL based). In terms of sharpness intensity adjustment, the plan is to provide users with the ability to customize and regulate sharpness levels. We suggest setting a minimum and maximum strength range from 1 to 255, where a value of 0 signifies that the sharpness feature is disabled, indicated by a u8 data type. For now we have mapped the strength value 0.0 to 14.9375 to 0-239 but as the datatype is u8 user can give value upto 255 which is gets clamped to 239. We are also open to have alternative scales, such as 1-100 or 1-10, as long as a general consensus is reached within the community comprising OEMs and vendors. > -Original Message- > From: Simon Ser > Sent: Thursday, February 15, 2024 2:03 PM > To: Garg, Nemesa > Cc: intel-...@lists.freedesktop.org; dri-devel@lists.freedesktop.org > Subject: Re: [RFC 0/5] Introduce drm sharpening property > > How much of this is Intel-specific? Are there any hardware vendors with a > similar > feature? How much is the "sharpness" knob tied to Intel hardware?
