Independent EDID parsing library
Hello Pekka, Daniel As discussed over IRC, I have prepared the first version of the EDID parsing library, which is hosted here: https://github.com/contactshashanksharma/libedid/tree/master This is a simple C library, and I have created this library keeping a compositor's context in mind, so its easy for a compositor to use it. There are only 2 APIs in this library: - libedid_process_edid_info: Get EDID information from raw_edid, returns filled struct edid_info ptr - libedid_destroy_edid_info: Free the EDID information I have provided much information in the README file, and have also added two simple test apps, with sample EDID, to test it. Please have a look and let me know your opinion on this. Regards Shashank ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Enabling HDR support in Wayland / Weston
Hello All, We have raised a new merge request, for the HDR implementation in Weston: https://gitlab.freedesktop.org/wayland/weston/merge_requests/124 This patch series is in continuation to the design we published here: https://lists.freedesktop.org/archives/wayland-devel/2019-January/039808.html We have added a basic and limited focus implementation of HDR video playback stack, some of the testing specs are: -HDR Format: P010 -Some of the videos tested: https://4kmedia.org/samsung-wonderland-two-hdr-uhd-4k-demo ; and others from the same site. -Display / Overlays HW: Icelake -Monitors: LG 32UD99-W / 27UK650-W This patch-set targets: - HDR framework in drm-backend, which will allow a single plane HDR playback using Icelake HW overlays. - HDR support in Weston's GL backend, which will allow GL based rendering for HDR playback as a fallback method for other Hardwares. The Wayland protocol for HDR/Color management is already being discussed at various forums and lists. Please note that, this patch-set contains a temporary implementation of HDR metadata and colorspace protocol, which is under tag do-not-merge, and is for the sake of completion of testing stack. We are using this as a placeholder for actual color management protocol. https://lists.freedesktop.org/archives/wayland-devel/2019-March/040264.html To give everyone some background of these parallel threads of work, in my team, we have divided the total HDR development work into 3 parts: - The HDR protocol development as a color management subset: (Driven by Ankit Nautiyal) - Adding HDR support in GL backend : (Driven by Harish Krupo) - Compositor and framework changes in DRM-backend : (Driven by me, Shashank Sharma) Please provide us the feedback on the implementation. Regards Shashank ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
RE: [PATCH] unstable: add HDR Mastering Meta-data Protocol
Regards Shashank > -Original Message- > From: Graeme Gill [mailto:grae...@argyllcms.com] > Sent: Thursday, March 7, 2019 7:05 AM > To: Sharma, Shashank ; gra...@argyllcms.com; > Pekka Paalanen ; Nautiyal, Ankit K > > Cc: e.bur...@gmail.com; Kps, Harish Krupo ; > niels_...@salscheider-online.de; sebast...@sebastianwick.net; wayland- > de...@lists.freedesktop.org > Subject: Re: [PATCH] unstable: add HDR Mastering Meta-data Protocol > > Sharma, Shashank wrote: > > >> From: Graeme Gill [mailto:grae...@argyllcms.com] > > >>>> From: Ankit Nautiyal > >>>> > >>>> This protcol enables a client to send the hdr meta-data: MAX-CLL, > >>>> MAX-FALL, Max Luminance and Min Luminance as defined by SMPTE ST.2086. > >> > >> Hmm. I'm wondering what the intent of this is. > > > The main reason is to pass it to the compositor so that it can do the > > tone mapping properly. As you know, there could be cases where we are > > dealing with one HDR and multiple SDR frames. Now, the HDR content > > comes with its own metadata information, which needs to be passed to > > the display using the AVI infoframes, for the correct HDR experience, > > but we need to make others non-HDR frames also compatible to this brightness > range of metadata, so we do tone mapping. > > I've been thinking a bit about that, and currently my view is that this is > not the best way > of arranging things. MAX-CLL and MAX-FALL are specific to certain current > source of > HD imagery, and not all HDR sources will have those specific numbers, and > video > standards may change in the future (i.e. frame by frame meta-data). Exactly > how they > can be used is unclear - i.e. they are observations from a video stream, not > parameters > for a specific tone mapping. So punting them to the Compositor doesn't seem > such a > good approach. I kindof agree on this point, that, the usage model is not very clear right now, and this stack and usage model will mature with time. But I would like to still keep this here, for following reasons: - If you closely see the HDMI AVI infoframes specified in CEA-861-G spec, the HDR metadata section is expecting MAX-CLL and MAX-FALL, this means many monitor might be using this already to provide you better viewing experience. So if a content can set these values, no harm in hand carrying that. - The tone mapping API implemented by libVA expects this data, as they are using it to weight the brightness values in frame. So the compositor will pass these values to libVA too. > > Instead what I'd suggest is providing some concrete tone mapping operators > for HDR > handling (at least for V1), with well defined behavior and parameters. The > video > playback application can do the conversion from source format specific > observations > like MAX-CLL and MAX-FALL into the specific compositor tone mapping > parameters, > or it has the option of determining those parameters some other way (user > setting > perhaps, or it's own analysis of the video stream), or it can even do the > adaptation of > the source to the display itself, since it has access to the display > characteristics via the > ICC profile. > > I think it's quite possible to have the Compositor do sane HDR conversions > with no > other extra information than the HDR nominal diffuse white brightness for > each HDR > profile, and perhaps this could be a base Compositor tone mapping behavior, > with > other tone mapping algorithms (and their associated parameters) added as they > are > identified and defined or standardized. > Yep, this is very close to what we are doing in our compositor code. > >> If the idea is that somehow the compositor is to do dynamic HDR > >> rendering, then I have my doubts about its suitability and > >> practicality, unless you are prepared to spell out in detail the algorithm > >> it should > use for performing this. > >> > > The GL tone mapping algorithms will be opensource methods, and will be > > available for the code review. HW tone mapping is using the libVA interface > > and > media-driver handler. > > Right, but there is a distinction between a particular implementation and a > Compositor > specification. > > Can you write a specification of how the tone curves you are proposing to use > work, in > such a way that someone else can implement them, and that someone dealing with > other HDR sources can use them ? Sure, in fact, soon we will publish the code, and you can have a look and provide us feedback. > (i.e. HDR photographs or synthetic imagery from games etc.) >
RE: [RFC wayland-protocols v2 1/1] Add the color-management protocol
Regards Shashank > -Original Message- > From: wayland-devel [mailto:wayland-devel-boun...@lists.freedesktop.org] On > Behalf Of Pekka Paalanen > Sent: Wednesday, March 6, 2019 7:10 PM > To: Graeme Gill ; Niels Ole Salscheider > ; Sebastian Wick > ; > Nautiyal, Ankit K > Cc: Kai-Uwe ; wayland-devel@lists.freedesktop.org; Adam > Jackson ; gra...@argyllcms.com; Chris Murphy > ; Sharma, Shashank > Subject: Re: [RFC wayland-protocols v2 1/1] Add the color-management protocol > > On Wed, 6 Mar 2019 21:26:56 +1100 > Graeme Gill wrote: > > > I'm maintaining a summary of my current thinking here: > > <http://www.argyllcms.com/WaylandCM_v1.txt> for anyone interested. > > Hi Graeme, > > so very good you made a write-up! The email threads are growing unwieldy to > search > for a specific topic or question. While your write-up has some things I'm > unsure of and > some things I cannot claim to understand yet, I do largely agree with it at > this point. > > Sebastian, Niels, Ankit, Shashank, > > I hope that you will cooperate and work on a common protocol proposal, so > that I do > not have to try to pick a better one. ;-) I also hope that you cooperate on > the Weston > implementation side to avoid duplicated work. > > Chris, Adam, > > thank you very much for your input! > > I have a feeling that there has been enough discussion for the moment, that I > would > like to see the next joint effort protocol proposal before my input could be > useful > again. Agree, Pekka. We were already thinking of publishing a sample merge of HDR protocol patches, built on top of Sebastian's color management protocol, so that all can comment directly on the code. Ankit will soon publish the code. - Shashank > > > Thank you everyone, > pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
RE: [PATCH] unstable: add HDR Mastering Meta-data Protocol
Regards Shashank > -Original Message- > From: Graeme Gill [mailto:grae...@argyllcms.com] > Sent: Tuesday, March 5, 2019 9:35 AM > To: Pekka Paalanen ; Nautiyal, Ankit K > > Cc: e.bur...@gmail.com; niels_...@salscheider-online.de; wayland- > de...@lists.freedesktop.org; sebast...@sebastianwick.net; Kps, Harish Krupo > ; Sharma, Shashank > Subject: Re: [PATCH] unstable: add HDR Mastering Meta-data Protocol > > Pekka Paalanen wrote: > > On Wed, 27 Feb 2019 10:27:16 +0530 > > "Nautiyal, Ankit K" wrote: > > > >> From: Ankit Nautiyal > >> > >> This protcol enables a client to send the hdr meta-data: > >> MAX-CLL, MAX-FALL, Max Luminance and Min Luminance as defined by > >> SMPTE ST.2086. > > Hmm. I'm wondering what the intent of this is. > > If it is to feed these values through to the display itself so that it can do > dynamic HDR > rendering, then this tends to cut across color management, since color > management > as it is currently formulated depends on reproducability in color devices. If > really > desired, I guess this would have to be some kind of special mode that > bypasses the > usual rendering pipeline. > The main reason is to pass it to the compositor so that it can do the tone mapping properly. As you know, there could be cases where we are dealing with one HDR and multiple SDR frames. Now, the HDR content comes with its own metadata information, which needs to be passed to the display using the AVI infoframes, for the correct HDR experience, but we need to make others non-HDR frames also compatible to this brightness range of metadata, so we do tone mapping. We are writing compositor code, which will take a call on target outputs content brightness level too (apart with colorspace) by applying appropriate tone mapping (SDR to HDR, HDR to HDR, or HDR to SDR) using libVA or GL extensions. This will make sure that everything being shown on the display falls into same brightness range. > If the idea is that somehow the compositor is to do dynamic HDR rendering, > then I > have my doubts about its suitability and practicality, unless you are > prepared to spell > out in detail the algorithm it should use for performing this. > The GL tone mapping algorithms will be opensource methods, and will be available for the code review. HW tone mapping is using the libVA interface and media-driver handler. > My assumption was that the compositor would only be expected to do well > understood fallback color conversions, and that anything that needed special > functionality or that was going to do experimental color handling (such as > dynamic > HDR rendering) would do so in the client. > Yes, we are trying our best here to make compositor fair and intelligent, and I believe with proper discussions and code reviews we should be able to reach that level. > For the purposes of mixing SDR and HDR in the compositor as a fallback, a much > simpler set of transforms should be sufficient, namely a brightness for > converting > SDR into HDR, and a tone mapping curve or equivalent settings for compressing > HDR > into SDR. > Agree. > > Is there a case for compositors that implement color management but > > not HDR support? Would it be unreasonable to make HDR parameters a > > part of the color management extension? Such that it would be optional > > for a client to set the HDR parameters. > Right now we are not targeting this, but eventually we want to be there. As we all know, it's a huge area to cover in an single attempt, and gets very cumbersome. So the plan is to first implement a modular limited capability HDR video playback, and use that as a driving and testing vehicle for color management, and then later enhance it with adding more and more modules. > I think that at this point in time it is entirely reasonable that a > compositing system have > some level of HDR awareness. > Yes, the idea is to make it completely aware of HDR scenarios over the period, over a slowly maturing stack. - Shashank > > Btw. do HDR videos contain ICC profiles? Where would a client get the > > ICC profile for a video it wants to show? Do we need to require the > > compositor to expose some commonly used specific profiles? > > As I understand it, no they don't. They either rely on a stated encoding > (i.e. REC2020 > based), or sometimes have simple meta-data such as primary coordinates & > transfer > curve specifications. A client wanting to support such meta-data can simply > create a > matching ICC profile on the fly if it wishes. > > Cheers, > Graeme Gill. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
RE: [RFC wayland-protocols 1/1] unstable: add color management protocol
Regards Shashank > -Original Message- > From: wayland-devel [mailto:wayland-devel-boun...@lists.freedesktop.org] On > Behalf Of Sebastian Wick > Sent: Thursday, February 14, 2019 12:13 PM > To: wayland-devel@lists.freedesktop.org > Subject: Re: [RFC wayland-protocols 1/1] unstable: add color management > protocol > > On 2019-02-14 05:55, Sharma, Shashank via wayland-devel wrote: > > Hello Sebastian, > > My comments inline. > > > > Regards > > Shashank > >> -Original Message- > >> From: wayland-devel > >> [mailto:wayland-devel-boun...@lists.freedesktop.org] On Behalf Of > >> Sebastian Wick > >> Sent: Thursday, February 14, 2019 8:16 AM > >> To: wayland-devel@lists.freedesktop.org > >> Cc: Sebastian Wick > >> Subject: [RFC wayland-protocols 1/1] unstable: add color management > >> protocol > >> > >> This protocol allows clients to attach a color space and rendering > >> intent to a wl_surface. It further allows the client to be informed > >> which color spaces a wl_surface was converted to on the last > >> presented. > >> > >> Signed-off-by: Sebastian Wick > >> --- > >> Makefile.am | 1 + > >> unstable/color-management/README | 4 + > >> .../color-management-unstable-v1.xml | 183 > >> ++ > >> 3 files changed, 188 insertions(+) > >> create mode 100644 unstable/color-management/README create mode > >> 100644 > >> unstable/color-management/color-management-unstable-v1.xml > >> > >> diff --git a/Makefile.am b/Makefile.am index 345ae6a..80abc1d 100644 > >> --- a/Makefile.am > >> +++ b/Makefile.am > >> @@ -23,6 +23,7 @@ unstable_protocols = > >>\ > >>unstable/xdg-decoration/xdg-decoration-unstable-v1.xml \ > >> > >> > >> unstable/linux-explicit-synchronization/linux-explicit-synchronizatio > >> n- > >> unstable-v1.xml \ > >>unstable/primary-selection/primary-selection-unstable-v1.xml > >> \ > >> + unstable/color-management/color-management-unstable-v1.xml > >> \ > >>$(NULL) > >> > >> stable_protocols = > >> \ > >> diff --git a/unstable/color-management/README b/unstable/color- > >> management/README new file mode 100644 index 000..140f1e9 > >> --- /dev/null > >> +++ b/unstable/color-management/README > >> @@ -0,0 +1,4 @@ > >> +Color management protocol > >> + > >> +Maintainers: > >> +Sebastian Wick > >> diff --git > >> a/unstable/color-management/color-management-unstable-v1.xml > >> b/unstable/color-management/color-management-unstable-v1.xml > >> new file mode 100644 > >> index 000..1615fe6 > >> --- /dev/null > >> +++ b/unstable/color-management/color-management-unstable-v1.xml > >> @@ -0,0 +1,183 @@ > >> + >> +name="color_management_unstable_v1"> > >> + > >> + > >> +Copyright © 2019 Sebastian Wick > >> +Copyright © 2019 Erwin Burema > >> + > >> +Permission is hereby granted, free of charge, to any person > >> obtaining a > >> +copy of this software and associated documentation files (the > >> "Software"), > >> +to deal in the Software without restriction, including without > >> limitation > >> +the rights to use, copy, modify, merge, publish, distribute, > >> sublicense, > >> +and/or sell copies of the Software, and to permit persons to > >> + whom > >> the > >> +Software is furnished to do so, subject to the following > >> conditions: > >> + > >> +The above copyright notice and this permission notice (including > >> the next > >> +paragraph) shall be included in all copies or substantial > >> portions of the > >> +Software. > >> + > >> +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > >> EXPRESS OR > >> +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > >> MERCHANTABILITY, > >> +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO > >> EVENT > >> SHALL > >> +THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, > >> + DAMAGES > >
RE: [RFC wayland-protocols 1/1] unstable: add color management protocol
Hello Sebastian, My comments inline. Regards Shashank > -Original Message- > From: wayland-devel [mailto:wayland-devel-boun...@lists.freedesktop.org] On > Behalf Of Sebastian Wick > Sent: Thursday, February 14, 2019 8:16 AM > To: wayland-devel@lists.freedesktop.org > Cc: Sebastian Wick > Subject: [RFC wayland-protocols 1/1] unstable: add color management protocol > > This protocol allows clients to attach a color space and rendering intent to a > wl_surface. It further allows the client to be informed which color spaces a > wl_surface > was converted to on the last presented. > > Signed-off-by: Sebastian Wick > --- > Makefile.am | 1 + > unstable/color-management/README | 4 + > .../color-management-unstable-v1.xml | 183 ++ > 3 files changed, 188 insertions(+) > create mode 100644 unstable/color-management/README create mode 100644 > unstable/color-management/color-management-unstable-v1.xml > > diff --git a/Makefile.am b/Makefile.am > index 345ae6a..80abc1d 100644 > --- a/Makefile.am > +++ b/Makefile.am > @@ -23,6 +23,7 @@ unstable_protocols = > \ > unstable/xdg-decoration/xdg-decoration-unstable-v1.xml \ > unstable/linux-explicit-synchronization/linux-explicit-synchronization- > unstable-v1.xml \ > unstable/primary-selection/primary-selection-unstable-v1.xml > \ > + unstable/color-management/color-management-unstable-v1.xml > \ > $(NULL) > > stable_protocols = > \ > diff --git a/unstable/color-management/README b/unstable/color- > management/README > new file mode 100644 > index 000..140f1e9 > --- /dev/null > +++ b/unstable/color-management/README > @@ -0,0 +1,4 @@ > +Color management protocol > + > +Maintainers: > +Sebastian Wick > diff --git a/unstable/color-management/color-management-unstable-v1.xml > b/unstable/color-management/color-management-unstable-v1.xml > new file mode 100644 > index 000..1615fe6 > --- /dev/null > +++ b/unstable/color-management/color-management-unstable-v1.xml > @@ -0,0 +1,183 @@ > + > + > + > + > +Copyright © 2019 Sebastian Wick > +Copyright © 2019 Erwin Burema > + > +Permission is hereby granted, free of charge, to any person obtaining a > +copy of this software and associated documentation files (the > "Software"), > +to deal in the Software without restriction, including without limitation > +the rights to use, copy, modify, merge, publish, distribute, sublicense, > +and/or sell copies of the Software, and to permit persons to whom the > +Software is furnished to do so, subject to the following conditions: > + > +The above copyright notice and this permission notice (including the next > +paragraph) shall be included in all copies or substantial portions of the > +Software. > + > +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > EXPRESS OR > +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > MERCHANTABILITY, > +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT > SHALL > +THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES > OR OTHER > +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, > ARISING > +FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR > OTHER > +DEALINGS IN THE SOFTWARE. > + > + > + > +This protocol provides the ability to specify the color space > +of a wl_surface. If further enumerates the color spaces currently > +in the system and allows to query feedback about which color spaces > +a wl_surface was converted to on the last present. > +The idea behind the feedback system is to allow the client to do color > +conversion to a color space which will likely be used to show the surface > +which allows the compositor to skip the color conversion step. > + > + > + > + > + The color manager is a singleton global object that provides access > + to outputs' color spaces, let's you create new color spaces and attach > + a color space to surfaces. > + > + > + > + > +render intents > + > + > + > + > + Would you elaborate a bit more on what do we want to achieve with render-intent ? How can the compositor utilize this filed during color conversion ? > + > + > + > + Its slightly difficult to define what is the definition of a "well-known" color space, as different use cases may have their own favorite. May be we can call it "general-purpose" or "standard" or something in those line. > +Well-known color spaces > + > + > + > + > + I guess we mean DCI-P3 here > + > + /> > + Well, this is slightly off the chart for me. If I understood this correctly, the cover letter talks about the intent is to focus on Colo
Re: HDR support in Wayland/Weston
Regards Shashank On 1/17/2019 5:33 PM, Pekka Paalanen wrote: On Wed, 16 Jan 2019 09:25:06 +0530 "Sharma, Shashank" wrote: On 1/14/2019 6:51 PM, Pekka Paalanen wrote: On Thu, 10 Jan 2019 20:32:18 +0530 "Sharma, Shashank" wrote: Hello All, This mail is to propose a design for enabling HDR support in Wayland/Weston stack, using display engine capabilities, and get more feedback and input from community. *snip* I understand your aim is to leverage display hardware capabilities to the fullest, but we must also consider hardware that lacks some or all of the conversion/mapping/other features while the monitor is well HDR-capable. We also need to consider what happens when a monitor is not HDR-capable or is somehow lacking. OTOH, whether a compositor implements HDR support at all would be obvious in the advertised Wayland globals and pixel formats. Very valid point. We have given good thoughts on how to handle such scenarios when we are getting into, and we can come-up with some kind of compositor policies which will decide if HDR video playback should be allowed or not, depending on the combination of Content type, SW stack(compositor and kernel), HW capabilities and connected Monitor capabilities. A sample such policy may look like (also attached as txt file, just in case if this table gets distorted): ++--+ |Content |SW (C|K)|HW |Monitor | HDR Playback *clip* Talking in terms of "allowed" and "not allowed" sounds very much like we would be needing "complicated" Wayland protocol to let applications fail gracefully at runtime, letting them know dynamically when things would work or not work. I believe we could do much simpler in protocol terms as follows: Does a compositor advertise HDR support extensions at all? - This would depend on the compositor implementation obviously, cannot advertise anything without. - Optionally, it could depend on the graphics card hardware/driver capabilities: if there is no card that could support HDR, then there is no reason advertise HDR support through Wayland, because it would always fall back to conversion to SDR. However, note that GPU hotplug might be a thing, which might bring HDR support later at runtime. - Third, optionally again, a compositor might choose to not advertise HDR support if it knows it will never had a HDR-capable monitor attached. This is much longer stretch, and probably only for embedded devices you cannot plug arbitrary monitors to. Once the Wayland HDR-related extensions have been advertised to clients at runtime, taking them away will be hard. You may want to consider to never revoke the extension interfaces if they have once been published in the lifetime of a compositor instance, because revoking Wayland globals has some caveats, mostly around clients still using HDR extensions until they actually notice the compositor wants to redact them. It can be made to work, but I'm not sure what the benefit would be. So, once a compositor advertises the extensions, they have to keep on working at all times. Specifically this means, that if a client has submitted a frame in HDR, and the compositor suddenly loses the ability to physically display HDR, e.g. the only HDR monitor gets unplugged and only SDR monitors remain, the compositor must still be able to show the window that has HDR content lingering. So converting HDR to SDR on-demand is a mandatory feature. The allowed vs. not allowed is only applicable with respect to what capabilities the compositor has advertised. A client is not "allowed" to submit HDR content if the compositor does not expose the Wayland extensions. Actually this is not about allowing, but being able to submit HDR content at all: if the interfaces are not advertised, a client simply has no interface to poke at. Pixel formats, color spaces, and so on are more interesting. The compositor should advertise what it supports by enumerating them explicitly or saying what description formats it supports. Then a client cannot use anything outside of those; if it attempts to, that will be a fatal protocol error, not a recoverable failure. If a client does everything according to what a compositor advertises, it must "work": the compositor must be able to consume the client content regardless of what will happen next, e.g. with monitor hot-unplug, or scenegraph changing such that overlay plane is no longer usable. This is why the fallback path through GL-renderer must exist, and it must be able to do HDR->SDR mapping, and so on. In summary, rather than dynamically allow or not allow, a compositor needs to live by its promise on what works. It cannot pull the rug from under a client by suddenly hiding the window or showing it corrupted. That is the baseline in behaviour. Makes a lot of sense, and sounds like a stable
Re: HDR support in Wayland/Weston
Hello Arnaud Thanks for your comments, mine inline. Regards Shashank On 1/17/2019 6:38 AM, Arnaud Vrac wrote: On Thu, Jan 10, 2019 at 4:02 PM Sharma, Shashank wrote: Hello All, This mail is to propose a design for enabling HDR support in Wayland/Weston stack, using display engine capabilities, and get more feedback and input from community. Here are few points (you might already know these), about HDR framebuffers, videos and displays: - HDR content/buffers are composed in REC2020 colorspace, with bit depth 10/12/16 BPC. Some of the popular formats are P010,P012,P016. - HDR content come with their own Metadata to be applied to get the right luminance at the display device. - The metadata can be of two type 1. static 2. dynamic . For simplicity, this solution is focusing on static HDR only (HDR10 standard) - HDR content also provide its supported EOTF (electro optical transfer function) information, which is a curve (like SRGB gamma curve). One popular EOTF is PQ(ST2084). - HDR capable displays mention their EOTF and HDR metadata support information in EDID CEA-861-G blocks. - Normal SRGB buffers are composed in SRGB color space following REC709 specifications. - For accurate blending in display engines, we need to make sure following: - All the buffers are in same colorspace (Rec 709 or Rec 2020) - All the buffers are liner (gamma/EOTF removed) - All the buffers are tone mapped in same zone (HDR or SDR) Please refer to the block diagram below, which presents a simple case of a HDR P010 movie playback, with HDR buffers as video buffers, and SDR buffers as subtitles. The subsystem looks and works like this: - A client decodes the buffer (using FFMpeg for example) and gets the two buffers, one with video (HDR) and one subtitles (SDR) - Client passes following information to the compositor: - The actual buffers - Their colorspace infromation, BT2020 for HDR buffer, REC709 for SDR buffer (planning to add a new protocol extension for this) - The HDR metadata of the content (planning to add new protocol for this) - Compositors actions: - Reads the End display's HDR capabilities from display EDID. Assume its an HDR HDMI monitor. - Compositor tone maps every view's framebuffer to match tone of end display, applying a libVA filter. In this example: - The SDR subtitles frame will go through SDR to HDR tone mapping (called S2H) - The HDR video frame will go through HDR to HDR tone mapping (called H2H) if the HDR capabilities of monitor and content are different. - Now both the buffers and the monitor are in the same tone mapped range. - As the end display is HDR capable, and one of the content frame is HDR, the compositor will prepare all other planes for color space conversion (CSC) from REC709->REC2020 using plane CSC property. - As the CSC and blending should be done in liner space, compositor will also use plane level degamma to make the buffers linear. - These actions will make sure that, during blending: - All the buffers are in same colorspace (REC2020) - All the buffers are linear - All the buffers are tone mapped (HDR) - The plane level color properties patch, for DRM can be found here: https://patchwork.freedesktop.org/series/30875/ - Now, in order to re-apply the HDR curve, compositor will apply CRTC level gamma, so that the output buffer is non-linear again. - To pass the output HDR information to kernel, so that it can create and send AVI-info-frames to HDMI, compositor will set Connector HDR metadata property. - Code for the same can be found here: https://patchwork.freedesktop.org/series/25091/ - And they will ever live happily after :). Please provide inputs, feedbacks and suggestions for this design and plan, so that we can improve out half cooked solution, and start sending the patches. +--+ +---+ | SDR Buffer subtitles | HDR Buffer video | (REC 709 colorsp) | (REC 2020 colorsp | | | | | +---+--+ +---+---+ || || || +--v---v+ +--+ | Compositor: v | | LibVA| | - assigns views to overlays +-> Tone mapping | | - prepare plane/CRTC color properties <-+ SDR to HDR | | for linear blending in display|
Re: HDR support in Wayland/Weston
Hello Graeme, Thanks for your inputs and comment, please find mine inline. Regards Shashank On 1/11/2019 12:55 PM, Graeme Gill wrote: Sharma, Shashank wrote: Hi, While I'm sure you could hard code various color space assumptions into such an implementation (and perhaps this is a pretty reasonable way of doing a proof of concept), it's not a good long term solution, and could end up being something of a millstone. What's missing is any serous Color Management in Wayland. It's a bigger project to fix that, but HDR would then be able to slot into a much more usable framework. I agree, honestly, HDR might be the biggest consumer of color management. We can very well use this stack to drive color management design. But the only worry is, when to target too big and too generic, the actual purpose gets diluted and we get stuck into long chain of mail communication. So probably the most important thing for us, as a team, would be to break this implementation into small measurable steps which slowly targets respective areas of Weston development, keeping the end goal alive. - HDR content/buffers are composed in REC2020 colorspace, with bit depth 10/12/16 BPC. Some of the popular formats are P010,P012,P016. While REC2020 based HDR colorspaces are very popular, they aren't the only ones out there. Agree. My sole purpose of proposing this design was to talk to people, and come up with a small, scalable target use case, which we can slowly expand and shape into a bigger, more generic framework, over the period of time and maturity. So the target was to start with one of the broadly used cases, and try to expand this range, into all possible formats/cases without regressing into existing framework. - Normal SRGB buffers are composed in SRGB color space following REC709 specifications. As far as I'm aware (I haven't been closely monitoring this mailing list since the last rather long and unsatisfactory discussion about color management), Wayland works in display dependent colorspace, since there is no facility for it to know how to convert from anything else to the display space (i.e. no knowledge of display profiles so it doesn't know what sRGB is). In all other computer graphic display systems, it's up to the client to be informed about each display colorspace is, and to do color conversion to the display space either itself, or by using operating system libraries. The operating system provides the display profile information to allow this. As far as I was informed, Wayland is architected in such a way that this is not possible, since clients have no knowledge of which display the pixels they send will end up on. Yes, this is very accurate. We are planning to add a protocol extension, which will allow a client to pass the buffers colorspace information to the compositor. And compositor already has HW's color capabilities (drm properties per plane and CRTC), and monitor color capabilities (EDID). So if compositor gets all of this, with some good policies, it might be able to take call on colorspace accurately. Also, only from HDR implementation point-of-view, we need not to really know, that which REC709 implementation is this buffer actually. We just need to know if this is a BT2020 buffer or not, so that we an scale up/down the color gamut, for a proper blending scenario. So again, strictly for HDR playback scenario, a non-2020 buffer can be considered as 709 buffer, and its almost safe. But if we want to provide a proper and accurate color management solution, we definitely need to know more about the buffer's color-space, than if its 2020 or not. Also note that while the use of an intermediate compositing space is essential when combining different colorspace sources together, it's not desirable in other situations where maximum fidelity and gamut are desired i.e. Photography. (The double conversions are a possible accuracy loss, and it makes it very difficult to achieve good gamut mapping from large gamut sources.) Very valid point, and that's why I was incline towards using the display HW's capabilities, because modern day HWs are investing well on the color pipeline to handle cases like HDR. I guess once we have a mature basic stack working, we can think of adding something like a uses's preference, which could be another entry in our blending decision making inputs. - For accurate blending in display engines, we need to make sure following: - All the buffers are in same colorspace (Rec 709 or Rec 2020) - All the buffers are liner (gamma/EOTF removed) - All the buffers are tone mapped in same zone (HDR or SDR) Is that something that Wayland should really know about though ? i.e. shouldn't that be an application issue, where Wayland provides the necessary mechanisms to achieve correct composition ? (Or in fact is that what you are suggesting ?) Not really, I was in favo
Re: HDR support in Wayland/Weston
Hello Pekka, Thanks a lot for your comments, and inputs on design, stability and general accessibility across all platforms. Please find my comments, inline. Regards Shashank On 1/14/2019 6:51 PM, Pekka Paalanen wrote: On Thu, 10 Jan 2019 20:32:18 +0530 "Sharma, Shashank" wrote: Hello All, This mail is to propose a design for enabling HDR support in Wayland/Weston stack, using display engine capabilities, and get more feedback and input from community. Here are few points (you might already know these), about HDR framebuffers, videos and displays: - HDR content/buffers are composed in REC2020 colorspace, with bit depth 10/12/16 BPC. Some of the popular formats are P010,P012,P016. - HDR content come with their own Metadata to be applied to get the right luminance at the display device. - The metadata can be of two type 1. static 2. dynamic . For simplicity, this solution is focusing on static HDR only (HDR10 standard) - HDR content also provide its supported EOTF (electro optical transfer function) information, which is a curve (like SRGB gamma curve). One popular EOTF is PQ(ST2084). - HDR capable displays mention their EOTF and HDR metadata support information in EDID CEA-861-G blocks. - Normal SRGB buffers are composed in SRGB color space following REC709 specifications. - For accurate blending in display engines, we need to make sure following: - All the buffers are in same colorspace (Rec 709 or Rec 2020) - All the buffers are liner (gamma/EOTF removed) - All the buffers are tone mapped in same zone (HDR or SDR) Please refer to the block diagram below, which presents a simple case of a HDR P010 movie playback, with HDR buffers as video buffers, and SDR buffers as subtitles. The subsystem looks and works like this: - A client decodes the buffer (using FFMpeg for example) and gets the two buffers, one with video (HDR) and one subtitles (SDR) - Client passes following information to the compositor: - The actual buffers - Their colorspace infromation, BT2020 for HDR buffer, REC709 for SDR buffer (planning to add a new protocol extension for this) - The HDR metadata of the content (planning to add new protocol for this) - Compositors actions: - Reads the End display's HDR capabilities from display EDID. Assume its an HDR HDMI monitor. - Compositor tone maps every view's framebuffer to match tone of end display, applying a libVA filter. In this example: - The SDR subtitles frame will go through SDR to HDR tone mapping (called S2H) - The HDR video frame will go through HDR to HDR tone mapping (called H2H) if the HDR capabilities of monitor and content are different. - Now both the buffers and the monitor are in the same tone mapped range. - As the end display is HDR capable, and one of the content frame is HDR, the compositor will prepare all other planes for color space conversion (CSC) from REC709->REC2020 using plane CSC property. - As the CSC and blending should be done in liner space, compositor will also use plane level degamma to make the buffers linear. - These actions will make sure that, during blending: - All the buffers are in same colorspace (REC2020) - All the buffers are linear - All the buffers are tone mapped (HDR) - The plane level color properties patch, for DRM can be found here: https://patchwork.freedesktop.org/series/30875/ - Now, in order to re-apply the HDR curve, compositor will apply CRTC level gamma, so that the output buffer is non-linear again. - To pass the output HDR information to kernel, so that it can create and send AVI-info-frames to HDMI, compositor will set Connector HDR metadata property. - Code for the same can be found here: https://patchwork.freedesktop.org/series/25091/ - And they will ever live happily after :). Please provide inputs, feedbacks and suggestions for this design and plan, so that we can improve out half cooked solution, and start sending the patches. Hi Shashank, this is a major feature that would be awesome to have in Weston, but it's also a big effort to design, implement, and maintain. To ease the maintenance, we will need some serious work on the test suite, which currently cannot even run the GL-renderer. I agree, Weston's stability and maintenance must be high priority. That's why I was thinking that instead of enabling such a huge feature in a single shot, and trying to cover all possible cases and combinations, we can split it into small modular subset of features, in form of small patch series, review it well, and add it slowly in the mainstream with proper precaution. These subsets could be ( just example) - enabling single HDR view (like a movie playback), in fullscreen mode, targeting only one target colorspace (say BT2020) and one type of HDR metadata (say Static HDR metadata, PQ ST 2084 EOTF, HDR 10 type)
Re: HDR support in Wayland/Weston
Hello Ole, On 1/10/2019 9:31 PM, Niels Ole Salscheider wrote: Hello, on a first glance this sounds sensible. Would it work well with the last color management protocol proposal that I made or do you see issues there? We could add REC2020 as another predefined profile. https://lists.freedesktop.org/archives/wayland-devel/2017-January/032769.html I think the last proposal was mostly sane and usable for everybody, but there was not much interest afterwards. However, there was a lot of discussion with wishes from different sides that went into this. The relevant mailing list threads are the following, but you have to follow the discussion over the next months: https://lists.freedesktop.org/archives/wayland-devel/2016-November/031728.html https://lists.freedesktop.org/archives/wayland-devel/2014-March/013951.html Thanks for sharing the links, let me go through the discussion history, I will come back with my feedback for this stack, soon. - Shashank Best regards, Ole Am Donnerstag, 10. Januar 2019, 16:02:18 CET schrieb Sharma, Shashank: Hello All, This mail is to propose a design for enabling HDR support in Wayland/Weston stack, using display engine capabilities, and get more feedback and input from community. Here are few points (you might already know these), about HDR framebuffers, videos and displays: - HDR content/buffers are composed in REC2020 colorspace, with bit depth 10/12/16 BPC. Some of the popular formats are P010,P012,P016. - HDR content come with their own Metadata to be applied to get the right luminance at the display device. - The metadata can be of two type 1. static 2. dynamic . For simplicity, this solution is focusing on static HDR only (HDR10 standard) - HDR content also provide its supported EOTF (electro optical transfer function) information, which is a curve (like SRGB gamma curve). One popular EOTF is PQ(ST2084). - HDR capable displays mention their EOTF and HDR metadata support information in EDID CEA-861-G blocks. - Normal SRGB buffers are composed in SRGB color space following REC709 specifications. - For accurate blending in display engines, we need to make sure following: - All the buffers are in same colorspace (Rec 709 or Rec 2020) - All the buffers are liner (gamma/EOTF removed) - All the buffers are tone mapped in same zone (HDR or SDR) Please refer to the block diagram below, which presents a simple case of a HDR P010 movie playback, with HDR buffers as video buffers, and SDR buffers as subtitles. The subsystem looks and works like this: - A client decodes the buffer (using FFMpeg for example) and gets the two buffers, one with video (HDR) and one subtitles (SDR) - Client passes following information to the compositor: - The actual buffers - Their colorspace infromation, BT2020 for HDR buffer, REC709 for SDR buffer (planning to add a new protocol extension for this) - The HDR metadata of the content (planning to add new protocol for this) - Compositors actions: - Reads the End display's HDR capabilities from display EDID. Assume its an HDR HDMI monitor. - Compositor tone maps every view's framebuffer to match tone of end display, applying a libVA filter. In this example: - The SDR subtitles frame will go through SDR to HDR tone mapping (called S2H) - The HDR video frame will go through HDR to HDR tone mapping (called H2H) if the HDR capabilities of monitor and content are different. - Now both the buffers and the monitor are in the same tone mapped range. - As the end display is HDR capable, and one of the content frame is HDR, the compositor will prepare all other planes for color space conversion (CSC) from REC709->REC2020 using plane CSC property. - As the CSC and blending should be done in liner space, compositor will also use plane level degamma to make the buffers linear. - These actions will make sure that, during blending: - All the buffers are in same colorspace (REC2020) - All the buffers are linear - All the buffers are tone mapped (HDR) - The plane level color properties patch, for DRM can be found here: https://patchwork.freedesktop.org/series/30875/ - Now, in order to re-apply the HDR curve, compositor will apply CRTC level gamma, so that the output buffer is non-linear again. - To pass the output HDR information to kernel, so that it can create and send AVI-info-frames to HDMI, compositor will set Connector HDR metadata property. - Code for the same can be found here: https://patchwork.freedesktop.org/series/25091/ - And they will ever live happily after :). Please provide inputs, feedbacks and suggestions for this design and plan, so that we can improve out half cooked solution, and start sending the patches. +--+ +---+ | SDR Buffer subtitles | HDR
Re: HDR support in Wayland/Weston
If the block diagram is not aligned due to mail client, please refer to the attached .txt file. Hope thats slightly better :). Regards Shashank On 1/10/2019 8:32 PM, Sharma, Shashank wrote: Hello All, This mail is to propose a design for enabling HDR support in Wayland/Weston stack, using display engine capabilities, and get more feedback and input from community. Here are few points (you might already know these), about HDR framebuffers, videos and displays: - HDR content/buffers are composed in REC2020 colorspace, with bit depth 10/12/16 BPC. Some of the popular formats are P010,P012,P016. - HDR content come with their own Metadata to be applied to get the right luminance at the display device. - The metadata can be of two type 1. static 2. dynamic . For simplicity, this solution is focusing on static HDR only (HDR10 standard) - HDR content also provide its supported EOTF (electro optical transfer function) information, which is a curve (like SRGB gamma curve). One popular EOTF is PQ(ST2084). - HDR capable displays mention their EOTF and HDR metadata support information in EDID CEA-861-G blocks. - Normal SRGB buffers are composed in SRGB color space following REC709 specifications. - For accurate blending in display engines, we need to make sure following: - All the buffers are in same colorspace (Rec 709 or Rec 2020) - All the buffers are liner (gamma/EOTF removed) - All the buffers are tone mapped in same zone (HDR or SDR) Please refer to the block diagram below, which presents a simple case of a HDR P010 movie playback, with HDR buffers as video buffers, and SDR buffers as subtitles. The subsystem looks and works like this: - A client decodes the buffer (using FFMpeg for example) and gets the two buffers, one with video (HDR) and one subtitles (SDR) - Client passes following information to the compositor: - The actual buffers - Their colorspace infromation, BT2020 for HDR buffer, REC709 for SDR buffer (planning to add a new protocol extension for this) - The HDR metadata of the content (planning to add new protocol for this) - Compositors actions: - Reads the End display's HDR capabilities from display EDID. Assume its an HDR HDMI monitor. - Compositor tone maps every view's framebuffer to match tone of end display, applying a libVA filter. In this example: - The SDR subtitles frame will go through SDR to HDR tone mapping (called S2H) - The HDR video frame will go through HDR to HDR tone mapping (called H2H) if the HDR capabilities of monitor and content are different. - Now both the buffers and the monitor are in the same tone mapped range. - As the end display is HDR capable, and one of the content frame is HDR, the compositor will prepare all other planes for color space conversion (CSC) from REC709->REC2020 using plane CSC property. - As the CSC and blending should be done in liner space, compositor will also use plane level degamma to make the buffers linear. - These actions will make sure that, during blending: - All the buffers are in same colorspace (REC2020) - All the buffers are linear - All the buffers are tone mapped (HDR) - The plane level color properties patch, for DRM can be found here: https://patchwork.freedesktop.org/series/30875/ - Now, in order to re-apply the HDR curve, compositor will apply CRTC level gamma, so that the output buffer is non-linear again. - To pass the output HDR information to kernel, so that it can create and send AVI-info-frames to HDMI, compositor will set Connector HDR metadata property. - Code for the same can be found here: https://patchwork.freedesktop.org/series/25091/ - And they will ever live happily after :). Please provide inputs, feedbacks and suggestions for this design and plan, so that we can improve out half cooked solution, and start sending the patches. +--+ +---+ | SDR Buffer subtitles | HDR Buffer video | (REC 709 colorsp) | (REC 2020 colorsp | | | | | +---+--+ +---+---+ || || || +--v---v+ +--+ | Compositor: v | | LibVA| | - assigns views to overlays +-> Tone mapping | | - prepare plane/CRTC color properties <-+ SDR to HDR | | for linear blending in display |
HDR support in Wayland/Weston
Hello All, This mail is to propose a design for enabling HDR support in Wayland/Weston stack, using display engine capabilities, and get more feedback and input from community. Here are few points (you might already know these), about HDR framebuffers, videos and displays: - HDR content/buffers are composed in REC2020 colorspace, with bit depth 10/12/16 BPC. Some of the popular formats are P010,P012,P016. - HDR content come with their own Metadata to be applied to get the right luminance at the display device. - The metadata can be of two type 1. static 2. dynamic . For simplicity, this solution is focusing on static HDR only (HDR10 standard) - HDR content also provide its supported EOTF (electro optical transfer function) information, which is a curve (like SRGB gamma curve). One popular EOTF is PQ(ST2084). - HDR capable displays mention their EOTF and HDR metadata support information in EDID CEA-861-G blocks. - Normal SRGB buffers are composed in SRGB color space following REC709 specifications. - For accurate blending in display engines, we need to make sure following: - All the buffers are in same colorspace (Rec 709 or Rec 2020) - All the buffers are liner (gamma/EOTF removed) - All the buffers are tone mapped in same zone (HDR or SDR) Please refer to the block diagram below, which presents a simple case of a HDR P010 movie playback, with HDR buffers as video buffers, and SDR buffers as subtitles. The subsystem looks and works like this: - A client decodes the buffer (using FFMpeg for example) and gets the two buffers, one with video (HDR) and one subtitles (SDR) - Client passes following information to the compositor: - The actual buffers - Their colorspace infromation, BT2020 for HDR buffer, REC709 for SDR buffer (planning to add a new protocol extension for this) - The HDR metadata of the content (planning to add new protocol for this) - Compositors actions: - Reads the End display's HDR capabilities from display EDID. Assume its an HDR HDMI monitor. - Compositor tone maps every view's framebuffer to match tone of end display, applying a libVA filter. In this example: - The SDR subtitles frame will go through SDR to HDR tone mapping (called S2H) - The HDR video frame will go through HDR to HDR tone mapping (called H2H) if the HDR capabilities of monitor and content are different. - Now both the buffers and the monitor are in the same tone mapped range. - As the end display is HDR capable, and one of the content frame is HDR, the compositor will prepare all other planes for color space conversion (CSC) from REC709->REC2020 using plane CSC property. - As the CSC and blending should be done in liner space, compositor will also use plane level degamma to make the buffers linear. - These actions will make sure that, during blending: - All the buffers are in same colorspace (REC2020) - All the buffers are linear - All the buffers are tone mapped (HDR) - The plane level color properties patch, for DRM can be found here: https://patchwork.freedesktop.org/series/30875/ - Now, in order to re-apply the HDR curve, compositor will apply CRTC level gamma, so that the output buffer is non-linear again. - To pass the output HDR information to kernel, so that it can create and send AVI-info-frames to HDMI, compositor will set Connector HDR metadata property. - Code for the same can be found here: https://patchwork.freedesktop.org/series/25091/ - And they will ever live happily after :). Please provide inputs, feedbacks and suggestions for this design and plan, so that we can improve out half cooked solution, and start sending the patches. +--+ +---+ | SDR Buffer subtitles | HDR Buffer video | (REC 709 colorsp) | (REC 2020 colorsp | | | | | +---+--+ +---+---+ || || || +--v---v+ +--+ | Compositor: v | | LibVA| | - assigns views to overlays +-> Tone mapping | | - prepare plane/CRTC color properties <-+ SDR to HDR | | for linear blending in display | | HDR to SDR | +--+-+--+ +--+ | | | Tone mapped | Tone mapped | non-linear-Rec709 | non-linear Rec2020
Re: [PATCH v5] compositor-drm: Add aspect-ratio parsing support
Regards Shashank On 7/3/2018 3:26 PM, Pekka Paalanen wrote: On Tue, 3 Jul 2018 14:42:06 +0530 "Sharma, Shashank" wrote: Hello Pekka, Thanks for your attention and code review of the patch series. Ankit will respond on rest of the review comments, I would take one related to mode with implicit aspect ratio information. Please find my comment inline. Regards Shashank On 6/28/2018 7:09 PM, Pekka Paalanen wrote: On Sun, 20 May 2018 18:33:01 +0530 "Nautiyal, Ankit K" wrote: From: Ankit Nautiyal The flag bits 19-22 of the connector modes, provide the aspect-ratio information. This information can be stored in flags bits of the weston mode structure, so that it can used for setting a mode with a particular aspect-ratio. Currently, DRM layer supports aspect-ratio with atomic-modesetting by default. For legacy modeset path, the user-space needs to set the drm client cap for aspect-ratio, if it wants aspect-ratio information in modes. This patch: - preserves aspect-ratio flags from kernel video modes and accommodates it in wayland mode. - uses aspect-ratio to pick the appropriate mode during modeset. - changes the mode format in configuration file weston.ini to accommodate aspect-ratio information as: WIDTHxHEIGHT at REFRESH-RATE ASPECT-RATIO The aspect-ratio should be given as . - modifies the man pages to explain the usage of different mode format configurations in weston.ini. v2: As per recommendation from Pekka Paalanen, Quentin Glidic, Daniel Stone, dropped the aspect-ratio info from wayland protocol, thereby avoiding exposure of aspect-ratio to the client. v3: As suggested by Pekka Paalanen, added aspect_ratio field to store aspect-ratio information from the drm. Also added drm client capability for aspect-ratio, as recommended by Daniel Vetter. v4: Minor modifications and fixes as suggested by Pekka Paalanen. v5: Rebased, fixed some styling issues, and added aspect-ratio information while printing weston_modes. Signed-off-by: Ankit Nautiyal Acked-by: Pekka Paalanen (v4) --- libweston/compositor-drm.c | 115 + libweston/compositor-drm.h | 21 +++ libweston/compositor.h | 1 + man/weston.ini.man | 13 +++-- 4 files changed, 136 insertions(+), 14 deletions(-) Hi Ankit, it's been a long while since I last looked at this, so forgive me if I go in circles with my review comments. I see the aspect ratio bits are in Linus' RC kernel. I would like to get this merged for the next release which basically means during the next week if you can. diff --git a/man/weston.ini.man b/man/weston.ini.man index f237fd60..e982cac9 100644 --- a/man/weston.ini.man +++ b/man/weston.ini.man @@ -371,10 +371,15 @@ The DRM backend accepts different modes: .PP .RS 10 .nf -.BR "WIDTHxHEIGHT" "Resolution size width and height in pixels" -.BR "preferred " "Uses the preferred mode" -.BR "current " "Uses the current crt controller mode" -.BR "off " "Disables the output" +.BR "WIDTHxHEIGHT " "Resolution size width and height in pixels" +.BR "WIDTHxHEIGHT@RR" "Resolution as above and refresh-rate in Hertz" +.BR "WIDTHxHEIGHT@RR RATIO " "Resolution as above and aspect-ratio as length:breadth" I think the length:breadth could be confusing. H:V? hor:vert? Or simply A:B, assuming that aspect ratio is general knowledge. It would also be worth to list all the acceptable values, because there are only few. One cannot give e.g. 8:6 and assume Weston figures out it means 4:3. If they are explicitly listed, we could just use RATIO and not split the value further. Since these are specific to the DRM-backend, they would be more logical in weston-drm.man. + +e.g. 720x576@50 4:3, 1920x1080@60 16:9, 2560x1080@60 64:27, 4096x2160@60 256:135 etc. What is the point of giving the aspect ratio explicitly if the resolution directly results in the same aspect ratio? Shouldn't the examples show cases where it actually matters, i.e. when the display shows non-square pixels? E.g. 1024x768 16:9 The implementation also makes a difference between implicit aspect ratio (that is, NONE) and explicit aspect ratio even if the two are equal. Is that intentional? Is there an actual difference? Yes there is an actual difference, let me try to explain this. For this, we should first understand difference between CEA-modes Vs non-CEA-modes. As you might already know, CEA defines timings of a video mode, which is considered as standard for HDMI certification and compliance testing. CEA defines each and every parameter of a video mode, like h/vactive, h/vfront, h/vback including aspect ratio information. This unique videmode is given a Video Identification Code (VIC) which is unique. For example, VIC=4 is 1280x720@
Re: [PATCH v5] compositor-drm: Add aspect-ratio parsing support
Hello Pekka,
Thanks for your attention and code review of the patch series.
Ankit will respond on rest of the review comments, I would take one
related to mode with implicit aspect ratio information.
Please find my comment inline.
Regards
Shashank
On 6/28/2018 7:09 PM, Pekka Paalanen wrote:
On Sun, 20 May 2018 18:33:01 +0530
"Nautiyal, Ankit K" wrote:
From: Ankit Nautiyal
The flag bits 19-22 of the connector modes, provide the aspect-ratio
information. This information can be stored in flags bits of the
weston mode structure, so that it can used for setting a mode with a
particular aspect-ratio.
Currently, DRM layer supports aspect-ratio with atomic-modesetting by
default. For legacy modeset path, the user-space needs to set the
drm client cap for aspect-ratio, if it wants aspect-ratio information
in modes.
This patch:
- preserves aspect-ratio flags from kernel video modes and
accommodates it in wayland mode.
- uses aspect-ratio to pick the appropriate mode during modeset.
- changes the mode format in configuration file weston.ini to
accommodate aspect-ratio information as:
WIDTHxHEIGHT at REFRESH-RATE ASPECT-RATIO
The aspect-ratio should be given as .
- modifies the man pages to explain the usage of different mode format
configurations in weston.ini.
v2: As per recommendation from Pekka Paalanen, Quentin Glidic,
Daniel Stone, dropped the aspect-ratio info from wayland protocol,
thereby avoiding exposure of aspect-ratio to the client.
v3: As suggested by Pekka Paalanen, added aspect_ratio field to store
aspect-ratio information from the drm. Also added drm client
capability for aspect-ratio, as recommended by Daniel Vetter.
v4: Minor modifications and fixes as suggested by Pekka Paalanen.
v5: Rebased, fixed some styling issues, and added aspect-ratio
information while printing weston_modes.
Signed-off-by: Ankit Nautiyal
Acked-by: Pekka Paalanen (v4)
---
libweston/compositor-drm.c | 115 +
libweston/compositor-drm.h | 21 +++
libweston/compositor.h | 1 +
man/weston.ini.man | 13 +++--
4 files changed, 136 insertions(+), 14 deletions(-)
Hi Ankit,
it's been a long while since I last looked at this, so forgive me if I
go in circles with my review comments. I see the aspect ratio bits are
in Linus' RC kernel. I would like to get this merged for the next
release which basically means during the next week if you can.
I didn't see the aspect ratio bits in libdrm master yet, I guess it
should be ok to re-import the headers from kernel to libdrm by now.
diff --git a/libweston/compositor-drm.c b/libweston/compositor-drm.c
index 287431eb..e1d79e49 100644
--- a/libweston/compositor-drm.c
+++ b/libweston/compositor-drm.c
@@ -73,6 +73,10 @@
#define DRM_CLIENT_CAP_UNIVERSAL_PLANES 2
#endif
+#ifndef DRM_CLIENT_CAP_ASPECT_RATIO
+#define DRM_CLIENT_CAP_ASPECT_RATIO4
+#endif
+
#ifndef DRM_CAP_CURSOR_WIDTH
#define DRM_CAP_CURSOR_WIDTH 0x8
#endif
@@ -272,6 +276,8 @@ struct drm_backend {
uint32_t pageflip_timeout;
bool shutting_down;
+
+ bool aspect_ratio_supported;
};
struct drm_mode {
@@ -3049,6 +3055,19 @@ err:
drmModeSetCursor(b->drm.fd, output->crtc_id, 0, 0, 0);
}
+/*
+ * Get the aspect-ratio from drmModeModeInfo mode flags.
+ *
+ * @param drm_mode_flags- flags from drmModeModeInfo structure.
+ * @returns aspect-ratio as encoded in enum 'weston_mode_aspect_ratio'.
+ */
+static uint8_t
+drm_to_weston_mode_aspect_ratio(uint32_t drm_mode_flags)
+{
+ return (drm_mode_flags & DRM_MODE_FLAG_PIC_AR_MASK) >>
+ DRM_MODE_FLAG_PIC_AR_BITS_POS;
+}
+
static void
drm_assign_planes(struct weston_output *output_base, void *repaint_data)
{
@@ -3179,25 +3198,44 @@ drm_assign_planes(struct weston_output *output_base,
void *repaint_data)
static struct drm_mode *
choose_mode (struct drm_output *output, struct weston_mode *target_mode)
{
- struct drm_mode *tmp_mode = NULL, *mode;
+ struct drm_mode *tmp_mode = NULL, *mode_fall_back = NULL, *mode;
+ uint8_t src_aspect = 0;
+ uint8_t target_aspect = 0;
+ struct drm_backend *b;
+ b = to_drm_backend(output->base.compositor);
+ target_aspect = target_mode->aspect_ratio;
+ src_aspect = output->base.current_mode->aspect_ratio;
if (output->base.current_mode->width == target_mode->width &&
output->base.current_mode->height == target_mode->height &&
(output->base.current_mode->refresh == target_mode->refresh ||
-target_mode->refresh == 0))
- return to_drm_mode(output->base.current_mode);
+target_mode->refresh == 0)) {
+ if (!b->aspect_ratio_supported || src_aspect == target_aspect)
+ return (struct drm_mode *)output->base.current_mode;
Please keep to_drm_mode() instead of converting it back to a cast.
+ }
wl_list_for_each(mode, &output->base.mode_li
Re: [PATCH] Add aspect-ratio support in wayland layer.
Hello Pekka, Sorry for the delay in response, I was OOO. I had a quick syncup with Ankit, where he told me about the discussions you guys had, over this topic. Please find my comments inline. On 9/12/2017 2:49 PM, Pekka Paalanen wrote: On Mon, 11 Sep 2017 14:36:13 +0530 "Sharma, Shashank" wrote: Hello Pekka Thanks for your review comments and constructive discussions with Ankit on IRC. I am Shashank from Intel, I am guiding Ankit on this activity. I have few comments (inline), it would be great if you can let us know your view on this. Hi, nice to meet you! Thanks, same here. Please bear with my ignorance below. On 9/8/2017 7:43 PM, Pekka Paalanen wrote: On Wed, 6 Sep 2017 17:50:00 +0530 Nautiyal Ankit wrote: From: Ankit Nautiyal This patch series adds video mode aspect ratio support for wayland protocol and weston drm compositor. This patch series adds two patches: 1- Adds aspect ratio flag bits bits in wl_output_mode flags as per DRM layer implementation. 2- Preserves and uses the aspect ratio flags while selecting the mode for modeset in weston drm compositor. The corresponding kernel implementation for the aspect ratio support in DRM Layer can be found here: https://patchwork.freedesktop.org/series/10850/ This is a four patch series, in which two patches are already merged, and two are waiting for user space implementation. Hi, we discussed these patches on #wayland in IRC, and I believe we came to the following conclusion: - Let's not add to wayland.xml, and not advertise the aspect ratio to clients. There is no need to do this, and clients are currently not able to take advantage of it either. - In this case, how to decide what should be the aspect ratio of the output mode and based on what ? For example If a CEA mode is available for 16:9 and 4:3 aspects, which one to pick and how ? If we don't get inputs from the client, we would be picking the first mode in the list which might not be the best practice. Clients have no part in picking the video mode in general. Like I said, there is no standard Wayland protocol to tell the compositor to pick a video mode at this time. I'm also starting to think that exposing the list of possible video modes was not well thought through and would have been better omitted completely. It already breaks apart on nested compositors where instead of a few discrete modes one can pick freely from a range of widths and heights, for instance. The policy and configuration to choose the appropriate video mode lies with each compositor. In Weston's case those are weston.ini for configuration and the DRM-backend has the policy. Weston's policy is simple and probably inadequate as it is. The way to fix that is to patch Weston. How to pick the right mode is a question that has no answer without a specific use case and the environment. I envision moving the policy from the DRM-backend to the users of libweston (into e.g. Weston) in the future. For example for a desktop, I struggle to imagine why anyone would want to pick a video mode with non-square pixels. All generic software assumes pixels are square. I would assume that also display panels are built with square pixels, no? For TV and such uses, I know nothing at all except for the feeling that the video standards are still crippled by the designs from the 1960s. If a compositor is being used for a special, non-desktop purpose, I would also imagine it encodes its own configuration and policy for choosing video modes. The choice requires case specific knowledge. Clients could also be involved via domain specific Wayland protocol extensions. These are some very valid points, and I agree to most of them. But probably we should discus about little bit about the theory of aspect ratio here. There are three variants of aspect ratios which affect the screen display 1. Display aspect ratio (DAR): Its the ratio of physical dimension of TV/monitor (width mm / height mm) 2. Storage aspect ratio (SAR): Its the ratio of the pixels of the picture captured at the source. For example, a 1920x1080 image, SAR would be 1920:1080 = 16:9 3. Pixel aspect ratio (PAR): The aspect ratio of the pixels being displayed on the monitor. So an image of resolution 1920x1080 (SAR 16:9) when being displayed on a TV of (DAR 16:9) will have square pixels of (PAR 1:1). So from the definition, we can deduce that PAR = DAR/SAR While sending the AVI info-frames from source to sink, the source set aspect ratio field, and the monitor prepares itself accordingly. In fact, there is a HDMI compliance certification test (7-27) which checks if the selected aspect ratio is being reflected in AVI IF correctly. Now, there are many widescreen TVs and gaming monitors available in market, where a selected gaming app / media player / dvd player / set-top box would want to show non-square pixels for: - widescreen effect / theater effect. - gaming experience. - VR/AR
Re: [PATCH] Add aspect-ratio support in wayland layer.
Hello Pekka Thanks for your review comments and constructive discussions with Ankit on IRC. I am Shashank from Intel, I am guiding Ankit on this activity. I have few comments (inline), it would be great if you can let us know your view on this. Regards Shashank On 9/8/2017 7:43 PM, Pekka Paalanen wrote: On Wed, 6 Sep 2017 17:50:00 +0530 Nautiyal Ankit wrote: From: Ankit Nautiyal This patch series adds video mode aspect ratio support for wayland protocol and weston drm compositor. This patch series adds two patches: 1- Adds aspect ratio flag bits bits in wl_output_mode flags as per DRM layer implementation. 2- Preserves and uses the aspect ratio flags while selecting the mode for modeset in weston drm compositor. The corresponding kernel implementation for the aspect ratio support in DRM Layer can be found here: https://patchwork.freedesktop.org/series/10850/ This is a four patch series, in which two patches are already merged, and two are waiting for user space implementation. Hi, we discussed these patches on #wayland in IRC, and I believe we came to the following conclusion: - Let's not add to wayland.xml, and not advertise the aspect ratio to clients. There is no need to do this, and clients are currently not able to take advantage of it either. - In this case, how to decide what should be the aspect ratio of the output mode and based on what ? For example If a CEA mode is available for 16:9 and 4:3 aspects, which one to pick and how ? If we don't get inputs from the client, we would be picking the first mode in the list which might not be the best practice. - While supporting advance display outputs like HDMI 2.0 / UHD, when the content can be in super wide aspect like 64:27 for movies/game, wouldn't it be under utilization of display capabilities not to pick it ? - Shashank - With the wl_output bits dropped, the idea in the Weston patch is good. You are adding a way to choose non-square pixel modes via weston.ini, and to choose the refresh rate as well. The purpose of the Weston patch is to provide a real test vehicle for the remaining kernel patches. The above mentioned should suffice, I believe. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
