Re: Call for Discussion : New Project to support the Wayland display server on Linux

[Mario Torre]

I tend to agree with this, we certainly need to identify the underlying APIs 
that are problematic (screenshot and mouse/keyboard control as far as I am 
aware), then see what we can do from there, and I would like to first address 
the XWayland use case before turning into a fully fledged new toolkit 
implementation (but I agree with Phil that this is the long term necessity).

What I would like to do however is to see if it makes sense to create a GTK 
toolkit rather than a Wayland one, and let GTK deal with every abstraction. We 
can be pretty confident that GTK will always be there, and will work on X11 and 
Wayland transparently, so even a user on a pure KDE desktop won’t really need 
to deal directly with Wayland.

Ideally, users with pure KDE distributions (OpenSuSE maybe?) may help here 
analyse what the requirements are. One problem that comes to mind when using 
GTK as a toolkit is the interaction with other GTK based applications that may 
be embedded, i.e. Swing views in Eclipse, Firefox based web views, etc.. etc… 
But I think this is already today an issue and is dealt with in the current 2d 
and JFX code already.

Cheers,
Mario


> On 7. Jul 2021, at 23:52, Alexander Zvegintsev 
>  wrote:
> 
> (adding awt-dev)
> 
> Let me add a few comments.
> 
>> already Wayland is the default on RHEL 8, OL 8, Ubuntu 21.04 and I am sure 
>> others too.
> It is the default, but not in every case. Wayland may be turned off 
> deliberately if you are using a Nvidia graphics card, so you need to take 
> extra steps to get it working 
> .
> 
> I faced this issue on Ubuntu 21.04.
> 
> However things getting better 
> with
>  Nvidia 470 drivers, its beta released 
>  on 
> 2021.6.22. So probably it won't be a problem in the near future.
> 
>> Consequently we expect quite shortly to propose an OpenJDK project that will 
>> consider the goals of
>> - a short to medium term solution for JDK running on Wayland in X11 
>> compatibility mode
>> - a medium to long term solution for JDK running as a native Wayland client.
> 
> Both goals having a common task: we will need to implement java.awt.Robot 
> functionality for Wayland(at least).
> 
> So it makes sense to make XToolkit's java.awt.Robot work under Wayland first, 
> and then reuse this code for native Wayland client.
> 
> I see two major tasks here: taking screenshots and mouse/keyboard control. As 
> far as I know there is no standard way to implement they across all display 
> servers yet.
> 
> Possible ways to implement this:
> 
> * taking screenshot:
> o open issue against Wayland, may be resolved
>   someday:https://gitlab.freedesktop.org/wayland/wayland/-/issues/32
>   
> o 
> https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.portal.Screenshot
>   
> 
> o via DBUS interface(display server dependent),
>   
> e.g.https://github.com/flameshot-org/flameshot/blob/master/src/utils/screengrabber.cpp#L43
>   
> 
> * generating key/mouse events:
> o 
> https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.impl.portal.RemoteDesktop
>   
> 
> o generating new virtual input device, uinput, superuser
>   privileges required, looks too flaky
>   
> https://unix.stackexchange.com/questions/422698/how-to-set-absolute-mouse-cursor-position-in-wayland-without-using-mouse
>   
> 
> 
> This still need more investigation, butxdg-desktop-portal 
> looks more preferable way for 
> now.
> 
> 
> Please see below some caveats for OpenJDK native Wayland client:
> 
> You can't control a position for a top-level window. 
> 
> 
> This will also affect a splashscreen windows. It is still possible to control 
> position under XWayland though.
> 
> Looks like we will just accept it.
> 
> 
> Top-level window decorations.
> 
> Initially Wayland had only client-side decorations(you have to draw it by 
> yourself).
> 
> As of now server-side decorations are available by XDG-Decoration protocol 
> .
> 
> However server-side window decorations are not mandatory and not all 
> compositors are supporting 

Re: Call for Discussion : New Project to support the Wayland display server on Linux

[Alexey Ushakov]

> I would be very interested in your experience here, when you say you
> have a lot of users on different desktops, what are the combinations
> you encounter, can you share this information, can we be sure GTK is
> not part of those distributions?

I’ve looked through the list of issues that we have resolved in the past and it 
looks like that many of the WMs indeed based on GTK. The rest of the issues are 
related to KDE. 
I’ll try to find some more info but it looks like that waste majority of users 
are either on GNOME based env or KDE based. However, we have a lot of different 
versions of GNOME and KDE in use. So, if we want to base our code on Gnome or 
KDE we need to provide some kind of compatibility layers (like we already have 
in JDK) and these layers should be continuously supported.

Best Regards,
Alexey

> On 9 Jul 2021, at 14:04, Mario Torre  wrote:
> 
> On Fri, Jul 9, 2021 at 12:17 PM Alexey Ushakov
>  wrote:
>> 
>> 
>> 
>> On 8 Jul 2021, at 19:11, Mario Torre  wrote:
>> 
>> [resending with hopefully the correct email address as I have
>> completely lost which mailing list I'm subscribed to with which
>> email!!!]
>> 
>> I tend to agree with this, we certainly need to identify the
>> underlying APIs that are problematic (screenshot and mouse/keyboard
>> control as far as I am aware), then see what we can do from there, and
>> I would like to first address the XWayland use case before turning
>> into a fully fledged new toolkit implementation (but I agree with Phil
>> that this is the long term necessity).
>> 
>> What I would like to do however is to see if it makes sense to create
>> a GTK toolkit rather than a Wayland one, and let GTK deal with every
>> abstraction. We can be pretty confident that GTK will always be there,
>> and will work on X11 and Wayland transparently, so even a user on a
>> pure KDE desktop won’t really need to deal directly with Wayland.
>> 
>> 
>> I’m not sure, that it’s right direction. We had a similar situation in early 
>> days of java. We had a Motif toolkit that was then replaced by XAWT 
>> implementation. I think in long term we need to be as close to low level 
>> interfaces as possible. We (JetBrains) have a lot of users on different 
>> desktops and having only GTK is not an option for us. So, I think that we 
>> need to have something like XAWT for Wayland.
> 
> Yeah, but the experience with that in the past led to the rather
> interesting result of having a lot of unmanageable special casing code
> to support all sorts of corner cases with window managers and
> toolkits. Also, Wayland is not a window manager but a protocol for a
> compositor, as such it doesn't really do much itself, we do rely on
> existing implementations. This means that if we have a pure Wayland
> toolkit it won't work on every KDE instance, for example, unless where
> KDE supports Wayland (but yes, that may not be an issue, since the X11
> variant will be there anyway).
> 
> I'm not advocating GTK as a sole solution, though, I fully agree with
> Phil that we need to first explore the options, and it may turn out
> that GTK isn't what we want after all and indeed a pure Wayland
> toolkit is the right approach as you suggest, but I think we should
> explore first existing abstractions and implementation and decide
> where to go.
> 
> I would be very interested in your experience here, when you say you
> have a lot of users on different desktops, what are the combinations
> you encounter, can you share this information, can we be sure GTK is
> not part of those distributions?
> 
> Cheers,
> Mario
> 
> -- 
> Mario Torre
> Manager, Software Engineering
> Red Hat GmbH 
> 9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898
> 

Re: Call for Discussion : New Project to support the Wayland display server on Linux

[Mario Torre]

On Fri, Jul 9, 2021 at 12:17 PM Alexey Ushakov
 wrote:
>
>
>
> On 8 Jul 2021, at 19:11, Mario Torre  wrote:
>
> [resending with hopefully the correct email address as I have
> completely lost which mailing list I'm subscribed to with which
> email!!!]
>
> I tend to agree with this, we certainly need to identify the
> underlying APIs that are problematic (screenshot and mouse/keyboard
> control as far as I am aware), then see what we can do from there, and
> I would like to first address the XWayland use case before turning
> into a fully fledged new toolkit implementation (but I agree with Phil
> that this is the long term necessity).
>
> What I would like to do however is to see if it makes sense to create
> a GTK toolkit rather than a Wayland one, and let GTK deal with every
> abstraction. We can be pretty confident that GTK will always be there,
> and will work on X11 and Wayland transparently, so even a user on a
> pure KDE desktop won’t really need to deal directly with Wayland.
>
>
> I’m not sure, that it’s right direction. We had a similar situation in early 
> days of java. We had a Motif toolkit that was then replaced by XAWT 
> implementation. I think in long term we need to be as close to low level 
> interfaces as possible. We (JetBrains) have a lot of users on different 
> desktops and having only GTK is not an option for us. So, I think that we 
> need to have something like XAWT for Wayland.

Yeah, but the experience with that in the past led to the rather
interesting result of having a lot of unmanageable special casing code
to support all sorts of corner cases with window managers and
toolkits. Also, Wayland is not a window manager but a protocol for a
compositor, as such it doesn't really do much itself, we do rely on
existing implementations. This means that if we have a pure Wayland
toolkit it won't work on every KDE instance, for example, unless where
KDE supports Wayland (but yes, that may not be an issue, since the X11
variant will be there anyway).

I'm not advocating GTK as a sole solution, though, I fully agree with
Phil that we need to first explore the options, and it may turn out
that GTK isn't what we want after all and indeed a pure Wayland
toolkit is the right approach as you suggest, but I think we should
explore first existing abstractions and implementation and decide
where to go.

I would be very interested in your experience here, when you say you
have a lot of users on different desktops, what are the combinations
you encounter, can you share this information, can we be sure GTK is
not part of those distributions?

Cheers,
Mario

-- 
Mario Torre
Manager, Software Engineering
Red Hat GmbH 
9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898

Re: Call for Discussion : New Project to support the Wayland display server on Linux

[Mario Torre]

[resending with hopefully the correct email address as I have
completely lost which mailing list I'm subscribed to with which
email!!!]

I tend to agree with this, we certainly need to identify the
underlying APIs that are problematic (screenshot and mouse/keyboard
control as far as I am aware), then see what we can do from there, and
I would like to first address the XWayland use case before turning
into a fully fledged new toolkit implementation (but I agree with Phil
that this is the long term necessity).

What I would like to do however is to see if it makes sense to create
a GTK toolkit rather than a Wayland one, and let GTK deal with every
abstraction. We can be pretty confident that GTK will always be there,
and will work on X11 and Wayland transparently, so even a user on a
pure KDE desktop won’t really need to deal directly with Wayland.

Ideally, users with pure KDE distributions (OpenSuSE maybe?) may help
here analyse what the requirements are. One problem that comes to mind
when using GTK as a toolkit is the interaction with other GTK based
applications that may be embedded, i.e. Swing views in Eclipse,
Firefox based web views, etc.. etc… But I think this is already today
an issue and is dealt with in the current 2d and JFX code already.

Cheers,
Mario

On Wed, Jul 7, 2021 at 11:56 PM Alexander Zvegintsev
 wrote:
>
> (adding awt-dev)
>
> Let me add a few comments.
>
> > already Wayland is the default on RHEL 8, OL 8, Ubuntu 21.04 and I am
> > sure others too.
> It is the default, but not in every case. Wayland may be turned off
> deliberately if you are using a Nvidia graphics card, so you need to
> take extra steps to get it working
> .
>
> I faced this issue on Ubuntu 21.04.
>
> However things getting better
> with
> Nvidia 470 drivers, its beta released
>  on
> 2021.6.22. So probably it won't be a problem in the near future.
>
> > Consequently we expect quite shortly to propose an OpenJDK project
> > that will consider the goals of
> > - a short to medium term solution for JDK running on Wayland in X11
> > compatibility mode
> > - a medium to long term solution for JDK running as a native Wayland
> > client.
>
> Both goals having a common task: we will need to implement
> java.awt.Robot functionality for Wayland(at least).
>
> So it makes sense to make XToolkit's java.awt.Robot work under Wayland
> first, and then reuse this code for native Wayland client.
>
> I see two major tasks here: taking screenshots and mouse/keyboard
> control. As far as I know there is no standard way to implement they
> across all display servers yet.
>
> Possible ways to implement this:
>
>   * taking screenshot:
>   o open issue against Wayland, may be resolved
> someday:https://gitlab.freedesktop.org/wayland/wayland/-/issues/32
> 
>   o 
> https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.portal.Screenshot
> 
> 
>   o via DBUS interface(display server dependent),
> 
> e.g.https://github.com/flameshot-org/flameshot/blob/master/src/utils/screengrabber.cpp#L43
> 
> 
>   * generating key/mouse events:
>   o 
> https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.impl.portal.RemoteDesktop
> 
> 
>   o generating new virtual input device, uinput, superuser
> privileges required, looks too flaky
> 
> https://unix.stackexchange.com/questions/422698/how-to-set-absolute-mouse-cursor-position-in-wayland-without-using-mouse
> 
> 
>
> This still need more investigation, butxdg-desktop-portal
> looks more preferable way
> for now.
>
>
> Please see below some caveats for OpenJDK native Wayland client:
>
> You can't control a position for a top-level window.
> 
>
> This will also affect a splashscreen windows. It is still possible to
> control position under XWayland though.
>
> Looks like we will just accept it.
>
>
> Top-level window decorations.
>
> Initially Wayland had only client-side decorations(you have to draw it
> by yourself).
>
> As of now server-side decorations are available by XDG-Decoration
> protocol 

Re: Call for Discussion : New Project to support the Wayland display server on Linux

[Alexey Ushakov]

Hi,

> Possible ways to implement this:
> 
> taking screenshot:
> open issue against Wayland, may be resolved someday: 
> https://gitlab.freedesktop.org/wayland/wayland/-/issues/32 
> 
> https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.portal.Screenshot
>  
> 
> via DBUS interface(display server dependent), e.g. 
> https://github.com/flameshot-org/flameshot/blob/master/src/utils/screengrabber.cpp#L43
>  
> 

I’ve already did an implementation of AWT Robot via DBUS interface. It works 
well for single snapshot but if you need to implement something like color 
picker (you need to obtain Color many times per second) this approach severely 
slows things down.

Best Regards,
Alexey 

> On 8 Jul 2021, at 00:52, Alexander Zvegintsev 
>  wrote:
> 
> (adding awt-dev)
> 
> Let me add a few comments.
> 
>> already Wayland is the default on RHEL 8, OL 8, Ubuntu 21.04 and I am sure 
>> others too.
> It is the default, but not in every case. Wayland may be turned off 
> deliberately if you are using a Nvidia graphics card, so you need to take 
> extra steps to get it working 
> .
> 
> I faced this issue on Ubuntu 21.04. 
> 
> However things getting better  
> with
>  Nvidia 470 drivers, its beta released 
>  on 
> 2021.6.22. So probably it won't be a problem in the near future.
> 
>> Consequently we expect quite shortly to propose an OpenJDK project that will 
>> consider the goals of 
>> - a short to medium term solution for JDK running on Wayland in X11 
>> compatibility mode 
>> - a medium to long term solution for JDK running as a native Wayland client.
> 
> Both goals having a common task: we will need to implement java.awt.Robot 
> functionality for Wayland(at least).
> 
> So it makes sense to make XToolkit's java.awt.Robot work under Wayland first, 
> and then reuse this code for native Wayland client.
> 
> I see two major tasks here: taking screenshots and mouse/keyboard control. As 
> far as I know there is no standard way to implement they across all display 
> servers yet.
> 
> Possible ways to implement this:
> 
> taking screenshot:
> open issue against Wayland, may be resolved someday: 
> https://gitlab.freedesktop.org/wayland/wayland/-/issues/32 
> 
> https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.portal.Screenshot
>  
> 
> via DBUS interface(display server dependent), e.g. 
> https://github.com/flameshot-org/flameshot/blob/master/src/utils/screengrabber.cpp#L43
>  
> 
> generating key/mouse events:
> https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.impl.portal.RemoteDesktop
>  
> 
> generating new virtual input device, uinput, superuser privileges required, 
> looks too flaky 
> https://unix.stackexchange.com/questions/422698/how-to-set-absolute-mouse-cursor-position-in-wayland-without-using-mouse
>  
> 
> This still need more investigation, but xdg-desktop-portal 
>  looks more preferable way for 
> now.
> 
> 
> 
> Please see below some caveats for OpenJDK native Wayland client:
> 
> You can't control a position for a top-level window. 
>  
> 
> This will also affect a splashscreen windows. It is still possible to control 
> position under XWayland though.
> 
> Looks like we will just accept it.
> 
> 
> 
> Top-level window decorations. 
> 
> Initially Wayland had only client-side decorations(you have to draw it by 
> yourself).
> 
> As of now server-side decorations are available by XDG-Decoration protocol 
> .
> 
> However server-side window decorations are not mandatory and not all 
> compositors are supporting it, e.g. Mutter(Gnome Shell's compositor).
> 
> Gnome Shell is the default on Ubuntu, so we will need to provide window 
> decorations somehow. One of possible solutions is to use Gtk to create a 
> window for us.
> 
> 
> 
> --
> Thanks,
> Alexander.
> On 7/7/21 6:24 

Re: Call for Discussion : New Project to support the Wayland display server on Linux

[Alexander Zvegintsev]

(adding awt-dev)

Let me add a few comments.

already Wayland is the default on RHEL 8, OL 8, Ubuntu 21.04 and I am 
sure others too. 
It is the default, but not in every case. Wayland may be turned off 
deliberately if you are using a Nvidia graphics card, so you need to 
take extra steps to get it working 
.


I faced this issue on Ubuntu 21.04.

However things getting better 
with 
Nvidia 470 drivers, its beta released 
 on 
2021.6.22. So probably it won't be a problem in the near future.


Consequently we expect quite shortly to propose an OpenJDK project 
that will consider the goals of
- a short to medium term solution for JDK running on Wayland in X11 
compatibility mode
- a medium to long term solution for JDK running as a native Wayland 
client. 


Both goals having a common task: we will need to implement 
java.awt.Robot functionality for Wayland(at least).


So it makes sense to make XToolkit's java.awt.Robot work under Wayland 
first, and then reuse this code for native Wayland client.


I see two major tasks here: taking screenshots and mouse/keyboard 
control. As far as I know there is no standard way to implement they 
across all display servers yet.


Possible ways to implement this:

 * taking screenshot:
 o open issue against Wayland, may be resolved
   someday:https://gitlab.freedesktop.org/wayland/wayland/-/issues/32
   
 o 
https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.portal.Screenshot
   

 o via DBUS interface(display server dependent),
   
e.g.https://github.com/flameshot-org/flameshot/blob/master/src/utils/screengrabber.cpp#L43
   

 * generating key/mouse events:
 o 
https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.impl.portal.RemoteDesktop
   

 o generating new virtual input device, uinput, superuser
   privileges required, looks too flaky
   
https://unix.stackexchange.com/questions/422698/how-to-set-absolute-mouse-cursor-position-in-wayland-without-using-mouse
   


This still need more investigation, butxdg-desktop-portal 
looks more preferable way 
for now.



Please see below some caveats for OpenJDK native Wayland client:

You can't control a position for a top-level window. 



This will also affect a splashscreen windows. It is still possible to 
control position under XWayland though.


Looks like we will just accept it.


Top-level window decorations.

Initially Wayland had only client-side decorations(you have to draw it 
by yourself).


As of now server-side decorations are available by XDG-Decoration 
protocol .


However server-side window decorations are not mandatory and not all 
compositors are supporting it, e.g. Mutter(Gnome Shell's compositor).


Gnome Shell is the default on Ubuntu, so we will need to provide window 
decorations somehow. One of possible solutions is to use Gtk to create a 
window for us.



--
Thanks,
Alexander.

On 7/7/21 6:24 AM, Philip Race wrote:


For a number of years the Linux community has been working on a 
complete replacement for the 1980's era X11 desktop display server 
protocol
with new protocols and libraries that support client-side rendering 
and a compositing desktop windowing system.
This work is being done under the auspices of the Wayland project [1] 
and there is a reference

implementation of a Wayland compositor called "Weston".

A new client written for  the Wayland desktop has no dependency at all 
on X11, but Wayland also provides a compatibility
mode where the X.org X11 server runs along side Wayland, so that 
thousands of X11 applications can continue to run.


Presently all distros that ship the Wayland server, also still ship 
the pure X11 server and the user can select
which one to use on the login screen. However there will come a time 
when Wayland is the only choice and
already Wayland is the default on RHEL 8, OL 8, Ubuntu 21.04 and I am 
sure others too.


At that time Java for Linux will "mostly" run via the X11 
compatibility layer, but would not pass the JCK,
since some important APIs,  notably