The issue is the inability to move the mouse cursor using `java.awt.Robot` on 
Oracle Linux 10 (which has not yet been released). FIY, the OL10 is no longer 
provides an Xorg session, so only Wayland is available.
It is a JCK conformance failure. 

---

In our robot implementation, we emulate mouse/keyboard actions using the XTEST 
extension protocol.

The XTEST calls are now wired to 
[libEI](https://libinput.pages.freedesktop.org/libei/api/index.html):

1.

It now asks for a user confirmation to allow interaction, so we should now be 
able to click on native applications, window titles, etc.

However, our robot implementation is mostly uses XTEST, but there is an 
exception:
for mouse move we use XWarpPointer, which is not part of the XTEST, thus it is 
not propagated outside of Xwayland server.

So we have a situation where mouse is moving only inside Xwayland server, and 
we see the mouse enter/exit events,
on mouse press we call XTestFakeButtonEvent and it is handled by libei,
so the click happens where is the actual mouse cursor is in the system. That is 
why the test fails.

To avoid this situation we should use XTestFakeMotionEvent for mouseMove in 
Xwayland case.

It would be a simple solution, but


2.

This permission granted by a user doesn't persist across reboots, it timeouts 
after a period of inactivity.

It still seems to conform to the specifications, but we will not be able to 
conveniently run automated tests in this case.

---

So as a solution we can switch the [Remote Desktop XDG 
portal](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.RemoteDesktop.html).
It has session and `restore_token` functionality to keep permissions granted by 
a user, similar to what we use in a 
[Screencast](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html)
 for taking screenshots.
Most of the code can be reused, and it has everything we need.


* It is only enabled by default on Wayland and Gnome Shell 47 and higher due to 
XTEST API complications that first arose in this version. For earlier versions, 
Screencast is used instead.
* A new value has been introduced for the `awt.robot.screenshotMethod` system 
property - `dbusRemoteDesktop`. It works on Ubuntu 24.04 with Gnome 46, but is 
not enabled by default.
* extra mouse buttons are not supported yet, 
[JDK-8357142](https://bugs.openjdk.org/browse/JDK-8357142)
* Since we are now moving the mouse using a non-X11 API and 
`MouseInfo.getPointerInfo().getLocation()` uses the X11 API to get the mouse 
pointer location, `XRobotPeer#mouseMove` still calls the old API to update the 
mouse position within the XWayland server.
This workaround is necessary to pass the another conformance test.

-------------

Commit messages:
 - 8351907: [XWayland] [OL10] Robot.mousePress() is delivered to wrong place

Changes: https://git.openjdk.org/jdk/pull/25265/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=25265&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8351907
  Stats: 941 lines in 13 files changed: 804 ins; 51 del; 86 mod
  Patch: https://git.openjdk.org/jdk/pull/25265.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/25265/head:pull/25265

PR: https://git.openjdk.org/jdk/pull/25265

Reply via email to