https://bugs.kde.org/show_bug.cgi?id=449273
Bug ID: 449273
Summary: Mouse clicks from openQA (automated test framework) in
Fedora installer on KDE 5.23.90 stop working shortly
after it starts
Product: kwin
Version: 5.23.90
Platform: Fedora RPMs
OS: Linux
Status: REPORTED
Severity: normal
Priority: NOR
Component: input
Assignee: kwin-bugs-n...@kde.org
Reporter: ad...@happyassassin.net
Target Milestone: ---
SUMMARY
I maintain Fedora's instance of an automated test system called openQA:
https://openqa.fedoraproject.org . openQA allows for "human-like" testing of
graphical applications by matching areas of screenshots and performing mouse
and keyboard input operations via a VNC connection. We run a suite of openQA
tests on Fedora composes and updates. Since KDE/Plasma 5.23.90 landed in Fedora
Rawhide, the test that runs a Fedora installation from the KDE live image is
failing.
The test boots the live image, double-clicks an icon on the desktop to launch
the installer - which runs maximized and without a window title, see attached
screenshot - then starts trying to navigate through it with keyboard and mouse
inputs. For example, the first page of the installer is a language selection
screen (as shown in the screenshot). The test first checks that it's at this
screen and clicks on a non-active area of it (the word "FEDORA" in "WELCOME TO
FEDORA 36") to ensure the window is active. Then it clicks in the text entry
box at bottom left and types the word "english". Then it looks for "English
(United States)" on the right-hand side and clicks on that. Then it looks for
the button labelled "Continue" and clicks on that.
Ordinarily this would indeed continue the installation process, and that's what
happens on the same test run on the GNOME live image, and on the same test run
on our dedicated installer images (which use gnome-kiosk), and what used to
happen when running the test on KDE. But with Plasma 5.23.90, when openQA
clicks on "Continue", nothing happens.
We've established a few things by experimentation. It seems that once we reach
the broken state, clicking on anything in the installer no longer works. For
instance, I tried modifying the test to click on a different language, which
should lead to that language being selected - but it doesn't. It seems the
first one or two clicks in the installer window work OK, but after that, they
stop working. However, clicking on something outside the installer window -
e.g. the KDE "kicker" icon - works fine. We also found that if we hijack the
VNC connection the openQA test runner uses to inject input events, we can click
on things in the installer successfully by hand. The bug for some reason only
affects the events as input by openQA's test runner.
Here are some more details about exactly how this is implemented in os-autoinst
(the openQA test runner). The test is run on a qemu VM, with the built-in VNC
server enabled. os-autoinst sends mouse and keyboard events via this VNC
server. This is precisely what happens when it wants to "click on something":
* Match the relevant area and determine its centre point
* Set the cursor to that precise position; it does not "move" the cursor the
way a human would, it just zaps it there with a single VNC event (see
https://github.com/os-autoinst/os-autoinst/blob/master/consoles/VNC.pm#L745-L760)
* Send a 'mouse button down' event
* Wait 0.15 seconds
* Send a 'mouse button up' event
* Wait 1 second
* Set the cursor to 1023,767: this is called a "mouse hide" event, the purpose
is to position the cursor somewhere we hope it won't happen to interfere with a
subsequent match attempt (i.e. the bottom right corner of the screen)
Notably, this means that between clicks, the cursor is instantly repositioned
to 1023x767 and then instantly positioned to the click target location. This is
the most obvious part of the runner's behaviour that a human cannot replicate -
when we take over the VNC connection and interact with the VM manually,
obviously, we can't instantly zap the cursor wherever we want, we're just
moving it there with our human hands and human mice. So this repositioning
behaviour may be significant (it was in a similar bug we found in GNOME
recently).
STEPS TO REPRODUCE
This is trivial to reproduce if you have an instance of openQA deployed with
Fedora's test configuration, and probably close to impossible to reproduce if
you don't, unfortunately. At least you'd probably have to replicate the whole
'instant cursor repositioning' thing. I can reproduce it on demand and provide
whatever debugging information would be useful. I'm available on libera.chat
IRC and chat.fedoraproject.org (they're bridged) as adamw.
OBSERVED RESULT
Mouse clicks stop working shortly after installer runs.
EXPECTED RESULT
Mouse clicks should keep working.
SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Fedora Rawhide
(available in About System)
KDE Plasma Version: 5.23.90
KDE Frameworks Version: 5.23.90
Qt Version: 5.15.2
ADDITIONAL INFORMATION
--
You are receiving this mail because:
You are watching all bug changes.
