https://bugs.kde.org/show_bug.cgi?id=449273
Bug ID: 449273 Summary: Mouse clicks from openQA (automated test framework) in Fedora installer on KDE 5.23.90 stop working shortly after it starts Product: kwin Version: 5.23.90 Platform: Fedora RPMs OS: Linux Status: REPORTED Severity: normal Priority: NOR Component: input Assignee: kwin-bugs-n...@kde.org Reporter: ad...@happyassassin.net Target Milestone: --- SUMMARY I maintain Fedora's instance of an automated test system called openQA: https://openqa.fedoraproject.org . openQA allows for "human-like" testing of graphical applications by matching areas of screenshots and performing mouse and keyboard input operations via a VNC connection. We run a suite of openQA tests on Fedora composes and updates. Since KDE/Plasma 5.23.90 landed in Fedora Rawhide, the test that runs a Fedora installation from the KDE live image is failing. The test boots the live image, double-clicks an icon on the desktop to launch the installer - which runs maximized and without a window title, see attached screenshot - then starts trying to navigate through it with keyboard and mouse inputs. For example, the first page of the installer is a language selection screen (as shown in the screenshot). The test first checks that it's at this screen and clicks on a non-active area of it (the word "FEDORA" in "WELCOME TO FEDORA 36") to ensure the window is active. Then it clicks in the text entry box at bottom left and types the word "english". Then it looks for "English (United States)" on the right-hand side and clicks on that. Then it looks for the button labelled "Continue" and clicks on that. Ordinarily this would indeed continue the installation process, and that's what happens on the same test run on the GNOME live image, and on the same test run on our dedicated installer images (which use gnome-kiosk), and what used to happen when running the test on KDE. But with Plasma 5.23.90, when openQA clicks on "Continue", nothing happens. We've established a few things by experimentation. It seems that once we reach the broken state, clicking on anything in the installer no longer works. For instance, I tried modifying the test to click on a different language, which should lead to that language being selected - but it doesn't. It seems the first one or two clicks in the installer window work OK, but after that, they stop working. However, clicking on something outside the installer window - e.g. the KDE "kicker" icon - works fine. We also found that if we hijack the VNC connection the openQA test runner uses to inject input events, we can click on things in the installer successfully by hand. The bug for some reason only affects the events as input by openQA's test runner. Here are some more details about exactly how this is implemented in os-autoinst (the openQA test runner). The test is run on a qemu VM, with the built-in VNC server enabled. os-autoinst sends mouse and keyboard events via this VNC server. This is precisely what happens when it wants to "click on something": * Match the relevant area and determine its centre point * Set the cursor to that precise position; it does not "move" the cursor the way a human would, it just zaps it there with a single VNC event (see https://github.com/os-autoinst/os-autoinst/blob/master/consoles/VNC.pm#L745-L760) * Send a 'mouse button down' event * Wait 0.15 seconds * Send a 'mouse button up' event * Wait 1 second * Set the cursor to 1023,767: this is called a "mouse hide" event, the purpose is to position the cursor somewhere we hope it won't happen to interfere with a subsequent match attempt (i.e. the bottom right corner of the screen) Notably, this means that between clicks, the cursor is instantly repositioned to 1023x767 and then instantly positioned to the click target location. This is the most obvious part of the runner's behaviour that a human cannot replicate - when we take over the VNC connection and interact with the VM manually, obviously, we can't instantly zap the cursor wherever we want, we're just moving it there with our human hands and human mice. So this repositioning behaviour may be significant (it was in a similar bug we found in GNOME recently). STEPS TO REPRODUCE This is trivial to reproduce if you have an instance of openQA deployed with Fedora's test configuration, and probably close to impossible to reproduce if you don't, unfortunately. At least you'd probably have to replicate the whole 'instant cursor repositioning' thing. I can reproduce it on demand and provide whatever debugging information would be useful. I'm available on libera.chat IRC and chat.fedoraproject.org (they're bridged) as adamw. OBSERVED RESULT Mouse clicks stop working shortly after installer runs. EXPECTED RESULT Mouse clicks should keep working. SOFTWARE/OS VERSIONS Linux/KDE Plasma: Fedora Rawhide (available in About System) KDE Plasma Version: 5.23.90 KDE Frameworks Version: 5.23.90 Qt Version: 5.15.2 ADDITIONAL INFORMATION -- You are receiving this mail because: You are watching all bug changes.