Hello KDE and XFCE teams, I am writing to you on behalf of the Fedora QA team, because you are the stakeholders of release-blocking desktop environments in Fedora.
We believe that there is need to revisit and review some of the approaches we have in desktop testing. I hope that I can clarify in the text. *How do we test desktop applications?* Currently, there are three desktop environments, which block the Fedora releases: GNOME, KDE, and XFCE (on ARM). Among others, there is a test case called “Desktop Menus” [ https://fedoraproject.org/wiki/QA:Testcase_desktop_menus ] which is used for all three desktop environments. *What is the purpose of the Desktop Menus test case?* The purpose of the “Desktop Menus” test case is to test that applications listed in the menu do generally work. That means that each application in the default system menu must launch without crashing and withstand basic usage. [2] *What problems can arise when trying to follow the test case?* This approach leaves a lot of space for variability, because the terms are not clearly given, especially when talking about “basic functionality” of the desktop applications. In the past, there have been some disputes during Blocker Review meetings, or during Go/No go meetings whether basic functionality is broken or not. It's usually up to group judgement to decide what is a basic functionality and what it beyond that, which means as testers we need to err on the side of doing more testing rather than less. Testing the “Desktop Menus” test case is one of the most time consuming test case we currently have. In GNOME, there are currently about 40 applications in the default menu and all need to be started and their basic functionality tested. In KDE, the number of applications is even higher, and some application types (like web browsers) are even present as several different applications. The scope of this test case also covers testing the subcomponents in the Control Center (or a similar equivalent) and, usually, such subcomponents are numerous. Testing XFCE on ARM is also rather time consuming, because the ARM devices we can use for testing are very slow and there are some application that make little sense on an ARM motherboard (e.g. a CD writing software). Thus, doing a thorough Desktop Menus test case requires a rather longish time frame and we lose time we could devote to different problems. Besides that, the test case is also very boring, so it usually gets tested by Fedora QA members only, with few inputs from the community. So, before the Beta and Final release we are able to do such tests a couple of times, but the overall coverage is not big. *What could we do about it?* At Fedora QA, we have been trying to come up with some ideas to make the situation a little better than it is now. The goal is that the important areas get enough attention and testing they deserve, and we avoid shipping broken and under-tested software. See the ideas below and please tell us if you could adopt all/some of them, or if you have different suggestions to improve the current state, we'd love to hear them too. - The list of pre-installed applications could be pruned of those which don't make much sense for that particular purpose (e.g. CD writing software on ARM, or perhaps on any system nowadays). - Pre-installed applications could be deduplicated where possible (e.g. not having several web browsers pre-installed by default). - We could define a list of "important" applications that would be tested thoroughly and the basic functionality criterion would apply just to them. That would tell QA where to focus. (This would obviously contain applications like system settings, file browser, text editor, browser, terminal emulator, etc. But e.g. a screenshot utility would probably not fall into the list. Also, in case of duplicated apps, only one of those could be marked as "important", and the others would* not). The rest of the applications would be just best effort in terms of QA, and wouldn't block the release. - Last but not least, you could help us more with release validation before relevant milestones (Beta, Final) :-) If there's anything that should be improved on our part (communication, instructions, etc), we'd love to hear that. Please, let me know what you think about the ideas proposed above. Do you have any other ideas that could help us improve the quality (with the same amount of testing force)? Best regards, -- Lukáš Růžička FEDORA QE, RHCE Red Hat <https://www.redhat.com> Purkyňova 115 612 45 Brno - Královo Pole [email protected] TRIED AND PERSONALLY TESTED, ERGO TRUSTED. <https://redhat.com/trusted>
_______________________________________________ xfce mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
