Re: [announcement] ATTENTION PLASMA THEME, WIDGET, & ADD-ON DEVELOPERS
Hello Joseph, thank you very much for the e-mail, with my extension developer hat on: I am currently trying to port my hoppla plasmoid. And "trying" is a pretty good description. The documentation is unfortunately quite lacking in some parts and incomplete / outdated in others e.g. also the tool to convert the .desktop to json (desktoptojson) does not add lines that are required by plasma but not documented, and the only way to find that out is to either start plasma or a plasmoid viewer on a command line and look for stdout / stderr output. (inb4 "then pull request a documentation fix" answers. I am already short on time as per below, so unfortunately I can't also, at least for now, write down all the obstracles I ran into and get them fixed, I am really sorry.) Changes made to Qt6 also come into play, e.g. QtGraphicalEffects. So you need multiple sources of documentation open at the same time, and e.g. the KDE API documentation (https://api.kde.org/ etc) seems to lack an easy option to switch between versions and the current docs linked there seem to be for plasma / frameworks / apps 5, not 6. Layouting / containers have undergone quite a lot of changes, and even when porting everything to the equivalent stuff in KF6, my plasmoid is completely unusable because controls end up out of view / too small / too big etc. And of course there are real time chats (IRC, Matrix, Telegram, whathaveyou) and forums, but trying to get in touch with the right people at the right time is hard, and given that I work on KDE stuff in my spare time, the delay between when I need the information, when someone is around, gives the information, I can answer their questions etc. is massive and there is no way my plasmoid will be even remotely ready in time. So the best option is to look at existing code. Problem there is: even if you look at official, first party plasmoids, you find out that the very same thing is achieved quite differently between different plasmoids. Given there is very very very few actual deep documentation that covers more than the very basics, there is no way for me to know which way is proper. There are very few third party extensions available on the KDE store, and those that are are rather simple on the GUI end. I assume that quite some of the other third party extension developers are facing similar issues that I am, so it will take quite a while until more are available, both for users and fellow devs. (Not to mention that coding and testing is currently quite hard, I had to set up a VM so I don't have to install a distro that even has plasma 6 or dual boot all the time, both of which is also not easy) tl;dr: the experience for third party developers is, at least in my opinion, sub par, the documenation far from sufficient and this can be quite frustrating. I don't want to point fingers and I know that people doing all of this are volunteers, too. Just trying to give KDE developers an insight from a third party extension dev. I hope that things will get better over time and with more people on plasma 6 and more documentation available, and I will try my best to still be able to port my plasmoid over in a decent timeframe, definitely not ready for release though. And given I already started a while ago (given people asked on GitHub for me to port it), I assume other devs catching your e-mail now won't be even nearly ready for release. That aside, I am looking forward to the plasma 6 release (I definitely won't switch myself before 6.1 / 6.2), and I wish us all great success and I want to send a huge thank you out to all developers, documentation writers, designers and all other helpful people whose work will make this big release possible. Kind regards, Christian (Fuchs) Am Freitag, 23. Februar 2024, 13:19:27 CET schrieb Joseph P. De Veaugh-Geiss: > Apologies for the all caps, but now that I have your attention :) > > With Plasma 6, various breaking changes affect existing Plasma > look-and-feel themes, widgets, and ad-ons. > > We love all your stuff, but you need to port it for it to work in Plasma > 6. We have created some handy, easy-to-follow guides: > >https://develop.kde.org/docs/plasma/widget/porting_kf6/ > >https://develop.kde.org/docs/plasma/theme/theme-porting-to-plasma6/ > > Porting is quite straightforward and should not take you long. > > Keep our users happy! Port your stuff! :) > > Cheers, > Joseph > > (H/t Paul Brown and Nate Graham for the above announcement)
Re: Post-MegaRelease projects
On Sat, Feb 24, 2024 at 11:35 PM Konstantin Kharlamov wrote: > On Sat, 2024-02-24 at 23:07 +1300, Ben Cooksley wrote: > > On Sat, Feb 24, 2024 at 10:27 PM Konstantin Kharlamov > wrote: > > On Sat, 2024-02-24 at 22:16 +1300, Ben Cooksley wrote: > > On Sat, Feb 24, 2024 at 8:37 PM Konstantin Kharlamov > wrote: > > On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley wrote: > > On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela wrote: > > On 2024-02-22, Nate Graham wrote: > > I've started pondering post-megarelease projects. We've spent so long on > > porting and bugfixing that I think it might be useful to shift gears to > > feature work, and I'd like to brainstorm potential large-scale projects > > and gauge the level of interest in putting resources into them soon. > > A bit more from the devops end that I'd love to see people tackle: > > - Ensure frameworks and app unit tests interacting with windows can run >on Windows. >More details: The following fails on our windows CI >https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp >I find it weird that we are spending resources on putting things in >the windows store and making apps available on windows, but we can't >actually have passing tests in our CI. > > > This unfortunately will not be easy to solve. > > One of the key things that we've learned out of doing CI, as has been > showcased by FreeBSD in particular, is that the builders need to be > ephemeral - that is only around for the build in question that is being run. > We're currently accomplishing this by using containers - via Podman (for > Linux/Android/FreeBSD) and Docker (for Windows). > > Containers also offer us the advantage of allowing people to easily > reproduce the CI environment on their local system without too much trouble. > > For Windows however, Microsoft has limited the container stack to not > allow anything GUI related to work. The underlying libraries may be there, > but the equivalent display server components are not operational. > > To complicate things further, on Windows certain permissions are > restricted to the interactive console and are not possible to do as either > a scheduled task or as a system service. > Usage of existing Windows automation frameworks such as Powershell > Remoting or SSH will therefore not work if we want things to perfectly > replicate a end user environment - because those will run the command(s) as > part of a non-interactive session (even if the user we connect as is the > same one logged in on the desktop console). > > > Idk if it's a silly question, but… If Windows native containers have so > many restrictions, why not just use Linux containers with WINE inside? > > > Because Wine is not Windows either, and there could be subtle differences > in how things run / interact with the system. > > > Fair point, no sw is bugless. Although, from my shallow experience WINE > seems to have an excellent test suite. I remember reporting a regression > once, which turned out to be due to some obscure surfaces manipulation by > an old Heroes Ⅲ game. WINE devs not only quickly fixed that, but they also > added tests for that case, so I'd presume such regression is just not gonna > happen anymore. > > Plus some of our software would like to test certain system level > infrastructure (like say KDE Connect). > > > Out of curiosity, what does this infrastructure include? I thought KDE > connect only uses network sockets and system tray. > > > No idea, I saw their commentary on debugging issues they were having in > their unit tests in #kde-devel. > Those issues were due to a lack of permissions, specifically around the > interactive console - that is how I know some of our tests need those > additional permissions and why running as a scheduled task / system service > will not be sufficient for "fully working" CI tests on Windows. > > > Sorry, it seems there's some misunderstanding… Judging by what you say you > seem to be referring to the restrictions that Windows containers have on > Windows. But that was the point that started this thread, to which I > replied that running Linux containers with WINE might be a solution 😊 > What i'm referring to here is that KDE Connect interacts with various components of the system in order to do what it does. See for instance the ability to share Notifications between devices, or ability to act as a presenter device. That requires accessing various system level APIs which WINE very well may not support - and we wouldn't support a scenario of using it under WINE because there is a native Linux version already. > > Plus, we have to have native Windows to compile things anyway as we need > to use MSVC (otherwise you have no Qt Web Engine support, as that cannot be > built with MingW) > > > But I presume it can be built with Clang? In particular, Google Chrome on > Windows is being built with Clang — and Web Engine is basically a fork of > Chromium. > > > Qt 6 as a whole does not lis
Re: Post-MegaRelease projects
On Sat, 2024-02-24 at 23:07 +1300, Ben Cooksley wrote: > On Sat, Feb 24, 2024 at 10:27 PM Konstantin Kharlamov > wrote: > > On Sat, 2024-02-24 at 22:16 +1300, Ben Cooksley wrote: > > > On Sat, Feb 24, 2024 at 8:37 PM Konstantin Kharlamov > > > wrote: > > > > On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley wrote: > > > > > On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela > > > > > wrote: > > > > > > On 2024-02-22, Nate Graham wrote: > > > > > > > I've started pondering post-megarelease projects. We've > > > > > > spent so long on > > > > > > > porting and bugfixing that I think it might be useful to > > > > > > shift gears to > > > > > > > feature work, and I'd like to brainstorm potential large- > > > > > > scale projects > > > > > > > and gauge the level of interest in putting resources into > > > > > > them soon. > > > > > > > > > > > > A bit more from the devops end that I'd love to see people > > > > > > tackle: > > > > > > > > > > > > - Ensure frameworks and app unit tests interacting with > > > > > > windows can run > > > > > > on Windows. > > > > > > More details: The following fails on our windows CI > > > > > > > > > > > > > > > > > > https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp > > > > > > I find it weird that we are spending resources on > > > > > > putting things in > > > > > > the windows store and making apps available on windows, > > > > > > but we can't > > > > > > actually have passing tests in our CI. > > > > > > > > > > > > > > > > > > > > > This unfortunately will not be easy to solve. > > > > > > > > > > One of the key things that we've learned out of doing CI, as > > > > > has been showcased by FreeBSD in particular, is that the > > > > > builders need to be ephemeral - that is only around for the > > > > > build in question that is being run. > > > > > We're currently accomplishing this by using containers - via > > > > > Podman (for Linux/Android/FreeBSD) and Docker (for Windows). > > > > > > > > > > Containers also offer us the advantage of allowing people to > > > > > easily reproduce the CI environment on their local system > > > > > without too much trouble. > > > > > > > > > > For Windows however, Microsoft has limited the container > > > > > stack to not allow anything GUI related to work. The > > > > > underlying libraries may be there, but the equivalent display > > > > > server components are not operational. > > > > > > > > > > To complicate things further, on Windows certain permissions > > > > > are restricted to the interactive console and are not > > > > > possible to do as either a scheduled task or as a system > > > > > service. > > > > > Usage of existing Windows automation frameworks such as > > > > > Powershell Remoting or SSH will therefore not work if we want > > > > > things to perfectly replicate a end user environment - > > > > > because those will run the command(s) as part of a non- > > > > > interactive session (even if the user we connect as is the > > > > > same one logged in on the desktop console). > > > > > > > > > > > > Idk if it's a silly question, but… If Windows native containers > > > > have so many restrictions, why not just use Linux containers > > > > with WINE inside? > > > > > > > > > > > > > Because Wine is not Windows either, and there could be subtle > > > differences in how things run / interact with the system. Fair point, no sw is bugless. Although, from my shallow experience WINE seems to have an excellent test suite. I remember reporting a regression once, which turned out to be due to some obscure surfaces manipulation by an old Heroes Ⅲ game. WINE devs not only quickly fixed that, but they also added tests for that case, so I'd presume such regression is just not gonna happen anymore. > > > Plus some of our software would like to test certain system level > > > infrastructure (like say KDE Connect). > > > > > > Out of curiosity, what does this infrastructure include? I thought > > KDE connect only uses network sockets and system tray. > > > > > No idea, I saw their commentary on debugging issues they were having > in their unit tests in #kde-devel. > Those issues were due to a lack of permissions, specifically around > the interactive console - that is how I know some of our tests need > those additional permissions and why running as a scheduled task / > system service will not be sufficient for "fully working" CI tests on > Windows. Sorry, it seems there's some misunderstanding… Judging by what you say you seem to be referring to the restrictions that Windows containers have on Windows. But that was the point that started this thread, to which I replied that running Linux containers with WINE might be a solution 😊 > > > Plus, we have to have native Windows to compile things anyway as > > > we need to use MSVC (otherwise you have no Qt Web Engine support, > > > as that cannot be built with MingW) > > > > > > But I presume it can be built with Clang? In pa
Re: Post-MegaRelease projects
On Sat, 2024-02-24 at 22:16 +1300, Ben Cooksley wrote: > On Sat, Feb 24, 2024 at 8:37 PM Konstantin Kharlamov > wrote: > > On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley wrote: > > > On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela > > > wrote: > > > > On 2024-02-22, Nate Graham wrote: > > > > > I've started pondering post-megarelease projects. We've spent > > > > so long on > > > > > porting and bugfixing that I think it might be useful to > > > > shift gears to > > > > > feature work, and I'd like to brainstorm potential large- > > > > scale projects > > > > > and gauge the level of interest in putting resources into > > > > them soon. > > > > > > > > A bit more from the devops end that I'd love to see people > > > > tackle: > > > > > > > > - Ensure frameworks and app unit tests interacting with > > > > windows can run > > > > on Windows. > > > > More details: The following fails on our windows CI > > > > > > > > https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp > > > > I find it weird that we are spending resources on putting > > > > things in > > > > the windows store and making apps available on windows, but > > > > we can't > > > > actually have passing tests in our CI. > > > > > > > > > > > > > This unfortunately will not be easy to solve. > > > > > > One of the key things that we've learned out of doing CI, as has > > > been showcased by FreeBSD in particular, is that the builders > > > need to be ephemeral - that is only around for the build in > > > question that is being run. > > > We're currently accomplishing this by using containers - via > > > Podman (for Linux/Android/FreeBSD) and Docker (for Windows). > > > > > > Containers also offer us the advantage of allowing people to > > > easily reproduce the CI environment on their local system without > > > too much trouble. > > > > > > For Windows however, Microsoft has limited the container stack to > > > not allow anything GUI related to work. The underlying libraries > > > may be there, but the equivalent display server components are > > > not operational. > > > > > > To complicate things further, on Windows certain permissions are > > > restricted to the interactive console and are not possible to do > > > as either a scheduled task or as a system service. > > > Usage of existing Windows automation frameworks such as > > > Powershell Remoting or SSH will therefore not work if we want > > > things to perfectly replicate a end user environment - because > > > those will run the command(s) as part of a non-interactive > > > session (even if the user we connect as is the same one logged in > > > on the desktop console). > > > > > > Idk if it's a silly question, but… If Windows native containers > > have so many restrictions, why not just use Linux containers with > > WINE inside? > > > > > Because Wine is not Windows either, and there could be subtle > differences in how things run / interact with the system. > Plus some of our software would like to test certain system level > infrastructure (like say KDE Connect). Out of curiosity, what does this infrastructure include? I thought KDE connect only uses network sockets and system tray. > Plus, we have to have native Windows to compile things anyway as we > need to use MSVC (otherwise you have no Qt Web Engine support, as > that cannot be built with MingW) But I presume it can be built with Clang? In particular, Google Chrome on Windows is being built with Clang — and Web Engine is basically a fork of Chromium. > (and you cannot really mix MingW / MSVC binaries due to incompatible > compiler ABIs for C++ code) Well, if for testing purposes Qt was pre-built with Clang, I guess there won't be any ABI issues.
Re: Post-MegaRelease projects
On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley wrote: > On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela > wrote: > > On 2024-02-22, Nate Graham wrote: > > > I've started pondering post-megarelease projects. We've spent so > > long on > > > porting and bugfixing that I think it might be useful to shift > > gears to > > > feature work, and I'd like to brainstorm potential large-scale > > projects > > > and gauge the level of interest in putting resources into them > > soon. > > > > A bit more from the devops end that I'd love to see people tackle: > > > > - Ensure frameworks and app unit tests interacting with windows > > can run > > on Windows. > > More details: The following fails on our windows CI > > > > https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp > > I find it weird that we are spending resources on putting things > > in > > the windows store and making apps available on windows, but we > > can't > > actually have passing tests in our CI. > > > > > This unfortunately will not be easy to solve. > > One of the key things that we've learned out of doing CI, as has been > showcased by FreeBSD in particular, is that the builders need to be > ephemeral - that is only around for the build in question that is > being run. > We're currently accomplishing this by using containers - via Podman > (for Linux/Android/FreeBSD) and Docker (for Windows). > > Containers also offer us the advantage of allowing people to easily > reproduce the CI environment on their local system without too much > trouble. > > For Windows however, Microsoft has limited the container stack to not > allow anything GUI related to work. The underlying libraries may be > there, but the equivalent display server components are not > operational. > > To complicate things further, on Windows certain permissions are > restricted to the interactive console and are not possible to do as > either a scheduled task or as a system service. > Usage of existing Windows automation frameworks such as Powershell > Remoting or SSH will therefore not work if we want things to > perfectly replicate a end user environment - because those will run > the command(s) as part of a non-interactive session (even if the user > we connect as is the same one logged in on the desktop console). Idk if it's a silly question, but… If Windows native containers have so many restrictions, why not just use Linux containers with WINE inside?
Re: Post-MegaRelease projects
On Sat, Feb 24, 2024 at 10:27 PM Konstantin Kharlamov wrote: > On Sat, 2024-02-24 at 22:16 +1300, Ben Cooksley wrote: > > On Sat, Feb 24, 2024 at 8:37 PM Konstantin Kharlamov > wrote: > > On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley wrote: > > On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela wrote: > > On 2024-02-22, Nate Graham wrote: > > I've started pondering post-megarelease projects. We've spent so long on > > porting and bugfixing that I think it might be useful to shift gears to > > feature work, and I'd like to brainstorm potential large-scale projects > > and gauge the level of interest in putting resources into them soon. > > A bit more from the devops end that I'd love to see people tackle: > > - Ensure frameworks and app unit tests interacting with windows can run >on Windows. >More details: The following fails on our windows CI >https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp >I find it weird that we are spending resources on putting things in >the windows store and making apps available on windows, but we can't >actually have passing tests in our CI. > > > This unfortunately will not be easy to solve. > > One of the key things that we've learned out of doing CI, as has been > showcased by FreeBSD in particular, is that the builders need to be > ephemeral - that is only around for the build in question that is being run. > We're currently accomplishing this by using containers - via Podman (for > Linux/Android/FreeBSD) and Docker (for Windows). > > Containers also offer us the advantage of allowing people to easily > reproduce the CI environment on their local system without too much trouble. > > For Windows however, Microsoft has limited the container stack to not > allow anything GUI related to work. The underlying libraries may be there, > but the equivalent display server components are not operational. > > To complicate things further, on Windows certain permissions are > restricted to the interactive console and are not possible to do as either > a scheduled task or as a system service. > Usage of existing Windows automation frameworks such as Powershell > Remoting or SSH will therefore not work if we want things to perfectly > replicate a end user environment - because those will run the command(s) as > part of a non-interactive session (even if the user we connect as is the > same one logged in on the desktop console). > > > Idk if it's a silly question, but… If Windows native containers have so > many restrictions, why not just use Linux containers with WINE inside? > > > Because Wine is not Windows either, and there could be subtle differences > in how things run / interact with the system. > Plus some of our software would like to test certain system level > infrastructure (like say KDE Connect). > > > Out of curiosity, what does this infrastructure include? I thought KDE > connect only uses network sockets and system tray. > No idea, I saw their commentary on debugging issues they were having in their unit tests in #kde-devel. Those issues were due to a lack of permissions, specifically around the interactive console - that is how I know some of our tests need those additional permissions and why running as a scheduled task / system service will not be sufficient for "fully working" CI tests on Windows. > > Plus, we have to have native Windows to compile things anyway as we need > to use MSVC (otherwise you have no Qt Web Engine support, as that cannot be > built with MingW) > > > But I presume it can be built with Clang? In particular, Google Chrome on > Windows is being built with Clang — and Web Engine is basically a fork of > Chromium. > Qt 6 as a whole does not list Clang as a supported compiler - see https://doc.qt.io/qt-6/supported-platforms.html Given Windows is a bit strange in the first place, i'd be quite reluctant to step outside of what they list as supported. > > (and you cannot really mix MingW / MSVC binaries due to incompatible > compiler ABIs for C++ code) > > > Well, if for testing purposes Qt was pre-built with Clang, I guess there > won't be any ABI issues. > Would require that everything else we have is rebuilt, and that all downstream users of our Craft cache also switch to Clang. Note that like most open source projects, we don't know the full extent to which Craft is used outside of our own project. Historically we have seen through Microsoft provided data that dependencies which we built and signed have shown up in surprising places (such as Snoretoast - which was used by something Node.js based at one point or another I believe) Cheers, Ben
Re: Post-MegaRelease projects
On Sat, Feb 24, 2024 at 9:21 PM Volker Krause wrote: > On Saturday, 24 February 2024 04:31:49 CET Ben Cooksley wrote: > > On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela wrote: > > > On 2024-02-22, Nate Graham wrote: > > > > I've started pondering post-megarelease projects. We've spent so > long on > > > > porting and bugfixing that I think it might be useful to shift gears > to > > > > feature work, and I'd like to brainstorm potential large-scale > projects > > > > and gauge the level of interest in putting resources into them soon. > > > > > > A bit more from the devops end that I'd love to see people tackle: > > > - Ensure frameworks and app unit tests interacting with windows can > run > > > > > >on Windows. > > >More details: The following fails on our windows CI > > > > https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp > > >I find it weird that we are spending resources on putting things in > > >the windows store and making apps available on windows, but we can't > > >actually have passing tests in our CI. > > > > This unfortunately will not be easy to solve. > > And that's fine, we are in dreaming territory here anyway :) > > > One of the key things that we've learned out of doing CI, as has been > > showcased by FreeBSD in particular, is that the builders need to be > > ephemeral - that is only around for the build in question that is being > run. > > We're currently accomplishing this by using containers - via Podman (for > > Linux/Android/FreeBSD) and Docker (for Windows). > > > > Containers also offer us the advantage of allowing people to easily > > reproduce the CI environment on their local system without too much > trouble. > > > > For Windows however, Microsoft has limited the container stack to not > allow > > anything GUI related to work. The underlying libraries may be there, but > > the equivalent display server components are not operational. > > > > To complicate things further, on Windows certain permissions are > restricted > > to the interactive console and are not possible to do as either a > scheduled > > task or as a system service. > > Usage of existing Windows automation frameworks such as Powershell > Remoting > > or SSH will therefore not work if we want things to perfectly replicate a > > end user environment - because those will run the command(s) as part of a > > non-interactive session (even if the user we connect as is the same one > > logged in on the desktop console). > > > > Resolving this will therefore not only require running full sized Windows > > VMs (on an ephemeral basis to avoid the recently resolved issues that > used > > to plague FreeBSD), but would also need the VM to automatically login and > > spawn an agent as part of the interactive desktop session that we would > > connect to in order to run the CI tests. > > > > The spawning of a VM would require refactoring the setup of our poor CI > > workers (again - after they were only just recently completely rebuilt to > > allow the transition to Podman to fix flatpak-builder) to make use of > > something like: > > - > > > https://forum.gitlab.com/t/custom-executor-into-windows-vm-using-linux-kvm-l > > ibvirt/72713/5 - > > https://docs.gitlab.com/runner/executors/custom_examples/libvirt.html > > > > We would still have to write the agent that receives our commands > > (something like > > https://gist.github.com/cschwede/3e2c025408ab4af531651098331cce45 maybe) > > > > We would still have to solve the issue of how to share disk images among > > our nodes so they were built (ideally would be able to use Gitlab's > > Container Registry for this, which is something Tart can do - Tart being > a > > virtualisation management utility for ARM based Macs), as well as > > automating the installation of the various OSes we need to run this way. > > See > > > https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/autom > > ate-windows-setup?view=windows-11 for some notes on that for Windows. > > > > The big downside to all of that of course is that the worker nodes would > > take longer to startup, and it would make sharing build artifacts much > more > > difficult and/or less efficient. > > > > > - Find a way to run unit tests on android CI. > > > > Likewise, this is very non-trivial - from my understanding our tooling > > currently for building Android software is centered around it all being > > cross compiled rather than being built on the native architecture it will > > be run on. > > Naturally tests cannot be run (unless you emulate, which I guess we could > > consider...) if the CPU architectures are not compatible. > > > > Even if you emulate though, I imagine we would need to provide a full > > Android system setup (including all of it's relevant bits of system > > infrastructure, such as it's display server DisplayFlinger) in order for > > tests to truly be of use. > > The path of least resistance is probably by making use of Google's > existing > >
Re: Post-MegaRelease projects
On Sat, Feb 24, 2024 at 8:37 PM Konstantin Kharlamov wrote: > On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley wrote: > > On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela wrote: > > On 2024-02-22, Nate Graham wrote: > > I've started pondering post-megarelease projects. We've spent so long on > > porting and bugfixing that I think it might be useful to shift gears to > > feature work, and I'd like to brainstorm potential large-scale projects > > and gauge the level of interest in putting resources into them soon. > > A bit more from the devops end that I'd love to see people tackle: > > - Ensure frameworks and app unit tests interacting with windows can run >on Windows. >More details: The following fails on our windows CI >https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp >I find it weird that we are spending resources on putting things in >the windows store and making apps available on windows, but we can't >actually have passing tests in our CI. > > > This unfortunately will not be easy to solve. > > One of the key things that we've learned out of doing CI, as has been > showcased by FreeBSD in particular, is that the builders need to be > ephemeral - that is only around for the build in question that is being run. > We're currently accomplishing this by using containers - via Podman (for > Linux/Android/FreeBSD) and Docker (for Windows). > > Containers also offer us the advantage of allowing people to easily > reproduce the CI environment on their local system without too much trouble. > > For Windows however, Microsoft has limited the container stack to not > allow anything GUI related to work. The underlying libraries may be there, > but the equivalent display server components are not operational. > > To complicate things further, on Windows certain permissions are > restricted to the interactive console and are not possible to do as either > a scheduled task or as a system service. > Usage of existing Windows automation frameworks such as Powershell > Remoting or SSH will therefore not work if we want things to perfectly > replicate a end user environment - because those will run the command(s) as > part of a non-interactive session (even if the user we connect as is the > same one logged in on the desktop console). > > > Idk if it's a silly question, but… If Windows native containers have so > many restrictions, why not just use Linux containers with WINE inside? > Because Wine is not Windows either, and there could be subtle differences in how things run / interact with the system. Plus some of our software would like to test certain system level infrastructure (like say KDE Connect). Plus, we have to have native Windows to compile things anyway as we need to use MSVC (otherwise you have no Qt Web Engine support, as that cannot be built with MingW) (and you cannot really mix MingW / MSVC binaries due to incompatible compiler ABIs for C++ code) Cheers, Ben
Re: Post-MegaRelease projects
On Saturday, 24 February 2024 04:31:49 CET Ben Cooksley wrote: > On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela wrote: > > On 2024-02-22, Nate Graham wrote: > > > I've started pondering post-megarelease projects. We've spent so long on > > > porting and bugfixing that I think it might be useful to shift gears to > > > feature work, and I'd like to brainstorm potential large-scale projects > > > and gauge the level of interest in putting resources into them soon. > > > > A bit more from the devops end that I'd love to see people tackle: > > - Ensure frameworks and app unit tests interacting with windows can run > > > >on Windows. > >More details: The following fails on our windows CI > >https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp > >I find it weird that we are spending resources on putting things in > >the windows store and making apps available on windows, but we can't > >actually have passing tests in our CI. > > This unfortunately will not be easy to solve. And that's fine, we are in dreaming territory here anyway :) > One of the key things that we've learned out of doing CI, as has been > showcased by FreeBSD in particular, is that the builders need to be > ephemeral - that is only around for the build in question that is being run. > We're currently accomplishing this by using containers - via Podman (for > Linux/Android/FreeBSD) and Docker (for Windows). > > Containers also offer us the advantage of allowing people to easily > reproduce the CI environment on their local system without too much trouble. > > For Windows however, Microsoft has limited the container stack to not allow > anything GUI related to work. The underlying libraries may be there, but > the equivalent display server components are not operational. > > To complicate things further, on Windows certain permissions are restricted > to the interactive console and are not possible to do as either a scheduled > task or as a system service. > Usage of existing Windows automation frameworks such as Powershell Remoting > or SSH will therefore not work if we want things to perfectly replicate a > end user environment - because those will run the command(s) as part of a > non-interactive session (even if the user we connect as is the same one > logged in on the desktop console). > > Resolving this will therefore not only require running full sized Windows > VMs (on an ephemeral basis to avoid the recently resolved issues that used > to plague FreeBSD), but would also need the VM to automatically login and > spawn an agent as part of the interactive desktop session that we would > connect to in order to run the CI tests. > > The spawning of a VM would require refactoring the setup of our poor CI > workers (again - after they were only just recently completely rebuilt to > allow the transition to Podman to fix flatpak-builder) to make use of > something like: > - > https://forum.gitlab.com/t/custom-executor-into-windows-vm-using-linux-kvm-l > ibvirt/72713/5 - > https://docs.gitlab.com/runner/executors/custom_examples/libvirt.html > > We would still have to write the agent that receives our commands > (something like > https://gist.github.com/cschwede/3e2c025408ab4af531651098331cce45 maybe) > > We would still have to solve the issue of how to share disk images among > our nodes so they were built (ideally would be able to use Gitlab's > Container Registry for this, which is something Tart can do - Tart being a > virtualisation management utility for ARM based Macs), as well as > automating the installation of the various OSes we need to run this way. > See > https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/autom > ate-windows-setup?view=windows-11 for some notes on that for Windows. > > The big downside to all of that of course is that the worker nodes would > take longer to startup, and it would make sharing build artifacts much more > difficult and/or less efficient. > > > - Find a way to run unit tests on android CI. > > Likewise, this is very non-trivial - from my understanding our tooling > currently for building Android software is centered around it all being > cross compiled rather than being built on the native architecture it will > be run on. > Naturally tests cannot be run (unless you emulate, which I guess we could > consider...) if the CPU architectures are not compatible. > > Even if you emulate though, I imagine we would need to provide a full > Android system setup (including all of it's relevant bits of system > infrastructure, such as it's display server DisplayFlinger) in order for > tests to truly be of use. > The path of least resistance is probably by making use of Google's existing > Android emulator - although i've no idea how you would tie that into CI. Right, the Android emulator is probably the first thing to look at for this. That alone isn't enough though, we can't just copy over a unit test executable and run it there, t
