Re: How to get dependencies into freebsd ci?
On Samstag, 16. September 2023 22:46:43 CEST Ben Cooksley wrote: > On Sun, Sep 17, 2023 at 8:37 AM Ingo Klöcker wrote: > > On Samstag, 16. September 2023 22:08:46 CEST Milian Wolff wrote: > > > And cppcheck is nice, but it's a long shot from clang-tidy and clazy. I > > > understand that's the way it is for now and it's not possible to get > > > additional checks enabled for projects? > > > > I'm not sure what you mean by "not possible to get additional checks > > enabled > > for projects", but okular has a custom job which runs clang-tidy and > > clazy. > > This job does a few things which are very specific for okular. I've just > > started to set up a similar job for libkleo: > > https://invent.kde.org/pim/libkleo/-/merge_requests/45 > > Where possible do try to avoid too many custom jobs - especially where they > duplicate functions like we're seeing here. > In the case of both Okular and libkleo here i'm seeing that they install > stuff from distribution repos - which can result in unpredictable failures > due to network issues. The idea is to make a standard job out of this once it's clear how this would look like. I think other people were interested in running clazy (and maybe clang-tidy) for their projects. Regards, Ingo signature.asc Description: This is a digitally signed message part.
Re: How to get dependencies into freebsd ci?
On Sun, Sep 17, 2023 at 8:37 AM Ingo Klöcker wrote: > On Samstag, 16. September 2023 22:08:46 CEST Milian Wolff wrote: > > And cppcheck is nice, but it's a long shot from clang-tidy and clazy. I > > understand that's the way it is for now and it's not possible to get > > additional checks enabled for projects? > > I'm not sure what you mean by "not possible to get additional checks > enabled > for projects", but okular has a custom job which runs clang-tidy and > clazy. > This job does a few things which are very specific for okular. I've just > started to set up a similar job for libkleo: > https://invent.kde.org/pim/libkleo/-/merge_requests/45 Where possible do try to avoid too many custom jobs - especially where they duplicate functions like we're seeing here. In the case of both Okular and libkleo here i'm seeing that they install stuff from distribution repos - which can result in unpredictable failures due to network issues. > > > Regards, > Ingo Cheers, Ben
Re: How to get dependencies into freebsd ci?
On Sun, Sep 17, 2023 at 8:08 AM Milian Wolff wrote: > On Samstag, 16. September 2023 22:00:37 CEST Ben Cooksley wrote: > > On Sun, Sep 17, 2023 at 3:16 AM Milian Wolff wrote: > > > Hey all, > > > > Hi Milian, > > > > > While looking at the kate CI setup, I saw that it gets build on > freebsd. I > > > would like to get that coverage too for heaptrack, but when I try to > add > > > > > > > https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/ > > > freebsd.yml > > > > > > Then heaptrack fails to find the elfutils dependency. This works on > linux, > > > but > > > I have zero clue how that is done. > > > > > > Can I add dependencies covered by `craft` to `.kde-ci.yml`? > > > > Dependencies are managed in different ways depending on the platform in > > question. > > > > For Linux, Android and Windows builds, this starts with a Docker image > that > > can be found at https://invent.kde.org/sysadmin/ci-images > > You can have dependencies added to those by filing a merge request > against > > that repository. > > > > As you'll see, in the case of Linux we source just about everything from > > the distro package manager (in our case, it's SUSE Tumbleweed). > > For Android and Windows, the majority of the dependencies come from > Craft. > > > > FreeBSD is the exception to all of this, as those builders are fixed > > permanent machines rather than ephemeral containers that are only around > > for a single build. > > For these, please file a Sysadmin ticket. > > > > > Where can I find documentation on what to put in there? Furthermore, is > > > there > > > some best practices when it comes to CI configuration for KDE > projects? By > > > chance I found https://community.kde.org/Infrastructure/GitLab/CI/ > > > Static_Code_Analysis which is interesting - is there more like it > > > somewhere? > > > > I'm afraid we've not done a terribly good job at documenting things, > > however if you are using the templates available at > > > https://invent.kde.org/sysadmin/ci-utilities/-/tree/master/gitlab-templates > > then i'd refer you to > > > https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/config-template.y > > ml?ref_type=heads for the options that the system supports. Note that > these > > aren't used by the craft-* or flatpak templates as those are CD jobs > rather > > than CI jobs. > > Thanks. Where can I find out what these options actually do? `force-inject- > asan` sounds interesting. > You'll need to read the code of the actual CI tooling in that repository. > Generally, is it possible to compile and run tests with asan enabled - > ideally > configured through a CMakePreset? > ASAN is enabled by default for Linux builds - see https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/run-ci-build.py?ref_type=heads#L202 Due to the contagion aspect of ASAN (where binaries have to be built with it if any library that it uses has been built with ASAN), it is not recommended to try to enable or disable it within individual projects. (This is why force-inject-asan exists - to allow for distribution provided binaries that load KDE project plugins/libraries to be used) > > > The Static Code Analysis job referred to there is based on legacy Jenkins > > infrastructure which has been shutdown and it therefore will no longer > > function, however the new system provides analysis jobs that work with > > cppcheck to provide similar functionality if enabled. > > So the wiki page is obsolete and should be removed? > Correct. > > And cppcheck is nice, but it's a long shot from clang-tidy and clazy. I > understand that's the way it is for now and it's not possible to get > additional checks enabled for projects? > Nobody has expressed any interest in working on those within the CI system, and I don't have the bandwidth i'm afraid. Note that if clang-tidy and clazy need to recompile everything we would need to carefully consider how that is implemented within the CI system, both from a resource utilisation as well as compiler coverage perspective. Currently gcc compiler coverage is the responsibility of the Linux CI jobs (while Clang compiler coverage is the responsibility of FreeBSD, and MSVC of Windows). If it is just needing to be compiled alongside then we can probably make that work without too much pain, although it would be FreeBSD only option. > > -- > Milian Wolff > m...@milianw.de > http://milianw.de Cheers, Ben
Re: How to get dependencies into freebsd ci?
On Samstag, 16. September 2023 22:08:46 CEST Milian Wolff wrote: > And cppcheck is nice, but it's a long shot from clang-tidy and clazy. I > understand that's the way it is for now and it's not possible to get > additional checks enabled for projects? I'm not sure what you mean by "not possible to get additional checks enabled for projects", but okular has a custom job which runs clang-tidy and clazy. This job does a few things which are very specific for okular. I've just started to set up a similar job for libkleo: https://invent.kde.org/pim/libkleo/-/merge_requests/45 Regards, Ingo signature.asc Description: This is a digitally signed message part.
Re: How to get dependencies into freebsd ci?
On Samstag, 16. September 2023 22:00:37 CEST Ben Cooksley wrote: > On Sun, Sep 17, 2023 at 3:16 AM Milian Wolff wrote: > > Hey all, > > Hi Milian, > > > While looking at the kate CI setup, I saw that it gets build on freebsd. I > > would like to get that coverage too for heaptrack, but when I try to add > > > > https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/ > > freebsd.yml > > > > Then heaptrack fails to find the elfutils dependency. This works on linux, > > but > > I have zero clue how that is done. > > > > Can I add dependencies covered by `craft` to `.kde-ci.yml`? > > Dependencies are managed in different ways depending on the platform in > question. > > For Linux, Android and Windows builds, this starts with a Docker image that > can be found at https://invent.kde.org/sysadmin/ci-images > You can have dependencies added to those by filing a merge request against > that repository. > > As you'll see, in the case of Linux we source just about everything from > the distro package manager (in our case, it's SUSE Tumbleweed). > For Android and Windows, the majority of the dependencies come from Craft. > > FreeBSD is the exception to all of this, as those builders are fixed > permanent machines rather than ephemeral containers that are only around > for a single build. > For these, please file a Sysadmin ticket. > > > Where can I find documentation on what to put in there? Furthermore, is > > there > > some best practices when it comes to CI configuration for KDE projects? By > > chance I found https://community.kde.org/Infrastructure/GitLab/CI/ > > Static_Code_Analysis which is interesting - is there more like it > > somewhere? > > I'm afraid we've not done a terribly good job at documenting things, > however if you are using the templates available at > https://invent.kde.org/sysadmin/ci-utilities/-/tree/master/gitlab-templates > then i'd refer you to > https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/config-template.y > ml?ref_type=heads for the options that the system supports. Note that these > aren't used by the craft-* or flatpak templates as those are CD jobs rather > than CI jobs. Thanks. Where can I find out what these options actually do? `force-inject- asan` sounds interesting. Generally, is it possible to compile and run tests with asan enabled - ideally configured through a CMakePreset? > The Static Code Analysis job referred to there is based on legacy Jenkins > infrastructure which has been shutdown and it therefore will no longer > function, however the new system provides analysis jobs that work with > cppcheck to provide similar functionality if enabled. So the wiki page is obsolete and should be removed? And cppcheck is nice, but it's a long shot from clang-tidy and clazy. I understand that's the way it is for now and it's not possible to get additional checks enabled for projects? -- Milian Wolff m...@milianw.de http://milianw.de signature.asc Description: This is a digitally signed message part.
Re: How to get dependencies into freebsd ci?
On Sun, Sep 17, 2023 at 3:16 AM Milian Wolff wrote: > Hey all, > Hi Milian, > > While looking at the kate CI setup, I saw that it gets build on freebsd. I > would like to get that coverage too for heaptrack, but when I try to add > > https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/ > freebsd.yml > > Then heaptrack fails to find the elfutils dependency. This works on linux, > but > I have zero clue how that is done. > > Can I add dependencies covered by `craft` to `.kde-ci.yml`? Dependencies are managed in different ways depending on the platform in question. For Linux, Android and Windows builds, this starts with a Docker image that can be found at https://invent.kde.org/sysadmin/ci-images You can have dependencies added to those by filing a merge request against that repository. As you'll see, in the case of Linux we source just about everything from the distro package manager (in our case, it's SUSE Tumbleweed). For Android and Windows, the majority of the dependencies come from Craft. FreeBSD is the exception to all of this, as those builders are fixed permanent machines rather than ephemeral containers that are only around for a single build. For these, please file a Sysadmin ticket. > > > Where can I find documentation on what to put in there? Furthermore, is > there > some best practices when it comes to CI configuration for KDE projects? By > chance I found https://community.kde.org/Infrastructure/GitLab/CI/ > Static_Code_Analysis which is interesting - is there more like it > somewhere? > I'm afraid we've not done a terribly good job at documenting things, however if you are using the templates available at https://invent.kde.org/sysadmin/ci-utilities/-/tree/master/gitlab-templates then i'd refer you to https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/config-template.yml?ref_type=heads for the options that the system supports. Note that these aren't used by the craft-* or flatpak templates as those are CD jobs rather than CI jobs. The Static Code Analysis job referred to there is based on legacy Jenkins infrastructure which has been shutdown and it therefore will no longer function, however the new system provides analysis jobs that work with cppcheck to provide similar functionality if enabled. Cheers, Ben > > Thanks > -- > Milian Wolff > m...@milianw.de > http://milianw.de
Re: How to get dependencies into freebsd ci?
On Samstag, 16. September 2023 17:47:28 CEST Tobias C. Berner wrote: > Moin moin > > I now installed elfutils-0.187 on the FreeBSD builders. Thanks Tobias! Can you help me a bit more please? Apparently libunwind is found - including support for unw_backtrace - yet I still run into a compile error later on, see: https://invent.kde.org/sdk/heaptrack/-/jobs/1196913 ``` -- Performing Test LIBUNWIND_HAS_UNW_BACKTRACE -- Performing Test LIBUNWIND_HAS_UNW_BACKTRACE - Success ... /var/tmp/gitlab_runner/builds/TY-7Z6G7M/0/sdk/heaptrack/src/track/ trace_libunwind.cpp:67:12: error: use of undeclared identifier 'unw_backtrace' return unw_backtrace(data, MAX_SIZE); ^ 1 error generated. ``` I don't understand how it could pass for the cmake check yet fail later on? Are there multiple `libunwind.h` on freebsd, and it finds the wrong one here based on an unfortunate ordering of the include directories? Or is there some other way for me to setup a freebsd environment without too much pain, so I can debug this issue directly? Thanks -- Milian Wolff m...@milianw.de http://milianw.de signature.asc Description: This is a digitally signed message part.
Re: How to get dependencies into freebsd ci?
Moin moin I now installed elfutils-0.187 on the FreeBSD builders. mfg Tobias On Sat, 16 Sept 2023 at 16:58, Milian Wolff wrote: > > Hey all, > > While looking at the kate CI setup, I saw that it gets build on freebsd. I > would like to get that coverage too for heaptrack, but when I try to add > > https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/ > freebsd.yml > > Then heaptrack fails to find the elfutils dependency. This works on linux, but > I have zero clue how that is done. > > Can I add dependencies covered by `craft` to `.kde-ci.yml`? > > Where can I find documentation on what to put in there? Furthermore, is there > some best practices when it comes to CI configuration for KDE projects? By > chance I found https://community.kde.org/Infrastructure/GitLab/CI/ > Static_Code_Analysis which is interesting - is there more like it somewhere? > > Thanks > -- > Milian Wolff > m...@milianw.de > http://milianw.de
