Re: [OE-core] Set the CMake BUILD_TESTING option to OFF as the default setting
On Mon, 3 Jun 2024 at 11:40, Fabio Berton via lists.openembedded.org wrote: > Based on this, I would say that we don't want to build tests unless ptests > are enabled (so the argument that building tests also tests the code is > moot). In turn, it may be sensible to use the default CMake option to disable > building of tests. It's actually more subtle. If a recipe has ptest support, then it can use that to enable/disable building of tests. If a recipe has no ptest support, then there's not global rule on what to do. Some recipes disable them, others simply keep things at default settings. I wouldn't want to change a global default for all cmake recipes unless there is compelling reason, because users may be relying on that default, and it will break things for them needlessly. The correct thing to do, if you observe tests that are being built but are not being run, is to add ptest support to the recipe. Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#200243): https://lists.openembedded.org/g/openembedded-core/message/200243 Mute This Topic: https://lists.openembedded.org/mt/105688417/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Set the CMake BUILD_TESTING option to OFF as the default setting
On Mon, 3 Jun 2024 at 11:40, Fabio Berton via lists.openembedded.org wrote: > Based on this, I would say that we don't want to build tests unless ptests > are enabled (so the argument that building tests also tests the code is > moot). In turn, it may be sensible to use the default CMake option to disable > building of tests. Sorry, but I do not follow. Building tests does test the code, as it exposes API problems and incompatible versions etc. Why should we do something else than what cmake does by default? Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#200241): https://lists.openembedded.org/g/openembedded-core/message/200241 Mute This Topic: https://lists.openembedded.org/mt/105688417/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Set the CMake BUILD_TESTING option to OFF as the default setting
Hi Alex,
I ran a few builds disabling CMake DBUILD_TESTING and the build time was about
the same. I'm not sure if by building plenty of recipes we can see a difference
in build time, since bitbake needs to build all the recipe dependencies that
use cmake bbclass.
Looking for some recipes that use ptest, I see examples like this:
- meta/recipes-multimedia/gstreamer/gstreamer1.0_1.24.3.bb: PACKAGECONFIG ??=
"${@bb.utils.contains('PTEST_ENABLED', '1', 'tests', '', d)}"
- meta/recipes-core/glib-networking/glib-networking_2.78.1.bb: PACKAGECONFIG
??= "gnutls environment ${@bb.utils.contains('PTEST_ENABLED', '1', 'tests', '',
d)}"
- meta/recipes-support/libnl/libnl_3.9.0.bb: do_compile_ptest to compile only
when PTEST_ENABLED is enabled
- ... (and several more)
- one interesting one is meta/recipes-devtools/flex/flex_2.6.4.bb:
${@bb.utils.contains('PTEST_ENABLED', '1', '', 'file://disable-tests.patch', d)}
they decided to apply a patch to disable tests when ptest is off
Based on this, I would say that we don't want to build tests unless ptests are
enabled (so the argument that building tests also tests the code is moot). In
turn, it may be sensible to use the default CMake option to disable building of
tests.
Thanks,
Fabio
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#200240):
https://lists.openembedded.org/g/openembedded-core/message/200240
Mute This Topic: https://lists.openembedded.org/mt/105688417/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Set the CMake BUILD_TESTING option to OFF as the default setting
You can build everything with 'bitbake world' (and it's going to be long).
Build times are recorded into tmp/buildstats, if that is enabled (see
default local.conf to see how if you're not already using that).
Alex
On Wed, 24 Apr 2024 at 09:56, Fabio Berton wrote:
>
> Hi Alex,
>
> What should I test to measure build time, all recipes that use cmake from
> OE-Core and meta-openembedded? Is there a way to build all recipes, or should
> I find all recipes that use cmake and run bitbake ?
>
> Regards,
>
> Fabio Berton
>
> On 4/23/2024 2:01 PM, Alexander Kanavin wrote:
>
> I'm not sure I like the idea. This is going to break ptests or other
> usage of CTest in cmake-based recipes that aren't under our control.
> Building tests is also a test in itself even if you don't run them.
> I'd say unless this adds significantly to build times we should leave
> it as it is.
>
> Alex
>
> On Tue, 23 Apr 2024 at 14:31, Fabio Berton via lists.openembedded.org
> wrote:
>
> I've noticed that some recipes, like json-c [1], which use the cmake bbclass,
> are generating files that aren't being used. This is because CMake sets the
> BUILD_TESTING option to ON by default. According to the CMake documentation
> [2], when CTest is included, as json-c does here [3], the module
> automatically creates a BUILD_TESTING option. This option determines whether
> to enable testing support, and it is ON by default.
>
> In the json-c example, the tests used by the do_install_ptest task are always
> generated. Thus, ${B}/tests/ exists even when ptests are not included in
> DISTRO_FEATURES.
>
> As the behavior of CTest/CMake to build tests by default is rather
> surprising, we are wondering whether disabling this option by default is
> feasible in the OpenEmbedded context. Recipes expecting tests to be built
> (e.g. because of ptest support) would then turn on the flag explicitly.
>
> For example, adding this to cmake.bbclass:
>
> OECMAKE_BUILD_TESTING ??= "false"
>
> EXTRA_OECMAKE:append = "\\
> ${@bb.utils.contains('OECMAKE_BUILD_TESTING', 'false',
> '-DBUILD_TESTING=OFF', '', d)} \\
> "
>
> and then set the OECMAKE_BUILD_TESTING in the recipe that requires
> BUILD_TESTING=ON
>
> Do you have any suggestions on which steps we should take to verify that this
> does not introduce any regressions?
>
>
> 1 -
> https://git.openembedded.org/openembedded-core/tree/meta/recipes-devtools/json-c/json-c_0.17.bb?h=master
>
> 2 - https://cmake.org/cmake/help/latest/module/CTest.html
>
> 3 - https://github.com/json-c/json-c/blob/master/CMakeLists.txt#L19
>
>
> Regards,
>
> Fabio Berton
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198642):
https://lists.openembedded.org/g/openembedded-core/message/198642
Mute This Topic: https://lists.openembedded.org/mt/105688417/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Set the CMake BUILD_TESTING option to OFF as the default setting
Hi Alex,
What should I test to measure build time, all recipes that use cmake
from OE-Core and meta-openembedded? Is there a way to build all recipes,
or should I find all recipes that use cmake and run bitbake ?
Regards,
Fabio Berton
On 4/23/2024 2:01 PM, Alexander Kanavin wrote:
I'm not sure I like the idea. This is going to break ptests or other
usage of CTest in cmake-based recipes that aren't under our control.
Building tests is also a test in itself even if you don't run them.
I'd say unless this adds significantly to build times we should leave
it as it is.
Alex
On Tue, 23 Apr 2024 at 14:31, Fabio Berton via lists.openembedded.org
wrote:
I've noticed that some recipes, like json-c [1], which use the cmake bbclass,
are generating files that aren't being used. This is because CMake sets the
BUILD_TESTING option to ON by default. According to the CMake documentation
[2], when CTest is included, as json-c does here [3], the module automatically
creates a BUILD_TESTING option. This option determines whether to enable
testing support, and it is ON by default.
In the json-c example, the tests used by the do_install_ptest task are always
generated. Thus, ${B}/tests/ exists even when ptests are not included in
DISTRO_FEATURES.
As the behavior of CTest/CMake to build tests by default is rather surprising,
we are wondering whether disabling this option by default is feasible in the
OpenEmbedded context. Recipes expecting tests to be built (e.g. because of
ptest support) would then turn on the flag explicitly.
For example, adding this to cmake.bbclass:
OECMAKE_BUILD_TESTING ??= "false"
EXTRA_OECMAKE:append = "\\
${@bb.utils.contains('OECMAKE_BUILD_TESTING', 'false',
'-DBUILD_TESTING=OFF', '', d)} \\
"
and then set the OECMAKE_BUILD_TESTING in the recipe that requires
BUILD_TESTING=ON
Do you have any suggestions on which steps we should take to verify that this
does not introduce any regressions?
1
-https://git.openembedded.org/openembedded-core/tree/meta/recipes-devtools/json-c/json-c_0.17.bb?h=master
2 -https://cmake.org/cmake/help/latest/module/CTest.html
3 -https://github.com/json-c/json-c/blob/master/CMakeLists.txt#L19
Regards,
Fabio Berton
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198640):
https://lists.openembedded.org/g/openembedded-core/message/198640
Mute This Topic: https://lists.openembedded.org/mt/105688417/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Set the CMake BUILD_TESTING option to OFF as the default setting
I'm not sure I like the idea. This is going to break ptests or other
usage of CTest in cmake-based recipes that aren't under our control.
Building tests is also a test in itself even if you don't run them.
I'd say unless this adds significantly to build times we should leave
it as it is.
Alex
On Tue, 23 Apr 2024 at 14:31, Fabio Berton via lists.openembedded.org
wrote:
>
> I've noticed that some recipes, like json-c [1], which use the cmake bbclass,
> are generating files that aren't being used. This is because CMake sets the
> BUILD_TESTING option to ON by default. According to the CMake documentation
> [2], when CTest is included, as json-c does here [3], the module
> automatically creates a BUILD_TESTING option. This option determines whether
> to enable testing support, and it is ON by default.
>
> In the json-c example, the tests used by the do_install_ptest task are always
> generated. Thus, ${B}/tests/ exists even when ptests are not included in
> DISTRO_FEATURES.
>
> As the behavior of CTest/CMake to build tests by default is rather
> surprising, we are wondering whether disabling this option by default is
> feasible in the OpenEmbedded context. Recipes expecting tests to be built
> (e.g. because of ptest support) would then turn on the flag explicitly.
>
> For example, adding this to cmake.bbclass:
>
> OECMAKE_BUILD_TESTING ??= "false"
>
> EXTRA_OECMAKE:append = "\\
> ${@bb.utils.contains('OECMAKE_BUILD_TESTING', 'false',
> '-DBUILD_TESTING=OFF', '', d)} \\
> "
>
> and then set the OECMAKE_BUILD_TESTING in the recipe that requires
> BUILD_TESTING=ON
>
> Do you have any suggestions on which steps we should take to verify that this
> does not introduce any regressions?
>
>
> 1 -
> https://git.openembedded.org/openembedded-core/tree/meta/recipes-devtools/json-c/json-c_0.17.bb?h=master
>
> 2 - https://cmake.org/cmake/help/latest/module/CTest.html
>
> 3 - https://github.com/json-c/json-c/blob/master/CMakeLists.txt#L19
>
>
> Regards,
>
> Fabio Berton
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198624):
https://lists.openembedded.org/g/openembedded-core/message/198624
Mute This Topic: https://lists.openembedded.org/mt/105688417/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] Set the CMake BUILD_TESTING option to OFF as the default setting
I've noticed that some recipes, like json-c [1], which use the cmake
bbclass, are generating files that aren't being used. This is because
CMake sets the BUILD_TESTING option to ON by default. According to the
CMake documentation [2], when CTest is included, as json-c does here
[3], the module automatically creates a |BUILD_TESTING| option. This
option determines whether to enable testing support, and it is ON by
default.
In the json-c example, the tests used by the do_install_ptest task are
always generated. Thus, ${B}/tests/ exists even when ptests are not
included in DISTRO_FEATURES.
As the behavior of CTest/CMake to build tests by default is rather
surprising, we are wondering whether disabling this option by default is
feasible in the OpenEmbedded context. Recipes expecting tests to be
built (e.g. because of ptest support) would then turn on the flag
explicitly.
For example, adding this to cmake.bbclass:
|OECMAKE_BUILD_TESTING ??= "false" EXTRA_OECMAKE:append = "\\
${@bb.utils.contains('OECMAKE_BUILD_TESTING', 'false',
'-DBUILD_TESTING=OFF', '', d)} \\ " |
and then set the OECMAKE_BUILD_TESTING in the recipe that requires
BUILD_TESTING=ON
Do you have any suggestions on which steps we should take to verify that
this does not introduce any regressions?
1 -
https://git.openembedded.org/openembedded-core/tree/meta/recipes-devtools/json-c/json-c_0.17.bb?h=master
2 - https://cmake.org/cmake/help/latest/module/CTest.html
3 - https://github.com/json-c/json-c/blob/master/CMakeLists.txt#L19
Regards,
Fabio Berton
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198621):
https://lists.openembedded.org/g/openembedded-core/message/198621
Mute This Topic: https://lists.openembedded.org/mt/105688417/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
