Re: [Qbs] tst_blackbox-qt fails
On Wed, 30 Jan 2019 07:20:30 +0100 Richard Weickelt wrote: > >> This is how I build and test now: > >> > >> qbs build profile:qt > >> qbs run -p qbs_app profile:qt -- setup-toolchains --detect > >> qbs run -p qbs_app profile:qt -- setup-qt > >> /usr/local/Qt/5.9.7/gcc_64/bin/qmake qt > >> QBS_AUTOTEST_PROFILE=qt qbs build -p autotest-runner profile:qt > >> > >> Is this how it is supposed to be done? > > > Looks fine to me. Correction: Unless you explicitly re-resolve, qbs always uses the environment stored in the build graph. So either you set QBS_AUTOTEST_PROFILE before doing the initial build, or you re-resolve before "building" the autotest runner: $ QBS_AUTOTEST_PROFILE=qt qbs resolve $ qbs build -p autotest-runner profile:qt > 3. How about adding a property "runEnvironment" to QbsAutotest that would > to set the profile on the command line when building > autotest-runner? Those variables would be appended to > qbs.commonRunEnvironment in the product. https://doc.qt.io/qbs/qml-qbsconvenienceitems-autotestrunner.html#environment-prop Christian ___ Qbs mailing list Qbs@qt-project.org https://lists.qt-project.org/listinfo/qbs
Re: [Qbs] tst_blackbox-qt fails
Christian, >> This is how I build and test now: >> >> qbs build profile:qt >> qbs run -p qbs_app profile:qt -- setup-toolchains --detect >> qbs run -p qbs_app profile:qt -- setup-qt >> /usr/local/Qt/5.9.7/gcc_64/bin/qmake qt >> QBS_AUTOTEST_PROFILE=qt qbs build -p autotest-runner profile:qt >> >> Is this how it is supposed to be done? > > Looks fine to me. For extra safety, e.g. if simultaneous execution with > other qbs instances is required, one would also pass a dedicated > --settings-path, as to have truly unique profiles for the respective > run. I have to bring this up again because I noticed that autotest-runner above does only work if I export QBS_AUTOTEST_PROFILE to the system environment. Exporting it in the current user session as shown above, does not work. The only way I was able to do it was, to override project.autotestWrapper with a custom script that exports QBS_AUTOTEST_PROFILE. But isn't that a little awkward? Therefore I would like to know: 1. Is that the way how developers are supposed to inject profiles into the autotest-runner product? 2. How are you doing it? Could you publish your Jenkins scripts? 3. How about adding a property "runEnvironment" to QbsAutotest that would allow to set the profile on the command line when building autotest-runner? Those variables would be appended to qbs.commonRunEnvironment in the product. Thank you very much Richard ___ Qbs mailing list Qbs@qt-project.org https://lists.qt-project.org/listinfo/qbs
Re: [Qbs] tst_blackbox-qt fails
On Wed, 2 Jan 2019 17:27:29 +0100 Richard Weickelt wrote: > This is how I build and test now: > > qbs build profile:qt > qbs run -p qbs_app profile:qt -- setup-toolchains --detect > qbs run -p qbs_app profile:qt -- setup-qt > /usr/local/Qt/5.9.7/gcc_64/bin/qmake qt > QBS_AUTOTEST_PROFILE=qt qbs build -p autotest-runner profile:qt > > Is this how it is supposed to be done? Looks fine to me. For extra safety, e.g. if simultaneous execution with other qbs instances is required, one would also pass a dedicated --settings-path, as to have truly unique profiles for the respective run. > >> I need some guidance how to properly setup the autotests. In addition, I > >> would like to know how The Qt Company is running Qbs regression tests today > >> on which platforms and when. > > > > This is currently done on a local Jenkins-based CI. Tests run on Linux (32 > and 64 bit), macOS (different versions) and Windows. On Windows, MSVC and > mingw are tested. For the Qt autotests, we also test against a static Qt > installation on Windows. > > Is this visible outside your company? > How could external contributors leverage that infrastructure? E.g. how could > they invoke tests and see the results? Not at all. > Would it be possible to integrate that into gerrit? And if yes, would you do > it? I suppose it would be possible to put qbs under the CI regime just like Qt. Jake has actually tried that at some point, but ran into problems. I don't remember the details. I have a suspicion that this is one of those tedious tasks that everybody likes to leave to someone else... > >> The page at > >> http://doc.qt.io/qbs/building-qbs.html does not talk about autotests and > >> the > >> docker images are outdated. Thus I assume that the information on this page > >> is not used. > > > > I don't know what you mean by that. > > 1. The page is titled "Building Qbs" but doesn't explain how to run > autotests and the requirements on the environment. If it was explained I > wouldn't have asked above. That's because it's aimed at users of qbs who want to build from source, not developers of qbs. > 2. It is not possible to build and test the Qbs master branch with the > provided Docker images without issues. They come with Qt 5.9.3 and Qbs 1.9.1. I have close to zero knowledge about the Docker images, unfortunately. So anyone investigating in this area for a couple of minutes will already likely have a better understanding of it than I do. > These things are easy to fix, but I assume that nobody has tested the > instructions on this page for a while and nobody is using those Docker > images. Sounds plausible. > I am going to make a patch. Thanks. Christian ___ Qbs mailing list Qbs@qt-project.org https://lists.qt-project.org/listinfo/qbs
Re: [Qbs] tst_blackbox-qt fails
On Sat, 29 Dec 2018 08:48:37 +0100 Richard Weickelt wrote: > I tried to build Qbs from source and to run the autotests on Debian stretch > with Qt 5.9.7 and 5.12.0. After creating a Qt profile (with the built Qbs) > and setting the undocumented QBS_AUTOTEST_PROFILE variable I still see some > tests failing in tst_blackbox-qt (see log below). I strongly suspect that you did not run the autotests with the exact same build that you called setup-qt with. Please double-check. > I need some guidance how to properly setup the autotests. In addition, I > would like to know how The Qt Company is running Qbs regression tests today > on which platforms and when. This is currently done on a local Jenkins-based CI. Tests run on Linux (32 and 64 bit), macOS (different versions) and Windows. On Windows, MSVC and mingw are tested. For the Qt autotests, we also test against a static Qt installation on Windows. > The page at > http://doc.qt.io/qbs/building-qbs.html does not talk about autotests and the > docker images are outdated. Thus I assume that the information on this page > is not used. I don't know what you mean by that. > I would be willing to update the Docker infrastructure and the > documentation. It might be beneficial if anybody could easily run all > autotests without setting anything up on the local system. Most of the tests do not strictly require a profile, and the Qt tests skip themselves if no Qt profile was given, so this should work already. Christian ___ Qbs mailing list Qbs@qt-project.org https://lists.qt-project.org/listinfo/qbs
[Qbs] tst_blackbox-qt fails
Hi,
I tried to build Qbs from source and to run the autotests on Debian stretch
with Qt 5.9.7 and 5.12.0. After creating a Qt profile (with the built Qbs)
and setting the undocumented QBS_AUTOTEST_PROFILE variable I still see some
tests failing in tst_blackbox-qt (see log below).
Looking closer at TestBlackboxQt::includedMocCpp() this test fails even with
Qbs 1.12.2.
I need some guidance how to properly setup the autotests. In addition, I
would like to know how The Qt Company is running Qbs regression tests today
on which platforms and when. The page at
http://doc.qt.io/qbs/building-qbs.html does not talk about autotests and the
docker images are outdated. Thus I assume that the information on this page
is not used.
I would be willing to update the Docker infrastructure and the
documentation. It might be beneficial if anybody could easily run all
autotests without setting anything up on the local system.
Best regards
Richard
* Start testing of TestBlackboxQt *
Config: Using QtTest library 5.12.0, Qt 5.12.0 (x86_64-little_endian-lp64
shared (dynamic) release build; by GCC 5.3.1 20160406 (Red Hat 5.3.1-6))
PASS : TestBlackboxQt::initTestCase()
PASS : TestBlackboxQt::addQObjectMacroToGeneratedCppFile()
PASS : TestBlackboxQt::autoQrc()
PASS : TestBlackboxQt::cachedQml()
PASS : TestBlackboxQt::combinedMoc()
PASS : TestBlackboxQt::createProject()
PASS : TestBlackboxQt::dbusAdaptors()
PASS : TestBlackboxQt::dbusInterfaces()
QDEBUG : TestBlackboxQt::includedMocCpp() ERROR: /usr/bin/g++
-Wl,-m,elf_x86_64,-rpath,/usr/local/Qt/5.12.0/gcc_64/lib
-L/usr/local/Qt/5.12.0/gcc_64/lib -m64 -o
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/included-moc-cpp/default/included-moc-cpp.d2f3350b/included-moc-cpp
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/included-moc-cpp/default/included-moc-cpp.d2f3350b/3a52ce780950d4d9/main.cpp.o
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/included-moc-cpp/default/included-moc-cpp.d2f3350b/3a52ce780950d4d9/myobject.cpp.o
/usr/local/Qt/5.12.0/gcc_64/lib/libQt5Core.so.5.12.0 -lpthread
ERROR:
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/included-moc-cpp/default/included-moc-cpp.d2f3350b/3a52ce780950d4d9/main.cpp.o:
In function `MyObject::MyObject()':
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/included-moc-cpp/myobject.h:6:
undefined reference to `vtable for MyObject'
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/included-moc-cpp/default/included-moc-cpp.d2f3350b/3a52ce780950d4d9/main.cpp.o:
In function `MyObject::~MyObject()':
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/included-moc-cpp/myobject.h:6:
undefined reference to `vtable for MyObject'
collect2: error: ld returned 1 exit status
ERROR: Process failed with exit code 1.
The following products could not be built for configuration default:
included-moc-cpp
QDEBUG : TestBlackboxQt::includedMocCpp() Restoring build graph from disk
Building for configuration default
compiling myobject.cpp [included-moc-cpp]
linking included-moc-cpp [included-moc-cpp]
FAIL! : TestBlackboxQt::includedMocCpp() Compared values are not the same
Actual (runQbs()): 1
Expected (0) : 0
Loc: [/project/tests/auto/blackbox/tst_blackboxqt.cpp(155)]
PASS : TestBlackboxQt::lrelease()
PASS : TestBlackboxQt::mixedBuildVariants()
PASS : TestBlackboxQt::mocAndCppCombining()
PASS : TestBlackboxQt::mocFlags()
PASS : TestBlackboxQt::mocSameFileName()
SKIP : TestBlackboxQt::pkgconfig() pkgconfig or Qt not found
Loc: [/project/tests/auto/blackbox/tst_blackboxqt.cpp(243)]
PASS : TestBlackboxQt::pluginMetaData()
QDEBUG : TestBlackboxQt::pluginSupport(request valid plugins) ERROR:
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/plugin-support/plugin-support.qbs:1:1
Error while handling product 'plugin-support':
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/plugin-support/modules/m1/m1.qbs:3:5
Dependency 'Qt.plugin_support' not found for product 'plugin-support'.
Please create a Qt profile using the qbs-setup-qt tool if you haven't
already done so.
/project/default/install-root/tests/auto/blackbox-qt/testWorkDir/plugin-support/modules/m2/m2.qbs:2:5
Dependency 'Qt.plugin_support' not found for product 'plugin-support'.
Please create a Qt profile using the qbs-setup-qt tool if you haven't
already done so.
QDEBUG : TestBlackboxQt::pluginSupport(request valid plugins) Build graph
does not yet exist for configuration 'default'. Starting from scratch.
Resolving project for configuration default
FAIL! : TestBlackboxQt::pluginSupport(request valid plugins) Compared
values are not the same
Actual (runQbs(resolveParams) == 0): 0
Expected (!invalidPlugin): 1
Loc: [/project/tests/auto/blackbox/tst_blackboxqt.cpp(275)]
FAIL! : TestBlackboxQt::pluginSupport(request invalid plugin)
'm_qbsStderr.contains("Plugin 'dummy' of type
