To emphasize on aspect of this PSA: This only works for tests in Qt 5.9 and onwards (once qtbase merged), not for older branches of Qt.
Simon ________________________________ From: Development <development-bounces+simon.hausmann=qt...@qt-project.org> on behalf of Tor Arne Vestbø <tor.arne.ves...@qt.io> Sent: Thursday, March 23, 2017 3:55:30 PM To: development@qt-project.org Subject: [Development] PSA: Blacklisting test for CI-runs only Hi everyone, It's now possible to blacklist tests only when run in the CI. You do this by adding the 'ci' keyword to the normal list of keywords when blacklisting a test, eg: [someTestFunction] osx-10.8 ci See also https://bugreports.qt.io/browse/QTBUG-59564 This results in the test being blacklisted when running on OSX 10.8 in the CI (BPASS/BFAIL), but not on local machines (PASS/FAIL). This should typically be used for tests that are flakey, but can only be reproduced in the CI. That way we'll detect when the test fails in local testing and have a higher chance of fixing it. Please consider if the 'ci' keyword should be added when blacklisting new tests (or reviewing blacklistings). In most cases it should, as tests that can be reproduced locally should ideally be fixed instead of blacklisted. Cheers, Tor Arne _______________________________________________ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
_______________________________________________ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
