hashar added a comment.
T288396#8979619 <https://phabricator.wikimedia.org/T288396#8979619> is a very nice dig :) > If you omit the `$ext/tests/phpunit argument`, it runs 36438 tests (presumably all tests from all installed extensions). > > Actually it's unclear to me how CI can ever run only the tests from a specific extension. It looks as if non-coverage test jobs will always run tests from extension dependencies. When a patch is triggered for a MediaWiki extension, Quibble invokes `composer run phpunit:entrypoint -- --testsuite extensions`. The suite is defined in `mediawiki/core` `tests/phpunit/suite.xml` and finds tests extensions registered via the `onUnitTestsList` hook. As a result, Quibble triggers PHPUnit tests for every single extensions that are dependencies. It is a brute force approach to run the cross extensions integration tests. There are a lot of issues: we run way too many tests and given the dependencies are not bi directional, some integration tests are not run when an extension is not defined as a dependency. But I digress. The coverage is rendered with solely for tests of the triggering extension (and thus the coverage reports lacks any tests exercised by other repositories). It is probably not a big issue. ---- > Alternatively, you can just specify extensions/Wikibase as the path. It seems to work. That is the root cause I guess, congratulation. `tests/phpunit` suffix probably rely on the convention used for most extension but clashes with Wikibase (and possibly other extensions). The path suffix comes from 2018 patch //Only run that extension's tests in coverage job// https://gerrit.wikimedia.org/r/c/integration/config/+/405393 The scripts can be altered: name=dockerfiles/quibble-buster-php74-coverage/mwext-phpunit-coverage.sh - "$MW_INSTALL_PATH/extensions/$EXT_NAME/tests/phpunit" & + "$MW_INSTALL_PATH/extensions/$EXT_NAME" & name=dockerfiles/quibble-buster-php74-coverage/mwskin-phpunit-coverage.sh - "$MW_INSTALL_PATH/extensions/$SKIN_NAME/tests/phpunit" & + "$MW_INSTALL_PATH/extensions/$SKIN_NAME" & ---- > If I fix the test path, I still get "Warning: Incorrect filter configuration, code coverage will not be processed". This is probably because src/, includes/ and maintenance/ are all non-existent, and so no files are included in the coverage filter. Perhaps CI could get the list of directories to cover from some configuration file in the extension, instead of guessing. MediaWiki core has `composer run phpunit:coverage-edit` which invokes `includes/composer/ComposerPhpunitXmlCoverageEdit.php` which looks similar to the CI script `phpunit-suite-edit.py` uses the same wrong assumption. Looks like it can be replaced by the MediaWiki core equivalent code. I have found T235031: phpunit:coverage-edit - Add configuration flag so it can replace phpunit-suite-edit.py <https://phabricator.wikimedia.org/T235031>. TASK DETAIL https://phabricator.wikimedia.org/T288396 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hashar Cc: tstarling, WMDE-leszek, hashar, Lucas_Werkmeister_WMDE, Jdforrester-WMF, Aklapper, Reedy, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, Peteosx1x, ItamarWMDE, Mgagat, Akuckartz, Totolinototo3, Zanziii, Sadisticturd, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
