[Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Antoine Musso
Hello, Too long, dont want to read: https://integration.wikimedia.org/cover/mediawiki-core/master/php/ Jenkins is nowadays producing a nightly coverage report. The idea is to run our full PHPUnit tests, track which lines in MediaWiki core have been executed and show up metrics regarding code

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Brad Jorsch (Anomie)
On Fri, Oct 18, 2013 at 5:52 AM, Antoine Musso hashar+...@free.fr wrote: track which lines in MediaWiki core have been executed Does it count the whole line as covered for something like $var = $alwaysTrue ? $something : $something_else; or just the true case? In June I enforced a PHPUnit

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Erik Bernhardson
On Fri, Oct 18, 2013 at 8:23 AM, Brad Jorsch (Anomie) bjor...@wikimedia.org wrote: On Fri, Oct 18, 2013 at 5:52 AM, Antoine Musso hashar+...@free.fr wrote: In June I enforced a PHPUnit feature which force us to mention which MediaWiki function is covered by a test method [FORCE COVER].

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Brad Jorsch (Anomie)
On Fri, Oct 18, 2013 at 12:13 PM, Erik Bernhardson ebernhard...@wikimedia.org wrote: On Fri, Oct 18, 2013 at 8:23 AM, Brad Jorsch (Anomie) bjor...@wikimedia.org wrote: On Fri, Oct 18, 2013 at 5:52 AM, Antoine Musso hashar+...@free.fr wrote: In June I enforced a PHPUnit feature which force

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Erik Bernhardson
different definitions of test ;-) code touched seems like a much less useful metric than code specifically tested, but i could be convinced otherwise. On Fri, Oct 18, 2013 at 9:31 AM, Brad Jorsch (Anomie) bjor...@wikimedia.org wrote: On Fri, Oct 18, 2013 at 12:13 PM, Erik Bernhardson

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Niklas Laxström
2013/10/18 Antoine Musso hashar+...@free.fr: Hello, Too long, dont want to read: https://integration.wikimedia.org/cover/mediawiki-core/master/php/ This is very cool, apart from the numbers of course. Is it hard to setup for extensions as well? Lots of development happens in extensions

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Jeroen De Dauw
Hey, Thanks for setting this up hashar! It's a very useful metric to have. I'd like to bring some attention to the dashboard link at the top of coverage pages, which leads to for instance https://integration.wikimedia.org/cover/mediawiki-core/master/php/index.dashboard.html This page is very

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Krinkle
True, but there's two sides of this. When needing the explicit annotation, it will keep out stuff that shouldn't be included (I agree on the wfRunHooks example). However, depending on how it's implemented, it might also remove the very essence of line coverage. We wouldn't want to disable

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Jan Zerebecki
On 10/18/2013 05:23 PM, Brad Jorsch (Anomie) wrote: On Fri, Oct 18, 2013 at 5:52 AM, Antoine Musso hashar+...@free.fr wrote: track which lines in MediaWiki core have been executed Does it count the whole line as covered for something like $var = $alwaysTrue ? $something : $something_else;

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Victor Vasiliev
On 10/18/2013 05:52 AM, Antoine Musso wrote: Hello, Too long, dont want to read: https://integration.wikimedia.org/cover/mediawiki-core/master/php/ Uh, what does test coverage on tests/ directory even mean? Who swatches the watchers/s tests the tests? — Victor.

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Jeroen De Dauw
Hey, Uh, what does test coverage on tests/ directory even mean? Who swatches the watchers/s tests the tests? Good point. Non-production code typically ought to be excluded. This is pretty simple to do with some PHPUnit config:

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Jan Zerebecki
Thank you! On 10/18/2013 11:52 AM, Antoine Musso wrote: Now I am wondering where on mw.org I can document our coverage system. Should we get a page under [[Continuous integration]] or maybe another section in [[Manual:PHP unit testing]] ?? Its specific to PHPUnit so I went ahead and added

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Jan Zerebecki
On 10/18/2013 07:31 PM, Jeroen De Dauw wrote: Uh, what does test coverage on tests/ directory even mean? Who swatches the watchers/s tests the tests? Good point. Non-production code typically ought to be excluded. This is pretty simple to do with some PHPUnit config:

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Antoine Musso
Le 18/10/13 19:20, Jan Zerebecki a écrit : The whole line. This is a limitation of the PHP extension Xdebug. See http://phpunit.de/manual/current/en/code-coverage-analysis.html#code-coverage-analysis.edge-cases Based on my experience, there will also be some false negatives. And also note

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Antoine Musso
Le 18/10/13 18:42, Niklas Laxström a écrit : Too long, dont want to read: https://integration.wikimedia.org/cover/mediawiki-core/master/php/ This is very cool, apart from the numbers of course. Is it hard to setup for extensions as well? Lots of development happens in extensions

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Antoine Musso
Le 18/10/13 18:52, Jeroen De Dauw a écrit : Hey, Thanks for setting this up hashar! It's a very useful metric to have. I'd like to bring some attention to the dashboard link at the top of coverage pages, which leads to for instance

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Jeroen De Dauw
Hey, For those interested in having coverage reports for their extensions, you can look at the source of any of the extensions listed here: https://coveralls.io/r/wikimedia These extensions allow running their tests by executing phpunit in their root directory. That makes creating the coverage

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Antoine Musso
Le 18/10/13 20:18, Jan Zerebecki a écrit : Its specific to PHPUnit so I went ahead and added https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests#Coverage Excellent! And I think that is the first time ever I validate a diff on mediawiki.org \O/ Thank you! -- Antoine

Re: [Wikitech-l] MediaWiki core code coverage report

2013-10-18 Thread Michelle Grover
Mobile is definitely interested in running this for MobileFrontend. I'll read up as well. On Fri, Oct 18, 2013 at 1:34 PM, Antoine Musso hashar+...@free.fr wrote: Le 18/10/13 18:42, Niklas Laxström a écrit : Too long, dont want to read: