Re: [Wikitech-l] MediaWiki core code coverage report
On 10/18/2013 05:23 PM, Brad Jorsch (Anomie) wrote: > On Fri, Oct 18, 2013 at 5:52 AM, Antoine Musso wrote: >> track which lines in MediaWiki core have been executed > > Does it count the whole line as covered for something like > > $var = $alwaysTrue ? $something : $something_else; > > or just the "true" case? The whole line. This is a limitation of the PHP extension Xdebug. See http://phpunit.de/manual/current/en/code-coverage-analysis.html#code-coverage-analysis.edge-cases Based on my experience, there will also be some false negatives. -- Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] MediaWiki core code coverage report
Thank you! On 10/18/2013 11:52 AM, Antoine Musso wrote: > Now I am wondering where on mw.org I can document our coverage system. > Should we get a page under [[Continuous integration]] or maybe another > section in [[Manual:PHP unit testing]] ?? Its specific to PHPUnit so I went ahead and added https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests#Coverage . -- Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] MediaWiki core code coverage report
On 10/18/2013 07:31 PM, Jeroen De Dauw wrote: >> Uh, what does test coverage on tests/ directory even mean? Who >> watches the watchers tests the tests? > > Good point. Non-production code typically ought to be excluded. This is > pretty simple to do with some PHPUnit config: https://gerrit.wikimedia.org/r/#/c/90584/ -- Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Tor exemption process (was: Re: Jake requests enabling access and edit access to Wikipedia via TOR)
On 01/18/2014 12:33 AM, Erik Moeller wrote: > I'm not arguing for open editing from Tor. I do think it would be nice > if global exemptions could in fact be obtained reasonably easily be > emailing stewa...@wikimedia.org. While it's true that such requests > could be misused, the following are also true: What would be involved for someone to misuse _without_ Tor?: 1) Find a free WiFi spot you haven't used before. 2) Create account (no need to enter any Email). 3) Abuse and repeat if you get banned. I tried 1 and 2, obviously not doing 3. It worked on my first try. I wouldn't have thought it would be that easy. To summarize: It is currently easier for someone to abuse without Tor than with it. An abuser doesn't need that strong anonymity. For someone who wants to do good face editing and needs the stronger anonymity provided by Tor it is currently much more work than for an abuser. Why should account creation be harder for Tor users than for free WiFi users? -- Regards, Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Invitation to beta-test HHVM
On 2014-09-21 09:51, Pine W wrote: > Thanks, but I'm looking for something that is more specific to MediaWiki > and that commits development teams to specific, standardized, and > transparently measured quality metrics as products advance or regress in > their path to production release. Yes I would be interested to know about that, too. In case there are none yet: Any ideas what those metrics should be? The only really useful one I can come up is open bugs that are marked with keyword "code-update-regression" after a certain amount of time of it being available for public testing. (Currently there are 60 of those, not all of them on Wikimedia sites: https://bugzilla.wikimedia.org/buglist.cgi?keywords=code-update-regression&list_id=345862&resolution=--- ) These are more for software maintenance quality than directly for end user quality: Code coverage of unit tests over a certain percentage. Amount of use cases (or user stories) that have integration tests that can be run directly against an unmodified installation over a certain percentage. Perhaps certain limits on metrics generated by tools like phpdepend. Perhaps being clean of certain phpmd or PHP_CodeSniffer tests failing. Is there interest in committing to only deploying in production if any metrics are met? (Obviously there needs to be a way to have a grace period for code that was already on the way to deployment before that was decided.) For the software quality metrics it probably makes sense to already enforce them before merge instead of before deployment. I know requiring unit tests was discussed in the past, anyone remembers where and what the result was? -- Regards, Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] MediaWiki Technical Debt
Actual results for mediawiki core: https://insight.sensiolabs.com/projects/8b5d527f-f70c-4fc0-977f-de99743eae06/analyses/16 Over all I think making stuff more testable with phpunit is much more important than going through any parts of that report. On 2014-10-23 17:16, Jeroen De Dauw wrote: >> It's probably a useless statistic > > It's based on SensioLabsInsight, which I personally quite dislike. > ScrutinizerCI is hands down better at spotting issues (way to many false > positives on SensioLabsInsight). I think PHP_CodeSniffer (possibly with a few new sniffs) can replace all of their tests. There are many important metrics they do not consider at all. Something quite better than what they have can be obtained with the following tools: phpunit (code coverage), phpcpd (copied code), warnings from your documentation generator like phpdox, PHP_Depend, phploc, PHPMD, PHP_CodeSniffer. See http://jenkins-php.org for how to integrate that into our CI infrastructure. Yes, many false positives. Many of their tests are only of the type: search for where $function is called. In fact I think none of their tests can be classified as static analysis (not sure about "Database queries should use parameter binding", though). Many of their tests can only be used as hints because there are enough instances where that is exactly what one needs to do. Examples: - exit() and die() functions should be avoided - PHP configuration should not be changed dynamically Some more examples that might be useful in some form: - Global variable or function should never be used (This is probably the only really useful one, because removing global functions/variables would result in better testable code.) - PHP debug statements found - Logical operators should be avoided - sleep() should not be used - Source code should not contain FIXME comments -- Regards, Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] RFC meeting this week
On 2015-08-05 11:55, Daniel Kinzler wrote: > Am 05.08.2015 um 13:21 schrieb Tim Starling: >> In the next RFC meeting, we will discuss the following RFC: >> >> * Streamlining Composer usage >> <https://www.mediawiki.org/wiki/Requests_for_comment/Streamlining_Composer_usage> > > Hm... Jan is on vacation, do you know if he's aware of this happening? I think > he's actually in SF, so he could join live from the WMF office :) I will take part on IRC, but won't be in the office at that time. -- Regards, Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Gerrit +1 now executes the code you reviewed
I just merged and deployed https://gerrit.wikimedia.org/r/#/c/184886/ , which means: A +1 in gerrit.w.o didn't have any technical effect until now. Now it submits the patch for testing. That means if you +1 a patch from a non-whitelisted user that was not yet tested, it will then, just as if recheck was issued. Thus executing the code that you reviewed to not steal secrets or compromise security in other ways. -- Regards, Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Gerrit +1 now executes the code you reviewed
On 2015-11-17 19:30, Chris Steipp wrote: > Just to clarify, this is a +1 from a user who has +2 rights? Whereas a +1 > from some random user will not initiate the tests? No. The permission here is the CI white list in zuul, which is a different one from gerrit +2. The permission and who has it didn't change. (Before a user in that white list could comment recheck on a repo where they didn't have +2 to get the same effect.) -- Regards, Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] mediawiki +2 for Jan Zerebecki
As is unwritten custom, I announce here on behalf of the Gerrit Administrators that per https://phabricator.wikimedia.org/T116506 Antoine "hashar" Musso gave Jan Zerebecki membership in the gerrit wikimedia group. Thx hashar. -- Regards, Jan Zerebecki ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
