I have no argument with your points. However, figuring out the code coverage of parserTests is low hanging fruit, i.e., relatively easy to determine and at least somewhat valuable. By knowing which files are touched by parserTests and how much of their code is covered is a first step in figuring out what needs to be done (e.g., identifying those files that parserTests doesn't even visit). After all, parserTests is all we have at the moment.
--- On Thu, 7/30/09, Happy-melon <happy-me...@live.com> wrote: > From: Happy-melon <happy-me...@live.com> > Subject: Re: [Wikitech-l] parserTests code coverage statistics > To: wikitech-l@lists.wikimedia.org > Date: Thursday, July 30, 2009, 4:26 AM > I suspect that much of this has to do > with the way Parser.php is eleven > thousand lines of programmatic sewage, and the way the > ParserTests > infrastructure requires a lot of the rest of MediaWiki to > be initialised in > order to run the tests. As long as the rest of the > system 'works' well > enough to allow the parser to parse, ParserTests is happy: > I wouldn't be > confident to say that it "tests" anything outside of > Parser.php; as you say, > it only marks lines 'visited'. > > --HM > > "dan nessett" <dness...@yahoo.com> > wrote in message > news:922893.25678...@web32507.mail.mud.yahoo.com... > > > > I failed to mention that xdebug ignores non-executable > lines of code. So, > > the statistics are for executable lines of code and do > not include lines > > like comments (in either the covered or uncovered > counts). > > > > --- On Wed, 7/29/09, dan nessett <dness...@yahoo.com> > wrote: > > > >> From: dan nessett <dness...@yahoo.com> > >> Subject: [Wikitech-l] parserTests code coverage > statistics > >> To: wikitech-l@lists.wikimedia.org > >> Date: Wednesday, July 29, 2009, 4:36 PM > >> > >> I decided to investigate how well parserTests > exercises the > >> MW code. So, I threw together a couple of MacGyver > tools > >> that use xdebug's code coverage capability and > analyzed the > >> results. The results are very, very preliminary, > but I > >> thought I would get them out so others can look > them over. > >> In the next couple of days I hope to post more > detailed > >> results and the tools themselves on the Mediawiki > wiki. (If > >> someone could tell me the appropriate page to use > that would > >> be useful. Otherwise, I will just create a page in > my own > >> namespace). > >> > >> The statistics (again very preliminary) are: > >> > >> Number of files exercised: 141 Number of > lines in > >> those files: 85606 > >> Lines covered: 59489 Lines not covered: > 26117 > >> Percentage covered: 0.694916244188 > >> > >> So, parserTests is getting (at best) about 70% > code > >> coverage. This is better than I expected, but > still it means > >> parserTests does not test 26117 lines of code. > What I mean > >> by "at best" is xdebug just notes whether a line > of code is > >> visited. It doesn't do any logic analysis on which > branches > >> are taken. Furthermore, parserTests may not visit > some files > >> that are critical to the operation of the MW > software. > >> Obviously, xdebug can only gather statistics on > visited > >> files. > >> > >> I want to emphasize that there may be errors in > these > >> results due to bad assumptions on my part or bad > coding. > >> However, it is a place to start. > >> > >> > >> > >> > >> _______________________________________________ > >> Wikitech-l mailing list > >> Wikitech-l@lists.wikimedia.org > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l > >> > > > > > > = > > > > _______________________________________________ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
