Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Aryeh Gregor
On Tue, Jul 21, 2009 at 6:05 PM, dan nessett wrote: > Not sure the post-commit hook running the parser is a good idea. The software > could have been broken by a previous committer. From that point on > parserTests will report errors until the problem is fixed, so committers will > just learn to

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread dan nessett
an > Subject: Re: [Wikitech-l] "known to fail" switch added to parserTests > To: "Wikimedia developers" > Date: Tuesday, July 21, 2009, 11:10 AM > > Right, well, a pre-commit hook that rejects all commits > which break the > software. Or a memory

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Brian
On Tue, Jul 21, 2009 at 12:05 PM, dan nessett wrote: > > Not sure the post-commit hook running the parser is a good idea. The > software could have been broken by a previous committer. From that point on > parserTests will report errors until the problem is fixed, so committers > will just learn

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread dan nessett
: > From: Brian > Subject: Re: [Wikitech-l] "known to fail" switch added to parserTests > To: "Wikimedia developers" > Date: Tuesday, July 21, 2009, 9:57 AM > On Tue, Jul 21, 2009 at 10:51 AM, > Aryeh Gregor < > simetrical+wikil...@gmail.com > > >

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Aryeh Gregor
On Tue, Jul 21, 2009 at 12:57 PM, Brian wrote: > The tests have never passed - they should be commented out for usability > reasons. Well, I have no objections, but apparently that's not acceptable. An "expected fail" flag would be about as usable. > And ideally there would be a post-commit hook

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Brian
On Tue, Jul 21, 2009 at 10:51 AM, Aryeh Gregor < simetrical+wikil...@gmail.com > wrote: > But if we're going to keep the known-to-fail tests at all, it doesn't make > a lot of sense to report > them as passing when they're actually failing . . . if we do that we may as > well just drop them. Th

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Aryeh Gregor
On Tue, Jul 21, 2009 at 10:45 AM, dan nessett wrote: > The change I made was to add a "flipresult" option that simply turns a > success into a failure and a failure into a success. This is what I > understood I was asked to do. On the plus side, this approach also allows the > addition of parser

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Stephen Bain
On Wed, Jul 22, 2009 at 12:45 AM, dan nessett wrote: > > I just looked at the code and it shouldn't be hard to add a knowntofail > option that acts like flipresult and then add a new category of test result > that specifies how many known to fail results occurred. However, one issue is > whether

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread dan nessett
page processing flow? --- On Tue, 7/21/09, Kwan Ting Chan wrote: > From: Kwan Ting Chan > Subject: Re: [Wikitech-l] "known to fail" switch added to parserTests > To: "Wikimedia developers" > Date: Tuesday, July 21, 2009, 4:43 AM > Aryeh Gregor wrote: > &

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Magnus Manske
On Tue, Jul 21, 2009 at 3:21 PM, Gerard Meijssen wrote: > Hoi, > That is exactly the problem. You report that they pass and in reality they > still fail. Someone should change them to pass ie fix the software. I think the appropriate English reply in this context would be "No s**t, Sherlock"... _

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Gerard Meijssen
Hoi, That is exactly the problem. You report that they pass and in reality they still fail. Someone should change them to pass ie fix the software. Thanks, GerardM 2009/7/21 Aryeh Gregor > > On Tue, Jul 21, 2009 at 9:56 AM, Gerard > Meijssen wrote: > > There is no point having a perfect sco

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Chad
On Tue, Jul 21, 2009 at 7:43 AM, Kwan Ting Chan wrote: > Aryeh Gregor wrote: >> >> On Tue, Jul 21, 2009 at 9:56 AM, Gerard >> Meijssen wrote: >>> >>> There is no point having a perfect score when it is actually a lie. It >>> seems >>> to me that Brion is against the removal of these tests because h

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Kwan Ting Chan
Aryeh Gregor wrote: On Tue, Jul 21, 2009 at 9:56 AM, Gerard Meijssen wrote: There is no point having a perfect score when it is actually a lie. It seems to me that Brion is against the removal of these tests because he wants them to pass. Having a third state of "known to fail" makes sense, just

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Aryeh Gregor
On Tue, Jul 21, 2009 at 9:56 AM, Gerard Meijssen wrote: > There is no point having a perfect score when it is actually a lie. It seems > to me that Brion is against the removal of these tests because he wants them > to pass. Having a third state of "known to fail" makes sense, just changing > them

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Gerard Meijssen
Hoi, There is no point having a perfect score when it is actually a lie. It seems to me that Brion is against the removal of these tests because he wants them to pass. Having a third state of "known to fail" makes sense, just changing them to pass makes it necessary to add a "citation needed" becau

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Aryeh Gregor
On Tue, Jul 21, 2009 at 7:19 AM, Gerard Meijssen wrote: > Suppose that someone fixes a test that has been always failing... one of > those "known to fail". It makes no difference right ?? Difference in what sense? It means we have one less failing test reported, presumably. > Giving them the > s

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-21 Thread Gerard Meijssen
Hoi, Suppose that someone fixes a test that has been always failing... one of those "known to fail". It makes no difference right ?? Giving them the status of pass is imho dead wrong because they should not fail in the first place.. now a status of KNOWN TO FAIL makes sense. Thanks, GeardM 2

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-20 Thread dan nessett
serTest under it. > > --- On Mon, 7/20/09, dan nessett > wrote: > > > From: dan nessett > > Subject: [Wikitech-l] "known to fail" switch added to > parserTests > > To: wikitech-l@lists.wikimedia.org > > Date: Monday, July 20, 2009, 8:09 AM > > >

Re: [Wikitech-l] "known to fail" switch added to parserTests

2009-07-20 Thread dan nessett
work for 1.16a under a 1.14 schema. I am now checking out REL1_14 and will run the modified parserTest under it. --- On Mon, 7/20/09, dan nessett wrote: > From: dan nessett > Subject: [Wikitech-l] "known to fail" switch added to parserTests > To: wikitech-l@lists.wikimedia

[Wikitech-l] "known to fail" switch added to parserTests

2009-07-20 Thread dan nessett
I have modified parserTests to take a "known to fail" switch so those tests that have always failed now pass. It was pretty simple. It only required 3 changes to parserTests.inc and some editing on parserTests.txt. I added an option for each test called flipresult. When this option is specified