Re: [Twisted-Python] Spurious failure on winxp32-py2.7

[Glyph Lefkowitz]

> On Jan 26, 2015, at 7:56 AM, Adi Roiban  wrote:
> 
> On 26 January 2015 at 04:00, Glyph  wrote:
>> 
>> On Jan 25, 2015, at 7:28 AM, exar...@twistedmatrix.com wrote:
>> 
>> 
>> This test has intermittently failed on Windows since disttrial was merged
>> into trunk.
>> 
>> 
>> And, as you yourself pointed out on the ticket, sometimes on Linux too.
>> 
>> I still have absolutely no clue why this happens, by the way.  If anyone
>> else could be prompted to investigate I would be much obliged
> 
> I have updated my findings here: https://twistedmatrix.com/trac/ticket/6027 
> 

Thanks for looking into it.

>> The default state of a test should *not* be to skip.  It should be to run.
>> A skipped test is a useless test.
>> 
>> As a matter of general principle, the only reason a test should ever be
>> skipped within Twisted is if an optional dependency is not available.  And,
>> just to be clear, the presence of "TODO'd" tests is pretty much just a bug,
>> fix old ones and never add new ones :-).
> 
> I think that these todo are just another way of creating a ticket ...
> with a promise to annoy you forever

And to annoy every contributor ever who works on something unrelated :).

> Would it help if I extract them into proper Trac tickes and clean the code?

Some of them are already filed, but... yes, having a ticket for every one would 
be helpful.

In fact, we might want to have two tickets for each one: one "delete this 
todo'd test" ticket, and one "actually fix this issue" ticket, since we don't 
need to keep hanging on to the tests in the repo themselves while we wait for 
someone to take care of the issues.

> 
> 
> Maybe update docs and stress that todo tests should not be committed
> ... or never mention about such thing as TODO markers,
> I just write tests and expect them to fail without marking them as todo.
> 
> http://twistedmatrix.com/documents/current/core/development/policy/test-standard.html
> 
> (not sure why some docs are on wiki and others in the repo)

There's no really good reason, that's just where things are.  Probably 
everything should move into the repo.

> And maybe also add a twistedchecker check and check that no new tests
> are commited with todo.

If you wanted to add that it would definitely be helpful.

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious failure on winxp32-py2.7

[Adi Roiban]

On 26 January 2015 at 04:00, Glyph  wrote:
>
> On Jan 25, 2015, at 7:28 AM, exar...@twistedmatrix.com wrote:
>
>
> This test has intermittently failed on Windows since disttrial was merged
> into trunk.
>
>
> And, as you yourself pointed out on the ticket, sometimes on Linux too.
>
> I still have absolutely no clue why this happens, by the way.  If anyone
> else could be prompted to investigate I would be much obliged

I have updated my findings here: https://twistedmatrix.com/trac/ticket/6027


> The default state of a test should *not* be to skip.  It should be to run.
> A skipped test is a useless test.
>
> As a matter of general principle, the only reason a test should ever be
> skipped within Twisted is if an optional dependency is not available.  And,
> just to be clear, the presence of "TODO'd" tests is pretty much just a bug,
> fix old ones and never add new ones :-).

I think that these todo are just another way of creating a ticket ...
with a promise to annoy you forever

Would it help if I extract them into proper Trac tickes and clean the code?



Maybe update docs and stress that todo tests should not be committed
... or never mention about such thing as TODO markers,
I just write tests and expect them to fail without marking them as todo.

http://twistedmatrix.com/documents/current/core/development/policy/test-standard.html

(not sure why some docs are on wiki and others in the repo)

And maybe also add a twistedchecker check and check that no new tests
are commited with todo.


-- 
Adi Roiban

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious failure on winxp32-py2.7

[Glyph]

> On Jan 25, 2015, at 7:28 AM, exar...@twistedmatrix.com wrote:

> This test has intermittently failed on Windows since disttrial was merged 
> into trunk.

And, as you yourself pointed out on the ticket, sometimes on Linux too.

I still have absolutely no clue why this happens, by the way.  If anyone else 
could be prompted to investigate I would be much obliged

> The default state of a test should *not* be to skip.  It should be to run.  A 
> skipped test is a useless test.

As a matter of general principle, the only reason a test should ever be skipped 
within Twisted is if an optional dependency is not available.  And, just to be 
clear, the presence of "TODO'd" tests is pretty much just a bug, fix old ones 
and never add new ones :-).

-glyph

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious failure on winxp32-py2.7

[exarkun]

On 02:19 pm, a...@roiban.ro wrote:

While running tests I see that the following tests fails randomly...
it it was failing  from some time


twisted.words.test.test_xpath.XPathTests.test_position


This is an unrelated test.  I'm not sure if this is a simple copy/paste 
mistake or if you're mis-reading trial output.  This is the test before 
the test that failed.


The test that failed is named at the bottom of the failure report:

twisted.trial._dist.test.test_disttrial.DistTrialRunnerTestCase.test_runUntilFailure

This test has intermittently failed on Windows since disttrial was 
merged into trunk.


The ticket tracking this problem is 
https://twistedmatrix.com/trac/ticket/6027

===
[FAIL]
Traceback (most recent call last):
 File 
"C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\trial\_dist\test\test_disttrial.py",

line 371, in test_runUntilFailure
   self.assertEqual(5, len(called))
 File 
"C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\trial\_synctest.py",

line 447, in assertEqual
   % (msg, pformat(first), pformat(second)))
twisted.trial.unittest.FailTest: not equal:
a = 5
b = 2

--

On my local computer (Ubuntu 14.04) the test is always skipped.


The skip report should tell you why it's skipped.  Presumably you're 
missing one of the dependencies - though off the top of my head I'm not 
sure what third-party dependencies this code has.

Does anyone has any idea why the test is not skipped ?


The default state of a test should *not* be to skip.  It should be to 
run.  A skipped test is a useless test.


Jean-Paul

I would like to have stable results from this tests as with the
current state I always have to double check if a failure on XP is a
new one or is just this bad tests.

Thanks!
--
Adi Roiban


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Spurious failure on winxp32-py2.7

[Adi Roiban]

While running tests I see that the following tests fails randomly...
it it was failing  from some time


twisted.words.test.test_xpath.XPathTests.test_position
===
[FAIL]
Traceback (most recent call last):
  File 
"C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\trial\_dist\test\test_disttrial.py",
line 371, in test_runUntilFailure
self.assertEqual(5, len(called))
  File "C:\buildslave\twisted\winxp32-py2.7\Twisted\twisted\trial\_synctest.py",
line 447, in assertEqual
% (msg, pformat(first), pformat(second)))
twisted.trial.unittest.FailTest: not equal:
a = 5
b = 2

--

On my local computer (Ubuntu 14.04) the test is always skipped.

Does anyone has any idea why the test is not skipped ?

I would like to have stable results from this tests as with the
current state I always have to double check if a failure on XP is a
new one or is just this bad tests.

Thanks!
-- 
Adi Roiban

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python