[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Chris Jerdonek]

Change by Chris Jerdonek :


--
nosy:  -chris.jerdonek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Chris Jerdonek]

Chris Jerdonek  added the comment:

See also: https://bugs.python.org/issue40735 (test_with_statement)

--
nosy: +chris.jerdonek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Berker Peksag  added the comment:

PR 9461 converts Martin's nntp-server.patch. I've made some tweaks and fixed a 
"threading._dangling was modified by test_nntplib" warning. I will try to 
convert multi-connect.patch later.

I will attribute Martin as the patch author before merging the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Change by Berker Peksag :


--
pull_requests: +8875

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Berker Peksag  added the comment:

Thanks for the reviews, Victor. I'm planning to work on them tonight. I will 
close this issue and open a new one if I decide to do further work on them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[STINNER Victor]

STINNER Victor  added the comment:

Thanks Berker! Do you want to do something with Martin's patches? If not, can 
you please close the issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[miss-islington]

miss-islington  added the comment:


New changeset 170ea8ccd4235d28538ab713041502d07ad1cacd by Miss Islington (bot) 
in branch '2.7':
bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446)
https://github.com/python/cpython/commit/170ea8ccd4235d28538ab713041502d07ad1cacd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[miss-islington]

miss-islington  added the comment:


New changeset 1eabe19c57938dd70b66b97239be337294e15cba by Miss Islington (bot) 
in branch '3.6':
bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446)
https://github.com/python/cpython/commit/1eabe19c57938dd70b66b97239be337294e15cba


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[miss-islington]

Change by miss-islington :


--
pull_requests: +8865

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 476177005e8c8d4ece3070c8c378f8b8c068e76f by Berker Peksag (Miss 
Islington (bot)) in branch '3.7':
bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446)
https://github.com/python/cpython/commit/476177005e8c8d4ece3070c8c378f8b8c068e76f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 8213eaddf3ce8e87564d2949454903a1484748b5 by Berker Peksag in 
branch 'master':
bpo-19756: Prevent test failures due to EADDRNOTAVAIL (GH-9446)
https://github.com/python/cpython/commit/8213eaddf3ce8e87564d2949454903a1484748b5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[miss-islington]

Change by miss-islington :


--
pull_requests: +8863

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[miss-islington]

Change by miss-islington :


--
pull_requests: +8862

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Change by Berker Peksag :


--
pull_requests: +8861
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Berker Peksag  added the comment:

Yeah, I will submit a pull request today. The reason I reopened this issue is 
that I didn't want to file another issue for a similar problem and wanted to 
use this as a meta issue for issues with test_nntplib. Also, I'd like to play 
Martin's patches a bit and see if they still needed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[STINNER Victor]

STINNER Victor  added the comment:

> Sure, that sounds like a reasonable change? 

Oh, typo: you should read "Sure, that sounds like a reasonable change." :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[STINNER Victor]

STINNER Victor  added the comment:

> I think it would be nice if tweaked support.transient_internet() to catch 
> EADDRNOTAVAIL.

Sure, that sounds like a reasonable change? Do you want to work on a PR? If 
not, maybe we should open a new issue, since this issue is tidly linked to 
test_nntplib, whereas modifying support.transient_internet() is a more generic 
change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Berker Peksag  added the comment:

I think it would be nice if tweaked support.transient_internet() to catch 
EADDRNOTAVAIL. See 
https://github.com/python/cpython/commit/e4dcbbd7f4ac18d01c0ec85f64ae98b8281ed403#diff-7fa374913c0ecbdc621db8ea79a1212bR78
 for another example.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Change by Berker Peksag :


--
Removed message: https://bugs.python.org/msg325858

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Berker Peksag  added the comment:

==
ERROR: setUpClass (test.test_nntplib.NetworkedNNTPTests)
--
Traceback (most recent call last):
  File "/home/travis/build/python/cpython/Lib/test/test_nntplib.py", line 292, 
in setUpClass
usenetrc=False)
  File "/home/travis/build/python/cpython/Lib/nntplib.py", line 1043, in 
__init__
self.sock = socket.create_connection((host, port), timeout)
  File "/home/travis/build/python/cpython/Lib/socket.py", line 724, in 
create_connection
raise err
  File "/home/travis/build/python/cpython/Lib/socket.py", line 713, in 
create_connection
sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address

https://travis-ci.org/python/cpython/jobs/430974933

--
status:  -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[STINNER Victor]

STINNER Victor  added the comment:

I didn't see test_nntplib failing recently, so I close the issue as outdated.

--
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[STINNER Victor]

STINNER Victor added the comment:

Any progress on this issue?

Martin: Can you please convert your latest patch into a Pull Request?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[STINNER Victor]

STINNER Victor added the comment:

> ERROR: setUpClass (test.test_nntplib.NetworkedNNTP_SSLTests)

I wrote an obvious fix for this one: issue #30188.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Martin Panter]

Martin Panter added the comment:

Multi-connect.patch is a smaller patch that changes setUpClass() → setUp(), so 
that each test method creates a new NNTP connection. The downside is all the 
reconnecting slows the test execution from 42 s down to 115 s, which is why I 
would like to move the testing to a local server.

--
versions: +Python 3.7
Added file: http://bugs.python.org/file46018/multi-connect.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Martin Panter]

Martin Panter added the comment:

Here is a diff that shows my changes better. I temporarily moved some classes 
back to their original positions to reduce the diff size, though it won’t work 
in practice because Python needs the classes defined before using them.

--
Added file: http://bugs.python.org/file43478/original-order.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Changes by Berker Peksag :


--
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Martin Panter]

Martin Panter added the comment:

Here is a patch that changes all affected tests to use a custom server running 
on localhost:

* Local server is based on the existing NNTPv2Handler class, and the 
LocalServerTests class
* Reintegrate test_starttls() back into NetworkedNNTPTestsMixin
* Only test_with_statement() talks to the remote NNTP_HOST now (not affected by 
this bug)
* Make tests in NetworkedNNTPTestsMixin stricter, because the server responses 
are predetermined now
* Add mock comp.lang.python support to NNTPv1Handler.handle_GROUP()
* Add support for rejecting passwords in NNTPv1Handler.handle_AUTHINFO()
* New message number with non-UTF-8 message body for ARTICLE, HEAD, BODY, and 
non-UTF-8 subject for OVER, XOVER

--
keywords: +patch
Added file: http://bugs.python.org/file43464/nntp-server.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Mark Lawrence]

Changes by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Martin Panter]

Martin Panter added the comment:

Thanks Berker. In this case the previous tests using the same NNTP connection 
object were skipped. First run:

test_unknown_command (test.test_nntplib.NetworkedNNTPTests) ... ok
test_welcome (test.test_nntplib.NetworkedNNTPTests) ... ok
test_with_statement (test.test_nntplib.NetworkedNNTPTests) ... skipped 
"Resource 'news.trigofacile.com' is not available"
test_xhdr (test.test_nntplib.NetworkedNNTPTests) ... skipped "Resource 
'news.trigofacile.com' is not available"
test_xover (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_zlogin (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_zzquit (test.test_nntplib.NetworkedNNTPTests) ... ERROR

Subsequent retry:

test_xover (test.test_nntplib.NNTPv2Tests) ... ok
test_article_head_body (test.test_nntplib.NetworkedNNTPTests) ... skipped 
"Resource 'news.trigofacile.com' is not available"
test_capabilities (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_date (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_description (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_descriptions (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_group (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_help (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_list (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_list_active (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_newgroups (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_over (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_unknown_command (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_welcome (test.test_nntplib.NetworkedNNTPTests) ... ok
test_with_statement (test.test_nntplib.NetworkedNNTPTests) ... ok
test_xhdr (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_xover (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_zlogin (test.test_nntplib.NetworkedNNTPTests) ... ERROR
test_zzquit (test.test_nntplib.NetworkedNNTPTests) ... ERROR

Test_with_statement() is a special case because it starts a fresh NNTP 
connection, rather than reusing self.server. Test_welcome() is also different 
because it does not send any new commands to the server. But I presume the 
other tests all try to reuse the old timed-out self.server object. The skipped 
messages are evidence of this.

The easy solution which I mentioned above would be to change setUpClass() to 
setUp(), although this might slow the tests down a bit, reconnecting to the 
remote server for each test method.

The solution that I prefer would be as Antoine suggested, to run our own 
server. I think expanding the server I created in Issue 25859 with more 
commands would be good enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Berker Peksag]

Berker Peksag added the comment:

I just saw the same test failures at 
http://buildbot.python.org/all/builders/AMD64%20Debian%20root%203.5/builds/998/steps/test/logs/stdio

I'm attaching the test logs for future reference.

--
nosy: +berker.peksag
stage:  -> needs patch
type:  -> behavior
versions: +Python 3.5, Python 3.6 -Python 3.3, Python 3.4
Added file: http://bugs.python.org/file43441/test_nntplib_logs.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Martin Panter]

Martin Panter added the comment:

See Issue 1186900 about the NNTP client raising EOFError.

As for the “cannot read from timed out object” errors (also reported in Issue 
19613), I think the test classes are bad. The test logs don’t seem to be 
available, but I reckon that the previous test to the failing test would have 
been skipped due to the transient Internet handler. The test classes are 
essentially coded like this:

class NetworkedNNTP_SSLTests(TestCase):
@classmethod
def setUpClass(cls):
with support.transient_internet(cls.NNTP_HOST):
cls.server = NNTP_SSL(cls.NNTP_HOST, timeout=TIMEOUT)

def test_that_triggered_the_failure(self):
with support.transient_internet(self.NNTP_HOST):
...  # Suppose this timed out and the test was skipped

def test_capabilities(self):
# Now this is reusing the existing connection, left in a broken state by
# the previous test
self.server.capabilities()

Either the server connection should be freshly made for each individual test 
case (not once per class), or all the test cases grouped by setUpClass() should 
be skipped if the transient Internet handler kicks in. I’m not sure if it is 
practical to do the second option with the standard unittest module. Perhaps 
the method wrapping stage could add some extra checks to see if the test should 
be skipped.

--
nosy: +vadmium

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19756
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Mark Lawrence]

Mark Lawrence added the comment:

Are these sporadic failures still happening?  I've glanced over the logs and 
only found failures due to recent code changes.

--
nosy: +BreamoreBoy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19756
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[STINNER Victor]

New submission from STINNER Victor:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.x/builds/692/steps/test/logs/stdio

==
ERROR: test_capabilities (test.test_nntplib.NetworkedNNTP_SSLTests)
--
Traceback (most recent call last):
  File 
/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/test/test_nntplib.py, 
line 251, in wrapped
meth(self)
  File 
/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/test/test_nntplib.py, 
line 201, in test_capabilities
resp, caps = self.server.capabilities()
  File /usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/nntplib.py, 
line 558, in capabilities
resp, lines = self._longcmdstring(CAPABILITIES)
  File /usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/nntplib.py, 
line 525, in _longcmdstring
resp, list = self._getlongresp(file)
  File /usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/nntplib.py, 
line 476, in _getlongresp
resp = self._getresp()
  File /usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/nntplib.py, 
line 449, in _getresp
resp = self._getline()
  File /usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/nntplib.py, 
line 432, in _getline
line = self.file.readline(_MAXLINE +1)
  File /usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/socket.py, line 
368, in readinto
raise OSError(cannot read from timed out object)
OSError: cannot read from timed out object



http://buildbot.python.org/all/builders/x86%20OpenIndiana%203.x/builds/7147/steps/test/logs/stdio


==
ERROR: setUpClass (test.test_nntplib.NetworkedNNTP_SSLTests)
--
Traceback (most recent call last):
  File 
/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/test_nntplib.py,
 line 298, in setUpClass
cls.server = cls.NNTP_CLASS(cls.NNTP_HOST, timeout=TIMEOUT, usenetrc=False)
  File /export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/nntplib.py, 
line 1071, in __init__
readermode=readermode, timeout=timeout)
  File /export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/nntplib.py, 
line 334, in __init__
self.getcapabilities()
  File /export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/nntplib.py, 
line 390, in getcapabilities
resp, caps = self.capabilities()
  File /export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/nntplib.py, 
line 558, in capabilities
resp, lines = self._longcmdstring(CAPABILITIES)
  File /export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/nntplib.py, 
line 525, in _longcmdstring
resp, list = self._getlongresp(file)
  File /export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/nntplib.py, 
line 476, in _getlongresp
resp = self._getresp()
  File /export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/nntplib.py, 
line 449, in _getresp
resp = self._getline()
  File /export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/nntplib.py, 
line 437, in _getline
if not line: raise EOFError
EOFError

--
components: Tests
keywords: buildbot
messages: 204248
nosy: haypo
priority: normal
severity: normal
status: open
title: test_nntplib: sporadic failures, network isses? server down?
versions: Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19756
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19756] test_nntplib: sporadic failures, network isses? server down?

[Antoine Pitrou]

Antoine Pitrou added the comment:

The third-party server isn't extremely reliable indeed. Ideally, we would run 
our own NNTP server, but that's a lot more work.

--
nosy: +pitrou
versions: +Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19756
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com