[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Richard Oudkerk]

Changes by Richard Oudkerk :


--
resolution:  -> fixed
stage: needs patch -> committed/rejected
status: open -> closed
type: crash -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 4e85e4743757 by Richard Oudkerk in branch '2.7':
Issue #6056: Make multiprocessing use setblocking(True) on the sockets it uses.
http://hg.python.org/cpython/rev/4e85e4743757

New changeset 290f04722be3 by Richard Oudkerk in branch '3.2':
Issue #6056: Make multiprocessing use setblocking(True) on the sockets it uses.
http://hg.python.org/cpython/rev/290f04722be3

New changeset f03839401420 by Richard Oudkerk in branch 'default':
Issue #6056: Make multiprocessing use setblocking(True) on the sockets it uses.
http://hg.python.org/cpython/rev/f03839401420

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Richard Oudkerk]

Changes by Richard Oudkerk :


--
nosy: +sbt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Derek Wilson]

Derek Wilson  added the comment:

Thanks, Jim, here is an updated patch. 

1) I feel like it is clearly not-a-feature. Currently 2.7 crashes if remote 
managers are used and socket.setdefaulttimeout is anything other than None. 
Crashing seems bad and all this does is keep multiprocessing connection sockets 
non-blocking even if a default timeout is specified (so it maintains current 
behavior rather than crashing).

2) This problem is still evident on 2.7, 3.2 and 3.3 beta 1. This patch is 
against the current dev tip as of a few days ago.

3) here it is!

4) I agree that setblocking is more clear. I made the change.

My test modifications cover the entire suite twice, once without a default 
timeout and once with. This may be excessive? I'm not sure where non-blocking 
sockets might pop up as an issue since there is C code that relies on blocking 
sockets and I haven't dug that deep.

--
Added file: http://bugs.python.org/file26497/mp6056fix.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Jim Jewett]

Jim Jewett  added the comment:

The wording in 138415 suggested this patch was changing socket to not support 
timeouts -- which would be unacceptable.  

But the actual patch only seems to touch multiprocessing/connection.py -- a far 
more reasonable change.

Unfortunately, the patch no longer applies to the development tip.  I *think* 
the places you wanted to change are still there, and just moved.

(1)  Is it sufficiently clear that this is not-a-feature to justify a backport?

(2)  Are the problems already fixed by some of the other changes?  (It doesn't 
look like it, but I'm not sure.)

(3)  Can you produce an updated patch?  (The current tip is 
http://hg.python.org/cpython/file/fec45282dc28/Lib/multiprocessing/connection.py
  )

(4)  If I understand the intent, then s.setblocking(True) would be slightly 
more clear than s.settimeout(None), though that change obviously isn't 
essential.

--
nosy: +Jim.Jewett

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Derek Wilson]

Derek Wilson  added the comment:

Any chance this patch will be accepted (or at least reviewed) soon?

--
type: behavior -> crash
versions: +Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Derek Wilson]

Derek Wilson  added the comment:

I was wrong about exit behavior of a process that has put to a queue -- it 
seems to behave as expected. i had been playing with a proxy to a queue rather 
than a queue (to which, if you put, the process can exit right away because the 
actual put happens in the process that owns the queue).

I think this works as intended, but lmk. Also, I haven't really played with the 
tests that much, so that bit could use some review. It hasn't broken anything 
in any of my real world tests though.

Also, have I mentioned that the multiprocessing module is amazing? Cause it is. 
I sort of feel like antigravity == multiprocessing ...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Derek Wilson]

Derek Wilson  added the comment:

While having multiprocessing use a timeout would be great, I didn't really have 
the time to fiddle with the c code.

Instead of using the socket timeout, I'm modifying all the sockets created by 
the socket module to have no timeout (and thus to be blocking) which makes the 
multiprocessing module 'immune' to the socket module's default timeout.

For testing, I just run the test suite twice, once with the initial default 
socket timeout and once with a 60 second timeout. Nothing there failed with 
this issue.

It is worth noting, however, that when using a default socket timeout, for some 
reason processes that have have put data into a queue no longer block at exit 
waiting for the data to be consumed. I'm not sure if there is some additional 
cleanup code that uses sockets and might need to block? Or maybe whatever the 
issue was with blocking sockets is not an issue with non-blocking sockets?

--
keywords: +patch
Added file: http://bugs.python.org/file22376/mpsock.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Jesse Noller]

Jesse Noller  added the comment:

I agree derek, I think that would be a fine addition, however we lack a patch 
and I don't have the current bandwidth to add it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Derek Wilson]

Derek Wilson  added the comment:

This should be higher priority as one of the major benefits of the 
multiprocessing module is remote process management in a completely transparent 
manner. socket timeouts are very important in this context as blocking forever 
waiting for a connection is not always an option. 

The problem of not being able to use a default socket timeout for other 
purposes in combination with multiprocessing managers is definitely an issue, 
but making multiprocessing actually use the timeout itself if set would be a 
huge advantage.

This might not be the place to ask for it, but it would make sense for manager 
objects to gain a timeout attribute to be used as a timeout for local or remote 
communications. At the very least, the manager.connect() method should accept a 
timeout argument.

--
nosy: +Derek.Wilson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Mark Lawrence]

Changes by Mark Lawrence :


--
stage:  -> needs patch
versions: +Python 2.7, Python 3.1, Python 3.2 -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Ask Solem]

Changes by Ask Solem :


--
nosy: +asksol

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Jesse Noller]

Jesse Noller  added the comment:

Well; I'm pretty tapped out right now - I think your idea of checking to 
see if a timeout has been set elsewhere makes sense. If you have the time 
to put together a patch (with a unit test or three :)) I can review it. 
Might take me a bit of time to get to this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Ryan Leslie]

Ryan Leslie  added the comment:

Yeah, storing pickled queues in the file system makes for some easy IPC
:) It wasn't a very original idea, I took the pickling comments in the
documentation at face value:
http://docs.python.org/library/multiprocessing.html#proxy-objects

So, from what I can tell this issue is related to the mixing of standard
python socket I/O with multiprocessing socket I/O, with state not being
carried from the former to the latter.

In multiprocessing/connection.py, SocketClient() creates a familiar
python socket object which, when a default timeout has been set in the
module, will be made non-blocking. In addition, the timeout is
remembered in the socket object, and when calling socket.recv(), the
function internal_select() will use this to perform the expected poll()
or select().

However, after a connection is established, SocketClient() will not use
python's socket implementation any further, and instead pass its
low-level socket descriptor to a multiprocessing Connection object. This
object has its own specialized socket I/O implementation, which is not
at all aware of the timeout previously associated with the socket. As a
result no select/poll occurs and, due to the socket's non-blocking
status, recv() may return EAGAIN immediately. I suspect this is what's
happening.

There might be a number of ways to make SocketClient() more timeout
friendly, but possibility could be to simply check if the python socket
has a timeout associated, and if so, use connection.poll() in addition
to connection.recv(). There may be other places in the code where
similar changes would occur.

You obviously have more experience with this code base so I'll be
curious to see the outcome.

--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Jesse Noller]

Jesse Noller  added the comment:

Pickling the queue and then unpickling it in a new process is something I 
never thought of. That's interesting in and of itself ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Benjamin Peterson]

Changes by Benjamin Peterson :


--
assignee:  -> jnoller
nosy: +jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6056] socket.setdefaulttimeout affecting multiprocessing Manager

[Ryan Leslie]

New submission from Ryan Leslie :

Terminal 1:
Python 2.6.1 (r261:67515, Apr  2 2009, 18:25:55)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from multiprocessing.managers import SyncManager
>>> manager = SyncManager(authkey="mykey")
>>> manager.start()
>>> queue = manager.Queue()
>>> import pickle
>>> pickle.dump(queue, open("myqueue.pkl", "w"))
>>>

Terminal 2:
Python 2.6.1 (r261:67515, Apr  2 2009, 18:25:55)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.setdefaulttimeout(30)
>>> import multiprocessing, pickle
>>> multiprocessing.current_process().authkey = "mykey"
>>> queue = pickle.load(open("myqueue.pkl"))
Traceback (most recent call last):
  File "", line 1, in 
  File "python2.6/pickle.py", line 1370, in load
return Unpickler(file).load()
  File "python2.6/pickle.py", line 858, in load
dispatch[key](self)
  File "python2.6/pickle.py", line 1133, in load_reduce
value = func(*args)
  File "python2.6/multiprocessing/managers.py", line 845, in RebuildProxy
return func(token, serializer, incref=incref, **kwds)
  File "python2.6/multiprocessing/managers.py", line 894, in AutoProxy
incref=incref)
  File "python2.6/multiprocessing/managers.py", line 700, in __init__
self._incref()
  File "python2.6/multiprocessing/managers.py", line 749, in _incref
conn = self._Client(self._token.address, authkey=self._authkey)
  File "python2.6/multiprocessing/connection.py", line 140, in Client
answer_challenge(c, authkey)
  File "python2.6/multiprocessing/connection.py", line 376, in
answer_challenge
response = connection.recv_bytes(256)# reject large message
IOError: [Errno 11] Resource temporarily unavailable
>>> 

This works as expected without socket.setdefaulttimeout(). However, the
timeout is useful since if the listening process on terminal 1 goes to
sleep, e.g. ^Z, it would avoid blocking.

I suspect the cause is similar to http://bugs.python.org/issue976613

--
components: Library (Lib)
messages: 88040
nosy: ryles
severity: normal
status: open
title: socket.setdefaulttimeout affecting multiprocessing Manager
versions: Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com