A.M. Kuchling added the comment:
neologix: did you intend to re-open this ticket when you made your 2013-03-05
comment? It seems to me that you didn't intend to -- your comment doesn't say
're-opening because reason'. I'll close it again; if you want, please
re-open it and just explain why.
Changes by Charles-François Natali cf.nat...@gmail.com:
--
status: closed - open
superseder: reduce multiprocessing.Queue contention -
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10886
Charles-François Natali added the comment:
For the record, I'm posting thse benchmark numbers here (originally from issue
#17025):
with patch:
$ ./python /tmp/multi_queue.py
took 0.7945001125335693 seconds with 1 workers
took 0.7428359985351562 seconds with 2 workers
took 0.7897098064422607
Charles-François Natali added the comment:
I'm closing, since issue #17025 proposes to do this as part of performance
optimization.
--
nosy: +neologix
status: open - closed
superseder: - reduce multiprocessing.Queue contention
___
Python tracker
sbt shibt...@gmail.com added the comment:
mp_queue_pickle_in_main_thread.patch (against the default branch) fixes the
problem by doing the pickling in Queue.put(). It is version of a patch for
Issue 8037 (although I believe the behaviour complained about in Issue 8037 is
not an actual bug).
Antoine Pitrou pit...@free.fr added the comment:
This shouldn't be a problem in Python 3.3, where the Connection classes are
reimplemented in pure Python.
--
nosy: +pitrou
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10886
sbt shibt...@gmail.com added the comment:
This shouldn't be a problem in Python 3.3, where the Connection classes
are reimplemented in pure Python.
What should not be a problem?
Changes to the implementation of Connection won't affect whether Queue.put()
raises an error immediately if it
Antoine Pitrou pit...@free.fr added the comment:
Changes to the implementation of Connection won't affect whether
Queue.put() raises an error immediately if it gets an unpicklable
argument.
Ah, right. Then indeed it won't make a difference.
--
New submission from Torsten Landschoff t.landsch...@gmx.net:
When trying to send an object via a Queue that can't be pickled, one gets a
quite unhelpful traceback:
Traceback (most recent call last):
File /usr/lib/python2.6/multiprocessing/queues.py, line 242, in _feed
send(obj)
