[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Roundup Robot added the comment: New changeset 20fd13242a45 by Antoine Pitrou in branch 'default': Simplify code in multiprocessing.Connection.send_bytes(). http://hg.python.org/cpython/rev/20fd13242a45 -- ___ Python tracker

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Serhiy Storchaka added the comment: What would you say about multi_nagle_simplify.patch, Antoine? -- ___ Python tracker ___ ___ Python

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Changes by Antoine Pitrou : -- resolution: rejected -> fixed ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: ht

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Serhiy Storchaka added the comment: _send_bytes() now looks a little complicated. There are no need in separate branches for n==0. header + buf where buf is b'' is fast (it is not slower than additional check n > 0). So this microoptimization is not needed. The chunks list is not needed, we c

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Antoine Pitrou added the comment: I've committed the fix. If there's less low-hanging fruit to pick, we can do that in another issue :) -- resolution: -> rejected stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Roundup Robot added the comment: New changeset 4816ab0477d2 by Antoine Pitrou in branch '3.3': Issue #20540: Fix a performance regression (vs. Python 3.2) when layering a multiprocessing Connection over a TCP socket. http://hg.python.org/cpython/rev/4816ab0477d2 New changeset 125c24f47f3c by An

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Antoine Pitrou added the comment: > Note that we're still behind the pre-rewrite performance (this is on > 2.7): If I believe the `time` command, the difference is additional userspace CPU time, therefore probably the pure Python overhead. -- ___ Py

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Richard Oudkerk added the comment: BTW, I see little difference between 3.2 and the unpatched default branch on MacOSX: $ py-32/release/python.exe ~/Downloads/test_manager.py 0.0007331371307373047 8.20159912109375e-05 9.417533874511719e-05 8.082389831542969e-05 7.796287536621094e-05 0.00011587

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Richard Oudkerk added the comment: LGTM -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Charles-François Natali added the comment: > Updated patch with the 3.2 heuristic for deciding when to concatenate. LGTM. For posterity, a quick benchmark performing 100 trivial remote calls: Before: $ ./python ~/test_manager.py 8.202659845352173 After: $ ./python ~/test_manager.py 0.131465673

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Antoine Pitrou added the comment: Updated patch with the 3.2 heuristic for deciding when to concatenate. -- keywords: +3.3regression -patch priority: normal -> high stage: -> patch review Added file: http://bugs.python.org/file33967/multi_nagle_2.patch _

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Irvin Probst added the comment: FWIW, according to your comments I tried a quick and dirty fix in my code as I can't wait for a new Python release to make it work: The do_stuff function now does: """ def do_stuff(): client=make_client('',, b"foo") data_proxy=client.get_proxy()

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Antoine Pitrou added the comment: > Modern operation systems have new functions like writev() to send to > consecutive buffers. Can't we use them, and emulate them if they are > missing (use 3.2 algorithm: join if small, or send separatly)? What would the complication bring? -- ___

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

STINNER Victor added the comment: Modern operation systems have new functions like writev() to send to consecutive buffers. Can't we use them, and emulate them if they are missing (use 3.2 algorithm: join if small, or send separatly)? -- ___ Python

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Antoine Pitrou added the comment: I would favour reusing the 3.2 heuristic. Joining the header and the payload may be expensive if the payload is very large. -- ___ Python tracker _

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

Charles-François Natali added the comment: Thanks for the detailed report. > connections.py from multiprocessing has been rewrittend between 3.2 > and 3.3 but I can't see what's wrong in the way it has been done, > basic socket options seem to be exactly the same. Indeed, multiprocessing.connec

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

New submission from Irvin Probst: After checking with the kind people of h...@python.org I think I found a bug in the way connections are handled in the multiprocessing module. Using python 3.3 I've noticed a performance drop of about 70 times when running some code performing basic requests o