[issue20114] Sporadic failure of test_semaphore_tracker() of test_multiprocessing_forkserver on FreeBSD 9 buildbot

2014-01-03 Thread Richard Oudkerk
Richard Oudkerk added the comment: How often has this happened? If the machine was very loaded then maybe the timeout was not enough time for the semaphore to be cleaned up by the tracker process. But I would expect 1 second to be more than ample

[issue20114] Sporadic failure of test_semaphore_tracker() of test_multiprocessing_forkserver on FreeBSD 9 buildbot

2014-01-04 Thread Richard Oudkerk
Richard Oudkerk added the comment: It is probably harmless then. I don't think increasing the timeout is necessary -- the multiprocessing tests already take a long time. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20114

[issue20153] New-in-3.4 weakref finalizer doc section is already out of date.

2014-01-08 Thread Richard Oudkerk
Richard Oudkerk added the comment: The following from the docs is wrong: ... module globals are no longer forced to None during interpreter shutdown. Actually, in 3.4 module globals *sometimes* get forced to None during interpreter shutdown, so the version the __del__ method can still

[issue20414] Python 3.4 has two Overlapped types

2014-01-28 Thread Richard Oudkerk
Richard Oudkerk added the comment: _overlapped is linked against the socket library whereas _winapi is not so it can be bundled in with python3.dll. I did intend to switch multiprocessing over to using _overlapped but I did not get round to it. Since this is a private module the names

[issue20527] multiprocessing.Queue deadlocks after “reader” process death

2014-02-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: This is expected. Killing processes which use shared locks is never going to end well. Even without the lock deadlock, the data in the pipe would be liable to be corrupted if a processes is killed while putting or getting from a queue. If you want

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

2014-02-08 Thread Richard Oudkerk
Richard Oudkerk added the comment: LGTM -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20540 ___ ___ Python-bugs-list mailing list Unsubscribe

[issue20540] Python 3.3/3.4 regression in multiprocessing manager ?

2014-02-08 Thread Richard Oudkerk
Richard Oudkerk added the comment: BTW, I see little difference between 3.2 and the unpatched default branch on MacOSX: $ py-32/release/python.exe ~/Downloads/test_manager.py 0.0007331371307373047 8.20159912109375e-05 9.417533874511719e-05 8.082389831542969e-05 7.796287536621094e-05

[issue20660] Starting a second multiprocessing.Manager causes INCREF on all object created by the first one.

2014-02-18 Thread Richard Oudkerk
Richard Oudkerk added the comment: On Unix, using the fork start method (which was the only option till 3.4), every sub process will incref every shared object for which its parent has a reference. This is deliberate because there is not really any way to know which shared objects

[issue20660] Starting a second multiprocessing.Manager causes INCREF on all object created by the first one.

2014-02-19 Thread Richard Oudkerk
Richard Oudkerk added the comment: Thanks Richard. The set_start_method() call will affect any process started from that time on? Is it possible to change idea at some point in the future? You can use different start methods in the same program by creating different contexts: spawn_ctx

[issue7503] multiprocessing AuthenticationError digest sent was rejected when pickling proxy

2014-02-28 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- assignee: - sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7503 ___ ___ Python-bugs-list

[issue20633] SystemError: Parent module 'multiprocessing' not loaded, cannot perform relative import

2014-03-01 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- assignee: - sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20633 ___ ___ Python-bugs-list

[issue20854] multiprocessing.managers.Server: problem with returning proxy of registered object

2014-03-08 Thread Richard Oudkerk
Richard Oudkerk added the comment: I am not sure method_to_typeid and create_method were really intended to be public -- they are only used by Pool proxies. You can maybe work around the problem by registering a second typeid without specifying callable. That can be used in method_to_typeid

[issue20990] pyflakes: undefined names, get_context() and main(), in multiprocessing

2014-03-20 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- assignee: - sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20990 ___ ___ Python-bugs-list

[issue20980] In multiprocessing.pool, ExceptionWithTraceback should derive from Exception

2014-03-20 Thread Richard Oudkerk
Richard Oudkerk added the comment: We should only wrap the exception with ExceptionWithTraceback in the process case where it will be pickled and then unpickled. -- assignee: - sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue7503] multiprocessing AuthenticationError digest sent was rejected when pickling proxy

2014-03-23 Thread Richard Oudkerk
Richard Oudkerk added the comment: For reasons we all know unpickling unauthenticated data received over TCP is very risky. Sending an unencrypted authentication key (as part of a pickle) over TCP would make the authentication useless. When a proxy is pickled the authkey is deliberately

[issue20633] SystemError: Parent module 'multiprocessing' not loaded, cannot perform relative import

2014-03-23 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20633

[issue20980] In multiprocessing.pool, ExceptionWithTraceback should derive from Exception

2014-03-23 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- resolution: - fixed stage: test needed - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20980

[issue20990] pyflakes: undefined names, get_context() and main(), in multiprocessing

2014-03-25 Thread Richard Oudkerk
Richard Oudkerk added the comment: Testing the is_forking() requires cx_freeze or something similar, so it really cannot go in the test suite. I have tested it manually (after spending too long trying to get cx_freeze to work with a source build). It should be noted that on Unix freezing

[issue21078] multiprocessing.managers.BaseManager.__init__'s serializer argument is not documented

2014-03-29 Thread Richard Oudkerk
Richard Oudkerk added the comment: No, the argument will not go away now. However, I don't much like the API which is perhaps why I did not get round to documenting it. It does have tests. Currently 'xmlrpclib' is the only supported alternative, but JSON support could be added quite easily

[issue1191964] asynchronous Subprocess

2014-03-30 Thread Richard Oudkerk
Richard Oudkerk added the comment: Using asyncio and the IOCP eventloop it is not necessary to use threads. (Windows may use worker threads for overlapped IO, but that is hidden from Python.) See https://code.google.com/p/tulip/source/browse/examples/child_process.py for vaguely expect

[issue21116] Failure to create multiprocessing shared arrays larger than 50% of memory size under linux

2014-04-02 Thread Richard Oudkerk
Richard Oudkerk added the comment: Using truncate() to zero extend is not really portable: it is only guaranteed on XSI-compliant POSIX systems. Also, the FreeBSD man page for mmap() has the following warning: WARNING! Extending a file with ftruncate(2), thus creating a big hole

[issue1191964] asynchronous Subprocess

2014-04-03 Thread Richard Oudkerk
Richard Oudkerk added the comment: I would recommended using _overlapped instead of _winapi. I intend to move multiprocessing over in future. Also note that you can do nonblocking reads by starting an overlapped read then cancelling it immediately if it fails with incomplete. You will need

[issue21162] code in multiprocessing.pool freeze if inside some code from scikit-learn (and probably liblinear) executed on ubuntu 12.04 64 Bit

2014-04-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: I would guess that the problem is simply that LogisticRegression objects are not picklable. Does the problem still occur if you do not use freeze? -- ___ Python tracker rep...@bugs.python.org http

[issue21162] code in multiprocessing.pool freeze if inside some code from scikit-learn (and probably liblinear) executed on ubuntu 12.04 64 Bit

2014-04-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: Ah, I misunderstood: you meant that it freezes/hangs, not that you used a freeze tool. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21162

[issue21162] code in multiprocessing.pool freeze if inside some code from scikit-learn (and probably liblinear) executed on ubuntu 12.04 64 Bit

2014-04-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: Could you try pickling and unpickling the result of func(): import cPickle data = cPickle.dumps(func([1,2,3]), -1) print cPickle.loads(data) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue1191964] asynchronous Subprocess

2014-04-09 Thread Richard Oudkerk
Richard Oudkerk added the comment: Can you explain why you write in 512 byte chunks. Writing in one chunk should not cause a deadlock. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1191964

[issue1191964] asynchronous Subprocess

2014-04-12 Thread Richard Oudkerk
Richard Oudkerk added the comment: I added some comments. Your problem with lost data may be caused by the fact you call ov.cancel() and expect ov.pending to tell you whether the write has/will succeed. Instead you should use ov.getresult() and expect either success or an aborted error

[issue1191964] asynchronous Subprocess

2014-04-17 Thread Richard Oudkerk
Richard Oudkerk added the comment: If you use the short timeouts to make the wait interruptible then you can use waitformultipleobjects (which automatically waits on an extra event object) instead of waitforsingleobject. -- ___ Python tracker rep

[issue20147] multiprocessing.Queue.get() raises queue.Empty exception if even if an item is available

2014-06-08 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- assignee: - sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20147 ___ ___ Python-bugs-list

[issue21372] multiprocessing.util.register_after_fork inconsistency

2014-06-08 Thread Richard Oudkerk
Richard Oudkerk added the comment: register_after_fork() is intentionally undocumented and for internal use. It is only run when starting a new process using the fork start method whether on Windows or not -- the fork in its name is a hint. -- resolution: - not a bug stage

[issue10850] inconsistent behavior concerning multiprocessing.manager.BaseManager._Server

2014-06-28 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- assignee: - sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10850 ___ ___ Python-bugs-list

[issue9248] multiprocessing.pool: Proposal: waitforslot

2014-06-28 Thread Richard Oudkerk
Richard Oudkerk added the comment: Since there are no new features added to Python 2, this would be a Python 3 only feature. I think for Python 3 it is better to concentrate on developing concurrent.futures rather than multiprocessing.Pool

[issue21779] test_multiprocessing_spawn fails when ran with -Werror

2014-06-28 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- assignee: - sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21779 ___ ___ Python-bugs-list

[issue21664] multiprocessing leaks temporary directories pymp-xxx

2014-06-28 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- assignee: - sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21664 ___ ___ Python-bugs-list

[issue14953] Reimplement subset of multiprocessing.sharedctypes using memoryview

2014-07-08 Thread Richard Oudkerk
Richard Oudkerk added the comment: Updated version of the patch. Still needs docs. -- Added file: http://bugs.python.org/file35902/memoryview-array-value.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14953

[issue18174] Make regrtest with --huntrleaks check for fd leaks

2014-07-28 Thread Richard Oudkerk
Richard Oudkerk added the comment: I can't remember why I did not use fstat() -- probably it did not occur to me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18174

<    3   4   5   6   7   8