[issue40440] allow array.array construction from memoryview w/o copy

Davin Potts added the comment: Being able to create an array.array without making a copy of a memoryview's contents does sound valuable. We do not always want to modify the size of the array, as evidenced by array.array's existing functionality where its size-changing manipulations (like

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Davin Potts added the comment: My sense is that it would be nice if we can catch this before ftruncate does something nasty. Where else is ftruncate used in CPython that this could similarly trigger a problem? How is it handled

[issue33082] multiprocessing docs bury very important 'callback=' args

Davin Potts added the comment: I appreciate the functionality offered by the callbacks and have found good uses for them, as Chad clearly does/has. That said, the thought of expanding the documentation on the callbacks had not come up for me. Reading through the proposed changes

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

Davin Potts added the comment: I believe the mentality behind multiprocessing.Process triggering an exit code of 1 when sys.exit() is invoked inside its process is to indicate a non-standard exit out of its execution. There may yet be other side effects that could be triggered by having

[issue22393] multiprocessing.Pool shouldn't hang forever if a worker process dies unexpectedly

Change by Davin Potts : -- pull_requests: +15722 pull_request: https://github.com/python/cpython/pull/16103 ___ Python tracker <https://bugs.python.org/issue22

[issue37652] Multiprocessing shared_memory ValueError on race with ShareableList

Davin Potts added the comment: Apologies, one of the quotes in my previous response should have been attributed to @mental. I think @pierreglaser phrased it very nicely: > shared_memory is a low level python module. Precautions should be made when > handling concurrently the shared_

[issue37652] Multiprocessing shared_memory ValueError on race with ShareableList

Davin Potts added the comment: Short responses to questions/comments from @bjs, followed by hopefully helpful further comments: > Are you supposed to ever use a raw SharedMemory buffer directly? Yes. > What atomicity guarantees are there for ShareableList operations and > r

[issue37637] multiprocessing numpy.ndarray not transmitted properly

Davin Potts added the comment: Marking as closed after providing an example of how to send NumPy arrays as bytes with the send_bytes() function. -- resolution: -> not a bug stage: -> resolved status: -> closed ___ Python tracke

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

Change by Davin Potts : -- keywords: +patch pull_requests: +15618 stage: -> patch review pull_request: https://github.com/python/cpython/pull/15989 ___ Python tracker <https://bugs.python.org/issu

[issue37754] Consistency of Unix's shared_memory implementation with windows

Davin Potts added the comment: I have created issue38119 to track a fix to the inappropriate use of resource tracker with shared memory segments, but this does not replace or supersede what is discussed here. -- ___ Python tracker <ht

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

New submission from Davin Potts : The resource tracker currently destroys (via _posixshmem.shm_unlink) shared memory segments on posix systems when any independently created Python process with a handle on a shared memory segment exits (gracefully or otherwise). This breaks the expected

[issue35267] reproducible deadlock with multiprocessing.Pool

Davin Potts added the comment: I second what @vstinner already said in the comments for PR11143, that this should not merely be documented. -- nosy: +davin ___ Python tracker <https://bugs.python.org/issue35

[issue38084] multiprocessing cannot recover from crashed worker

Davin Potts added the comment: Agreed with @ppperry that this is a duplicate of issue22393. The proposed patch in issue22393 is, for the moment, out of sync with more recent changes. That patch's approach would result in the loss of all partial results from a Pool.map, but it may be faster

[issue38084] multiprocessing cannot recover from crashed worker

Davin Potts added the comment: Thanks to Pablo's good work with implementing the use of multiprocessing's Process.sentinel, the logic for handling PoolWorkers that die has been centralized into Pool._maintain_pool(). If _maintain_pool() can also identify which job died with the dead

[issue38084] multiprocessing cannot recover from crashed worker

Davin Potts added the comment: Sharing for the sake of documenting a few things going on in this particular example: * When a PoolWorker process exits in this way (os._exit(anything)), the PoolWorker never gets the chance to send a signal of failure (normally sent via the outqueue

[issue38018] Increase Code Coverage for multiprocessing.shared_memory

Davin Potts added the comment: Initial review of the test failure suggests a likely flaw in the mechanism used by the resource tracker. I will continue investigating more tomorrow. -- ___ Python tracker <https://bugs.python.org/issue38

[issue38018] Increase Code Coverage for multiprocessing.shared_memory

Davin Potts added the comment: New changeset d14e39c8d9a9b525c7dcd83b2a260e2707fa85c1 by Davin Potts (Vinay Sharma) in branch 'master': bpo-38018: Increase code coverage for multiprocessing.shared_memory (GH-15662) https://github.com/python/cpython/commit

[issue37185] use os.memfd_create in multiprocessing.shared_memory?

Davin Potts added the comment: Unless I am missing something, memfd_create appears to be specific to the Linux kernel still so we would need to replicate its behavior on all of the other unix systems. To your point, but quoting from the docs, "separate invocations of memfd_c

[issue37754] Consistency of Unix's shared_memory implementation with windows

Davin Potts added the comment: A shared semaphore approach for the resource tracker sounds appealing as a way to make the behavior on Windows and posix systems more consistent. However this might get implemented, we should not artificially prevent users from having some option to persist

[issue37754] alter size of segment using multiprocessing.shared_memory

Davin Potts added the comment: Attempts to alter the size of a shared memory segment are met with a variety of different, nuanced behaviors on systems we want to support. I agree that it would be valuable to be able to effectively realloc a shared memory segment, which thankfully the user

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: Victor raises an important question: should the *default* start behavior be made consistent across platforms? Assuming we change it on MacOS, the default start behavior on Windows and MacOS will be spawn but the default start behavior on Linux and FreeBSD

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: I believe we must change the default behavior on MacOS to use spawn instead of fork. Encouraging people to use fork by default on MacOS is encouraging them to create something that effectively will not work. Keeping fork as the default behavior when we have

[issue36364] errors in multiprocessing.shared_memory examples

Davin Potts added the comment: Very much agreed, they're moving over to the main docs. -- ___ Python tracker <https://bugs.python.org/issue36364> ___ ___ Pytho

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: As best as I can see, there is no magic bullet to help mitigate this. At a minimum, I am convinced we need to update the documentation to describe this behavior on MacOS and recommend alternatives. I continue to give serious thought to the idea of changing

[issue36102] TestSharedMemory fails on AMD64 FreeBSD CURRENT Shared 3.x

Davin Potts added the comment: Closing. Thank you Giampaolo for jumping in so quickly to review! Thank you Victor for catching this on the buildbot. Though what is this talk of "_if_ the color changes"? ;) -- resolution: -> fixed stage: patch review -> resol

[issue36102] TestSharedMemory fails on AMD64 FreeBSD CURRENT Shared 3.x

Davin Potts added the comment: New changeset aadef2b41600cb6a4f845cdc4cea001c916d8745 by Davin Potts in branch 'master': bpo-36102: Prepend slash to all POSIX shared memory block names (#12036) https://github.com/python/cpython/commit/aadef2b41600cb6a4f845cdc4cea001c916d8745

[issue36102] TestSharedMemory fails on AMD64 FreeBSD CURRENT Shared 3.x

Change by Davin Potts : -- stage: -> patch review ___ Python tracker <https://bugs.python.org/issue36102> ___ ___ Python-bugs-list mailing list Unsubscrib

[issue36102] TestSharedMemory fails on AMD64 FreeBSD CURRENT Shared 3.x

Davin Potts added the comment: I have locally tested GH-12036 on all 5 of the aforementioned OSes and all are made happy by the patch. Victor: If we want to go ahead and apply this patch right away to hopefully make the FreeBSD buildbot go green, the nature of this change is sufficiently

[issue36102] TestSharedMemory fails on AMD64 FreeBSD CURRENT Shared 3.x

Change by Davin Potts : -- keywords: +patch pull_requests: +12065 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue36102> ___ ___ Py

[issue36102] TestSharedMemory fails on AMD64 FreeBSD CURRENT Shared 3.x

Davin Potts added the comment: In local testing, I found the following systems to impose the leading slash as a requirement for simply creating a shared memory block: * NetBSD 8.0 * FreeBSD 12.x * TrueOS 18.12 (the OS formerly known as PC-BSD) I found the following systems to have

[issue36102] TestSharedMemory fails on AMD64 FreeBSD CURRENT Shared 3.x

Davin Potts added the comment: Though apparently undocumented on FreeBSD, their implementation of shm_open differs from others in the following way: all names for shared memory blocks *must* begin with a slash. This requirement does not exist on OpenBSD. According to its man page

[issue36099] Clarify the difference between mu and xbar in the statistics documentation

Davin Potts added the comment: Without necessarily defining what each means, perhaps it is sufficient to change this clause in the docs: it should be the mean of data For pvariance() it could read as: it should be the *population* mean of data And for variance() it could read

[issue36018] Add a Normal Distribution class to the statistics module

Davin Potts added the comment: Steven: Your point about population versus sample makes sense and your point that altering their names would be a breaking change is especially important. I think that pretty well puts an end to my suggestion of alternative names and says the current pattern

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @Giampaolo: The docstring in the shared_memory module currently marks the API as experimental. (You read my mind...) I will start a new PR where we can work on the better-integration-into-the-larger-multiprocessing-docs and add comments

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: New changeset e895de3e7f3cc2f7213b87621cfe9812ea4343f0 by Davin Potts in branch 'master': bpo-35813: Tests and docs for shared_memory (#11816) https://github.com/python/cpython/commit/e895de3e7f3cc2f7213b87621cfe9812ea4343f0

[issue36018] Add a Normal Distribution class to the statistics module

Davin Potts added the comment: There is an inconsistency worth paying attention to in the choice of names of the input parameters. Currently in the statistics module, pvariance() accepts a parameter named "mu" and pstdev() and variance() each accept a parameter named "x

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: > FWIW I bumped into this lib: http://semanchuk.com/philip/sysv_ipc/ The author of that lib, Philip Semanchuk, is one of the people participating in this effort -- he has posted above in msg334934 here on b.p.o. and has helped review the PR in GH-11816.

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: > Code looks much better now. I'm still not convinced > "SharedMemory(name=None, create=False, size=0)" is the best API. > How are you supposed to "create or attach" atomically? We are consciously choosing to not suppor

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: The simpler API is now implemented in GH-11816 as discussed previously. Notably: > * We go with this simpler API: SharedMemory(name=None, create=False, size=0) > * 'size' is ignored when create=False > * create=True acts like O_CREX and create=F

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: > I think we need the "create with exclusive behavior" option, even > though we don't know how to implement it on Windows right now. A fix to avoid the potential race condition on Windows is now part of GH-11816. > To support 1 & 2, we

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @giampaolo: > Also, what happens if you alter the size of an existing object with a smaller > value? Is the memory region overwritten? Attaching to an existing shared memory block with a size=N which is smaller than its allocated size (say it was c

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @giampaolo: > 1) As for SharedMemoryManager, I believe it should live in > multiprocessing.managers, not shared_memory.py. I am happy to see it live in multiprocessing.managers so long as we can provide a clean way of handling what happens on a platform

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @giampaolo: > Maybe something like this instead? > SharedMemory(name=None, attach_if_exists=False, size=0) I think this misses the use case when wanting to ensure we only attach to an existing shared memory block and if it does not exist, we should

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @giampaolo: > 1) it seems SharedMemory.close() does not destroy the memory region > (I'm able to re-attach to it via name). If I'm not mistaken only > the manager can do that. Correct, close() does not and should not destroy the memory region becau

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: Regarding the API of the SharedMemory class, its flags, mode, and read_only parameters are not universally enforced or simply not implemented on all platforms that offer POSIX Shared Memory or Windows Named Shared Memory. A simplified API for the SharedMemory

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: These questions (originally asked in comments on GH-11816) seemed more appropriate to discuss here: Why should the user want to use `SharedMemory` directly? Why not just go through the manager? Also, perhaps a naive question: don't you _always_ need

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @Antoine: SharedMemoryManager does not subclass SyncManager but it did previously. This is the source of the confusion. SharedMemoryManager subclasses BaseManager which does not provide Value, Array, list, dict, etc. Agreed that the manager facility does

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @giampaolo.rodola: It definitely helps. Conceptually, SyncManager provides "distributed shared memory" where lists, dicts, etc. are held in memory by one process but may be accessed remotely from another via a Proxy Object. Mutating a dict from o

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @terry.reedy and @ronaldoussoren: I have asked Van again to provide comments here clarifying the topics of (1) copyright notices and (2) requiring the BSD-licensed-work's author to sign a contributor agreement. Specifically regarding the appearance

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @giampaolo.rodola: Your patch from 3 days ago in issue35917 included additional tests around the SharedMemoryManager which are now causing test failures in my new PR. This is my fault because I altered SharedMemoryManager to no longer support functionality

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: Docs and tests are now available in a new PR. I have stayed focused on getting these docs and tests to everyone without delay but that means I have not yet had an opportunity to respond to the helpful comments, thoughtful questions, and threads that have

[issue35813] shared memory construct to avoid need for serialization between processes

Change by Davin Potts : -- pull_requests: +11834 ___ Python tracker <https://bugs.python.org/issue35813> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35813] shared memory construct to avoid need for serialization between processes

Change by Davin Potts : -- pull_requests: +11834, 11835, 11836 ___ Python tracker <https://bugs.python.org/issue35813> ___ ___ Python-bugs-list mailing list Unsub

[issue35813] shared memory construct to avoid need for serialization between processes

Change by Davin Potts : -- pull_requests: +11834, 11835 ___ Python tracker <https://bugs.python.org/issue35813> ___ ___ Python-bugs-list mailing list Unsub

[issue35903] Build of posixshmem.c should probe for required OS functions

Davin Potts added the comment: Agreed that the logic for building that code needs exactly this sort of change. Thanks for the patch! It looks like your patch does not happily detect the dependencies on MacOS for some reason, but all appears well on Windows & Linux. I will have a cl

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: @lukasz.langa: Missing tests and documentation will be in by alpha2. -- ___ Python tracker <https://bugs.python.org/issue35

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: This work is the result of ~1.5 years of development effort, much of it accomplished at the last two core dev sprints. The code behind it has been stable since September 2018 and tested as an independently installable package by multiple people. I

[issue35813] shared memory construct to avoid need for serialization between processes

Davin Potts added the comment: New changeset e5ef45b8f519a9be9965590e1a0a587ff584c180 by Davin Potts in branch 'master': bpo-35813: Added shared_memory submodule of multiprocessing. (#11664) https://github.com/python/cpython/commit/e5ef45b8f519a9be9965590e1a0a587ff584c180

[issue35813] shared memory construct to avoid need for serialization between processes

Change by Davin Potts : -- keywords: +patch, patch pull_requests: +11470, 11471 stage: -> patch review ___ Python tracker <https://bugs.python.org/issu

[issue35813] shared memory construct to avoid need for serialization between processes

Change by Davin Potts : -- keywords: +patch, patch, patch pull_requests: +11470, 11471, 11472 stage: -> patch review ___ Python tracker <https://bugs.python.org/issu

[issue35813] shared memory construct to avoid need for serialization between processes

Change by Davin Potts : -- keywords: +patch pull_requests: +11470 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35813> ___ ___ Py

[issue35813] shared memory construct to avoid need for serialization between processes

New submission from Davin Potts : A facility for using shared memory would permit direct, zero-copy access to data across distinct processes (especially when created via multiprocessing) without the need for serialization, thus eliminating the primary performance bottleneck in the most

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: @ned.deily: Apologies, I misread what you wrote -- I would like to see the random segfaults that you were seeing on Mojave if you can still point me to a few. -- ___ Python tracker <https://bugs.python.

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: Do we really need to disable the running of test_multiprocessing_fork entirely on MacOS? My understanding so far is that not *all* of the system libraries on the mac are spinning up threads and so we should expect that there are situations where fork alone

[issue35219] macOS 10.14 Mojave crashes in multiprocessing

Davin Potts added the comment: Resolution is marked dupe but status is still open. Are we closing this one or is there a more specific remedy for this situation (as opposed to what issue33725 discusses) that would be helpful to document? -- nosy: +davin

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: Given the original post mentioned 2.7.15, I wonder if it is feasible to fork near the beginning of execution, then maintain and pass around a multiprocessing.Pool to be used when needed instead of dynamically forking? Working with legacy code is almost always

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: Barry's effort as well as comments in other links seem to all suggest that OBJC_DISABLE_INITIALIZE_FORK_SAFETY is not comprehensive in its ability to make other threads "safe" before forking. "Objective-C classes defined by the OS frameworks rem

[issue35242] multiprocessing.Queue in an inconsistent state and a traceback silently suppressed if put an unpickable object and process's target function is finished

Change by Davin Potts : -- nosy: +davin ___ Python tracker <https://bugs.python.org/issue35242> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33196] multiprocessing: serialization must ensure that contexts are compatible (the same)

Change by Davin Potts : -- nosy: +davin ___ Python tracker <https://bugs.python.org/issue33196> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue31308] forkserver process isn't re-launched if it died

Davin Potts added the comment: I have two concerns with this: 1) The implicit restart of the forkserver process seems in conflict with the zen of making things explicit. 2) This would seem to make forkserver's behavior inconsistent with the behavior of things like the Manager which similarly

[issue20854] multiprocessing.managers.Server: problem with returning proxy of registered object

Davin Potts added the comment: It appears that the multiple workarounds proposed by the OP (@allista) address the original request and that there is no bug or unintended behavior arising from multiprocessing itself. Combined with the lack of activity in this discussion, I'm inclined

[issue30339] test_multiprocessing_main_handling: "RuntimeError: Timed out waiting for results" on x86 Windows7 3.x

Davin Potts added the comment: Patch on issue30317 also addresses this issue in a more flexible way. -- dependencies: +test_timeout() of test_multiprocessing_spawn.WithManagerTestBarrier fails randomly on x86 Windows7 3.x buildbot nosy: +davin

[issue30317] test_timeout() of test_multiprocessing_spawn.WithManagerTestBarrier fails randomly on x86 Windows7 3.x buildbot

Davin Potts added the comment: To better accommodate very slow buildbots, a parameter is added in PR-1722 to scale up the timeout durations where they are necessary on a per-machine basis. Relevant tests have a timeout set to some default number of seconds times a multiplier value

[issue30317] test_timeout() of test_multiprocessing_spawn.WithManagerTestBarrier fails randomly on x86 Windows7 3.x buildbot

Changes by Davin Potts <pyt...@discontinuity.net>: -- pull_requests: +1810 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30317> ___

[issue28053] parameterize what serialization is used in multiprocessing

Davin Potts added the comment: Docs need updating still. -- versions: +Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/i

[issue26434] multiprocessing cannot spawn grandchild from a Windows service

Davin Potts added the comment: Patch committed in 2.7 branch. Thanks for your help, Marc. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <rep...@bugs.python.org> <http://bugs.pyt

[issue26434] multiprocessing cannot spawn grandchild from a Windows service

Davin Potts added the comment: New changeset c47c315812b1fa9acb16510a7aa3b37d113def48 by Davin Potts (Marc Schlaich) in branch '2.7': bpo-26434: Fix multiprocessing grandchilds in a Windows service (GH-1167) https://github.com/python/cpython/commit/c47c315812b1fa9acb16510a7aa3b37d113def48

[issue30379] multiprocessing Array create for ctypes.c_char, TypeError unless 1 char string arg used

Davin Potts added the comment: Perhaps I should've used ctypes.c_uint8 in that example/question instead. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/i

[issue30379] multiprocessing Array create for ctypes.c_char, TypeError unless 1 char string arg used

Davin Potts added the comment: Maybe I missed your point but why would you not want to do this instead? >>> mp.Array(ctypes.c_int8, arr) > -- nosy: +davin ___ Python tracker <rep...@bugs.python.org> <http://bugs.

[issue30018] multiprocessing.Pool garbles call stack for __new__

Davin Potts added the comment: > I am unfortunately not at liberty to share the code I'm working on. I very much understand and am very thankful you took the time to create a simple example that you could share. Honestly, that's the reason I felt inspired to stop what I was doing to l

[issue30018] multiprocessing.Pool garbles call stack for __new__

Davin Potts added the comment: Expanding my above example to show how multiprocessing relates: >>> import multiprocessing >>> import os >>> class Floof(object): ... def __new__(cls): ... print("New via pid=%d" % os.getpid()) ... return o

[issue30018] multiprocessing.Pool garbles call stack for __new__

Davin Potts added the comment: It looks like the first 'Called Foo.__new__' is being reported by the child (pool of 1) process and the second 'Called Foo.__new__' is being reported by the parent process. In multiprocessing, because objects are by default serialized using pickle, this may

[issue29828] Allow registering after-fork initializers in multiprocessing

Davin Potts added the comment: Having a read through issue16500 and issue6721, I worry that this could again become bogged down with similar concerns. With the specific example of NumPy, I am not sure I would want its random number generator to be reseeded with each forked process

[issue29795] Clarify how to share multiprocessing primitives

Changes by Davin Potts <pyt...@discontinuity.net>: -- resolution: -> works for me stage: needs patch -> resolved status: open -> closed ___ Python tracker <rep...@bugs.python.org> <http://bu

[issue17560] problem using multiprocessing with really big objects?

Davin Potts added the comment: @artxyz: The current release of 2.7 is 2.7.13 -- if you are still using 2.7.5 you might consider updating to the latest release. As pointed out in the text of the issue, the multiprocessing pickler has been made pluggable in 3.3 and it's been made more

[issue29795] Clarify how to share multiprocessing primitives

Davin Potts added the comment: On Windows, because that OS does not support fork, multiprocessing uses spawn to create new processes by default. Note that in Python 3, multiprocessing provides the user with a choice of how to create new processes (i.e. fork, spawn, forkserver). When fork

[issue29701] Add close method to queue.Queue

Davin Potts added the comment: The example of AMQP is perhaps a stronger argument for why multiprocessing.Queue.close should (or does) exist, not as much a reason for queue.Queue. The strongest point, I think, is the argument that existing patterns are lacking. In the multiprocessing module

[issue29454] Shutting down consumer on a remote queue

Davin Potts added the comment: My understanding of other message queueing systems is that many are motivated by speed to the point that they will permit messages to be "lost" due to specific scenarios that would be overly costly to defend against. Other message queueing sys

[issue29454] Shutting down consumer on a remote queue

Changes by Davin Potts <pyt...@discontinuity.net>: -- stage: -> needs patch type: behavior -> enhancement ___ Python tracker <rep...@bugs.python.org> <http://bugs.

[issue29454] Shutting down consumer on a remote queue

Davin Potts added the comment: My understanding is that example uses a queue.Queue() to demonstrate how to create a custom, remote service from scratch. The implementation in this simple example lacks the sophistication of multiprocessing.Queue() for handling situations such as the one

[issue29575] doc 17.2.1: basic Pool example is too basic

Davin Potts added the comment: When passing judgement on what is "too basic", the initial example should be so basic as to be immediately digestible by as many people as possible. Some background: All too many examples mislead newcomers into believing that the number of proces

[issue19675] Pool dies with excessive workers, but does not cleanup

Davin Potts added the comment: For triggering the exception, supplying a Process target that deliberately fails sounds right. As for tests for the various start methods (fork/forkserver/spawn), if you are looking at the 3.x branches you'll find this was been consolidated so that one test

[issue19675] Pool dies with excessive workers, but does not cleanup

Davin Potts added the comment: @Winterflower: Thank you for encouraging @dsoprea to create the new PR and working to convert the previous patch. @dsoprea: Thank you for taking the time to create the PR especially after this has been sitting unloved for so long. Though the new workflow using

[issue19675] Pool dies with excessive workers, but does not cleanup

Changes by Davin Potts <pyt...@discontinuity.net>: -- versions: +Python 2.7, Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python

[issue9914] trace/profile conflict with the use of sys.modules[__name__]

Davin Potts added the comment: Though this issue is specifically concerned with runpy APIs and their impact especially in running unittest test scripts, it's worth commenting here for people who need a workaround in the short term: code such as that shared in http://stackoverflow.com/q

[issue29284] Include thread_name_prefix in the concurrent.futures.ThreadPoolExecutor example 17.4.2.1

Changes by Davin Potts <pyt...@discontinuity.net>: -- nosy: +davin ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29284> ___ _

[issue29345] More lost updates with multiprocessing.Value and .Array

Davin Potts added the comment: I'm having difficulty watching your video attachment. Would it be possible to instead describe, preferably with example code that others can similarly try to reproduce the behavior, what you're experiencing? Please keep in mind what the documentation repeatedly

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Davin Potts added the comment: Serhiy: The above discussion seemed to converge on the perspective that object identity should not survive pickling and that the point of a sentinel is object identity. While your proposed patch may mechanically work, I believe it is in conflict

[issue29229] incompatible: unittest.mock.sentinel and multiprocessing.Pool.map()

Davin Potts added the comment: I think this should be regarded as a duplicate of issue20804 though discussion in issue14577 is also related/relevant. -- superseder: -> Sentinels identity lost when pickled (unittest.mock) ___ Python tracker &

[issue29229] incompatible: unittest.mock.sentinel and multiprocessing.Pool.map()

Davin Potts added the comment: This arises from the behavior of pickle (which is used by default in multiprocessing to serialize objects sent to / received from other processes in data exchanges), as seen with Python 3.6: >>> import pickle >>> x = pickle.dumps(mock.sentinel

  1   2   3   4   >