On Wed, Sep 27, 2017 at 1:26 AM, Nick Coghlan <ncogh...@gmail.com> wrote: > It's also the case that unlike Go channels, which were designed from > scratch on the basis of implementing pure CSP,
FWIW, Go's channels (and goroutines) don't implement pure CSP. They provide a variant that the Go authors felt was more in-line with the language's flavor. The channels in the PEP aim to support a more pure implementation. > Python has an > established behavioural precedent in the APIs of queue.Queue and > collections.deque: they're unbounded by default, and you have to opt > in to making them bounded. Right. That's part of why I'm leaning toward support for buffered channels. > While the article title is clickbaity, > http://www.jtolds.com/writing/2016/03/go-channels-are-bad-and-you-should-feel-bad/ > actually has a good discussion of this point. Search for "compose" to > find the relevant section ("Channels don’t compose well with other > concurrency primitives"). > > The specific problem cited is that only offering unbuffered or > bounded-buffer channels means that every send call becomes a potential > deadlock scenario, as all that needs to happen is for you to be > holding a different synchronisation primitive when the send call > blocks. Yeah, that blog post was a reference for me as I was designing the PEP's channels. > The fact that the proposal now allows for M:N sender:receiver > relationships (just as queue.Queue does with threads) makes that > problem worse, since you may now have variability not only on the > message consumption side, but also on the message production side. > > Consider this example where you have an event processing thread pool > that we're attempting to isolate from blocking IO by using channels > rather than coroutines. > > Desired flow: > > 1. Listener thread receives external message from socket > 2. Listener thread files message for processing on receive channel > 3. Listener thread returns to blocking on the receive socket > > 4. Processing thread picks up message from receive channel > 5. Processing thread processes message > 6. Processing thread puts reply on the send channel > > 7. Sending thread picks up message from send channel > 8. Sending thread makes a blocking network send call to transmit the message > 9. Sending thread returns to blocking on the send channel > > When queue.Queue is used to pass the messages between threads, such an > arrangement will be effectively non-blocking as long as the send rate > is greater than or equal to the receive rate. However, the GIL means > it won't exploit all available cores, even if we create multiple > processing threads: you have to switch to multiprocessing for that, > with all the extra overhead that entails. > > So I see the essential premise of PEP 554 as being to ask the question > "If each of these threads was running its own *interpreter*, could we > use Sans IO style protocols with interpreter channels to separate > internally "synchronous" processing threads from separate IO threads > operating at system boundaries, without having to make the entire > application pervasively asynchronous?" +1 > If channels are an unbuffered blocking primitive, then we don't get > that benefit: even when there are additional receive messages to be > processed, the processing thread will block until the previous send > has completed. Switching the listener and sender threads over to > asynchronous IO would help with that, but they'd also end up having to > implement their own message buffering to manage the lack of buffering > in the core channel primitive. > > By contrast, if the core channels are designed to offer an unbounded > buffer by default, then you can get close-to-CSP semantics just by > setting the buffer size to 1 (it's still not exactly CSP, since that > has a buffer size of 0, but you at least get the semantics of having > to alternate sending and receiving of messages). Yep, I came to the same conclusion. >> By the way, I do think efficiency is a concern here. Otherwise >> subinterpreters don't even have a point (just use multiprocessing). > > Agreed, and I think the interaction between the threading module and > the interpreters module is one we're going to have to explicitly call > out as being covered by the provisional status of the interpreters > module, as I think it could be incredibly valuable to be able to send > at least some threading objects through channels, and have them be an > interpreter-specific reference to a common underlying sync primitive. Agreed. I'll add a note to the PEP. -eric _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com