Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-23 Thread MRAB
On 2018-10-23 06:13, Nathaniel Smith wrote: On Sun, Oct 21, 2018 at 8:31 PM, Guido van Rossum wrote: On Sun, Oct 21, 2018 at 6:08 PM Nathaniel Smith wrote: I'm not sure if this is an issue the way Queue is used in practice, but in general you have to be careful with this kind of circular flow

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-22 Thread Nathaniel Smith
On Sun, Oct 21, 2018 at 8:31 PM, Guido van Rossum wrote: > On Sun, Oct 21, 2018 at 6:08 PM Nathaniel Smith wrote: >> I'm not sure if this is an issue the way Queue is used in practice, but in >> general you have to be careful with this kind of circular flow because if >> your queue communicates b

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-22 Thread Vladimir Filipović
Nathaniel, thank you for the pointer to Trio. Its approach seems very robust. I'm relieved to see that a solution so fundamentally rebuilt has also settled on very similar semantics for its `.close_put()`. I think your `.clone()` idiom is very clear when the communication objects are treated as di

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread Guido van Rossum
On Sun, Oct 21, 2018 at 6:08 PM Nathaniel Smith wrote: I'm not sure if this is an issue the way Queue is used in practice, but in general you have to be careful with this kind of circular flow because if your queue communicates backpressure (which it should) then circular flows can deadlock. Nath

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread Cameron Simpson
On 21Oct2018 18:06, Nathaniel Smith wrote: On Sun, Oct 21, 2018, 16:48 MRAB wrote: On 2018-10-21 22:30, Antoine Pitrou wrote: > Ah. This is the one statement that makes me favorable to this > idea. > When there is a single consumer, it's easy enough to send a sentinel. > But when there are

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread Cameron Simpson
On 21Oct2018 21:19, Vladimir Filipović wrote: On Sun, Oct 21, 2018 at 8:45 PM MRAB wrote: FTR, this has been discussed before: [Python-ideas] `__iter__` for queues? https://mail.python.org/pipermail/python-ideas/2010-January/006711.html Thank you! Hmm, yes. My post there is this one: htt

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread Nathaniel Smith
On Sun, Oct 21, 2018, 16:48 MRAB wrote: > On 2018-10-21 22:30, Antoine Pitrou wrote: > > On Sun, 21 Oct 2018 19:58:05 +0200 > > Vladimir Filipović > > wrote: > >> > >> To anticipate a couple more possible questions: > >> > >> - What would this proposal do about multiple producers/consumers > >>

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread Terry Reedy
On 10/21/2018 2:42 PM, MRAB wrote: On 2018-10-21 18:58, Vladimir Filipović wrote: Hi! I originally submitted this as a pull request. Raymond Hettinger suggested it should be given a shakeout in python-ideas first. https://github.com/python/cpython/pull/10018 https://bugs.python.org/issue35034

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread MRAB
On 2018-10-21 22:30, Antoine Pitrou wrote: On Sun, 21 Oct 2018 19:58:05 +0200 Vladimir Filipović wrote: To anticipate a couple more possible questions: - What would this proposal do about multiple producers/consumers needing to jointly decide _when_ to close the queue? Explicitly nothing. T

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread Antoine Pitrou
On Sun, 21 Oct 2018 19:58:05 +0200 Vladimir Filipović wrote: > > To anticipate a couple more possible questions: > > - What would this proposal do about multiple producers/consumers > needing to jointly decide _when_ to close the queue? > > Explicitly nothing. > > The queue's state is either c

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread Nathaniel Smith
Hi Vladimir, It's great to see people revisiting these old stdlib tools. Closure tracking is definitely a big point of awkwardness for Queues. In Trio we started with a straight copy of threading.Queue, and this turned out to be a major friction point for users. We just deprecated our version of Q

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread Vladimir Filipović
On Sun, Oct 21, 2018 at 8:45 PM MRAB wrote: > FTR, this has been discussed before: > > [Python-ideas] `__iter__` for queues? > https://mail.python.org/pipermail/python-ideas/2010-January/006711.html Thank you! For the sake of clarity, I want to outline a few differences between that discussion a

Re: [Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread MRAB
On 2018-10-21 18:58, Vladimir Filipović wrote: Hi! I originally submitted this as a pull request. Raymond Hettinger suggested it should be given a shakeout in python-ideas first. https://github.com/python/cpython/pull/10018 https://bugs.python.org/issue35034 [snip] FTR, this has been discusse

[Python-ideas] Add closing and iteration to threading.Queue

2018-10-21 Thread Vladimir Filipović
Hi! I originally submitted this as a pull request. Raymond Hettinger suggested it should be given a shakeout in python-ideas first. https://github.com/python/cpython/pull/10018 https://bugs.python.org/issue35034 -- Briefly: Add a close() method to Queue, which should simplify many common u