Re: [Python-ideas] Thread-safe generators

2017-04-17 Thread Greg Ewing
Nick Coghlan wrote: but at the cost of changing the nature of the workload in a given thread, and hence messing with the working set of objects it has active. Does the working set of an individual thread matter to anything? Or even an individual process? As far as the virtual memory system is

Re: [Python-ideas] Thread-safe generators

2017-04-17 Thread Jim Baker
This is a bad idea in the generator itself, as commented earlier by others here. >From a cross implementation perspective, in Jython, different threads can call next on a non running generator, *so long as they coordinate with each other external to any use of this generator*, and this works

Re: [Python-ideas] Thread-safe generators

2017-04-16 Thread Nick Coghlan
On 17 April 2017 at 08:00, Paul Moore wrote: > On 15 April 2017 at 10:45, Nick Coghlan wrote: >> So I'd be opposed to trying to make generator objects natively thread >> aware - as Stephen notes, the GIL is an implementation detail of >> CPython, so it

Re: [Python-ideas] Thread-safe generators

2017-04-16 Thread Paul Moore
On 15 April 2017 at 10:45, Nick Coghlan wrote: > So I'd be opposed to trying to make generator objects natively thread > aware - as Stephen notes, the GIL is an implementation detail of > CPython, so it isn't OK to rely on it when defining changes to > language level semantics

Re: [Python-ideas] Thread-safe generators

2017-04-16 Thread Guido van Rossum
I think the two shouldn't be mixed. On Apr 16, 2017 7:58 AM, "Victor Stinner" wrote: > Thread safety is very complex and has an impact on performance. I dislike > the idea of providing such property to generators which can have a complex > next method. > > IMHO it's

Re: [Python-ideas] Thread-safe generators

2017-04-16 Thread Victor Stinner
Thread safety is very complex and has an impact on performance. I dislike the idea of providing such property to generators which can have a complex next method. IMHO it's better to put a generator in wrapper which adds thread safety. What do you think? Victor Le 14 avr. 2017 18:48, "Serhiy

Re: [Python-ideas] Thread-safe generators

2017-04-15 Thread Guido van Rossum
My 2 cent's worth, don't even think about it. On Apr 15, 2017 3:27 AM, "Serhiy Storchaka" wrote: > On 15.04.17 11:55, Stephen J. Turnbull wrote: > >> Serhiy Storchaka writes: >> >> > The first thread just sets the running flag (as in current code). Due >> to >> > GIL this

Re: [Python-ideas] Thread-safe generators

2017-04-15 Thread Serhiy Storchaka
On 15.04.17 11:55, Stephen J. Turnbull wrote: Serhiy Storchaka writes: > The first thread just sets the running flag (as in current code). Due to > GIL this doesn't need additional synchronization. Can we assume this lack of additional synchronization for other implementations? If not, do

Re: [Python-ideas] Thread-safe generators

2017-04-15 Thread Nick Coghlan
On 15 April 2017 at 02:47, Serhiy Storchaka wrote: > When use a generator from different threads you can get a ValueError > "generator already executing". Getting this exception with the single thread > is a programming error, it in case of different threads it could be

Re: [Python-ideas] Thread-safe generators

2017-04-15 Thread Stephen J. Turnbull
Serhiy Storchaka writes: > The first thread just sets the running flag (as in current code). Due to > GIL this doesn't need additional synchronization. Can we assume this lack of additional synchronization for other implementations? If not, do we care?

Re: [Python-ideas] Thread-safe generators

2017-04-14 Thread Serhiy Storchaka
On 15.04.17 01:42, Greg Ewing wrote: Serhiy Storchaka wrote: but should not affect performance since locking is used only when you faced with a generator running in other thread. I don't think that's true, because the first thread to use a generator has to lock it as well. And even if there

Re: [Python-ideas] Thread-safe generators

2017-04-14 Thread Greg Ewing
Serhiy Storchaka wrote: but should not affect performance since locking is used only when you faced with a generator running in other thread. I don't think that's true, because the first thread to use a generator has to lock it as well. And even if there is only one thread in existence when

[Python-ideas] Thread-safe generators

2017-04-14 Thread Serhiy Storchaka
When use a generator from different threads you can get a ValueError "generator already executing". Getting this exception with the single thread is a programming error, it in case of different threads it could be possible to wait until other thread finish executing the generator. The