Re: [Python-Dev] Pythonic concurrency - offtopic

2005-10-19 Thread Josiah Carlson
JanC <[EMAIL PROTECTED]> wrote: > > On 10/14/05, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > Until Microsoft adds kernel support for fork, don't expect standard > > Windows Python to support it. > > AFAIK the NT kernel has support for fork, but the Win32 subsystem > doesn't support it (you can

Re: [Python-Dev] Pythonic concurrency - offtopic

2005-10-19 Thread JanC
On 10/14/05, Josiah Carlson <[EMAIL PROTECTED]> wrote: > Until Microsoft adds kernel support for fork, don't expect standard > Windows Python to support it. AFAIK the NT kernel has support for fork, but the Win32 subsystem doesn't support it (you can only use it with the POSIX subsystem). -- JanC

Re: [Python-Dev] Pythonic concurrency - offtopic

2005-10-14 Thread Sokolov Yura
Josiah Carlson wrote: >Sokolov Yura <[EMAIL PROTECTED]> wrote: > > >>Offtopic: >> >>Microsoft Windows [Version 5.2.3790] >>(C) Copyright 1985-2003 Microsoft Corp. >> >>G:\Working\1>c:\Python24\python >>Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on >>win32 >>Type "help

Re: [Python-Dev] Pythonic concurrency - offtopic

2005-10-13 Thread Josiah Carlson
Sokolov Yura <[EMAIL PROTECTED]> wrote: > > Offtopic: > > Microsoft Windows [Version 5.2.3790] > (C) Copyright 1985-2003 Microsoft Corp. > > G:\Working\1>c:\Python24\python > Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on > win32 > Type "help", "copyright", "credits"

[Python-Dev] Pythonic concurrency - offtopic

2005-10-13 Thread Sokolov Yura
Offtopic: Microsoft Windows [Version 5.2.3790] (C) Copyright 1985-2003 Microsoft Corp. G:\Working\1>c:\Python24\python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from os import fork Tr

Re: [Python-Dev] Pythonic concurrency

2005-10-13 Thread Bruce Eckel
I don't know of anything that exists. There is an upcoming book that may help: Java Concurrency in Practice, by Brian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, and Doug Lea (Addison-Wesley 2006). I have had assistance from some of the authors, but don't know if it introduces

Re: [Python-Dev] Pythonic concurrency

2005-10-13 Thread Michael Hudson
Bruce Eckel <[EMAIL PROTECTED]> writes: > Not only are there significant new library components in > java.util.concurrent in J2SE5, but perhaps more important is the new > memory model that deals with issues that are (especially) revealed in > multiprocessor environments. The new memory model repr

Re: [Python-Dev] Pythonic concurrency

2005-10-12 Thread Kalle Anke
On Fri, 7 Oct 2005 18:47:51 +0200, Bruce Eckel wrote (in article <[EMAIL PROTECTED]>): > It's hard to know how to answer. I've met enough brilliant people to > know that it's just possible that the person posting really does > easily grok concurrency issues and thus I must seem irreconcilably > th

Re: [Python-Dev] Pythonic concurrency

2005-10-12 Thread Phillip J. Eby
At 02:35 AM 10/12/2005 +, Joshua Spoerri wrote: >that stm paper isn't the end. > >there's a java implementation which seems to be exactly what we want: >http://research.microsoft.com/~tharris/papers/2003-oopsla.pdf There's already a Python implementation of what's described in the paper. It'

Re: [Python-Dev] Pythonic concurrency

2005-10-11 Thread Joshua Spoerri
that stm paper isn't the end. there's a java implementation which seems to be exactly what we want: http://research.microsoft.com/~tharris/papers/2003-oopsla.pdf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/p

Re: [Python-Dev] Pythonic concurrency

2005-10-11 Thread Josiah Carlson
"Robert Brewer" <[EMAIL PROTECTED]> wrote: > "Somewhat alleviated" and somewhat worsened. I've had half a dozen > conversations in the last year about sharing data between threads; in > every case, I've had to work quite hard to convince the other person > that threading.local is *not* magic pixie

Re: [Python-Dev] Pythonic concurrency

2005-10-11 Thread Bruce Eckel
> Java's condition variables don't (didn't? has this been fixed?) quite > work. The emphasis on portability and the resulting notions of > red/green threading packages at the beginning didn't help either. > Read Allen Holub's book. And Doug Lea's book. I understand much of > this has been addre

Re: [Python-Dev] Pythonic concurrency

2005-10-11 Thread Nick Coghlan
Robert Brewer wrote: > "Somewhat alleviated" and somewhat worsened. I've had half a dozen > conversations in the last year about sharing data between threads; in > every case, I've had to work quite hard to convince the other person > that threading.local is *not* magic pixie thread dust. Each t

Re: [Python-Dev] Pythonic concurrency

2005-10-11 Thread Robert Brewer
Title: RE: [Python-Dev] Pythonic concurrency Steve Holden wrote: > The nice thing about Python threads (or rather threading.threads) is > that since each thread is an instance it's *relatively* easy to ensure > that a thread restricts itself to manipulating thread-local re

Re: [Python-Dev] Pythonic concurrency

2005-10-11 Thread Steve Holden
Bruce Eckel wrote: [Bill Janssen] >>Yes, there's a troublesome meme in the world: "threads are hard". >>They aren't, really. You just have to know what you're doing. > But that begs the question, because there is a significant amount of evidence that when it comes to threads "knowing what you ar

Re: [Python-Dev] Pythonic concurrency

2005-10-11 Thread Nick Coghlan
Bruce Eckel wrote: >>Yes, there's a troublesome meme in the world: "threads are hard". >>They aren't, really. You just have to know what you're doing. > > > I would say that the troublesome meme is that "threads are easy." I > posted an earlier, rather longish message about this. The gist of > w

Re: [Python-Dev] Pythonic concurrency

2005-10-11 Thread Nick Coghlan
Donovan Baarda wrote: > On Fri, 2005-10-07 at 23:54, Nick Coghlan wrote: > [...] > >>The few times I have encountered anyone saying anything resembling "threading >>is easy", it was because the full sentence went something like "threading is >>easy if you use message passing and copy-on-send or

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Bill Janssen
Guido writes: > Given the tendency of Python developers to build layers of > abstractions I don't think [non-preemptive threads] will help much. I think that's right, although I think adding priorities to Python's existing preemptive threads might be useful for real-time programmers (yes, as machi

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Christopher Armstrong
On 10/11/05, Guido van Rossum <[EMAIL PROTECTED]> wrote: > I recall using a non-preemptive system in the past; in Amoeba, to be precise. > > Initially it worked great. > > But as we added more powerful APIs to the library, we started to run > into bugs that were just as if you had preemptive schedu

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Guido van Rossum
On 10/10/05, Greg Ewing <[EMAIL PROTECTED]> wrote: > I'm wondering whether Python threads should be > non-preemptive by default. Preemptive threading is > massive overkill for many applications. You don't > need it, for example, if you just want to use threads > to structure your program, overlap p

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Greg Ewing
Ian Bicking wrote: > What the GIL-ranters don't get is that the GIL actually gives you just > enough determinism to be able to write threaded programs that don't crash, The GIL no doubt helps, but your threads can still get preempted between bytecodes, so I can't see it making much difference a

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Neil Hodgson
Bruce Eckel: > I would say that the troublesome meme is that "threads are easy." I > posted an earlier, rather longish message about this. The gist of > which was: "when someone says that threads are easy, I have no idea > what they mean by it." I think you are overcomplicating the issue by lo

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Ian Bicking
Phillip J. Eby wrote: > What the GIL-ranters don't get is that the GIL actually gives you just > enough determinism to be able to write threaded programs that don't crash, > and that maybe will even work if you treat every point of interaction > between threads as a minefield and program with ap

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Bruce Eckel
> Yes, there's a troublesome meme in the world: "threads are hard". > They aren't, really. You just have to know what you're doing. I would say that the troublesome meme is that "threads are easy." I posted an earlier, rather longish message about this. The gist of which was: "when someone says t

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Bill Janssen
Skip, > With the Mojam middleware stuff I suffered quite awhile with a > single-threaded implementation that would hang the entire webserver if a > backend query took too long. I realized I needed to do something (threads, > asyncore, whatever), but didn't think I understood the issues well enoug

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Donovan Baarda
On Mon, 2005-10-10 at 18:59, Bill Janssen wrote: > > The problem with threads is at first glance they appear easy... > > Anyone who thinks that a "glance" is enough to understand something is > too far gone to worry about. On the other hand, you might be > referring to a putative brokenness of th

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread skip
>> The hard part is knowing when and how to lock shared resources... Bill> Well, I might say the "careful part". With the Mojam middleware stuff I suffered quite awhile with a single-threaded implementation that would hang the entire webserver if a backend query took too long. I realize

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Bill Janssen
> The problem with threads is at first glance they appear easy... Anyone who thinks that a "glance" is enough to understand something is too far gone to worry about. On the other hand, you might be referring to a putative brokenness of the Python documentation on Python threads. I'm not sure the

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Michael Sparks
On Monday 10 Oct 2005 15:45, Donovan Baarda wrote: > Sounds like yet another reason to avoid threading and use processes > instead... effort spent on threading based message passing > implementations could instead be spent on inter-process messaging. I can't let that pass (even if our threaded com

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Donovan Baarda
On Fri, 2005-10-07 at 17:47, Bruce Eckel wrote: > Early in this thread there was a comment to the effect that "if you > don't know how to use threads, don't use them," which I pointedly > avoided responding to because it seemed to me to simply be > inflammatory. But Ian Bicking just posted a weblog

Re: [Python-Dev] Pythonic concurrency

2005-10-10 Thread Donovan Baarda
On Fri, 2005-10-07 at 23:54, Nick Coghlan wrote: [...] > The few times I have encountered anyone saying anything resembling "threading > is easy", it was because the full sentence went something like "threading is > easy if you use message passing and copy-on-send or release-reference-on-send >

Re: [Python-Dev] Pythonic concurrency

2005-10-08 Thread Josiah Carlson
Michael Sparks <[EMAIL PROTECTED]> wrote: > On Saturday 08 October 2005 04:05, Josiah Carlson wrote: > > I'm just offering the above as example benchmarks (you certainly don't > > need to do them to satisfy me, but I'll be doing those when my tuple > > space implementation is closer to being done)

Re: [Python-Dev] Pythonic concurrency

2005-10-08 Thread Michael Sparks
On Saturday 08 October 2005 04:05, Josiah Carlson wrote: [ simplistic, informal benchmark of a test optimised versioned of the system, based on bouncing scaing rotating sprites around the screen. ] > Single process?  Multi-process single machine?  Multiprocess multiple > machine? SIngle process,

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Josiah Carlson
Michael Sparks <[EMAIL PROTECTED]> wrote: > [ Possibly overlengthy reply. However given a multiple sets of cans of > worms... ] > On Friday 07 October 2005 07:25, Josiah Carlson wrote: > > One thing I notice is absent from the Kamaelia page is benchmarks. > > That's largely for one simple reaso

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Nick Coghlan
Bruce Eckel wrote: > I always have a problem with this. After many years of studying > concurrency on-and-off, I continue to believe that threading is very > difficult (indeed, the more I study it, the more difficult I > understand it to be). And I admit this. The comments I sometimes get > back ar

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Michael Sparks
On Thursday 06 October 2005 21:06, Bruce Eckel wrote: > So yes indeed, this is quite high on my list to research. Looks like > people there have been doing some interesting work. > > Right now I'm just trying to cast a net, so that people can put in > ideas, for when the Java book is done and I can

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Michael Sparks
On Friday 07 October 2005 23:26, Bruce Eckel wrote: > I think the ease of use issue opens up far greater possibilities if you > include multiprocessing ... > That's what I'm looking for. In which case that's an area we need to push our work into sooner rather than later. After all, the PS3 and

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Bruce Eckel
> //Theoretically// I suspect that the system /could/ perform as well as > traditional approaches to dealing with concurrent problems single threaded > (and multi-thread/process). I also think it's important to factor in the possibility of multiprocessors. If Kamaelia (for example) has a very safe

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Shane Hathaway
Antoine Pitrou wrote: >>I'd be happy to explain how >>ZODB solves those problems, if you're interested. > > > Well, yes, I'm interested :) > (I don't anything about Zope internals though, and I've never even used > it) Ok. Quoting your list: > To apply the same thing to Python you would at l

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Antoine Pitrou
> Well, I think you just described ZODB. ;-) *gasp* > I'd be happy to explain how > ZODB solves those problems, if you're interested. Well, yes, I'm interested :) (I don't anything about Zope internals though, and I've never even used it) ___ Pyth

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Jim Fulton
Shane Hathaway wrote: > Antoine Pitrou wrote: > >>A relational database has a very strict and regular data model. Also, it >>has transactions. This makes it easy to precisely define concurrency at >>the engine level. >> >>To apply the same thing to Python you would at least need : >> 1. a way to

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Michael Sparks
[ Possibly overlengthy reply. However given a multiple sets of cans of worms... ] On Friday 07 October 2005 07:25, Josiah Carlson wrote: > One thing I notice is absent from the Kamaelia page is benchmarks. That's largely for one simple reason: we haven't done any yet. At least not anything I'd

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Shane Hathaway
Antoine Pitrou wrote: > A relational database has a very strict and regular data model. Also, it > has transactions. This makes it easy to precisely define concurrency at > the engine level. > > To apply the same thing to Python you would at least need : > 1. a way to define a subset of the curr

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Antoine Pitrou
Hi, (my 2 cents, probably not very constructive) > Recently, I've been simulating high concurrency on a PostgreSQL > database, and I've discovered that the way you reason about row and > table locks is very similar to the way you reason about locking among > threads. The big difference is th

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Barry Warsaw
On Fri, 2005-10-07 at 14:42, Shane Hathaway wrote: > What's insanely difficult is really locking, and locking is driven by > concurrency in general, not just threads. It's hard to reason about > locks. I think that's a very interesting observation! I have not built a tremendous number of co

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Shane Hathaway
Bruce Eckel wrote: > But. I do happen to have contact with a lot of people who are at the > forefront of the threading world, and *none* of them (many of whom > have written the concurrency libraries for Java 5, for example) ever > imply that threading is easy. In fact, they generally go out of the

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Phillip J. Eby
At 10:47 AM 10/7/2005 -0600, Bruce Eckel wrote: >Also, look at the work that Scott Meyers, Andrei Alexandrescu, et al >did on the "Double Checked Locking" idiom, showing that it was broken >under threading. That was by no means "trivial and obvious" during all >the years that people thought that it

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Aahz
On Fri, Oct 07, 2005, Bruce Eckel wrote: > > I always have a problem with this. After many years of studying > concurrency on-and-off, I continue to believe that threading is very > difficult (indeed, the more I study it, the more difficult I > understand it to be). And I admit this. The comments I

[Python-Dev] Pythonic concurrency

2005-10-07 Thread Bruce Eckel
Early in this thread there was a comment to the effect that "if you don't know how to use threads, don't use them," which I pointedly avoided responding to because it seemed to me to simply be inflammatory. But Ian Bicking just posted a weblog entry: http://blog.ianbicking.org/concurrency-and-proce

Re: [Python-Dev] Pythonic concurrency

2005-10-06 Thread Josiah Carlson
Michael Sparks <[EMAIL PROTECTED]> wrote: > > On Thursday 06 October 2005 23:15, Josiah Carlson wrote: > [... 6 specific use cases ...] > > If Kamaelia is able to handle all of the above mechanisms in both a > > blocking and non-blocking fashion, then I would guess it has the basic > > requiremen

Re: [Python-Dev] Pythonic concurrency

2005-10-06 Thread Michael Sparks
On Thursday 06 October 2005 23:15, Josiah Carlson wrote: [... 6 specific use cases ...] > If Kamaelia is able to handle all of the above mechanisms in both a > blocking and non-blocking fashion, then I would guess it has the basic > requirements for most concurrent applications. It can. I can easi

Re: [Python-Dev] Pythonic concurrency

2005-10-06 Thread Josiah Carlson
Michael Sparks <[EMAIL PROTECTED]> wrote: > What I'd be interested in, is hearing how our system doesn't match with > the goals of the hypothetical concurrency system you'd like to see (if it > doesn't). The main reason I'm interested in hearing this, is because the > goals you listed are ones we

Re: [Python-Dev] Pythonic concurrency

2005-10-06 Thread Bruce Eckel
This does look quite fascinating, and I know there's a lot of really interesting work going on at the BBC now -- looks like some really pioneering stuff going on with respect to TV show distribution over the internet, new compression formats, etc. So yes indeed, this is quite high on my list to re

Re: [Python-Dev] Pythonic concurrency

2005-10-06 Thread Michael Sparks
Hi Bruce, On Thursday 06 October 2005 18:12, Bruce Eckel wrote: > Although I hope our conversation isn't done, as he suggests! ... > At some point when more ideas have been thrown about (and TIJ4 is > done) I hope to summarize what we've talked about in an article. I don't know if you saw my pre

Re: [Python-Dev] Pythonic concurrency

2005-10-06 Thread Paolo Invernizzi
Just to add another 2 cents http://www.erights.org/talks/promises/paper/tgc05.pdf --- Paolo Invernizzi Bruce Eckel wrote: > Jeremy Jones published a blog discussing some of the ideas we've > talked about here: > http://www.oreillynet.com/pub/wlg/8002 > Although I hope our conversation isn't

[Python-Dev] Pythonic concurrency

2005-10-06 Thread Bruce Eckel
Jeremy Jones published a blog discussing some of the ideas we've talked about here: http://www.oreillynet.com/pub/wlg/8002 Although I hope our conversation isn't done, as he suggests! At some point when more ideas have been thrown about (and TIJ4 is done) I hope to summarize what we've talked abou

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-10-02 Thread Martin Blais
On 10/2/05, Christopher Armstrong <[EMAIL PROTECTED]> wrote: > On 10/3/05, Martin Blais <[EMAIL PROTECTED]> wrote: > > On 10/1/05, Antoine <[EMAIL PROTECTED]> wrote: > > > > > > > like this with their "deferred objects", no? I figure they would > > > > need to do something like this too. I will h

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-10-02 Thread Christopher Armstrong
On 10/3/05, Martin Blais <[EMAIL PROTECTED]> wrote: > On 10/1/05, Antoine <[EMAIL PROTECTED]> wrote: > > > > > like this with their "deferred objects", no? I figure they would > > > need to do something like this too. I will have to check.) > > > > A Deferred object is just the abstraction of a c

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-10-02 Thread Martin Blais
On 10/1/05, Antoine <[EMAIL PROTECTED]> wrote: > > > like this with their "deferred objects", no? I figure they would > > need to do something like this too. I will have to check.) > > A Deferred object is just the abstraction of a callback - or, rather, two > callbacks: one for success and one f

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-10-01 Thread Christopher Armstrong
On 10/2/05, Martin Blais <[EMAIL PROTECTED]> wrote: > One of the problems that you have with using generators like > this, is that automatic "yield" on resource access does not occur > automatically, like it does in threading. With threads, the > kernel is invoked when access to a low-level resour

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-10-01 Thread Michael Sparks
On Saturday 01 October 2005 22:50, Martin Blais wrote: ... > because some people on the list appear to discuss generators as > a concurrency scheme, and as far as I know they really are not > addressing that at all. Our project started in the context of dealing with the task of a naturally concurr

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-10-01 Thread Antoine
Hi Martin, [snip] The "confusion" stems from the fact that two issues are mixed up in this discussion thread: - improving concurrency schemes to make it easier to write well-behaving applications with independent parallel flows - improving concurrency schemes to improve performance when there ar

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-10-01 Thread Martin Blais
Hi. I hear a confusion that is annoying me a bit in some of the discussions on concurrency, and I thought I'd flush my thoughts here to help me clarify some of that stuff, because some people on the list appear to discuss generators as a concurrency scheme, and as far as I know (and please correct

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Gustavo J. A. M. Carneiro
On Fri, 2005-09-30 at 18:33 +0200, Antoine Pitrou wrote: > Hi Jp, > > Le vendredi 30 septembre 2005 à 12:20 -0400, Jp Calderone a écrit : > > "Advocating" might be putting it too strongly :) "Experimenting with" > > describes the current state of things most accurately. > > Ok :) > > > The prob

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Bruce Eckel
> I was just saying that it can be useful to mix cooperative threading and > preemptive threading in the same app, i.e. have different domains of > cooperative threading which are preemptively scheduled by the OS. That > has nothing to do with the GIL, I think (but I don't know much in Python > int

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Antoine Pitrou
Hi Jp, Le vendredi 30 septembre 2005 à 12:20 -0400, Jp Calderone a écrit : > "Advocating" might be putting it too strongly :) "Experimenting with" > describes the current state of things most accurately. Ok :) > The problem it aims to solve is integration with cooperative threading > systems w

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Jp Calderone
On Fri, 30 Sep 2005 17:26:27 +0200, Antoine Pitrou <[EMAIL PROTECTED]> wrote: > >Hi, > >> I've never heard >> someone complain that the GIL is in the way for these types of apps. > >I've never said so either. >I was just saying that it can be useful to mix cooperative threading and >preemptive thre

Re: [Python-Dev] Pythonic concurrency

2005-09-30 Thread Paul Moore
On 9/30/05, Jeremy Maxfield <[EMAIL PROTECTED]> wrote: > Support for multiple interpreters already exists from the C API > (mod_python, Java Embedded Python a few other add-ons use them) I'm aware of that (didn't I mention it in my message - sorry). > But: > - it's not possible to create new inte

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Antoine Pitrou
Hi, > I've never heard > someone complain that the GIL is in the way for these types of apps. I've never said so either. I was just saying that it can be useful to mix cooperative threading and preemptive threading in the same app, i.e. have different domains of cooperative threading which are p

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Antoine Pitrou
Le vendredi 30 septembre 2005 à 07:32 -0700, Simon Wittber a écrit : > I use this approach extensively, using tasks which are defined using > generators. The scheduler I developed for this can be viewed here: > > http://metaplay.dyndns.org:82/svn/nanothreads/nanothreads.py FWIW, I've coded my ow

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Guido van Rossum
On 9/30/05, Antoine Pitrou <[EMAIL PROTECTED]> wrote: > Le vendredi 30 septembre 2005 à 08:32 -0500, [EMAIL PROTECTED] a écrit : > > >> (C) That scheduler is non-preemptive. A single greedy generator can > > >> starve all the others. > > > > Antoine> Instead of looking at this as

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Simon Wittber
On 9/30/05, Antoine Pitrou <[EMAIL PROTECTED]> wrote: > > > (C) That scheduler is non-preemptive. A single > > greedy generator can starve all the others. > > Instead of looking at this as a problem, you could look at it as a > feature. Since generators can't be switched at arbitrary places, the

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Antoine Pitrou
Le vendredi 30 septembre 2005 à 08:32 -0500, [EMAIL PROTECTED] a écrit : > >> (C) That scheduler is non-preemptive. A single greedy generator can > >> starve all the others. > > Antoine> Instead of looking at this as a problem, you could look at it > Antoine> as a feature. >

Re: [Python-Dev] Pythonic concurrency

2005-09-30 Thread Jeremy Maxfield
Support for multiple interpreters already exists from the C API (mod_python, Java Embedded Python a few other add-ons use them) But: - it's not possible to create new interpreter instances from within Python. - there's no mechanism for passing information between interpreters. - interaction wit

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread skip
>> (C) That scheduler is non-preemptive. A single greedy generator can >> starve all the others. Antoine> Instead of looking at this as a problem, you could look at it Antoine> as a feature. Apple looked at it as a feature for years. Not anymore. Skip _

Re: [Python-Dev] Pythonic concurrency

2005-09-30 Thread Paul Moore
On 9/30/05, Jim Jewett <[EMAIL PROTECTED]> wrote: > Bruce Eckel wrote: > > > 3) Tasks are cheap enough that I can make > > thousands of them, ... > > > 4) Tasks are "self-guarding," so they prevent > > other tasks from interfering with them. The > > only way tasks can communicate with each > > othe

Re: [Python-Dev] Pythonic concurrency - cooperative MT

2005-09-30 Thread Antoine Pitrou
> (C) That scheduler is non-preemptive. A single > greedy generator can starve all the others. Instead of looking at this as a problem, you could look at it as a feature. Since generators can't be switched at arbitrary places, the programmer has to define his/her synchronization points explicit

[Python-Dev] Pythonic concurrency

2005-09-29 Thread Jim Jewett
Bruce Eckel wrote: > 3) Tasks are cheap enough that I can make > thousands of them, ... > 4) Tasks are "self-guarding," so they prevent > other tasks from interfering with them. The > only way tasks can communicate with each > other is through some kind of formal > mechanism (something queue-ish,

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Jonathan LaCour
> I'd like to restart this discussion; I didn't mean to put forth active > objects as "the" solution, only that it seems to be one of the better, > more OO solutions that I've seen so far. Thanks for doing this. I think this is an issue that is going to be more and more important as Python conti

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Shane Hathaway
Bruce Eckel wrote: > I'd like to restart this discussion; I didn't mean to put forth active > objects as "the" solution, only that it seems to be one of the better, > more OO solutions that I've seen so far. > > What I'd really like to figure out is the "pythonic" solution for > concurrency. Guido

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Jim Fulton
Michael Hudson wrote: > Bruce Eckel <[EMAIL PROTECTED]> writes: > > >>I'd like to restart this discussion; I didn't mean to put forth active >>objects as "the" solution, only that it seems to be one of the better, >>more OO solutions that I've seen so far. >> >>What I'd really like to figure out

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Michael Hudson
"Phillip J. Eby" <[EMAIL PROTECTED]> writes: > At 09:30 AM 9/29/2005 -0600, Bruce Eckel wrote: >>This paper looks very interesting and promises some good ideas. It >>also looks like it will require time and effort to digest. >> >>I've only read the first few pages, but one thing that does leap out

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Phillip J. Eby
At 10:48 AM 9/29/2005 -0600, Bruce Eckel wrote: >I haven't spent the weekends on the paper yet (but it looks like that >is what it would take), but I had the impression that they were >talking about the lock-free techniques such as the ones used in Java >5. Basically, you start a write operation "i

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Bill Janssen
> 1) It works by default, so that novices can use it without falling > into the deep well of threading. That is, a program that you write > using threading is broken by default, and the tool you have to fix it > is "inspection." I want something that allows me to say "this is a > task. Go." and hav

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Bruce Eckel
> I spent a few weekends studying that paper earlier this year in order to > see if anything could be stolen for Python; my general impression was "not > easily" at the least. One notable feature of the presented concept was > that when code would otherwise block, they *rolled it back* to the last

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Phillip J. Eby
At 09:30 AM 9/29/2005 -0600, Bruce Eckel wrote: >This paper looks very interesting and promises some good ideas. It >also looks like it will require time and effort to digest. > >I've only read the first few pages, but one thing that does leap out >is at the beginning of section 3, they say: > >"..

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Michael Walter
FWIW, the Perl 6 community is also investigating STM, so it appears to be a worthwhile idea for an impure, multi-paradigm language as well. Regards, Michael On 9/29/05, Bruce Eckel <[EMAIL PROTECTED]> wrote: > This paper looks very interesting and promises some good ideas. It > also looks like it

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Bruce Eckel
This paper looks very interesting and promises some good ideas. It also looks like it will require time and effort to digest. I've only read the first few pages, but one thing that does leap out is at the beginning of section 3, they say: "... a purely-declarative language is a perfect setting fo

Re: [Python-Dev] Pythonic concurrency

2005-09-29 Thread Michael Hudson
Bruce Eckel <[EMAIL PROTECTED]> writes: > I'd like to restart this discussion; I didn't mean to put forth active > objects as "the" solution, only that it seems to be one of the better, > more OO solutions that I've seen so far. > > What I'd really like to figure out is the "pythonic" solution for

[Python-Dev] Pythonic concurrency

2005-09-29 Thread Bruce Eckel
I'd like to restart this discussion; I didn't mean to put forth active objects as "the" solution, only that it seems to be one of the better, more OO solutions that I've seen so far. What I'd really like to figure out is the "pythonic" solution for concurrency. Guido and I got as far as agreeing t