[Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-10 Thread dustin
Mostly for my own curiosity, I'm working on a PEP-342-based microthreading library with a similar api to threads and threading (coalesced into a single module). It uses coroutines and a trampoline scheduler, and provides basic async wrappers for common IO operations. It's not a framework/environm

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-10 Thread Brett Cannon
On 2/10/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Mostly for my own curiosity, I'm working on a PEP-342-based > microthreading library with a similar api to threads and threading > (coalesced into a single module). It uses coroutines and a trampoline > scheduler, and provides basic async

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-10 Thread dustin
On Sat, Feb 10, 2007 at 03:00:28PM -0800, Brett Cannon wrote: > 1. Write it > 2. Get the community to use it and like it > 3. Make it follow PEP 7/8 style guidelines > 4. Write docs > 5. Write tests > 6. Promise to help maintain the code. Thanks -- I hadn't really planned that far ahead yet. I ex

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-10 Thread Bill Janssen
> If this sounds like a terrible idea, let fly the n00b-seeking missiles. Sounds like a good idea. We did this with ILU, and it helped manage the overhead of threads quite a bit. Brett's comments on "the next step" are right on target. Bill ___ Python

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-10 Thread Aahz
On Sat, Feb 10, 2007, Brett Cannon wrote: > On 2/10/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> >> Is there any interest in including a simple microthreading module in >> Python's standard library? > > Basically, the list of things you need to do (typically, these are > just guidelines

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-10 Thread dustin
On Sun, Feb 11, 2007 at 03:35:29AM +0200, Yotam Rubin wrote: > Why don't you use Stackless? It's very simple, stable, and solves > quite completely the problems in writing concurrect code. That's a great point -- I'm not necessarily producing this to solve a problem I'm having. Rather, I think th

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-10 Thread Steve Holden
[EMAIL PROTECTED] wrote: > On Sun, Feb 11, 2007 at 03:35:29AM +0200, Yotam Rubin wrote: >> Why don't you use Stackless? It's very simple, stable, and solves >> quite completely the problems in writing concurrect code. > > That's a great point -- I'm not necessarily producing this to solve a > prob

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-11 Thread Brett Cannon
On 2/10/07, Steve Holden <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: > > On Sun, Feb 11, 2007 at 03:35:29AM +0200, Yotam Rubin wrote: > >> Why don't you use Stackless? It's very simple, stable, and solves > >> quite completely the problems in writing concurrect code. > > > > That's a grea

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-11 Thread Martin v. Löwis
Brett Cannon schrieb: >> Of course Stackless isn't quite fully integrated with 2.5 (yet). >> >> When did someone last suggest that Stackless become part of the core >> CPython implementation, and why didn't that ever happen? >> > > Don't remember the "when". The "why" has always been that Christi

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-11 Thread Richard Tew
On 2/11/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Brett Cannon schrieb: > >> Of course Stackless isn't quite fully integrated with 2.5 (yet). > >> > >> When did someone last suggest that Stackless become part of the core > >> CPython implementation, and why didn't that ever happen? > >> >

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-11 Thread Martin v. Löwis
Richard Tew schrieb: > If these generator coroutine microthreads went ahead and part > of it was improving the support for asynchronous calls in the > runtime and standard library, this would also be something > which also benefited Stackless Python. Even if they didn't go > ahead I would be inter

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-11 Thread Armin Rigo
Hi Martin, On Sun, Feb 11, 2007 at 07:09:29PM +0100, "Martin v. L?wis" wrote: > > hacks into the core were complicated and he didn't even think > > integration was worth it. > > With emphasis on the latter. Christian never proposed (to my knowledge) > that Stackless should be integrated. Of cours

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-11 Thread Martin v. Löwis
Armin Rigo schrieb: > The history as I remember it is that Christian tried hard to integrate > the first versions of Stackless with CPython, but was turned town by > python-dev. Are there public records of that? As I remember it, Christian never actually submitted a patch for inclusion (at least n

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Richard Tew
On 2/11/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Richard Tew schrieb: > > If these generator coroutine microthreads went ahead and part > > of it was improving the support for asynchronous calls in the > > runtime and standard library, this would also be something > > which also benefited

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Tristan Seligmann
* Richard Tew <[EMAIL PROTECTED]> [2007-02-12 13:46:43 +]: > I currently use Windows. By using asyncore and monkeypatching in a > replacement socket module based on it [1] I can give users of Stackless > socket operations which invisibly block at the microthread level allowing > the scheduler

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Richard Tew
On 2/12/07, Tristan Seligmann <[EMAIL PROTECTED]> wrote: > * Richard Tew <[EMAIL PROTECTED]> [2007-02-12 13:46:43 +]: > > Perhaps there is a better way. And I of course have no concept of > > how this might be done on other platforms. > > Building on an existing framework that does this seems

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Richard Tew
On 2/12/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Armin Rigo schrieb: > > The history as I remember it is that Christian tried hard to integrate > > the first versions of Stackless with CPython, but was turned town by > > python-dev. > > Are there public records of that? As I remember it,

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Talin
Richard Tew wrote: > See "A Bit Of History" > http://svn.python.org/view/stackless/trunk/Stackless/readme.txt I admit that I haven't given Stackless more than a cursory look over, but it seems to me that the real source of its complexity is because its trying to add a fundamental architectural f

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Martin v. Löwis
Richard Tew schrieb: > I can't point you to the posts, but I can point you to something > Christian has written on the subject which may indicate why > it was never actually submitted for inclusion. > > See "A Bit Of History" > http://svn.python.org/view/stackless/trunk/Stackless/readme.txt > > I

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Martin v. Löwis
Richard Tew schrieb: > It depends what you mean by mechanism. I can't go into low level details > because I do not have the necessary experience or knowledge to know > which approach would be best. Indeed, low-level details is what I'm interested in. > I currently use Windows. By using asyncore

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Josiah Carlson
"Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Richard Tew schrieb: > > but at that stage I need to > > poll two different resources for events. In order to avoid this it makes > > sense to stop using asyncore for sockets and to write a new > > replacement socket object based on IO completion por

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Greg Ewing
Richard Tew wrote: > The ideal mechanism at the high level would be expanding asyncore into > a "one-stop shop". Where all these things can be passed into it and > it can do the work to notify of events on the objects in a standard way. +1. This sounds like an excellent idea. It's downright sill

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Greg Ewing
Talin wrote: > What I am getting at is that rather that doing heroic efforts to add > stackless-ness to the current Python code base without changing it, > instead define a migration path which allows Python to eventually > acquire the characteristics of a stackless implementation. I think you

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread dustin
On Tue, Feb 13, 2007 at 12:33:46PM +1300, Greg Ewing wrote: > Richard Tew wrote: > > > The ideal mechanism at the high level would be expanding asyncore into > > a "one-stop shop". Where all these things can be passed into it and > > it can do the work to notify of events on the objects in a stan

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-12 Thread Martin v. Löwis
Greg Ewing schrieb: > The other thing is that, even if some kind of migration > path could be found, Guido et al wouldn't want to follow > that path anyway -- because the end result would be too > convoluted for ordinary people to understand and maintain. That very much depends on what the end res

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread Steve Holden
Martin v. Löwis wrote: > Richard Tew schrieb: >> I can't point you to the posts, but I can point you to something >> Christian has written on the subject which may indicate why >> it was never actually submitted for inclusion. >> >> See "A Bit Of History" >> http://svn.python.org/view/stackless/tru

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread Martin v. Löwis
Steve Holden schrieb: > The only things that concern me are a) whether it could make sense to > add Stackless in bits and pieces and b) whether the BDFL (or even the > developer community en masse) would object in principle, thereby > rendering such efforts useless. I think I need to try again.

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread Richard Tew
On 2/13/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Steve Holden schrieb: > > The only things that concern me are a) whether it could make sense to > > add Stackless in bits and pieces and b) whether the BDFL (or even the > > developer community en masse) would object in principle, thereby >

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread Martin v. Löwis
Richard Tew schrieb: > If there is no Stackless 'hard switching' available (the stack > switching done with assistance of assembler) for the platform it is > being compiled on, then compilation proceeds without Stackless > compiled into Python. The module is not available. The process > of adding

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread glyph
On 12 Feb, 05:11 pm, [EMAIL PROTECTED] wrote: >On 2/12/07, Tristan Seligmann <[EMAIL PROTECTED]> wrote: >> * Richard Tew <[EMAIL PROTECTED]> [2007-02-12 13:46:43 +]: >> > Perhaps there is a better way. And I of course have no concept of >> > how this might be done on other platforms. >> >> Bui

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread Greg Ewing
Martin v. Löwis wrote: > Greg Ewing schrieb: > > the end result would be too > > convoluted for ordinary people to understand and maintain. > > That very much depends on what the end result would be. True. What I should have said is that Guido *believes* the outcome would be too convoluted. Chris

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread Phillip J. Eby
At 08:41 PM 2/13/2007 +, [EMAIL PROTECTED] wrote: > and the microthreading features being discussed here are a trivial hack > somewhere in its mainloop machinery, not an entirely new subsystem that > it should be implemented in terms of. It doesn't even require hacking the mainloop; it can

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread Greg Ewing
Steve Holden wrote: > I know that there's work in progress (and indeed > almost complete) to put Stackless into 2.5 It might be less confusing to rename the current version of Stackless to "microthreads" or something, since it's not really stackless at all. -- Greg __

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread Greg Ewing
[EMAIL PROTECTED] wrote: > I have no problem with other, competing event-driven mechanisms being > developed; The need for different event-driven mechanisms to compete with each other is the very problem that needs to be addressed. If Twisted is designed so that it absolutely *has* to use its ow

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-13 Thread Jean-Paul Calderone
On Wed, 14 Feb 2007 15:20:13 +1300, Greg Ewing <[EMAIL PROTECTED]> wrote: > Greg, productive discussion is not furthered by the unsupported statement of one position or another. Instead of only stating what you believe to be a problem, explain why you believe it is a problem. A sentence like: >

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-14 Thread Richard Tew
On 2/13/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Tristan is correct: this should be a patch against Twisted, or perhaps as a > separate library that could implement a reactor. I think there is some confusion here. I am not writing a competing event driven mechanism. What I was doing wa

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-14 Thread Tristan Seligmann
* Richard Tew <[EMAIL PROTECTED]> [2007-02-14 16:49:03 +]: > I am not writing a competing event driven mechanism. What I was doing > was feeling out whether there was any interest in better support for > asynchronous calls. I interpreted your suggestions as being about enhancing asyncore wit

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-14 Thread Josiah Carlson
Tristan Seligmann <[EMAIL PROTECTED]> wrote: > * Richard Tew <[EMAIL PROTECTED]> [2007-02-14 16:49:03 +]: > > > I am not writing a competing event driven mechanism. What I was doing > > was feeling out whether there was any interest in better support for > > asynchronous calls. > > I interp

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-14 Thread glyph
On 04:49 pm, [EMAIL PROTECTED] wrote: >On 2/13/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >>Tristan is correct: this should be a patch against Twisted, or perhaps as a >>separate library that could implement a reactor. > >I think there is some confusion here. Quite. >I am not writing a comp

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-14 Thread Greg Ewing
Jean-Paul Calderone wrote: > Greg, productive discussion is not furthered by the > unsupported statement of one position or another. Sorry, I'll expand on that a bit. The "problem" I was referring to is the turf wars between all the event-driven frameworks that all want to do things their own way

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-14 Thread Greg Ewing
[EMAIL PROTECTED] wrote: > sounded like > monkeypatching the socket and asyncore modules to provide asynchronous > file I/O based on the platform-specific IOCP API for Windows. I don't think anyone's suggesting that any long-term solution to all this would involve monkeypatching. -- Greg __

Re: [Python-Dev] Trial balloon: microthreads library in stdlib

2007-02-15 Thread Andrew Dalke
On 2/14/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote in response to [EMAIL PROTECTED]: > As far as I can tell, you still haven't even clearly expressed what your > needs are, let alone whether or not Twisted is suitable. In the reply > you're citing, you said that "this" sounded like something