Thanks for the comments, I'll pass this along. On Mon, Jun 28, 2010 at 2:04 PM, Dino Viehland <[email protected]> wrote:
> I think that’s a fair summary and ties into what I was saying about cross > function switching. So basically this means every method needs to be > re-written to support this including dispatching calls via returning back to > an outer loop and then dispatching to the called object with the new > arguments (including the perf hit that would come with that). I think that > can certainly be done with a DLR tree AST re-write and replacing normal > invocation with this trampoline mechanism. Obviously it’ll come with a > significant perf hit as we need to allocate memory for local variables, > calls with be slower as they need to return and call, etc… > > > > But if you can get the tree re-write going to make the code entirely > stackless from there you could opt into to the tree re-write dynamically. > That could be done by doing a simpler tree re-write to track where you are > in the function and then throwing a .NET exception when the 1st switch > occurs. There would need to be try/catch/rethrow blocks to save all of the > relevant state (which would need to be in locals) and then all of the > functions on the stack would need to be re-written with the full stackless > re-write. > > > > I do agree that it sounds more difficult then what is going on with the > CPython implementation. > > > > *From:* [email protected] [mailto: > [email protected]] *On Behalf Of *Tristan Zajonc > *Sent:* Friday, June 25, 2010 4:45 AM > *To:* Discussion of IronPython > *Subject:* Re: [IronPython] Implementing Greenlets API in IronPython > > > > I have been trying to hire somebody to implement the greenlet extension > module, which would enable modules like eventlet and gevent to work on > IronPython. The person I currently have looking into it, however, isn't > hopeful. A condensed version of his perspective is below. Perhaps the > IronPython list (Dino?) has an idea for a solution? > > > > == Response to Greenlets on IronPython == > > > > I looked at various solutions but none of them seem to arrive at the > correct solution. I did try generating the DLR using AST again to make the > function stackless but the limitation is with python and .NET. > > > > Let me explain the issue. > > 1) .NET lanugages (C#. VB.NET etc.) are strongly typed. In other words, if > I get a variable I know what its type is at compile time. IronPython > actually compiles the functions into CLR and runs them on the fly. A > function is compiled once and can be executed again and again. One must know > that the function needs to be stackless at compile time and a function > cannot be made stackless at run time. > > 2) Python is a typeless language. A variable can assume any type at any > point in execution at run-time. Now consider the following code in Python > using the greenlet API: > > a = greenlet(foo) > > def abc(): > print "X" > a.switch() > print "Y" > > Although variable a is defined, it does not have a strong type and during > execution, a can be assigned a different object which can also implement the > switch function and the code will execute fine. > > As mentioned above, IronPython needs to know at compile time that the > function has to be made stackless. In python since a is not typed, a can be > greenlet in one execution of abc and any other object in the second call. > Also, there is no way to determine at compile that the function abc needs to > be made stackless since it needs to switch since the type of a is unknown > till it is actually called at runtime. > > Hence is it next to impossible to implement the switch because of the above > implementation. > > > CPython dos not compile the functions and does not use the yield keyword > (which is not avilable in C/C++ anyways) to do it. It does this by > using context switching which is built into python and so the API is > fairly straightforward in CPython. CPython basically has the stack of all > previous functions calls and saves the stack so that it can resume later > from that point. It needs to store the context and uses the inherent context > switching built into Python to do it. > > > > == End Response == > > > > Ideas? > > > > Tristan > > > > On Mon, May 31, 2010 at 1:28 PM, Dino Viehland <[email protected]> > wrote: > > You’ll want to first look at GeneratorRewriter.cs - it’s responsible for > taking a DLR AST and re-writing it so that it can re-start at an arbitrary > location and so that it’s stackless. There’s one of these for debugging and > one that’s used for IronPython’s generators, you can probably adapt either > one. You can get the AST to re-write from the function code object but > that’s not public right now (we’ll make it public if anyone has a compelling > scenario to do so – this could be it but you’ll probably need more public > surface area as well). > > > > From there I think you just need to worry about having to save multiple > frames (which I get the impression is supported from the docs – eg f() calls > g() and then g switches). For that you’ll probably want to re-write all of > the dynamic expressions in the generator (these show up in several forms > though based upon how we’re generating code- there’s a > ReduciableDynamicExpression and some PythonDynamicExpression’s floating > around now). If greenlets only works across calls then you just need to > intercept the ones which are using invoke binders. If you can have a > descriptor who’s __get__ or __set__ switches then you’d also need to handle > gets/sets/etc… If you can do something like import a module which hasn’t > been loaded yet and then have the top-level module code switch then you’ll > need to start getting really creative J You’ll then presumably need to > change these into using some trampoline to do the actual invocation. > > > > Hopefully that will get you started – I’m sure there’ll be lots of little > issues so if you have questions feel free to ask. If you get it all working > we can figure out what exactly we need to make public and how we should > expose that so you can do this as an extension – but I’m sure you’ll > initially be using a bunch of internal APIs. > > > > > > *From:* [email protected] [mailto: > [email protected]] *On Behalf Of *Tristan Zajonc > *Sent:* Sunday, May 30, 2010 6:59 PM > *To:* Discussion of IronPython > *Subject:* [IronPython] Implementing Greenlets API in IronPython > > > > Hi - > > > > The greenlets C-extension module (http://packages.python.org/greenlet/) > provides a very basic api that is used by eventlet and gevent to > provide asynchronous programming constructs. It would be nice to have a > reasonably performant version of the greenlet API in IronPython. > > > > Before I start playing around with possibilities, is there an obvious > approach to implement the greenlets API in IronPython? > > > > Thanks > > Tristan > > > _______________________________________________ > Users mailing list > [email protected] > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com > > > > _______________________________________________ > Users mailing list > [email protected] > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com > >
_______________________________________________ Users mailing list [email protected] http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
