>
> > Of course, this is the penultimate reason that going multi-threaded in
> > python is insane... not only do you get the opportunity to learn all
> > about synchronization and thread management, you also enjoy non-
> > deterministic bugs which only take days or weeks to solve whereas more
> > conventional logic bugs take many many minutes, sometimes even hours.
> > (From a Keynesian economics perspective, going multi-threaded is
> > justified just by the added work)
>
> My point was that there is benifit going mutli-threaded: you don't have to
> manually set up callbacks/fork every time you might block. Whether this
> simplification is worth the other complexities depends on the program at
> hand (and probably the programer as well).
I disagree. Once Twisted is in place (and I'm not a paid Twisted
spokesman) you'd be very surprised at how easy it becomes...
particularly given the capabilities of python.
A trivial (and likely syntactically incorrect) example
def doThings():
myDeferred = someting_that_blocks();
def doThingOne(result1):
print("doing ThingOne", result1)
return sqrt(result1)
myDeferred.addCallback(doThingOne)
def doThingTwo(result2):
print("doing ThingTwo", result2)
return another_thing_that_blocks(result2)
myDeferred.addCallback(doThingTwo)
def doThingThree(result3):
print("doing ThingThree", result3)
return result3*3.1415927
myDeferred.addCallback(doThingThree)
return myDeferred
Thats it. Note, that the implementation of doThingTwo uses something
that blocks and hence gets a deferred returned... but you don't worry
that part because the deferred return mechanism can take results or
deferreds. Once you start playing around, you find that the code is
very straightforward even if the callback chain gets complex... the
chain itself is managed by twisted... (the deferred mechanism actually
maintains two chains, one for exceptions and one for regular
results... start dealing with those issues and you quickly see why you
want a well thought out and robust infrastructure underlying all this)
All this magic is due to one key addition to the conventional notion
of callbacks... the deferred. It maintains state (btw, the deferred
can also take additional parameters to be used in the callback, hence
maintaining additional state should your application require
information available when the chain is constructed)
So, the code can be made to look very linear and intuitive. By
exploiting lambdas and closures, things become even cleaner but it
takes a slight bit more explanation.
As to the entire block of code required to initialize all this magic,
there is almost none and I reference a recent post where I list and
include a number of tutorials an introductions.
I claim, knowing quite about threading and using them heavily every
day in C++, that asynchronous programming is very simple relative to
threads. My C++ code uses both... and threads have their place. But
that place rarely exists in Python.
Finally, and this is critically important, virtually all of this can
be hidden from the vast majority of sage developers. Typically,
extensions to sage are compute bound. They're gonna get invoked and
return a result. Only folks who are doing IO or want timer callbacks
need touch this stuff. At that point, I claim you're already in need
of something and the system-wide architectural issues to support your
needs are likely much more straightforward with an asynchronous
core...
Which, by the way, is why virtually all systems of any degree of
complexity, especially distributed systems, have embraced this
approach... even as a front end to thread pools (called futures
typically... but they go by other names)
The idea that code is "cleaner" because a thread is allowed to block
misses the massive issues which become system-wide in larger codes...
and as sage is a huge code, this is even more true.
-glenn
>
> - Robert
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel-unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://www.sagemath.org
-~----------~----~----~----~------~----~------~--~---
