Re: python concurrency proposal
Corey Coughlin <[EMAIL PROTECTED]> writes: > Mike Meyer wrote: >> Calls to methods of a separate object are non-blocking. The calls are >> queued, and executed later. Reading an attribute from a separate >> object is a blocking action. Calling a method with a separate object >> as an argument causes the call to block until the separate arguments >> are all locked. There are rules - enforceable by the compiler - about >> what you can do with separate objects that prevent a lot of the things >> that cause headaches when doing concurrent programming. > Undoubtedly enforced by contracts, Eiffel is nice that way. Actually, no. "separate" is a property of a variable, and certain things you can do with normal variables you can't do with separate variables. The semantics of contracts do enter into things, though. When you pass a separate variable to a function, the function blocks until the separate variable is available, and all the preconditions that mention it are true. This gives you gaurded semaphores. > Yeah, the dynamic nature of python makes a straight translation of > SCOOP pretty problematic. But it's good stuff to consider, > definitely some great ideas in there. I'm not really sure I want a > solution that gets as complicated as SCOOP, but it may not be > something I can avoid. Ah well, some thinking to do. But just out > of curiosity, would you support a PEP with some language additions > to support and simplify concurrency? I'm certainly interested in seeing such a thing done to Python. Whether I'd support a specific PEP would depend on the PEP. http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information. -- http://mail.python.org/mailman/listinfo/python-list
Re: python concurrency proposal
Mike Meyer wrote: > [Rest elided] > > This really has a lot in common with SCOOP. SCOOP makes concurrent > stuff easier, but i'm not sure it fits well with Python. I'll describe > it, on the off chance you may get some ideas from it. See http://archive.eiffel.com/doc/manuals/technology/concurrency/ > for > full details. Eiffel! I should have thought of that. I may have to dig all my old Meyer books out of storage.. > > Like pardef, you use SCOOP by creating objects that are going to run > concurrently. Unlike pardef, the objects are - well, objects, not just > restricted to functions. This has deep implications. For instance, one > of the things that sets SCOOP apart from other concurrency systems > that I know if is that the program doesn't have code to create > "processors" (threads, process, whatever - places for the code to > run). From my reading of your description, this isn't the case for > pardef. Invoking a pardef causes a new processor to be created, and > when the pardef returns, the processor is finished. Well, not quite, the pardef is actually an object, and you can certainly put all kinds of objects inside it if you want to. The pardef object is really more of an interface specification, where the actual implementation of most of the methods is really defined in the base class. The body of the pardef really just defines what happens during a .run() call. The whole 'def' in pardef and the use of returns (and yields) may be kind of misleading, the return (and yield) are really just ways to send info back to the calling scope without having to come up with some wierd way of referencing it. What I'm envisioning can return a single value, or stay resident and return multiple values (through yield or .send calls) or whatever. > > Like pardef, SCOOP separates the specification of the kind of > processor from the rest of the code. It carries things further than > pardef though. You specify a set of processors at build time, and the > support system takes care of getting objects on the right processor. > > Synchronization and protection is all handled via method > invocation. There are no new primitives for locking objects, explicit > synchronization, or communications. Yeah, I've been trying to think of some way to setup channels in a better way. Tagged channels can certainly be used, but method (or method-like) invocations would be more elegant in more than a few situations. On the other hand, I've also been trying to avoid locking people into a fully OO style in order to use concurrency, it would be nice to allow people who don't want to define everything as an object to use parallelism. But it can get ugly that way. Maybe something like a "property" declaration to optionally simplify things, hmm > > Calls to methods of a separate object are non-blocking. The calls are > queued, and executed later. Reading an attribute from a separate > object is a blocking action. Calling a method with a separate object > as an argument causes the call to block until the separate arguments > are all locked. There are rules - enforceable by the compiler - about > what you can do with separate objects that prevent a lot of the things > that cause headaches when doing concurrent programming. Undoubtedly enforced by contracts, Eiffel is nice that way. Some things I do miss from stricter languages. But then again, having to define everything as an object does drive me nuts. > > So instead of "run", you instantiate a separate object. Instead of > ".send", you just invoke a method on the seperate object, and the > arguments get sent to it. Instead of ".receive", you read an attribute > from a separate object. Instead of ."kill", you let the object be > garbage collected (or maybe those aren't the same thing). Return is > missing, because it's an object, not a function. On the other hand, > you could define an attribute that's the "return value", and reading > it fetches that value. > > The problems with Python is that "separate" is really a property of a > variable, not an object. To see the difference, if I declare an object > separate, and it creates an attribute, and I read that attribute, then > that object is separate for me. But in the object that created it, it > isn't. This is one major clash with Python. The other is the > critical distinction that SCOOP places on reading vs. calling a > feature. What does: > > a = Seperate_Object() > x = a.thing > x() > > do with SCOOP semantics? > > Oh well. I think SCOOP does the job you want of "making things > easier", and fits well in an OO language. Maybe you can find good > ideas in it. > > http://mail.python.org/mailman/listinfo/python-list
Re: python concurrency proposal
[EMAIL PROTECTED] writes: > Alright, so I've been following some of the arguments about enhancing > parallelism in python, and I've kind of been struck by how hard things > still are. It seems like what we really need is a more pythonic > approach. I certainly agree, and have thought about it some myself. > pardef (self, , arguments...): > self.send(, , arguments) > self.receive(, arguments) > return arguments > yield arguments [Rest elided] This really has a lot in common with SCOOP. SCOOP makes concurrent stuff easier, but i'm not sure it fits well with Python. I'll describe it, on the off chance you may get some ideas from it. See http://archive.eiffel.com/doc/manuals/technology/concurrency/ > for full details. Like pardef, you use SCOOP by creating objects that are going to run concurrently. Unlike pardef, the objects are - well, objects, not just restricted to functions. This has deep implications. For instance, one of the things that sets SCOOP apart from other concurrency systems that I know if is that the program doesn't have code to create "processors" (threads, process, whatever - places for the code to run). From my reading of your description, this isn't the case for pardef. Invoking a pardef causes a new processor to be created, and when the pardef returns, the processor is finished. Like pardef, SCOOP separates the specification of the kind of processor from the rest of the code. It carries things further than pardef though. You specify a set of processors at build time, and the support system takes care of getting objects on the right processor. Synchronization and protection is all handled via method invocation. There are no new primitives for locking objects, explicit synchronization, or communications. Calls to methods of a separate object are non-blocking. The calls are queued, and executed later. Reading an attribute from a separate object is a blocking action. Calling a method with a separate object as an argument causes the call to block until the separate arguments are all locked. There are rules - enforceable by the compiler - about what you can do with separate objects that prevent a lot of the things that cause headaches when doing concurrent programming. So instead of "run", you instantiate a separate object. Instead of ".send", you just invoke a method on the seperate object, and the arguments get sent to it. Instead of ".receive", you read an attribute from a separate object. Instead of ."kill", you let the object be garbage collected (or maybe those aren't the same thing). Return is missing, because it's an object, not a function. On the other hand, you could define an attribute that's the "return value", and reading it fetches that value. The problems with Python is that "separate" is really a property of a variable, not an object. To see the difference, if I declare an object separate, and it creates an attribute, and I read that attribute, then that object is separate for me. But in the object that created it, it isn't. This is one major clash with Python. The other is the critical distinction that SCOOP places on reading vs. calling a feature. What does: a = Seperate_Object() x = a.thing x() do with SCOOP semantics? Oh well. I think SCOOP does the job you want of "making things easier", and fits well in an OO language. Maybe you can find good ideas in it. http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information. -- http://mail.python.org/mailman/listinfo/python-list
Re: python concurrency proposal
OK, thanks for all this criticism, you've obviously taken some time
here, guess I'll see if I can help clear some of this up
Michael wrote:
>
>
> On the surface of it, what you've described resembles Kamaelia[1] -
> specifically in the way used in the Axon Shell [2]. In other ways it
> differs wildy. I think it's interesting because this (or some of this)
> could be used as syntactic sugar for Kamaelia.
> [1] http://kamaelia.sourceforge.net/Home
> [2] http://kamaelia.sourceforge.net/AxonShell.html
>
> The similarity is this: a pardef appears to be a function that can run,
> and be made to run in parallel with other functions. In Kamaelia we use
> generators to achieve precisely this.
Hey, you got it, great!
> However, given pythonic is a loaded term (beauty is in the eye of the
> beholder), I'd personally say that there's some elements of your syntax
> that suprise me, especially given the (IMO valid) approach of not being
> able to access inside the pardef.
>
> Since there are some strong similarities between what you've written and
> Kamaelia it seems sensible to discuss them.
>
> If you have your pardef:
> pardef (self, , arguments...):
> self.send(, , arguments)
> self.receive(, arguments)
> return arguments
> yield arguments
>
> This maps to this: (in your terms :) )
>
> class ():
> Inboxes = [ , , ] # Explicit is better than implicit
> Outboxes = [ , , ] # There are defaults available...
> def __init__(self, arguments ...):
> // standard initialisation //
> super(, self).__init__()
>
> def main(self):
> // local
> // Initialisation
> while 1:
> do stuff
> yield //some value// (relinquish control to scheduler)
> return
> // We don't have the concept of a result, though this
>would be useful, though we do have something
>similar so this would be doable //
>
> Inboxes and Outboxes are used as declarations to allow the baseclass
> (component) to initialise some datastructures for communications. These
> declarations are actually any iterable, and these days we tend to use
> dictionaries because it simplifies documentation.
Yes, it would map in a more detailed level to a class of some kind with
the methods indicated. Although, I'm imagining that for different base classes, the init and main loops would be implemented in
different ways. Different types of communication channels (shared
memory copies, file based pickling, socket communication, and so on)
would also have different implementations as well, beyond Inboxes and
Outboxes. This could make for some much more complicated base classes.
It also suggests some problems, notably that of coercing different
messages from different types of base classes (i.e., suppose the
standard library os and system classes get implemented as threads, but
you still want to send and receive messages to and from them from
processes and clustered processes).
>
> An example here might control the behaviour of a sprite onscreen. So,
> suppose we have a sprite:
>
>
> Unlike your system, our components (your pardefs), don't know
> about each other and only talk to inboxes AND outboxes - these
> are connected at a higher level by something enclosing them.
>
> Graphline(
> Walker = SimpleSprite(walkerImage, 10,10),
> WalkerLogic = WalkInASquare(10,10,400,400,50),
> linkages = {
>("WalkerLogic", "position") : ("Walker", "position"),
>("WalkerLogic", "orientation") : ("Walker", "rotation"),
> }
> )
>
> This graphline can then be activated or run like any other component.
Yes, I've become a little worried about the casual way I'm pulling the
pardefs together. I did want to keep the definition fairly primitive,
and then try to pull the jobs together using standard python objects
like lists and dictionaries and such, but given that pympi and Kamaelia
use more custom methods, it may be something to think about. It might
simplify the blocking/non-blocking behavior, which I'll get to soon...
> [[ For more complete examples, grab Kamaelia from the website,
>download and run :) ]]
>
> You'll note from the above that clearly there are aspects to it where
> your ideas could be used as syntactic sugar. Some of the benefits (like
> collapsing __init__ and main together) can be gained by using decorators
> in python 2.4.
I've actually been thinking that decorators may be the way to go to set
the base class of the pardef, either that or have it be another class
modifier. Neither seems very ideal, but having it after self also seems
kludgy.
> You'll also note there are some areas where your ideas increase parallel
> safety (such as when sending mutable data structures), at the expense on
> increasing the cost of communication.
Yes, the hope is that the ability to have different base classes that
implement the same interface will allow peop
Re: python concurrency proposal
> Yes. Parallelism certainly deserves attention, and I believe > "amateurs" are likely to help in the breakthroughs to come. I > further suspect, though, that they'll be amateurs who benefit > from knowledge of existing research into the range of documented > concurrency concepts, including CSPs, tasks, guarded methods, > microthreads, weightless threads, chords, co-routines, and so on. Yes, there are lots of different concepts, even in python, there's pympi (as was mentioned), the standard python thread library, the subprocess library, generators, microthreads and stackless, not to mention Candygram, PyLinda, ATOM, Kamaelia (get to that in a minute), and other things you can search for on the web. My motivation here is just to see if I can find some lowest common denominator, to try to simplify this stuff to the point where the whole concept is a little easier to use, and the plumbing can be hidden away somewhere so "amateurs" don't have to worry about it (too much) if they don't want to. Now to be more specific, there does seem to be a lot of work with generators to set up concurrency, and that's fine, but it does seem like it takes a bunch of scaffolding and a different way of looking at things, and it's not really obvious to me how it can scale up on multiple processor systems with the GIL still in place. Now I'm not sure that this will be the answer to all the problems, but breaking the global address space and making it easy to break up jobs into small communicating chunks seems like it would be a good way to go. Or maybe I'm missing something? Is there anything you'd care to elaborate on? -- http://mail.python.org/mailman/listinfo/python-list
Re: python concurrency proposal
Hey, some responses, let's see... Peter Tillotson wrote: > I'd really like to see a concurrency system come into python based on > theories such as Communicating Sequential Processes (CSP) or its > derivatives lambda or pi calculus. These provide an analytic framework > for developing multi thread / process apps. CSP like concurrency is one > of the hidden gems in the Java Tiger release (java.util.concurrency). > The advantages of the analytic framework is that they minimise livelock, > deadlock and facilitate debugging. Yes, a CSP-like system would be kind of nice. Of course, to really pull it off, you'd probably need some kind of primitive to represent a simple process. Which is kind of what I'm proposing. It's kind of a primitive version, but an object to easily represent processes and communication channels would be a big help, I'd imagine. Once a primitive is in place, I believe it would be fairly easy to build up a full set of CSP-ish primitives to help assemble full systems. > I'm no expert on the theory but i've developed under these frameworks > and found them a more reliable way of developing distributed agent systems. > > You may also be interested in looking at > http://sourceforge.net/projects/pympi Ah yes, pympi, that's good stuff. It does suggest that I might need to add blocking and non-blocking version of send and receive, there might be a need for that. Especially the non-blocking receive, that looks very handy. And maybe a .status for polling the status of running jobs. It does go a lot further than I do with this proposal, it adds all that stuff for collecting the jobs, running them as a group, gathering results, and so on. This proposal could probably use an extra library to provide a bunch of those type of operations, but I wanted to start a little bit small here with just the proposal of a process primitive. That seems like it would be a good first step. Of course, what I'm really hoping for is that at some point you could do something like this: import mpi pardef vecadd(self, mpi.mpiproc, ...) and so on. I'm not really all that concerned about the communication channels and process distributions work, I suspect that many people will want to try different methods like MPI, or the standard pthread wrapper with some kind of standard queue communications channel, or pyro, maybe even Kamaelia. I'm just proposing the primitive for it. So, if there's anything else you'd like to see it work like, be sure to let me know. Thanks for the input! - Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: python concurrency proposal
[EMAIL PROTECTED] wrote:
> Alright, so I've been following some of the arguments about enhancing
> parallelism in python, and I've kind of been struck by how hard things
> still are. It seems like what we really need is a more pythonic
> approach.
[... major snippage ...]
> OK? So what do you all think?
On the surface of it, what you've described resembles Kamaelia[1] -
specifically in the way used in the Axon Shell [2]. In other ways it
differs wildy. I think it's interesting because this (or some of this)
could be used as syntactic sugar for Kamaelia.
[1] http://kamaelia.sourceforge.net/Home
[2] http://kamaelia.sourceforge.net/AxonShell.html
The similarity is this: a pardef appears to be a function that can run,
and be made to run in parallel with other functions. In Kamaelia we use
generators to achieve precisely this.
However, given pythonic is a loaded term (beauty is in the eye of the
beholder), I'd personally say that there's some elements of your syntax
that suprise me, especially given the (IMO valid) approach of not being
able to access inside the pardef.
Since there are some strong similarities between what you've written and
Kamaelia it seems sensible to discuss them.
If you have your pardef:
pardef (self, , arguments...):
self.send(, , arguments)
self.receive(, arguments)
return arguments
yield arguments
This maps to this: (in your terms :) )
class ():
Inboxes = [ , , ] # Explicit is better than implicit
Outboxes = [ , , ] # There are defaults available...
def __init__(self, arguments ...):
// standard initialisation //
super(, self).__init__()
def main(self):
// local
// Initialisation
while 1:
do stuff
yield //some value// (relinquish control to scheduler)
return
// We don't have the concept of a result, though this
would be useful, though we do have something
similar so this would be doable //
Inboxes and Outboxes are used as declarations to allow the baseclass
(component) to initialise some datastructures for communications. These
declarations are actually any iterable, and these days we tend to use
dictionaries because it simplifies documentation.
An example here might control the behaviour of a sprite onscreen. So,
suppose we have a sprite:
(This is a simplified example of an existing component)
class SimpleSprite(component, pygame.sprite.Sprite):
Inboxes = { "position" : "Expect to receive (x:int,y:int) values",
"rotation" : "Expect to an int in range 0..359",
}
def __init__(self, image, pos):
pygame.sprite.Sprite.__init__(self)
component.__init__(self) # Can't use super here because we inherit
# from pygame.sprite.Sprite as well
self.original = image
self.image = image # self.image is displayed by pygame
self.pos = pos
def main(self):
pos = self.pos
rotation = 0
image = self.image
while 1:
self.image = self.original
if not self.anyReady():
self.pause() # Ask scheduler not to run us until we
# receive data on an inbox
yield 1
if self.dataReady("position"):
pos = self.recv("position")
if self.dataReady("rotation"):
angle = self.recv("rotation")
self.image = pygame.transform.rotate(self.image, angle)
self.rect.center = pos
yield 1
We could then have some game logic that sends out information that controls
this sprite over time:
class WalkInASquare(component):
Outboxes = {
"position" : "Sends out an (x:int, y:int) pair",
"orientation" : "Sends out an angular orientation",
}
def __init__(self, left, top, right, bottom, steps):
# We assume here that left < right and top < bottom
# In something real you'd want to check or enforce this
# (eg asserts or swapping)
self.left = left
self.top = top
self.right = right
self.bottom = bottom
def main(self):
# You'd ideally want to check for shutdown messages as well
x = self.left
y = self.top
while 1: # do this forever, normally we'd shutdown
# Walk right
self.send(90, "orientation")
for i in xrange(self.left, self.right, steps):
self.send((i,y), "position")
yield 1
x = right
# Walk down
self.send(180, "orientation")
for i in xrange(self.top, self.bottom, steps):
self.send((x,i), "position")
yield 1
y = self.bottom
# Walk left
self.send(270, "orientation")
for i in xrange(self.right, self.left, -steps):
self.send((i,y), "position")
yield 1
x = self.left
Re: python concurrency proposal
In article <[EMAIL PROTECTED]>, Peter Tillotson <[EMAIL PROTECTED]> wrote: >I'd really like to see a concurrency system come into python based on >theories such as Communicating Sequential Processes (CSP) or its >derivatives lambda or pi calculus. These provide an analytic framework >for developing multi thread / process apps. CSP like concurrency is one >of the hidden gems in the Java Tiger release (java.util.concurrency). >The advantages of the analytic framework is that they minimise livelock, >deadlock and facilitate debugging. > >I'm no expert on the theory but i've developed under these frameworks >and found them a more reliable way of developing distributed agent systems. > >You may also be interested in looking at >http://sourceforge.net/projects/pympi . . . Yes. Parallelism certainly deserves attention, and I believe "amateurs" are likely to help in the breakthroughs to come. I further suspect, though, that they'll be amateurs who benefit from knowledge of existing research into the range of documented concurrency concepts, including CSPs, tasks, guarded methods, microthreads, weightless threads, chords, co-routines, and so on. -- http://mail.python.org/mailman/listinfo/python-list
Re: python concurrency proposal
I'd really like to see a concurrency system come into python based on
theories such as Communicating Sequential Processes (CSP) or its
derivatives lambda or pi calculus. These provide an analytic framework
for developing multi thread / process apps. CSP like concurrency is one
of the hidden gems in the Java Tiger release (java.util.concurrency).
The advantages of the analytic framework is that they minimise livelock,
deadlock and facilitate debugging.
I'm no expert on the theory but i've developed under these frameworks
and found them a more reliable way of developing distributed agent systems.
You may also be interested in looking at
http://sourceforge.net/projects/pympi
p
[EMAIL PROTECTED] wrote:
> Alright, so I've been following some of the arguments about enhancing
> parallelism in python, and I've kind of been struck by how hard things
> still are. It seems like what we really need is a more pythonic
> approach. One thing I've been seeing suggested a lot lately is that
> running jobs in separate processes, to make it easy to use the latest
> multiprocessor machines. Makes a lot of sense to me, those processors
> are going to be more and more popular as time goes on. But it would
> also be nice if it could also turn into a way to make standard
> threading a little easier and trouble free. But I'm not seeing an easy
> way to make it possible with the current constraints of the language,
> so it seems like we're going to need some kind of language improvement.
> Thinking of it from that perspective, I started thinking about how it
> would be easy to deal with in a more idealized sense. It would be nice
> to abstract out the concept of running something in parallel to
> something that can be easily customized, is flexible enough to use in a
> variety of concepts, and is resonably hard to screw up and fairly easy
> to use. Perhaps a new built-in type might be just the trick. Consider
> a new suite:
>
> pardef (self, , arguments...):
> self.send(, , arguments)
> self.receive(, arguments)
> return arguments
> yield arguments
>
> so the object would then be something you can create an instance of,
> and set up like a normal object, and it would have other interface
> functions as well. Consider your basic vector add operation:
>
> import concurrent
> import array
>
> pardef vecadd(self, concurrent.subprocess, veca, vecb, arrtype):
> import array
> output = array.array(arrtype)
> for a,b in zip(veca, vecb):
> output.append( a + b)
> return output
>
> a = array.array('d')
> b = array.array('d')
> for i in range(1000):
> a.append(float(i))
> b.append(float(i))
>
> h1 = vecadd(a[:500], b[:500], 'd')
> h2 = vecadd()
> h2.veca = a[500:]
> h2.vecb = b[500:]
> h2.arrtype = 'd'
>
> h1.run()
> h2.run()
> c = h1.result + h2.result
>
> You can see a few things in this example. First off, you'll notice
> that vecadd has the import for array inside it. One of the most
> important things about the pardef is that it must not inherit anything
> from the global scope, all variable passing must occur through either
> the arguments or .receive statements. You'll also notice that it's
> possible to set the arguments like instance values. This isn't as
> important in this case, but it could be very useful for setting
> arguments for other pardefs. Take this example of your basic SIMD-ish
> diffusion simulation:
>
> import concurrent
>
> pardef vecadd(self, concurrent.subprocess, right, left, up, down,
> initval):
> current = initval
> maxruns = 100
> updef = not (isinstance(up, int) or isintance(up, float))
> downdef = not (isinstance(down, int) or isintance(down, float))
> rightdef = not (isinstance(right, int) or isintance(right, float))
> leftdef = not (isinstance(left, int) or isintance(left, float))
> for i in range(maxruns):
> if updef:
> upval = self.receive(up, 'up')
> else:
> upval = up
> if downdef:
> downval = self.receive(down, 'down')
> else:
> downval = down
> if rightdef:
> rightval = self.receive(right, 'right')
> else:
> rightval = right
> if leftdef:
> leftval = self.receive(left, 'left')
> else:
> leftval = left
> current = (upval + downval + leftval + rightval) / 4
> if updef:
> up.send('down', current)
> if downdef:
> down.send('up', current)
> if rightdef:
> right.send('left', current)
> if leftdef:
> left.send('right', current)
> return current
>
> diffgrid = {}
> for x, y in zip(range(10), range(10)):
>
python concurrency proposal
Alright, so I've been following some of the arguments about enhancing
parallelism in python, and I've kind of been struck by how hard things
still are. It seems like what we really need is a more pythonic
approach. One thing I've been seeing suggested a lot lately is that
running jobs in separate processes, to make it easy to use the latest
multiprocessor machines. Makes a lot of sense to me, those processors
are going to be more and more popular as time goes on. But it would
also be nice if it could also turn into a way to make standard
threading a little easier and trouble free. But I'm not seeing an easy
way to make it possible with the current constraints of the language,
so it seems like we're going to need some kind of language improvement.
Thinking of it from that perspective, I started thinking about how it
would be easy to deal with in a more idealized sense. It would be nice
to abstract out the concept of running something in parallel to
something that can be easily customized, is flexible enough to use in a
variety of concepts, and is resonably hard to screw up and fairly easy
to use. Perhaps a new built-in type might be just the trick. Consider
a new suite:
pardef (self, , arguments...):
self.send(, , arguments)
self.receive(, arguments)
return arguments
yield arguments
so the object would then be something you can create an instance of,
and set up like a normal object, and it would have other interface
functions as well. Consider your basic vector add operation:
import concurrent
import array
pardef vecadd(self, concurrent.subprocess, veca, vecb, arrtype):
import array
output = array.array(arrtype)
for a,b in zip(veca, vecb):
output.append( a + b)
return output
a = array.array('d')
b = array.array('d')
for i in range(1000):
a.append(float(i))
b.append(float(i))
h1 = vecadd(a[:500], b[:500], 'd')
h2 = vecadd()
h2.veca = a[500:]
h2.vecb = b[500:]
h2.arrtype = 'd'
h1.run()
h2.run()
c = h1.result + h2.result
You can see a few things in this example. First off, you'll notice
that vecadd has the import for array inside it. One of the most
important things about the pardef is that it must not inherit anything
from the global scope, all variable passing must occur through either
the arguments or .receive statements. You'll also notice that it's
possible to set the arguments like instance values. This isn't as
important in this case, but it could be very useful for setting
arguments for other pardefs. Take this example of your basic SIMD-ish
diffusion simulation:
import concurrent
pardef vecadd(self, concurrent.subprocess, right, left, up, down,
initval):
current = initval
maxruns = 100
updef = not (isinstance(up, int) or isintance(up, float))
downdef = not (isinstance(down, int) or isintance(down, float))
rightdef = not (isinstance(right, int) or isintance(right, float))
leftdef = not (isinstance(left, int) or isintance(left, float))
for i in range(maxruns):
if updef:
upval = self.receive(up, 'up')
else:
upval = up
if downdef:
downval = self.receive(down, 'down')
else:
downval = down
if rightdef:
rightval = self.receive(right, 'right')
else:
rightval = right
if leftdef:
leftval = self.receive(left, 'left')
else:
leftval = left
current = (upval + downval + leftval + rightval) / 4
if updef:
up.send('down', current)
if downdef:
down.send('up', current)
if rightdef:
right.send('left', current)
if leftdef:
left.send('right', current)
return current
diffgrid = {}
for x, y in zip(range(10), range(10)):
diffgrid[(x, y)] = vecadd()
for x, y in zip(range(10), range(10)):
gridpt = diffgrid[(x, y)]
gridpt.initval = 50.0
if x == 0:
gridpt.left = 75.0
else:
gridpt.left = diffgrid[(x-1, y)]
if x == 10:
gridpt.right = 50.0
else:
gridpt.right = diffgrid[(x+1, y)]
if y == 0:
gridpt.down = 0.0
else:
gridpt.down = diffgrid[(x, y-1)]
if y == 10:
gridpt.up = 100.0
else:
gridpt.up = diffgrid[(x, y+1)]
for coord in diffgrid:
diffgrid[coord].run()
for x, y in zip(range(10), range(10)):
print '(%i, %i) = %f' % (x, y, diffgrid[(x,y)].return())
Now sure, this example is a little contrived, but it shows the utility
of allowing
