Paul Rubin wrote: ... > I don't see how generators substitute for microthreads. In your example > from another post:
I've done some digging and found what you mean by microthreads - specifically I suspect you're referring to the microthreads package for stackless? (I tend to view an activated generator as having a thread of control, and since it's not a true thread, but is similar, I tend to view that as a microthread. However your term and mine don't co-incide, and it appears to cause confusion, so I'll switch my definition to match yours, given the microthreads package, etc) You're right, generators aren't a substitue for microthreads. However I do see them as being a useful alternative to microthreads. Indeed the fact that you're limited to a single stack frame I think has actually helped our architecture. The reason I say this is because it naturally encourages small components which are highly focussed in what they do. For example, when I was originally looking at how to wrap network handling up, it was logical to want to do this: [ writing something probably implementable using greenlets, but definitely pseudocode ] @Nestedgenerator def runProtocol(...) while: data = get_data_from_connection( ... ) # Assume non-blocking socket def get_data_from_connection(...) try: data = sock.recv() return data except ... : Yield(WaitSocketDataReady(sock)) except ... : return failure Of something - you get the idea (the above code is naff, but that's because it's late here) - the operation that would block normally you yield inside until given a message. The thing about this is that we wouldn't have resulted in the structure we do have - which is to have components for dealing with connected sockets, listening sockets and so on. We've been able to reuse the connected socket code between systems much more cleanly that we would have done (I suspect) than if we'd been able to nest yields (as I once asked about here) or have true co-routines. At some point it would be interesing to rewrite our entire system based on greenlets and see if that works out with more or less reuse. (And more or less ability to make code more parallel or not) [re-arranging order slightly of comments ] > class encoder(component): > def __init__(self, **args): > self.encoder = unbreakable_encryption.encoder(**args) > def main(self): > while 1: > if self.dataReady("inbox"): > data = self.recv("inbox") > encoded = self.encoder.encode(data) > self.send(encoded, "outbox") > yield 1 > ... > In that particular example, the yield is only at the end, so the > generator isn't doing anything that an ordinary function closure > couldn't: > > def main(self): > def run_event(): > if self.dataReady("inbox"): > data = self.recv("inbox") > encoded = self.encoder.encode(data) > self.send(encoded, "outbox") > return run_event Indeed, in particular we can currently rewrite that particular example as: class encoder(component): def __init__(self, **args): self.encoder = unbreakable_encryption.encoder(**args) def mainLoop(self): if self.dataReady("inbox"): data = self.recv("inbox") encoded = self.encoder.encode(data) self.send(encoded, "outbox") return 1 That's a bad example though. A more useful example is probably something more like this: class Multicast_sender(Axon.Component.component): def __init__(self, local_addr, local_port, remote_addr, remote_port): super(Multicast_sender, self).__init__() self.local_addr = local_addr self.local_port = local_port self.remote_addr = remote_addr self.remote_port = remote_port def main(self): sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) sock.bind((self.local_addr,self.local_port)) sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 10) while 1: if self.dataReady("inbox"): data = self.recv() l = sock.sendto(data, (self.remote_addr,self.remote_port) ); yield 1 With a bit of fun with decorators, that can actually be collapsed into something more like: @component def Multicast_sender(self, local_addr, local_port, remote_addr, remote_port): sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) sock.bind((self.local_addr,self.local_port)) sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 10) while 1: if self.dataReady("inbox"): data = self.recv() l = sock.sendto(data, (self.remote_addr,self.remote_port) ); yield 1 > You've got the "main" method creating a generator that has its own > event loop that yields after each event it processes. Notice the kludge > > if self.dataReady("inbox"): > data = self.recv("inbox") > > instead of just saying something like: > > data = self.get_an_event("inbox") > > where .get_an_event "blocks" (i.e. yields) if no event is pending. > The reason for that is that Python generators aren't really coroutines > and you can't yield except from the top level function in the generator. > > > Now instead of calling .next on a generator every time you want to let > your microthread run, just call the run_event function that main has > returned. However, I suppose there's times when you'd want to read an > event, do something with it, yield, read another event, and do > something different with it, before looping. In that case you can use > yields in different parts of that state machine. But it's not that > big a deal; you could just use multiple functions otherwise. > > All in all, maybe I'm missing something but I don't see generators as > being that much help here. With first-class continuations like > Stackless used to have, the story would be different, of course. -- http://mail.python.org/mailman/listinfo/python-list