Yes and no: - State machine generators are not "true" coroutines in that they are single-level only. Yes, you can nest them, but it's rather tricky to do that efficiently and transparently to the programmer. - In order to use a generator, the caller has to be aware that he is dealing with one. Generators tend to "infect" the whole callstack. I mean look at C# - once you have async function at the leaf level, you need to make its' callers async as well, at least up to the level where you spawn a bunch of them and then can afford to block your thread. - Async I/O using generators requires different API than blocking I/O. With greenlets this *might *be avoidable. - Greenlets can accommodate even foreign frames being on the stack (say, a call into C library, which calls back into Rust), event though this is not optimal for greenlet switching performance.
That said, generators also have their uses. For example, greenlets would be too heavy to implement container iterators on top of them - because of the context switch overhead. Perhaps Rust needs both. I would like to hear what others think. PS: Incidentally, I am also brewing a proposal for how that Python-style generators could be done in Rust, but it isn't quite ready yet... On Wed, Nov 13, 2013 at 10:44 AM, Erick Tryzelaar <erick.tryzel...@gmail.com > wrote: > This approach is not all that different different from my plan to > implement generators: > > https://github.com/mozilla/rust/issues/7746 > > My current thinking is that a generator is really the combination between > storing the local variables in a structure instead of on the stack, and a > state machine to jump between each of the generator steps. Without thinking > too deeply into it, I suspect that the main difference between a generator > and a coroutine/greenlet is whether or not that structure is stored on the > stack or on the heap. I'm not quite sure how we'll handle passing data into > a coroutine, maybe we can reserve a slot in the environment structure for > passed in values, and the compiler can guarantee no one accesses that value > until it's been set. Unfortunately this'll requires a bunch of scary > compiler magic to get it all to work. But I don't think it's impossible. > > > > On Wed, Nov 13, 2013 at 10:20 AM, Daniel Micay <danielmi...@gmail.com>wrote: > >> On Wed, Nov 13, 2013 at 3:02 AM, Vadim <vadi...@gmail.com> wrote: >> > Hi, >> > I would like to float a proposal (or three :-), regarding "greenlets" in >> > Rust. For those unfamiliar with greenlets, they are a tool for writing >> > concurrent code, similar to Rust's tasks, but much more light-weight in >> > terms of memory consumption (especially now that segmented stacks are no >> > more). >> > >> > I think there are some scenarios where low memory consumption per-task >> is >> > still important, 64-bit address spaces notwithstanding. A typical one >> > would be a pub-sub server, which needs to maintain a massive number of >> > simple I/O workflows, where I/O channels are idle most of the time. >> > >> > So here we go (in the order of increasing craziness): >> > >> > 1. Recently I've learned how Python greenlets are implemented. I >> believe >> > that the same approach could work in Rust: >> > >> > Basically, greenlets are spawned using the same stack as the parent >> > greenlet, just like a normal function call. When a greenlet is >> suspended, >> > it copies the portion of the stack used up since its' spawning to the >> heap. >> > When one is re-activated, the saved memory is copied back where it came >> from >> > (having first saved stack of the previous active greenlet,- if they >> > overlap). >> > >> > Since greenlets don't need to save "red zone" of the stack, the amount >> of >> > data per instance is precisely what is actually used. >> > >> > There are also downsides, of course: >> > - greenlets are bound to the thread that spawned them, >> > - two memcpy's are needed when switching between them. >> > >> > In the case of Python, though, there's one further optimization: since >> > Python's stack frames live on the heap, in most cases, there nothing on >> the >> > hardware stack that a greenlet needs saving! As a bonus, it can now be >> > resumed at any stack position, so no saving of previous greenlet's >> stack is >> > needed. The only time when a full save occurs is when there are foreign >> > stack frames on the stack. >> > >> > >> > 2. Well, can Rust do the same? What if we came up with an attribute, >> say, >> > [#stackless], which causes a function to allocate it's stack frame on >> the >> > heap and put all local vars there? The only things on the actual >> hardware >> > stack would then be the function's arguments, the return address, the >> saved >> > base pointer and the pointer to that heap-alllocated frame. With the >> > exception of base pointers, all these things are position-independent, I >> > believe. And base pointer chain can be easily fixed up if/when stack >> is >> > moved. >> > >> > So if we had that, and the whole greenlet's stack consisted of such >> > functions, and there was a way for the "switch_to_greenlet()" function >> to >> > detect that, then such greenlet's stack would be relocatable and could >> be >> > resumed at any position in the thread's stack (or even in another >> thread!) >> > with minimal memory copying, just like in Python. >> > >> > Of course, the [#stackless] functions would be slower than the normal >> ones, >> > but in the scenario I've outlined in the beginning, it shouldn't be a >> > problem. >> > >> > >> > 3. Unfortunately, in order for the above scheme to work, all I/O >> methods, >> > (which are typically where yields happen), would need to be marked as >> > [#stackless]... This would affect the performance of "normal" code >> using >> > the same API, which is undesirable. >> > >> > Okay, but usually there are not that many things that point into the >> stack >> > in a typical program. I can think of only three things: references to >> > stack-allocated buffers, base pointer chains and references to >> > caller-allocated return values. >> > - The first one can be lived without - just allocate buffers on the >> heap. >> > - The second one - see above. >> > - The last one is more tricky, but for the sake of argument, let's >> assume >> > that we restricted function signatures such that only register-allocated >> > types can be returned. >> > >> > Let's say we came up with a way to mark up functions that may yield to >> > another greenlet, and also with a way to prohibit taking address of >> > stack-allocated variables for the duration of calls to yielding >> functions. >> > These restrictions would be annoying, but not overly so, as long as you >> had >> > to obey them only in functions that are intended to be run in a >> greenlet. >> > On the plus side, the hardware stack contents would now be relocatable. >> > >> > In this setup, everything could proceed as usual, using the hardware >> stack, >> > until execution came to a blocking I/O call. At that point, the >> scheduler >> > would check if running inside a greenlet, copy greenlet's stack to the >> heap >> > and switch off to another greenlet. Otherwise, it would do the same >> thing >> > it does now, i.e. switch tasks. >> > >> > >> > So, what do you say, rustafarians? Does any of that make any sense? >> > >> > Vadim >> >> Rust has no way to move a stack/value and update the pointers to it. A >> fundamental property of borrowed pointers is that they're just >> integers in memory at runtime without any boxing or type information. >> _______________________________________________ >> Rust-dev mailing list >> Rust-dev@mozilla.org >> https://mail.mozilla.org/listinfo/rust-dev >> > >
_______________________________________________ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
