Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
Jim Peters wrote: > > Threads are allocated on multiple cores nowadays and you can use > > lock-free data structures when possible. Still you didn't provide > > any benchmark, only thoughts. > > We are both putting forward "only thoughts". > > To prove my point I will have to implement simple coroutines in Vala > using low-level C stuff, i.e. setcontext/makecontext/etc. I guess a > Coroutine class could be useful for something someday. Give me a > moment ... Actually, even though Serge was starting off two threads in his example, and the GIO docs claim that async stuff uses GThread, the async code from Generator doesn't use them. It all runs in a single thread. I just traced through the loop with GDB and checked the GLib source. So everything I said about the cost of kernel context switches doesn't apply. Also the stuff you said about multiple cores and lock-free data structures doesn't apply either. So things aren't as they seemed. Sorry for the distraction. Incidentally, the rules for use of async functions aren't that clear. In your Generator you're calling an async function 'helper()' from a non-async function 'new Generator()'. Reading the C, I can kind of see how it is working. Is there any good web page on the theory of async stuff, either in Vala or C? Jim -- Jim Peters (_)/=\~/_(_) j...@uazu.net (_) /=\ ~/_ (_) Uazú (_)/=\~/_(_)http:// in Peru(_) /=\ ~/_ (_)uazu.net ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
Luca Bruno wrote: > > No, I don't have a benchmark, but a Thread is actually a kernel > > process on Linux, whereas a goroutine could be just an allocated > > mini-stack of 100 bytes (assuming enough static analysis to figure out > > required stack depth) and some housekeeping structures. Switching > > between threads means waiting for the Linux kernel to schedule it. > > Switching to/from a go/coroutine could be a longjmp (having read some > > of the Pth implementation stuff). > > > > To me it seems that they would not be comparable in efficiency. > > Threads are allocated on multiple cores nowadays and you can use lock-free > data structures when possible. Still you didn't provide any benchmark, > only thoughts. We are both putting forward "only thoughts". To prove my point I will have to implement simple coroutines in Vala using low-level C stuff, i.e. setcontext/makecontext/etc. I guess a Coroutine class could be useful for something someday. Give me a moment ... Jim -- Jim Peters (_)/=\~/_(_) j...@uazu.net (_) /=\ ~/_ (_) Uazú (_)/=\~/_(_)http:// in Peru(_) /=\ ~/_ (_)uazu.net ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
Christian Siefkes wrote: > Iterators already exist in Vala, so why not just use them? > (I didn't look in detail at the examples for couroutines and generators, so > if I missed something obvious, sorry.) The original poster wanted to write his iterator in the style of Go, i.e. with a coroutine style. Jim -- Jim Peters (_)/=\~/_(_) j...@uazu.net (_) /=\ ~/_ (_) Uazú (_)/=\~/_(_)http:// in Peru(_) /=\ ~/_ (_)uazu.net ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
On Tue, Jul 12, 2011 at 12:04:13PM -0500, Jim Peters wrote: > Luca Bruno wrote: > > > Threads are very heavy compared to an optimised implementation of > > > 'goroutines' or whatever. Certainly you can emulate them but it's not > > > going to be an efficient way of implementing algorithms unless someone > > > puts in the work to make it efficient under the hood. > > > > > > If convenience matters more than efficiency, then no problem. > > > > Do you have any benchmark? I wouldn't say that loudly in general. > > No, I don't have a benchmark, but a Thread is actually a kernel > process on Linux, whereas a goroutine could be just an allocated > mini-stack of 100 bytes (assuming enough static analysis to figure out > required stack depth) and some housekeeping structures. Switching > between threads means waiting for the Linux kernel to schedule it. > Switching to/from a go/coroutine could be a longjmp (having read some > of the Pth implementation stuff). > > To me it seems that they would not be comparable in efficiency. Threads are allocated on multiple cores nowadays and you can use lock-free data structures when possible. Still you didn't provide any benchmark, only thoughts. -- http://www.debian.org - The Universal Operating System ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
Hi Serge, Jim, all, On 07/12/2011 07:04 PM, Jim Peters wrote: > In this case we're talking about an Iterator. A traditional > non-coroutine implementation of an Iterator would hold state in the > Iterator object and generate new objects to return one by one on > demand, implemented with an internal generate_next() call or similar. > This will run in a single thread. A co-routine or generator type > implementation turns the code inside out, and holds state on its own > private stack (i.e. local variables) instead of inside the Iterator > object. If implemented as a coroutine the iteration will also run > inside a single thread, and the equivalent of the generate_next() call > is a longjmp switch to the coroutine and a longjmp back. This is > comparable to the non-coroutine implementation. However, if the > coroutine is running in a separate thread, then we require the kernel > to schedule a context switch. This HAS to be slower -- at least > that's how it seems to me. Iterators already exist in Vala, so why not just use them? (I didn't look in detail at the examples for couroutines and generators, so if I missed something obvious, sorry.) Best regards Christian -- |--- Dr. Christian Siefkes --- christ...@siefkes.net --- | Homepage: http://www.siefkes.net/ | Blog: http://www.keimform.de/ |Peer Production Everywhere: http://peerconomy.org/wiki/ |-- OpenPGP Key ID: 0x346452D8 -- A bug is a test case you haven't written yet. -- Mark Pilgrim signature.asc Description: OpenPGP digital signature ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
Luca Bruno wrote: > > Threads are very heavy compared to an optimised implementation of > > 'goroutines' or whatever. Certainly you can emulate them but it's not > > going to be an efficient way of implementing algorithms unless someone > > puts in the work to make it efficient under the hood. > > > > If convenience matters more than efficiency, then no problem. > > Do you have any benchmark? I wouldn't say that loudly in general. No, I don't have a benchmark, but a Thread is actually a kernel process on Linux, whereas a goroutine could be just an allocated mini-stack of 100 bytes (assuming enough static analysis to figure out required stack depth) and some housekeeping structures. Switching between threads means waiting for the Linux kernel to schedule it. Switching to/from a go/coroutine could be a longjmp (having read some of the Pth implementation stuff). To me it seems that they would not be comparable in efficiency. I think that the idea of goroutines is that they can be used everywhere in the code, just like an 'if' or a 'for'. So there could be 100s of them active. If they only need 100-200 bytes each, that is manageable. If they require an entire kernel process for each one, then it does not seem like a good approach. I guess if we're just talking about one or two extra threads running at the top level of the application, then I agree, that is fine. But using them as a fundamental control-flow structure within the language needs a more optimised implementation. In this case we're talking about an Iterator. A traditional non-coroutine implementation of an Iterator would hold state in the Iterator object and generate new objects to return one by one on demand, implemented with an internal generate_next() call or similar. This will run in a single thread. A co-routine or generator type implementation turns the code inside out, and holds state on its own private stack (i.e. local variables) instead of inside the Iterator object. If implemented as a coroutine the iteration will also run inside a single thread, and the equivalent of the generate_next() call is a longjmp switch to the coroutine and a longjmp back. This is comparable to the non-coroutine implementation. However, if the coroutine is running in a separate thread, then we require the kernel to schedule a context switch. This HAS to be slower -- at least that's how it seems to me. Maybe this could be made more workable using 'Pth' instead of kernel threads, or by using the async stuff already in Vala. (Sorry, I'm not up to speed on that yet.) Jim -- Jim Peters (_)/=\~/_(_) j...@uazu.net (_) /=\ ~/_ (_) Uazú (_)/=\~/_(_)http:// in Peru(_) /=\ ~/_ (_)uazu.net ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
On Tue, Jul 12, 2011 at 11:11, Serge Hulne wrote: > > Another reason why these speculations cannot possibly add anything > performance-wise is that I don't think (I might be wrong) that Vala can be > tailored specifically for a given multi-core architecture. In this case, > using threads can be convenient, but it will not process tasks in parallel, > therefore threads can only slow down the execution of an application. > Vala threads are based on GLib threads, so they do scale to multicore systems and they do process in parallel. Contrary to a few other languages, Vala does not emulate threads, it uses the system threads (with all the benefits and problems of that). But pay attention that if your application is not designed for concurrent processing, it won't matter. For instance, you can block one thread waiting for the other to finish processing or to free some resource, then you are just doing sequential processing anyway. *Alexandre Rosenfeld* ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
Absolutely correct ! Hence the term "speculation" Moreover, if efficiency was paramount, the whole construct would have to be implemented at lower level (in C not in Vala). Another reason why these speculations cannot possibly add anything performance-wise is that I don't think (I might be wrong) that Vala can be tailored specifically for a given multi-core architecture. In this case, using threads can be convenient, but it will not process tasks in parallel, therefore threads can only slow down the execution of an application. Last but not least goroutines are indeed more lightweight than threads (they use threads but they are not threads). So basically it was just for convenience (and for fun) ! Serge. On Tue, Jul 12, 2011 at 2:43 PM, Jim Peters wrote: > Serge Hulne wrote: > > Here is a further development of the idea of Luca Bruno about a Vala > > implementation for Generators: > > > > Simulating Go's goroutines and channels in Vala: > > > > Basically the idea is to start as many threads as needed (which play the > > role of Go' goroutines) and to recuperate their output from a "Generator" > > (which plays the role of the "Go" channel form which the result from a > given > > thread can be pulled): > > Threads are very heavy compared to an optimised implementation of > 'goroutines' or whatever. Certainly you can emulate them but it's not > going to be an efficient way of implementing algorithms unless someone > puts in the work to make it efficient under the hood. > > If convenience matters more than efficiency, then no problem. > > Jim > > -- > Jim Peters (_)/=\~/_(_) j...@uazu.net > (_) /=\ ~/_ (_) > Uazú (_)/=\~/_(_)http:// > in Peru(_) /=\ ~/_ (_)uazu.net > ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
On Tue, Jul 12, 2011 at 07:43:14AM -0500, Jim Peters wrote: > Serge Hulne wrote: > > Here is a further development of the idea of Luca Bruno about a Vala > > implementation for Generators: > > > > Simulating Go's goroutines and channels in Vala: > > > > Basically the idea is to start as many threads as needed (which play the > > role of Go' goroutines) and to recuperate their output from a "Generator" > > (which plays the role of the "Go" channel form which the result from a given > > thread can be pulled): > > Threads are very heavy compared to an optimised implementation of > 'goroutines' or whatever. Certainly you can emulate them but it's not > going to be an efficient way of implementing algorithms unless someone > puts in the work to make it efficient under the hood. > > If convenience matters more than efficiency, then no problem. Do you have any benchmark? I wouldn't say that loudly in general. -- http://www.debian.org - The Universal Operating System ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
Serge Hulne wrote: > Here is a further development of the idea of Luca Bruno about a Vala > implementation for Generators: > > Simulating Go's goroutines and channels in Vala: > > Basically the idea is to start as many threads as needed (which play the > role of Go' goroutines) and to recuperate their output from a "Generator" > (which plays the role of the "Go" channel form which the result from a given > thread can be pulled): Threads are very heavy compared to an optimised implementation of 'goroutines' or whatever. Certainly you can emulate them but it's not going to be an efficient way of implementing algorithms unless someone puts in the work to make it efficient under the hood. If convenience matters more than efficiency, then no problem. Jim -- Jim Peters (_)/=\~/_(_) j...@uazu.net (_) /=\ ~/_ (_) Uazú (_)/=\~/_(_)http:// in Peru(_) /=\ ~/_ (_)uazu.net ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] How to force window destroy?
On Tue, 2011-07-12 at 11:13 +0200, Tomasz Jankowski wrote: > Hello! > > I have a window with button inside it and I want to destroy window. Then in > button's click event handler i want to destroy window. The problem I that I > cannot find wrapper for gtk_widget_destroy() and I have no idea ho to do > this. Maybe it's because memory model based on reference counting, I'm not > sure, so I'm asking for explanation. I can see Gtk.Object.destroy() (Gtk.Widget inherits Gtk.Object) for gtk2 and Gtk.Widget.destroy() for gtk3. signature.asc Description: This is a digitally signed message part ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
[Vala] How to force window destroy?
Hello! I have a window with button inside it and I want to destroy window. Then in button's click event handler i want to destroy window. The problem I that I cannot find wrapper for gtk_widget_destroy() and I have no idea ho to do this. Maybe it's because memory model based on reference counting, I'm not sure, so I'm asking for explanation. ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] [Erratum]Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
Erratum, there was a omission in my previous post, here is the complete
snippet:
On Tue, Jul 12, 2011 at 9:25 AM, Serge Hulne wrote:
> If the code proposed by Luca was stored separately in its own file (or
> library), then all one would have to to to use use this
> goroutine/channnels/-ike construct would go like:
>
> /
> using Posix;
>
> class IntGenerator : Generator {
> protected override async void generate () {
> for (int i=0; i < 10; i++) {
> if (i%2 ==0) yield feed (i);
> }
> }
> }
>
> class IntGenerator_1 : Generator {
> protected override async void generate () {
> for (int i=0; i < 10; i++) {
> if (i%2 !=0) yield feed (i);
> }
> }
> }
>
>
> IntGeneratorgen;
> IntGenerator_1 gen_1;
>
> void* thread_func() {
> //Posix.stdout.printf("Thread running.\n");
> gen = new IntGenerator();
> return null;
> }
>
> void* thread_func_1() {
> //Posix.stdout.printf("Thread running.\n");
> gen_1 = new IntGenerator_1();
> return null;
> }
>
>
> int main(string[] args) {
>
> if (!Thread.supported()) {
> Posix.stderr.printf("Cannot run without threads.\n");
> return 1;
> }
>
> try {
> unowned Thread thread = Thread.create(thread_func,
> true);
> thread.join();
>
> unowned Thread thread_1 =
> Thread.create(thread_func_1, true);
> thread_1.join();
>
> } catch (ThreadError e) {
> return 1;
> }
>
> print("\n\nResults computed in first thread\n");
>
> var i=0;
> foreach (var item in gen) {
> if (i<10) Posix.stdout.printf("%i\n", item);
> i++;
> }
>
> print("\n\nResults computed in the second thread\n\n");
>
> i=0;
> foreach (var item in gen_1) {
> if (i<10) Posix.stdout.printf("%i\n", item);
> i++;
> }
>
>
> return 0;
> }
> /
>
>
>
>
>
>
___
vala-list mailing list
vala-list@gnome.org
http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
Obviously, if the code proposed by Luca was stored separately in its own
file (or in) library then all one would have to to to use use this
goroutine/channnels/-ike construct boils down to:
/
using Posix;
IntGeneratorgen;
IntGenerator_1 gen_1;
void* thread_func() {
//Posix.stdout.printf("Thread running.\n");
gen = new IntGenerator();
return null;
}
void* thread_func_1() {
//Posix.stdout.printf("Thread running.\n");
gen_1 = new IntGenerator_1();
return null;
}
int main(string[] args) {
if (!Thread.supported()) {
Posix.stderr.printf("Cannot run without threads.\n");
return 1;
}
try {
unowned Thread thread = Thread.create(thread_func,
true);
thread.join();
unowned Thread thread_1 = Thread.create(thread_func_1,
true);
thread_1.join();
} catch (ThreadError e) {
return 1;
}
print("\n\nResults computed in first thread\n");
var i=0;
foreach (var item in gen) {
if (i<10) Posix.stdout.printf("%i\n", item);
i++;
}
print("\n\nResults computed in the second thread\n\n");
i=0;
foreach (var item in gen_1) {
if (i<10) Posix.stdout.printf("%i\n", item);
i++;
}
return 0;
}
/
On Tue, July 12, 2011 at 5:55 AM, Serge Hulne wrote:
> Here is a further development of the idea of Luca Bruno about a Vala
> implementation for Generators:
>
> Simulating Go's goroutines and channels in Vala:
>
> Basically the idea is to start as many threads as needed (which play the
> role of Go' goroutines) and to recuperate their output from a "Generator"
> (which plays the role of the "Go" channel form which the result from a given
> thread can be pulled):
>
> See code hereunder,
> Serge.
>
>
>
> /
> using Posix;
>
> abstract class Generator {
> private bool consumed;
> private unowned G value;
> private SourceFunc callback;
>
> public Generator () {
> helper ();
> }
>
> private async void helper () {
> yield generate ();
> consumed = true;
> }
>
> protected abstract async void generate ();
>
> protected async void feed (G value) {
> this.value = value;
> this.callback = feed.callback;
> yield;
> }
>
> public bool next () {
> return !consumed;
> }
>
> public G get () {
> var result = value;
> callback ();
> return result;
> }
>
> public Generator iterator () {
> return this;
> }
> }
>
>
>
> class IntGenerator : Generator {
> protected override async void generate () {
> for (int i=0; i < 10; i++) {
> if (i%2 ==0) yield feed (i);
> }
> }
> }
>
> class IntGenerator_1 : Generator {
> protected override async void generate () {
> for (int i=0; i < 10; i++) {
> if (i%2 !=0) yield feed (i);
> }
> }
> }
>
> IntGeneratorgen;
> IntGenerator_1 gen_1;
>
> void* thread_func() {
> //Posix.stdout.printf("Thread running.\n");
> gen = new IntGenerator();
> return null;
> }
>
> void* thread_func_1() {
> //Posix.stdout.printf("Thread running.\n");
> gen_1 = new IntGenerator_1();
> return null;
> }
>
>
> int main(string[] args) {
>
> if (!Thread.supported()) {
> Posix.stderr.printf("Cannot run without threads.\n");
> return 1;
> }
>
> try {
> unowned Thread thread = Thread.create(thread_func,
> true);
> thread.join();
>
> unowned Thread thread_1 =
> Thread.create(thread_func_1, true);
> thread_1.join();
>
> } catch (ThreadError e) {
> return 1;
> }
>
> print("\n\nResults computed in first thread\n");
>
> var i=0;
> foreach (var item in gen) {
> if (i<10) Posix.stdout.printf("%i\n", item);
> i++;
> }
>
> print("\n\nResults computed in the second thread\n\n");
>
> i=0;
> foreach (var item in gen_1) {
> if (i<10) Posix.stdout.printf("%i\n", item);
> i++;
> }
>
>
> return 0;
> }
>
> /
>
___
vala-list mailing list
vala-list@gnome.org
http://mail.gnome.org/mailman/listinfo/vala-list
Re: [Vala] Further speculations on couroutines, generators and threads : Emulating Go's goroutines and channels in Vala
> Simulating Go's goroutines and channels in Vala: I was exposed a long time ago to CSP channels in Occam. I'm really surprised that it didn't get adopted more widely. Occam makes channels really easy. If your first exposure to channels is from C, you might be less convinced about the convenience of channels compared to other methods of synchronizing/inter-process communication. But, if you've done it in Occam, you will think everything else sucks. Go appears to have syntactic support for CSP channels too. But I haven't really gone far into Go (Vala is taking up a lot of my time now :-) Hopefully Vala will someday (soon) has syntactic support for channel that is as easy as in Occam. And hopefully Luca Bruno will continue the fantasy to episode 2. hand Nor Jaidi Tuah ___ vala-list mailing list vala-list@gnome.org http://mail.gnome.org/mailman/listinfo/vala-list
