Re: Revamped concurrency API
Andrei Alexandrescu wrote: Occasionally people here ask for ways in which they can help D. One thing that would be extremely helpful at this point would be to help defining and implementing D's new concurrency API. Unfortunately, Bartosz has declined to contribute. That leaves Walter, Sean, Don, and participants to this group to do it. I'm sure you know the three of us are overcommitted, and I'm also sure many of you are also in that situation. But if we could somehow focus all of the energy and good will in this group to one task of extraordinary urgency and importance, that would be awesome. If anyone has ideas, suggestions, and code to help defining a new concurrency API for D, please post. Andrei Don't fix it to one model, leave room for multiple concurrency models to be used side by side! I would begin by making the 'shared' specs usable, since the rest will be within the runtime and everything will rely on shared. I already made a post on this some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=96161 Jeremie
Re: Revamped concurrency API
== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> Occasionally people here ask for ways in which they can help D. One
> thing that would be extremely helpful at this point would be to help
> defining and implementing D's new concurrency API. Unfortunately,
> Bartosz has declined to contribute. That leaves Walter, Sean, Don, and
> participants to this group to do it.
> I'm sure you know the three of us are overcommitted, and I'm also sure
> many of you are also in that situation. But if we could somehow focus
> all of the energy and good will in this group to one task of
> extraordinary urgency and importance, that would be awesome.
> If anyone has ideas, suggestions, and code to help defining a new
> concurrency API for D, please post.
For what it's worth, I've been experimenting with message-passing for
one leg of the API. I decided to copy the Erlang API because it's very
simple and easy to build fancier stuff on top of. What I have so far is:
void sendmsg(T)( Pid pid, T val );
final void recvmsg(T...)( Pid pid, T ops );
Pid spawn(T)( T fun );
spawn() is pretty limited so far in that it only spawns threads--I'd
expect that function to end up with multiple overloads at some point.
Also 'ops' in recvmsg are delegates. Typical use would be:
// Thread A
sendmsg( pid, 5 );
sendmsg( pid, tuple(5) );
// Thread B
recvmsg( pid, (int val) { writefln( "got int: %s", val ); } );
recvmsg( pid, (Tuple!(int) val) { writefln( "got tuple: %s", val ); } );
I thought about using predefined types for "receive any" and
"abort after timeout" functions to pass to recvmsg. Pattern matching
in D is largely limited to type matching at the moment, which is kind
of a limitation, so I'd considered having the delegates return a bool
indicating whether the value passed was a match or not.
Anyway, thought I'd just add this to the thread in case it sparks
discussion. I have a working implementation of this around
somewhere if anyone is interested.
Re: Revamped concurrency API
Sean Kelly Wrote: > > void sendmsg(T)( Pid pid, T val ); > final void recvmsg(T...)( Pid pid, T ops ); Oops, I should mention that 'pid' shouldn't be in recvmsg. I just have it in there for now for testing purposes.
Re: Revamped concurrency API
Andrei Alexandrescu: >Unfortunately, Bartosz has declined to contribute.< I have read a good amount of his interesting blog posts, he has shown me many things I didn't know about. And he was very happy about the idea of helping D concurrency. Do you know why he has changed his mind regarding this? Maybe because Walter has refused the uniqueness/lend ideas? Bear hugs, bearophile
Re: Revamped concurrency API
bearophile wrote: Andrei Alexandrescu: Unfortunately, Bartosz has declined to contribute.< I have read a good amount of his interesting blog posts, he has shown me many things I didn't know about. And he was very happy about the idea of helping D concurrency. Do you know why he has changed his mind regarding this? Maybe because Walter has refused the uniqueness/lend ideas? You may want to email Bartosz and ask him. Andrei
Re: Revamped concurrency API
bearophile wrote: Andrei Alexandrescu: Unfortunately, Bartosz has declined to contribute.< I have read a good amount of his interesting blog posts, he has shown me many things I didn't know about. And he was very happy about the idea of helping D concurrency. Do you know why he has changed his mind regarding this? Maybe because Walter has refused the uniqueness/lend ideas? I believe that Bearophile is likely to be on target here. Bartosz has spent a lot of time and effort educating the D community on the subject of data races etc, and many thanks for him for doing this. And now he 'declined to contribute' ? Too busy at work, perhaps ? More likely, a difference in opinion, as to the direction this should proceed, without any discussion on various/conflicting proposals with the community. Is this the way serious architecture decisions should be made ? Would Walter/Bartosz/Andrei like to comment further Nick B
Re: Revamped concurrency API
Andrei Alexandrescu wrote: bearophile wrote: Andrei Alexandrescu: Unfortunately, Bartosz has declined to contribute.< I have read a good amount of his interesting blog posts, he has shown me many things I didn't know about. And he was very happy about the idea of helping D concurrency. Do you know why he has changed his mind regarding this? Maybe because Walter has refused the uniqueness/lend ideas? You may want to email Bartosz and ask him. Andrei I will ask him, via his blog, and then will post the link. Nick B.
Re: Revamped concurrency API
Nick B wrote: Andrei Alexandrescu wrote: bearophile wrote: Andrei Alexandrescu: Unfortunately, Bartosz has declined to contribute.< I have read a good amount of his interesting blog posts, he has shown me many things I didn't know about. And he was very happy about the idea of helping D concurrency. Do you know why he has changed his mind regarding this? Maybe because Walter has refused the uniqueness/lend ideas? You may want to email Bartosz and ask him. Andrei I will ask him, via his blog, and then will post the link. Great, in the meantime I've also let him know. FWIW, there's no falling out or other juicy details. Bartosz and I are still good friends, we review each other's work, and I eat the mean brioche he bakes whenever I'm given a chance. Andrei
Re: Revamped concurrency API
Nick B wrote: Andrei Alexandrescu wrote: bearophile wrote: Andrei Alexandrescu: Unfortunately, Bartosz has declined to contribute.< I have read a good amount of his interesting blog posts, he has shown me many things I didn't know about. And he was very happy about the idea of helping D concurrency. Do you know why he has changed his mind regarding this? Maybe because Walter has refused the uniqueness/lend ideas? You may want to email Bartosz and ask him. Andrei I will ask him, via his blog, and then will post the link. Nick B. here it is. http://bartoszmilewski.wordpress.com/2009/09/22/ownership-systems-against-data-races/#comment-922 Nick B.
Re: Revamped concurrency API
If anyone has ideas, suggestions, and code to help defining a new
concurrency API for D, please post.
I think that the correct way to handle concurrency is through a
library, D is flexible enough, so that a library can be used, and then
one can even write special handlers for special cases.
In blip (http://dsource.org/project/blip) I actually did exactly that,
in D 1.0 for two reasons:
1) I need 64 bits
2) concurrency is complex, and I had a hard time already with bugs in
the stable branch, furthermore the possibility of breakage in very
sensitive code that I don't want to touch too often by language changes
was not appealing to me.
That said there is a feature that would help much D 1.0, and that I
would really love to have.
I know that D 1.0 is frozen and stuff... but I will try to ask all the same...
real closures from delegates when you put a "new" in front of them
new delegate(ref int x){x+=y}
would create a closure (allocating y).
I know that D 2.0 has basically that by default (and scope to get the
previous behavior), but as I said D 2.0 is not yet an option for me (I
want to work on my projects, I think I am already doing enough for the
community), so I thought that asking for a feature that does not break
existing D 1.0 code and is in some way already implemented in D 2.0
could be worth trying :)
The other D 2.0 features are nice, I do like structure
constructors/destructors, post blits, template constraints, const...,
well shared I am not so sure about, but anyway all of them are not
needed for concurrency.
Now about the concurrency infrastructure, here I will discuss SMP
parallelization, there is also a more coarse grained parallelization
that needs another approach (MPI and agent based model, for now I just
wrapped mpi and serialization in a way that could be implemented
directly on tcp, and allow to easily communicate arbitrary objects).
The concurrency has two sides one is the user/programmer side and the
other the efficient realization, I will discuss the user level api, as
I realized it in Blip, which is optimized for recursive operations that
have to be finished (i.e computations to be performed, not to simulate
concurrent systems).
The idea is to separate each tasks in chunks that are as small as
possible, while still being large enough so that the switching time is
small wrt. to the computing time.
This subdivision typically is not directly dependent on the number of
processors
Task is a class the represents a task, it has a string name (for
debugging purposes) and can be initialized with a delegate, a function,
a fiber or a generator (in two flavors).
There are some optimizations to make allocation cheaper.
a task can spawn subtasks, and it "knows" how many subtasks there are
executing.
a task is considered finished when the task is complete and all its
subtasks are completed
you can append operations to be executed after a task has finished executing.
you can wait for a task to finish (but try avoiding it, addint the task
to the onFinish of the task is much more efficient).
a task has a given level, subtasks have level+1, and tasks that cannot
have subtasks have a very high level (int.max/2).
a new task can be submitted with
t.submit()
or t.submitYield()
submitYield submits the current task and possibly stops the current
one, this together with the fact that tasks with higher level are
executed before tasks with lower level means that execution is
preferentially a depth first reduction which minimizes the number of
suspended tasks (I have also task stealing that steals preferentially
the tasks with lowest level, but that part is not yet really used).
other important operations are delay and resubmitDelayed that allow one
to delay a task, for example when you have to do i/o and you use a
method like epoll you can delay the current task, add the file handler
to the one to control, and execute resubmitDelayed when that handler
has new data.
executeNow is useful to execute a task synchronously.
That is basically the core that one has to know, the library is
actually much more rich, but for a typical usage this is enough.
With it one can write code like:
import blip.parallel.WorkManager;
class SubProblem{
void execute(){
if(problem is large){
SubProblem firstHalf=...;
Task("subproblem1",&firstHalf.execute).autorelease.submitYield();
SubProblem secondHalf=...;
Task("subproblem2",&secondHalf.execute).autorelease.submitYield();
} else {
direct solver
}
}
}
solveProblem(){
if(problemIsLarge){
SubProblem wholeProblem=...;
Task("solveProblem",&wholeProblem.execute).executeNow(default);
} else {
direct solver
}
}
just to show a very basic divide and conquer approach
Re: Revamped concurrency API
On Mon, 12 Oct 2009 18:21:37 -0400, Fawzi Mohamed wrote: If anyone has ideas, suggestions, and code to help defining a new concurrency API for D, please post. I think that the correct way to handle concurrency is through a library, D is flexible enough, so that a library can be used, and then one can even write special handlers for special cases. In blip (http://dsource.org/project/blip) I actually did exactly that, in D 1.0 for two reasons: [snip] Looks very interesting. ( Corrected link: http://www.dsource.org/projects/blip ) It looks like it's under the Apache 2.0 license? Would you be willing to put it under the boost license, so that it might be considered for Phobos?
Re: Revamped concurrency API
Nick B Wrote: > Is this the way serious architecture decisions should be made ? > > Would Walter/Bartosz/Andrei like to comment further > > > Nick B Good point.
Re: Revamped concurrency API
On 2009-10-12 11:45:55 -0400, Andrei Alexandrescu said: Occasionally people here ask for ways in which they can help D. One thing that would be extremely helpful at this point would be to help defining and implementing D's new concurrency API. Unfortunately, Bartosz has declined to contribute. That leaves Walter, Sean, Don, and participants to this group to do it. So Bartosz quit? Not that much surprising given the all too many missing building blocks. I recently realized that you can easily implement unique in a library. It can work pretty much the same as auto_ptr in C++. But to guaranty uniqueness you need either to rely on convention (as in C++), or add support for lent to the compiler. I know we discussed at length how lent could be implemented, we called it 'scope' and 'escape analysis' at the time and the conclusion you (and Walter I guess) came with was that it was too much for D2. Well, without lent (and thus without unique), safe message passing systems across threads will be limited to immutable data, or copied data, which not at all useful in many situations. Now if you want a good concurrency API relying on convention, then that's great: there are plenty of examples to follow out there. But if you want it to be both safe (enforced) and useful at the same time, that's a mission impossible with the current set of tools available from the compiler. That's why I'm not surprised Bartosz declined to implement it. (And sorry if I sound pessimistic.) -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Revamped concurrency API
On Mon, 12 Oct 2009 21:45:20 -0400, Michel Fortin wrote: On 2009-10-12 11:45:55 -0400, Andrei Alexandrescu said: Occasionally people here ask for ways in which they can help D. One thing that would be extremely helpful at this point would be to help defining and implementing D's new concurrency API. Unfortunately, Bartosz has declined to contribute. That leaves Walter, Sean, Don, and participants to this group to do it. So Bartosz quit? Not that much surprising given the all too many missing building blocks. I recently realized that you can easily implement unique in a library. It can work pretty much the same as auto_ptr in C++. But to guaranty uniqueness you need either to rely on convention (as in C++), or add support for lent to the compiler. I know we discussed at length how lent could be implemented, we called it 'scope' and 'escape analysis' at the time and the conclusion you (and Walter I guess) came with was that it was too much for D2. Well, without lent (and thus without unique), safe message passing systems across threads will be limited to immutable data, or copied data, which not at all useful in many situations. Now if you want a good concurrency API relying on convention, then that's great: there are plenty of examples to follow out there. But if you want it to be both safe (enforced) and useful at the same time, that's a mission impossible with the current set of tools available from the compiler. That's why I'm not surprised Bartosz declined to implement it. (And sorry if I sound pessimistic.) I agree. Particularly about lent. Immutable and mutable weren't considered complete without const, so I'm surprised that local and shared are considered complete without lent. You can even implement it without escape analysis. Strangely, from what I remember of Bartosz's posts, it was unique that was the sticking point, though you can implement both unique and owned as library types (though they do become less efficient).
Re: Revamped concurrency API
Robert Jacques wrote: On Mon, 12 Oct 2009 21:45:20 -0400, Michel Fortin wrote: On 2009-10-12 11:45:55 -0400, Andrei Alexandrescu said: Occasionally people here ask for ways in which they can help D. One thing that would be extremely helpful at this point would be to help defining and implementing D's new concurrency API. Unfortunately, Bartosz has declined to contribute. That leaves Walter, Sean, Don, and participants to this group to do it. So Bartosz quit? Not that much surprising given the all too many missing building blocks. I recently realized that you can easily implement unique in a library. It can work pretty much the same as auto_ptr in C++. But to guaranty uniqueness you need either to rely on convention (as in C++), or add support for lent to the compiler. I know we discussed at length how lent could be implemented, we called it 'scope' and 'escape analysis' at the time and the conclusion you (and Walter I guess) came with was that it was too much for D2. Well, without lent (and thus without unique), safe message passing systems across threads will be limited to immutable data, or copied data, which not at all useful in many situations. Now if you want a good concurrency API relying on convention, then that's great: there are plenty of examples to follow out there. But if you want it to be both safe (enforced) and useful at the same time, that's a mission impossible with the current set of tools available from the compiler. That's why I'm not surprised Bartosz declined to implement it. (And sorry if I sound pessimistic.) I agree. Particularly about lent. Immutable and mutable weren't considered complete without const, so I'm surprised that local and shared are considered complete without lent. You can even implement it without escape analysis. Strangely, from what I remember of Bartosz's posts, it was unique that was the sticking point, though you can implement both unique and owned as library types (though they do become less efficient). I may sound ignorant, but that does lent means? I mean the word itself, I couldn't find something relevant on google. Anyways, I agree that we need a type qualifier between unshared (local) and shared, just like const sits between mutable and immutable. It would make shared handling just so much easier and convenient. If shared/lent were properly implemented and as easy to use as immutable/const, we'd already begin to see concurrency APIs, I tried a few times to implement different concurrency models and always failed due to shared requiring either too much casts or forces too many structures and methods to be shared. I vote for a 'lent' qualifier to be implemented in the compiler. Jeremie
Re: Revamped concurrency API
Jeremie Pelletier, el 12 de octubre a las 22:45 me escribiste: > >I agree. Particularly about lent. Immutable and mutable weren't > >considered complete without const, so I'm surprised that local and > >shared are considered complete without lent. You can even > >implement it without escape analysis. Strangely, from what I > >remember of Bartosz's posts, it was unique that was the sticking > >point, though you can implement both unique and owned as library > >types (though they do become less efficient). > > I may sound ignorant, but that does lent means? I mean the word > itself, I couldn't find something relevant on google. The terms "unique", "shared" and "lent" used here generaly refers to the terms used by Bartoz Milewski in his blog[1]. I think he defines lent in this particular blog post[2], but maybe you might need to read a couple of posts more to understand everything. [1] http://bartoszmilewski.wordpress.com/ [2] http://bartoszmilewski.wordpress.com/2009/05/21/unique_ptr-how-unique-is-it/ -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ -- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) -- Sometimes you got to suffer a little in your youth to motivate you to succeed later in life. Do you think if Bill Gates got laid in high school, do you think there'd be a Microsoft? Of course not. You gotta spend a lot of time stuffin your own locker with your underwear wedged up your arse before you think "I'm gona take over the world with computers! You'll see I'll show them."
Re: Revamped concurrency API
Jeremie Pelletier: > I vote for a 'lent' qualifier to be implemented in the compiler. You may be right, and Bartosz has explained me why and how to use lent. It sounds like a nice idea, but it has some costs too (in language complexity, for example). But I need to use such things in practice, in real (even small) programs to know/understand if I like and understand them enough. So I'd like to have an experimental version of D where such ideas (like lend, nonnull references, and few other things) can be tried, and if they are experimentally seen as not good (by me too), they can be discarded and removed. For me this is true for nonnull references, but such experimental need is even strong for lend. It seem all the long discussion regarding nonnull references has gone nowhere. That has made me a little sad. Only a bit of flow analysis has being accepted, to cover certain cases of a specific bug case of uninitialized objects. Bye, bearophile
Re: Revamped concurrency API
Sean Kelly wrote: > void sendmsg(T)( Pid pid, T val ); > final void recvmsg(T...)( Pid pid, T ops ); > Pid spawn(T)( T fun ); > > spawn() is pretty limited so far in that it only spawns threads--I'd > expect that function to end up with multiple overloads at some point. Interesting. Future spawn() may spawn the thread in a remote CPU that doesn't share the memory. In that case it perhaps helps to limit fun to be pure, and val to be immutable, as in Erlang. D already has pure and transitive const, so the first few steps for the massively-parallel D programs are already completed! :-)
Re: Revamped concurrency API
MIURA Masahiro wrote: > Sean Kelly wrote: >> void sendmsg(T)( Pid pid, T val ); >> final void recvmsg(T...)( Pid pid, T ops ); >> Pid spawn(T)( T fun ); >> >> spawn() is pretty limited so far in that it only spawns threads--I'd >> expect that function to end up with multiple overloads at some point. > > Interesting. Future spawn() may spawn the thread in a remote CPU that > doesn't share the memory. In that case it perhaps helps to limit fun to > be pure, and val to be immutable, as in Erlang. D already has pure and > transitive const, so the first few steps for the massively-parallel > D programs are already completed! :-) That's the plan, except that we weren't thinking of creating remote processes, only sending messages across process and machine boundaries. Here's where notions like value vs. reference and deep cloning become very important. Andrei
Re: Revamped concurrency API
Leandro Lucarella wrote: Jeremie Pelletier, el 12 de octubre a las 22:45 me escribiste: I agree. Particularly about lent. Immutable and mutable weren't considered complete without const, so I'm surprised that local and shared are considered complete without lent. You can even implement it without escape analysis. Strangely, from what I remember of Bartosz's posts, it was unique that was the sticking point, though you can implement both unique and owned as library types (though they do become less efficient). I may sound ignorant, but that does lent means? I mean the word itself, I couldn't find something relevant on google. The terms "unique", "shared" and "lent" used here generaly refers to the terms used by Bartoz Milewski in his blog[1]. I think he defines lent in this particular blog post[2], but maybe you might need to read a couple of posts more to understand everything. [1] http://bartoszmilewski.wordpress.com/ [2] http://bartoszmilewski.wordpress.com/2009/05/21/unique_ptr-how-unique-is-it/ I know of this article, he mentions the word but he doesn't say where it comes from. My native language is french (I'm from Quebec, Canada), I don't know every word of the english dictionary. That's why I asked, I'm just curious as to what it meant. I did some googling but I only landed on a religious page :/ I assume it means 'temporary shared' just like const is 'temporary immutable', but if i can make a link with the english language too its even better.
Re: Revamped concurrency API
Sean Kelly: > For what it's worth, I've been experimenting with message-passing for > one leg of the API. I decided to copy the Erlang API because it's very > simple and easy to build fancier stuff on top of. You may also want to take a look at how actors are in Scala (I think they are a little different than usual, so I think they even have a different name, something like Agents, or something like that): http://www.scala-lang.org/node/242 Bye, bearophile
Re: Revamped concurrency API
On 2009-10-13 03:17:13 +0200, "Robert Jacques" said: On Mon, 12 Oct 2009 18:21:37 -0400, Fawzi Mohamed wrote: If anyone has ideas, suggestions, and code to help defining a new concurrency API for D, please post. I think that the correct way to handle concurrency is through a library, D is flexible enough, so that a library can be used, and then one can even write special handlers for special cases. In blip (http://dsource.org/project/blip) I actually did exactly that, in D 1.0 for two reasons: [snip] Looks very interesting. ( Corrected link: http://www.dsource.org/projects/blip ) It looks like it's under the Apache 2.0 license? Would you be willing to put it under the boost license, so that it might be considered for Phobos? if there is interest, yes
Re: Revamped concurrency API
bearophile wrote: Jeremie Pelletier: I vote for a 'lent' qualifier to be implemented in the compiler. You may be right, and Bartosz has explained me why and how to use lent. It sounds like a nice idea, but it has some costs too (in language complexity, for example). But I need to use such things in practice, in real (even small) programs to know/understand if I like and understand them enough. I don't see how heavy that cost is since all of const/immutable/lent/shared are type qualifiers, the semantics are already there for const and immutable, and half there for shared. I would also much prefer compiler implementations to be a bit more complex and implement lent/shared properly than push that complexity to user code. If we're trying to get rid of data races and deadlocks and whatnot, the last thing we want are complex implementations in library/user code because they lack the proper language primitives. If we have just those two qualifiers, the rest of the concurrent models can be implemented entirely in the library. So I'd like to have an experimental version of D where such ideas (like lend, nonnull references, and few other things) can be tried, and if they are experimentally seen as not good (by me too), they can be discarded and removed. For me this is true for nonnull references, but such experimental need is even strong for lend. D2 is already 'experimental' if you consider the state of shared right now :) It seem all the long discussion regarding nonnull references has gone nowhere. That has made me a little sad. Only a bit of flow analysis has being accepted, to cover certain cases of a specific bug case of uninitialized objects. I never liked the Object vs Object? syntax, that would just create ambiguity with the ternary conditional. D already dropped the template syntax from C++ to drop the ambiguity with <> and comparison operators. I wouldn't mind something like nonnull(Object), ie yet another type qualifier, although that would get cumbersome in a few cases like const(shared(nonnull(Object))) :o) Bye, bearophile
Re: Revamped concurrency API
On Tue, 13 Oct 2009 01:59:57 -0400, Jeremie Pelletier wrote: Leandro Lucarella wrote: Jeremie Pelletier, el 12 de octubre a las 22:45 me escribiste: I agree. Particularly about lent. Immutable and mutable weren't considered complete without const, so I'm surprised that local and shared are considered complete without lent. You can even implement it without escape analysis. Strangely, from what I remember of Bartosz's posts, it was unique that was the sticking point, though you can implement both unique and owned as library types (though they do become less efficient). I may sound ignorant, but that does lent means? I mean the word itself, I couldn't find something relevant on google. The terms "unique", "shared" and "lent" used here generaly refers to the terms used by Bartoz Milewski in his blog[1]. I think he defines lent in this particular blog post[2], but maybe you might need to read a couple of posts more to understand everything. [1] http://bartoszmilewski.wordpress.com/ [2] http://bartoszmilewski.wordpress.com/2009/05/21/unique_ptr-how-unique-is-it/ I know of this article, he mentions the word but he doesn't say where it comes from. My native language is french (I'm from Quebec, Canada), I don't know every word of the english dictionary. That's why I asked, I'm just curious as to what it meant. I did some googling but I only landed on a religious page :/ I assume it means 'temporary shared' just like const is 'temporary immutable', but if i can make a link with the english language too its even better. Oh. Yes Lent is a religious holiday, but it's also simple past tense and past participle of lend. ( From http://en.wiktionary.org/wiki/lent ) Apparently it's based off of lentus from French.
Re: Revamped concurrency API
Jeremie Pelletier wrote: Leandro Lucarella wrote: Jeremie Pelletier, el 12 de octubre a las 22:45 me escribiste: I agree. Particularly about lent. Immutable and mutable weren't considered complete without const, so I'm surprised that local and shared are considered complete without lent. You can even implement it without escape analysis. Strangely, from what I remember of Bartosz's posts, it was unique that was the sticking point, though you can implement both unique and owned as library types (though they do become less efficient). I may sound ignorant, but that does lent means? I mean the word itself, I couldn't find something relevant on google. The terms "unique", "shared" and "lent" used here generaly refers to the terms used by Bartoz Milewski in his blog[1]. I think he defines lent in this particular blog post[2], but maybe you might need to read a couple of posts more to understand everything. [1] http://bartoszmilewski.wordpress.com/ [2] http://bartoszmilewski.wordpress.com/2009/05/21/unique_ptr-how-unique-is-it/ I know of this article, he mentions the word but he doesn't say where it comes from. My native language is french (I'm from Quebec, Canada), I don't know every word of the english dictionary. That's why I asked, I'm just curious as to what it meant. I did some googling but I only landed on a religious page :/ I assume it means 'temporary shared' just like const is 'temporary immutable', but if i can make a link with the english language too its even better. to lend loan lending are all of the same root. "lent" is the passive/simple past/past participle form of "to lend". I guess is the French word is one of "confer\'e" or "pr\^et\'e". Andrei P.S. Beer? Coffee? Marriage? I'll be in Quebec City on Dec 10th and 11th for a talk.
Re: Revamped concurrency API
bearophile wrote: Sean Kelly: For what it's worth, I've been experimenting with message-passing for one leg of the API. I decided to copy the Erlang API because it's very simple and easy to build fancier stuff on top of. You may also want to take a look at how actors are in Scala (I think they are a little different than usual, so I think they even have a different name, something like Agents, or something like that): http://www.scala-lang.org/node/242 Bye, bearophile I really like the actor model, it can scale very well to thousands of concurrent actors, I know the Unreal engine use them for all scriptable entities and can process thousands of them per frame. Bartosz also had an entry about actors: http://bartoszmilewski.wordpress.com/2009/07/16/on-actors-and-casting/ This is definitely one concurrent model I want to see in D.
Re: Revamped concurrency API
Jeremie Pelletier wrote: bearophile wrote: Sean Kelly: For what it's worth, I've been experimenting with message-passing for one leg of the API. I decided to copy the Erlang API because it's very simple and easy to build fancier stuff on top of. You may also want to take a look at how actors are in Scala (I think they are a little different than usual, so I think they even have a different name, something like Agents, or something like that): http://www.scala-lang.org/node/242 Bye, bearophile I really like the actor model, it can scale very well to thousands of concurrent actors, I know the Unreal engine use them for all scriptable entities and can process thousands of them per frame. Bartosz also had an entry about actors: http://bartoszmilewski.wordpress.com/2009/07/16/on-actors-and-casting/ This is definitely one concurrent model I want to see in D. Same here. If anyone would like to start putting together an actor API, that would be great. Andrei
Re: Revamped concurrency API
Fawzi Mohamed wrote: On 2009-10-13 03:17:13 +0200, "Robert Jacques" said: On Mon, 12 Oct 2009 18:21:37 -0400, Fawzi Mohamed wrote: If anyone has ideas, suggestions, and code to help defining a new concurrency API for D, please post. I think that the correct way to handle concurrency is through a library, D is flexible enough, so that a library can be used, and then one can even write special handlers for special cases. In blip (http://dsource.org/project/blip) I actually did exactly that, in D 1.0 for two reasons: [snip] Looks very interesting. ( Corrected link: http://www.dsource.org/projects/blip ) It looks like it's under the Apache 2.0 license? Would you be willing to put it under the boost license, so that it might be considered for Phobos? if there is interest, yes Thanks. There definitely is. I'll start looking at the lib soon, and I recommend others to do the same. Andrei
Re: Revamped concurrency API
Andrei Alexandrescu wrote: Jeremie Pelletier wrote: Leandro Lucarella wrote: Jeremie Pelletier, el 12 de octubre a las 22:45 me escribiste: I agree. Particularly about lent. Immutable and mutable weren't considered complete without const, so I'm surprised that local and shared are considered complete without lent. You can even implement it without escape analysis. Strangely, from what I remember of Bartosz's posts, it was unique that was the sticking point, though you can implement both unique and owned as library types (though they do become less efficient). I may sound ignorant, but that does lent means? I mean the word itself, I couldn't find something relevant on google. The terms "unique", "shared" and "lent" used here generaly refers to the terms used by Bartoz Milewski in his blog[1]. I think he defines lent in this particular blog post[2], but maybe you might need to read a couple of posts more to understand everything. [1] http://bartoszmilewski.wordpress.com/ [2] http://bartoszmilewski.wordpress.com/2009/05/21/unique_ptr-how-unique-is-it/ I know of this article, he mentions the word but he doesn't say where it comes from. My native language is french (I'm from Quebec, Canada), I don't know every word of the english dictionary. That's why I asked, I'm just curious as to what it meant. I did some googling but I only landed on a religious page :/ I assume it means 'temporary shared' just like const is 'temporary immutable', but if i can make a link with the english language too its even better. to lend loan lending are all of the same root. "lent" is the passive/simple past/past participle form of "to lend". I guess is the French word is one of "confer\'e" or "pr\^et\'e". Thanks! I now make the link, may I then suggest the keyword 'borrow', seems to make more sense to me. Do you speak any french or did you just google that? :o) Andrei P.S. Beer? Coffee? Marriage? I'll be in Quebec City on Dec 10th and 11th for a talk. I will most definitely try to be there, I'd love a beer or a coffee, as for marriage.. you'd need to be female from birth, sorry :) Jeremie
Re: Revamped concurrency API
Andrei Alexandrescu Wrote: > MIURA Masahiro wrote: > > Sean Kelly wrote: > >> void sendmsg(T)( Pid pid, T val ); > >> final void recvmsg(T...)( Pid pid, T ops ); > >> Pid spawn(T)( T fun ); > >> > >> spawn() is pretty limited so far in that it only spawns threads--I'd > >> expect that function to end up with multiple overloads at some point. > > > > Interesting. Future spawn() may spawn the thread in a remote CPU that > > doesn't share the memory. In that case it perhaps helps to limit fun to > > be pure, and val to be immutable, as in Erlang. D already has pure and > > transitive const, so the first few steps for the massively-parallel > > D programs are already completed! :-) > > That's the plan, except that we weren't thinking of creating remote > processes, only sending messages across process and machine boundaries. > Here's where notions like value vs. reference and deep cloning become > very important. Creating remote threads (maybe not OS-level processes) is certainly possible, and I wouldn't be surprised if we did that at some point, assuming something like spawn() is added. But this would be largely invisible at an API level. Like Andrei, I'm more interested in exploring what types should be allowed within messages, what language and library features are necessary, etc. Restricting messages to only shallow value types is certainly possible, but it's rather limiting. I've also experimented with ways to wrap a "shared" container in the same sendmsg/recvmsg interface, but haven't decided if it's an idea worth pursuing yet (or whether it can even be done in a sufficiently robust manner). In any case, that seemed like one way to eliminate obvious use of mutexes for accessing shared data.
Re: Revamped concurrency API
On Tue, 13 Oct 2009 02:25:07 -0400, Jeremie Pelletier wrote: Andrei Alexandrescu wrote: [snip] to lend loan lending are all of the same root. "lent" is the passive/simple past/past participle form of "to lend". I guess is the French word is one of "confer\'e" or "pr\^et\'e". Thanks! I now make the link, may I then suggest the keyword 'borrow', seems to make more sense to me. Borrow is a verb, borrowed would be correct noun (i.e. past tense). A lent object and a borrowed object have practically the same meaning, but one word has half the number of characters.
Re: Revamped concurrency API
Robert Jacques wrote: On Tue, 13 Oct 2009 02:25:07 -0400, Jeremie Pelletier wrote: Andrei Alexandrescu wrote: [snip] to lend loan lending are all of the same root. "lent" is the passive/simple past/past participle form of "to lend". I guess is the French word is one of "confer\'e" or "pr\^et\'e". Thanks! I now make the link, may I then suggest the keyword 'borrow', seems to make more sense to me. Borrow is a verb, borrowed would be correct noun (i.e. past tense). A lent object and a borrowed object have practically the same meaning, but one word has half the number of characters. I disagree that they have the same meaning, one side lends the object and the other borrows it :) But I agree that it makes more sense after reading what you said, maybe I just don't like the sound of past tense verbs in programming keywords, are there any other such keywords in D? Isn't there a qualifier name that would means lent or borrowed without being past-tense, without being a verb implying it also is a function (such as assert). Jeremie
Re: Revamped concurrency API
On 2009-10-13 07:45:41 +0200, Andrei Alexandrescu said: MIURA Masahiro wrote: Sean Kelly wrote: void sendmsg(T)( Pid pid, T val ); final void recvmsg(T...)( Pid pid, T ops ); Pid spawn(T)( T fun ); spawn() is pretty limited so far in that it only spawns threads--I'd expect that function to end up with multiple overloads at some point. Interesting. Future spawn() may spawn the thread in a remote CPU that doesn't share the memory. In that case it perhaps helps to limit fun to be pure, and val to be immutable, as in Erlang. D already has pure and transitive const, so the first few steps for the massively-parallel D programs are already completed! :-) That's the plan, except that we weren't thinking of creating remote processes, only sending messages across process and machine boundaries. Here's where notions like value vs. reference and deep cloning become very important. Andrei I think that there are different parallelization levels, and different strategies, there isn't ne that "rules them all". I am very suspicious of systems where data is moved around "magically", normally the cost of that cannot be ignored, so for coarse level parallelization the message passing approach with explicit data distribution done by the programmer is the way to go. On Numa, or really well connected machines having some kind of shared memory is an option, this works well especially if the data is immutable. Finally when one really has shared memory one can go to task scheduling without thinking too much about transfer of memory, it is this last thing that I did address in my previous post. I think that threads are the wrong approach to parallelization at that level, so what I did was to create Tasks that one can use to express more complex relationships between them, so that one can avoids locks almost always, which is both more efficient and less error prone. Fawzi
Re: Revamped concurrency API
Andrei Alexandrescu wrote: > MIURA Masahiro wrote: >> Sean Kelly wrote: >>> void sendmsg(T)( Pid pid, T val ); >>> final void recvmsg(T...)( Pid pid, T ops ); >>> Pid spawn(T)( T fun ); >>> >>> spawn() is pretty limited so far in that it only spawns threads--I'd >>> expect that function to end up with multiple overloads at some point. >> Interesting. Future spawn() may spawn the thread in a remote CPU that >> doesn't share the memory. In that case it perhaps helps to limit fun to >> be pure, and val to be immutable, as in Erlang. D already has pure and >> transitive const, so the first few steps for the massively-parallel >> D programs are already completed! :-) > > That's the plan, except that we weren't thinking of creating remote > processes, only sending messages across process and machine boundaries. > Here's where notions like value vs. reference and deep cloning become > very important. > > > Andrei With the popularity of cloud computing today and applications pushing toward a client frontend to a server backend, it would only be logical to access server threads from clients. I can see many uses of such a model, for example a cloud IDE that reads from and saves to a specialized version control server, showing what other devs are working on in realtime and being notified instantly of their changes, just like google docs do. That's why I keep saying we need multiple concurrent models in D, there are many usages to every single one of them, and if they can all live together its even better. Jeremie
Re: Revamped concurrency API
On Tue, 13 Oct 2009 02:50:41 -0400, Jeremie Pelletier wrote: Robert Jacques wrote: On Tue, 13 Oct 2009 02:25:07 -0400, Jeremie Pelletier wrote: Andrei Alexandrescu wrote: [snip] to lend loan lending are all of the same root. "lent" is the passive/simple past/past participle form of "to lend". I guess is the French word is one of "confer\'e" or "pr\^et\'e". Thanks! I now make the link, may I then suggest the keyword 'borrow', seems to make more sense to me. Borrow is a verb, borrowed would be correct noun (i.e. past tense). A lent object and a borrowed object have practically the same meaning, but one word has half the number of characters. I disagree that they have the same meaning, one side lends the object and the other borrows it :) But I agree that it makes more sense after reading what you said, maybe I just don't like the sound of past tense verbs in programming keywords, are there any other such keywords in D? shared? Isn't there a qualifier name that would means lent or borrowed without being past-tense, without being a verb implying it also is a function (such as assert). Jeremie
Re: Revamped concurrency API
Fawzi Mohamed wrote: On 2009-10-13 07:45:41 +0200, Andrei Alexandrescu said: MIURA Masahiro wrote: Sean Kelly wrote: void sendmsg(T)( Pid pid, T val ); final void recvmsg(T...)( Pid pid, T ops ); Pid spawn(T)( T fun ); spawn() is pretty limited so far in that it only spawns threads--I'd expect that function to end up with multiple overloads at some point. Interesting. Future spawn() may spawn the thread in a remote CPU that doesn't share the memory. In that case it perhaps helps to limit fun to be pure, and val to be immutable, as in Erlang. D already has pure and transitive const, so the first few steps for the massively-parallel D programs are already completed! :-) That's the plan, except that we weren't thinking of creating remote processes, only sending messages across process and machine boundaries. Here's where notions like value vs. reference and deep cloning become very important. Andrei I think that there are different parallelization levels, and different strategies, there isn't ne that "rules them all". I am very suspicious of systems where data is moved around "magically", normally the cost of that cannot be ignored, so for coarse level parallelization the message passing approach with explicit data distribution done by the programmer is the way to go. On Numa, or really well connected machines having some kind of shared memory is an option, this works well especially if the data is immutable. Finally when one really has shared memory one can go to task scheduling without thinking too much about transfer of memory, it is this last thing that I did address in my previous post. I think that threads are the wrong approach to parallelization at that level, so what I did was to create Tasks that one can use to express more complex relationships between them, so that one can avoids locks almost always, which is both more efficient and less error prone. I also don't believe one model is "ruling them all". I agree that threads aren't the best approach, even things like async I/O can be done with futures, other operations can be done with async methods, the actor model is perfect for lots of independent operations that can be executed in any order, message passing is great for event driven designs such as GUIs, software transactional memory works great with state management, and those are only a few models. Threads only need a platform abstraction in the library, from which all other concurrency models can be built. Jeremie
Re: Revamped concurrency API
Robert Jacques wrote: On Tue, 13 Oct 2009 02:50:41 -0400, Jeremie Pelletier wrote: Robert Jacques wrote: On Tue, 13 Oct 2009 02:25:07 -0400, Jeremie Pelletier wrote: Andrei Alexandrescu wrote: [snip] to lend loan lending are all of the same root. "lent" is the passive/simple past/past participle form of "to lend". I guess is the French word is one of "confer\'e" or "pr\^et\'e". Thanks! I now make the link, may I then suggest the keyword 'borrow', seems to make more sense to me. Borrow is a verb, borrowed would be correct noun (i.e. past tense). A lent object and a borrowed object have practically the same meaning, but one word has half the number of characters. I disagree that they have the same meaning, one side lends the object and the other borrows it :) But I agree that it makes more sense after reading what you said, maybe I just don't like the sound of past tense verbs in programming keywords, are there any other such keywords in D? shared? I stand corrected :x
Re: Revamped concurrency API
Jeremie Pelletier wrote: > I also don't believe one model is "ruling them all". Let me clarity this, just in case I have caused an unnecessary confusion: I think Sean's Erlang-like API is meant to coexist with the current core.thread, and that you can use one or both of them to build higher-level concurrency models. I think it's nice to have core.thread *and* message-passing API in Phobos. Thread alone is too primitive for daily use, but we don't want to have too many concurrency models.
Re: Revamped concurrency API
Jeremie Pelletier wrote: bearophile wrote: Jeremie Pelletier: I vote for a 'lent' qualifier to be implemented in the compiler. You may be right, and Bartosz has explained me why and how to use lent. It sounds like a nice idea, but it has some costs too (in language complexity, for example). But I need to use such things in practice, in real (even small) programs to know/understand if I like and understand them enough. I don't see how heavy that cost is since all of const/immutable/lent/shared are type qualifiers, the semantics are already there for const and immutable, and half there for shared. I would also much prefer compiler implementations to be a bit more complex and implement lent/shared properly than push that complexity to user code. If we're trying to get rid of data races and deadlocks and whatnot, the last thing we want are complex implementations in library/user code because they lack the proper language primitives. If we have just those two qualifiers, the rest of the concurrent models can be implemented entirely in the library. So I'd like to have an experimental version of D where such ideas (like lend, nonnull references, and few other things) can be tried, and if they are experimentally seen as not good (by me too), they can be discarded and removed. For me this is true for nonnull references, but such experimental need is even strong for lend. D2 is already 'experimental' if you consider the state of shared right now :) It seem all the long discussion regarding nonnull references has gone nowhere. That has made me a little sad. Only a bit of flow analysis has being accepted, to cover certain cases of a specific bug case of uninitialized objects. I never liked the Object vs Object? syntax, that would just create ambiguity with the ternary conditional. We tried really hard to find a situation that was ambiguous, but without success. I think it's OK. D already dropped the template syntax from C++ to drop the ambiguity with <> and comparison operators. I wouldn't mind something like nonnull(Object), ie yet another type qualifier, although that would get cumbersome in a few cases like const(shared(nonnull(Object))) :o) Bye, bearophile
Re: Revamped concurrency API
On Tue, 13 Oct 2009 10:10:31 +0400, Jeremie Pelletier wrote: I never liked the Object vs Object? syntax, that would just create ambiguity with the ternary conditional. D already dropped the template syntax from C++ to drop the ambiguity with <> and comparison operators. С# uses this syntax, and there is no ambiguity.
Re: Revamped concurrency API
On 2009-10-13 02:16:30 -0400, Andrei Alexandrescu said: P.S. Beer? Coffee? Marriage? I'll be in Quebec City on Dec 10th and 11th for a talk. So you'll be in my city too. :-) How many of us are in Quebec City? -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Revamped concurrency API
Jeremie Pelletier, el 13 de octubre a las 01:59 me escribiste: > Leandro Lucarella wrote: > >Jeremie Pelletier, el 12 de octubre a las 22:45 me escribiste: > >>>I agree. Particularly about lent. Immutable and mutable weren't > >>>considered complete without const, so I'm surprised that local and > >>>shared are considered complete without lent. You can even > >>>implement it without escape analysis. Strangely, from what I > >>>remember of Bartosz's posts, it was unique that was the sticking > >>>point, though you can implement both unique and owned as library > >>>types (though they do become less efficient). > >>I may sound ignorant, but that does lent means? I mean the word > >>itself, I couldn't find something relevant on google. > > > >The terms "unique", "shared" and "lent" used here generaly refers to > >the terms used by Bartoz Milewski in his blog[1]. > > > >I think he defines lent in this particular blog post[2], but maybe you > >might need to read a couple of posts more to understand everything. > > > >[1] http://bartoszmilewski.wordpress.com/ > >[2] > >http://bartoszmilewski.wordpress.com/2009/05/21/unique_ptr-how-unique-is-it/ > > > > I know of this article, he mentions the word but he doesn't say > where it comes from. > > My native language is french (I'm from Quebec, Canada), I don't know > every word of the english dictionary. That's why I asked, I'm just > curious as to what it meant. Woops! Sorry, I understood exactly the opposite from you mail =/ -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ -- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) -- Sus discipulos se miraron sin entended hasta que uno preguntose: Peperino, soy Daniel Q. de Olivos tengo 54 años y aún soy virgen. A lo que Peperino respondiole: Si sos ganso, ganso ser. Y lo frotó, y lo curó y lo sanó. A lo que todos dijeron: ¡¡¡Peperino se la come, Peperino se la come!!! -- Peperino Pómoro
Re: Revamped concurrency API
MIURA Masahiro wrote: > Jeremie Pelletier wrote: >> I also don't believe one model is "ruling them all". > > Let me clarity this, just in case I have caused an unnecessary > confusion: I think Sean's Erlang-like API is meant to coexist > with the current core.thread, and that you can use one or both > of them to build higher-level concurrency models. > > I think it's nice to have core.thread *and* message-passing API > in Phobos. Thread alone is too primitive for daily use, but > we don't want to have too many concurrency models. Absolutely! I agree. We need to attack the beast with everything we have. Andrei
Re: Revamped concurrency API
MIURA Masahiro Wrote: > Jeremie Pelletier wrote: > > I also don't believe one model is "ruling them all". > > Let me clarity this, just in case I have caused an unnecessary > confusion: I think Sean's Erlang-like API is meant to coexist > with the current core.thread, and that you can use one or both > of them to build higher-level concurrency models. Exactly. That's also why I settled on the Erlang-like API instead of something like coroutines. It's possible to build a coroutine API on top of the Erlang API, but not vice-versa. > I think it's nice to have core.thread *and* message-passing API > in Phobos. Thread alone is too primitive for daily use, but > we don't want to have too many concurrency models. I see core as holding language-essential stuff and "close to the metal" APIs, and have no inclination of hiding core.thread or core.sync from the user. If someone wants to work at this level and they understand the risks they should be able to. Any higher-level stuff like the message-passing API I proposed would live in Phobos and sit on top of what's in core. I disagree about poor performance though. With unique references or move semantics, a copy of even complex data isn't necessary to ensure that a message is passed safely.
Re: Revamped concurrency API
On 2009-10-13 11:39:21 -0400, Sean Kelly said: I disagree about poor performance though. With unique references or move semantics, a copy of even complex data isn't necessary to ensure that a message is passed safely. Yeah, but for unique reference to be usable you need lent semantics, otherwise you can't make sure you're the unique holder once you call a function to do something with your unique object. Anything that use the reference could be storing it elsewhere: unique!Object o = new Object; o.doSomething(); How in the example above can I enforce that doSomething() won't escape the 'o' reference elsewhere? 'doSomething' could be made 'pure', but that trick will only work for one-argument functions (in the case above, the argument is the implicit 'this') because otherwise a pure function could leak the reference through another argument. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Revamped concurrency API
On Tue, 13 Oct 2009 11:19:30 -0400, Andrei Alexandrescu wrote: MIURA Masahiro wrote: Jeremie Pelletier wrote: I also don't believe one model is "ruling them all". Let me clarity this, just in case I have caused an unnecessary confusion: I think Sean's Erlang-like API is meant to coexist with the current core.thread, and that you can use one or both of them to build higher-level concurrency models. I think it's nice to have core.thread *and* message-passing API in Phobos. Thread alone is too primitive for daily use, but we don't want to have too many concurrency models. Absolutely! I agree. We need to attack the beast with everything we have. Andrei I'd recommend reading Bartosz's blog on thread objects vs spawn (http://bartoszmilewski.wordpress.com/2009/09/01/spawning-a-thread-the-d-way/). It makes a really good case for why thread objects should never be sub-classed and therefore should be a private, hidden implementation detail and not a public API.
Re: Revamped concurrency API
Robert Jacques wrote: On Tue, 13 Oct 2009 11:19:30 -0400, Andrei Alexandrescu wrote: MIURA Masahiro wrote: Jeremie Pelletier wrote: I also don't believe one model is "ruling them all". Let me clarity this, just in case I have caused an unnecessary confusion: I think Sean's Erlang-like API is meant to coexist with the current core.thread, and that you can use one or both of them to build higher-level concurrency models. I think it's nice to have core.thread *and* message-passing API in Phobos. Thread alone is too primitive for daily use, but we don't want to have too many concurrency models. Absolutely! I agree. We need to attack the beast with everything we have. Andrei I'd recommend reading Bartosz's blog on thread objects vs spawn (http://bartoszmilewski.wordpress.com/2009/09/01/spawning-a-thread-the-d-way/). It makes a really good case for why thread objects should never be sub-classed and therefore should be a private, hidden implementation detail and not a public API. Threads should NOT be private, I don't mind making them final classes but some programs may not require, or some programmers may not want, the high level constructs. If you put the thread construct in core.thread and the concurrent implementations in std.concurrent you still need the std api to access the core threads. But threads being a core module makes it a "here it is should you want it, but check these standard modules first" kind of module. Threads being public also allow any programmer to build custom concurrent models if they're not in the library. Jeremie
Re: Revamped concurrency API
Robert Jacques wrote: On Tue, 13 Oct 2009 11:19:30 -0400, Andrei Alexandrescu wrote: MIURA Masahiro wrote: Jeremie Pelletier wrote: I also don't believe one model is "ruling them all". Let me clarity this, just in case I have caused an unnecessary confusion: I think Sean's Erlang-like API is meant to coexist with the current core.thread, and that you can use one or both of them to build higher-level concurrency models. I think it's nice to have core.thread *and* message-passing API in Phobos. Thread alone is too primitive for daily use, but we don't want to have too many concurrency models. Absolutely! I agree. We need to attack the beast with everything we have. Andrei I'd recommend reading Bartosz's blog on thread objects vs spawn (http://bartoszmilewski.wordpress.com/2009/09/01/spawning-a-thread-the-d-way/). It makes a really good case for why thread objects should never be sub-classed and therefore should be a private, hidden implementation detail and not a public API. I reviewed that article so I guess I read it :o). Bartosz's position is held by many people in the C++ threading community. Andrei
Re: Revamped concurrency API
Michel Fortin Wrote: > On 2009-10-13 11:39:21 -0400, Sean Kelly said: > > > I disagree about poor performance though. With unique references or > > move semantics, a copy of even complex data isn't necessary to ensure > > that a message is passed safely. > > Yeah, but for unique reference to be usable you need lent semantics, > otherwise you can't make sure you're the unique holder once you call a > function to do something with your unique object. > > Anything that use the reference could be storing it elsewhere: > > unique!Object o = new Object; > o.doSomething(); > > How in the example above can I enforce that doSomething() won't escape > the 'o' reference elsewhere? 'doSomething' could be made 'pure', but > that trick will only work for one-argument functions (in the case > above, the argument is the implicit 'this') because otherwise a pure > function could leak the reference through another argument. I honestly don't know how to enforce this-- I simply mentioned it because people have suggested it. Phobos already has assumeUnique() for converting to invariant, which is one option. I was hoping someone could suggest alternatives.
Re: Revamped concurrency API
Nick B Wrote: > Nick B wrote: > > Andrei Alexandrescu wrote: > >> bearophile wrote: > >>> Andrei Alexandrescu: > >>> > Unfortunately, Bartosz has declined to contribute.< > >>> > >>> I have read a good amount of his interesting blog posts, he has shown > >>> me many things I didn't know about. And he was very happy about the > >>> idea of helping D concurrency. Do you know why he has changed his > >>> mind regarding this? Maybe because Walter has refused the > >>> uniqueness/lend ideas? > >> > >> You may want to email Bartosz and ask him. > >> > >> Andrei > > > > I will ask him, via his blog, and then will post the link. > > > > Nick B. > > here it is. > > http://bartoszmilewski.wordpress.com/2009/09/22/ownership-systems-against-data-races/#comment-922 > > Nick B. It's good to know that my ideas are still circulating in the D community. Thanks, guys! Since I'm now put on the spot, I have to explain my recent detachment from D. I didn't so much "decline to contribute" as hit a wall. I'm a bit of a perfectionist and it's hard for me to subscribe to the "good enough" philosophy (as long as it's better that C++, it's fine for D). My impression is that, as the release of D2 and the publication of Andrei's book are nearing, this attitude is gaining ground. I try to fight this attitude but it's an uphill battle. Or, as Andrei puts it, I start whining and give up ;-). Particular issues that bother me are: The semantics of "shared." I can live with postponing the implementation of the race-free type system, but not with the compiler inserting barriers around all shared reads and writes, even inside synchronized sections. The C++-derived template metaprogramming mess. Especially when C++0x provides better support for variadic templates than D (see my upcoming blog). I fought successfully for non-functional approach to string mixins. The same is needed for more general templates. In my opinion, there should not be any part of the language that is understandable only by seasoned library writers. The "better than Java" aspect of D ignores the latest development in Java. In particular the work on non-null pointers and the use of annotations for type-system extensions. Annotations are being added to D as we speak because it finally became obvious that no amount of cleverness can make object properties work without additional syntax. So the patching begins, without a clear vision of the role of annotation in future D. As far as my thread work went, I had an almost working implementation of spawn, except for a nasty compiler bug which resisted all efforts to reduce it to a simple test case. Threads also required some clever work with templates (testing arguments to "spawn" for sharing violations). I implemented typelist.d in Phobos to make it easier, and learned how inflexible D templates were. At least they are better than C++, or are they?
Re: Revamped concurrency API
On 2009-10-13 18:26:04 -0400, Bartosz Milewski said: I didn't so much "decline to contribute" as hit a wall. I'm a bit of a perfectionist and it's hard for me to subscribe to the "good enough" philosophy (as long as it's better that C++, it's fine for D). My impression is that, as the release of D2 and the publication of Andrei's book are nearing, this attitude is gaining ground. I try to fight this attitude but it's an uphill battle. Or, as Andrei puts it, I start whining and give up ;-). I don't like that attitude either. I'd rather remove completely an unfinished feature than leave it there half-done. If the main feature of D2 is a new concurrency model, then leaving things in the state they are now isn't going to impress much. The semantics of "shared." I can live with postponing the implementation of the race-free type system, but not with the compiler inserting barriers around all shared reads and writes, even inside synchronized sections. I hadn't thought about that, but it's somewhat ridiculous. On my part I'm more preoccupied by the absence of lent semantics as it goes much deeper than concurrency: you need it to implement proper invariant constructors and unique references. The C++-derived template metaprogramming mess. Especially when C++0x provides better support for variadic templates than D (see my upcoming blog). I fought successfully for non-functional approach to string mixins. The same is needed for more general templates. In my opinion, there should not be any part of the language that is understandable only by seasoned library writers. I'm quite eager to read about that. The "better than Java" aspect of D ignores the latest development in Java. In particular the work on non-null pointers and the use of annotations for type-system extensions. Annotations are being added to D as we speak because it finally became obvious that no amount of cleverness can make object properties work without additional syntax. So the patching begins, without a clear vision of the role of annotation in future D. I'm not even sure what convinced Walter to put annotations in and what they're supposed to be capable of. I hope it's not just a new way to create keywords without them really being keywords. As far as my thread work went, I had an almost working implementation of spawn, except for a nasty compiler bug which resisted all efforts to reduce it to a simple test case. Threads also required some clever work with templates (testing arguments to "spawn" for sharing violations). I implemented typelist.d in Phobos to make it easier, and learned how inflexible D templates were. At least they are better than C++, or are they? I'm not that satisfied with D templates either. You can use them in mixins to achieve pretty amazing things, but I've been hindered by the visibility rules with non-mixin templates and how templates with alias arguments are difficult to instantiate in certain contexts. If it were me in charge of the D2 project, given the approaching deadline, I'd scrap the new concurrency model completely (including immutable, unless we are given a way to write safe constructor for it) and focus on everything else. In my view, that's better than leave it in the state it is now. (Note that I'm speaking of the model at the compiler level, creating new library code is a perfectly fine idea.) -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Revamped concurrency API
Bartosz Milewski: >It's good to know that my ideas are still circulating in the D community. >Thanks, guys!< They will keep circulating! And there's not just D in computer science, future computer languages may use your ideas. Someday I'll design my own C-like language, simpler (and more limited in scope) than D ;-) >Since I'm now put on the spot, I have to explain my recent detachment from D.< This makes me (and surely many other persons here) a little sad. D is an engineering project, and it's not small. And the first thing to learn about all sizeable real-world engineering projects (like the creation of a large bridge, or many other things) is that they are first of all politics. They are not the result of rationality, even if often some rationality is required. All large engineering project come out of a large amount of compromises. If you pretend to see one of such systems to come out of rationality alone, you will never see a real one. So every time you have to fight for the rationality to prevail, and you have to be ready to accept some compromises. It's life. >it's hard for me to subscribe to the "good enough" philosophy (as long as it's >better that C++, it's fine for D).< D language is largely grown, and not designed. D is not Scala or Haskell. I think there is not enough brain mass in this group to design rationally a C++-class language. Probably there's not even brain mass to design a Python-class language. D2 will surely have a big amount of design mistakes, every one here can list you some of them. But one of the best qualities of Walter is that he's not frozen, he sometimes seems to move very slowly, but he never stops moving, and usually he eventually sees the light (and when this happens it's usually a small surprise for everyone else, like the show of a rabbit pulled out of a hat). So I think some of the design mistakes of D2 will be fixed in D3 :-) >My impression is that, as the release of D2 and the publication of Andrei's >book are nearing, this attitude is gaining ground. I try to fight this >attitude but it's an uphill battle. Or, as Andrei puts it, I start whining and >give up ;-).< Andrei's book comes too much early, forcing rhythms that I think aren't good for D2. On the other hand no software is ever finished, it's just abandoned. And you must not think of D2 as the finished product :-) You may even think of D2 as just a bridge toward D3 :-) D2 is probably just an experiment, to experimentally see what design ideas are good and what to throw away and redesign. And such things are slow. So you may want to come back in few years, to try to improve the concurrency design of D3 :-) >The semantics of "shared." I can live with postponing the implementation of >the race-free type system, but not with the compiler inserting barriers around >all shared reads and writes, even inside synchronized sections.< This is a topic that can be discussed. If enough people think this is a problem, then eventually Walter may change his mind, or he may design something better, etc. If no one discussed problems they can't be fixed/improved. I know that discussing things and accepting compromises is a pain, but it's an inevitable pain. Rationality alone usually never wins, you need politics too. >The C++-derived template metaprogramming mess. Especially when C++0x provides >better support for variadic templates than D (see my upcoming blog). I fought >successfully for non-functional approach to string mixins. The same is needed >for more general templates. In my opinion, there should not be any part of the >language that is understandable only by seasoned library writers.< Again, this seems a small piece of D2, so it may be fixed if enough people think this is bad. You can explain what do you want to change here, and people will see and think. Eventually it may even be fixed. >The "better than Java" aspect of D ignores the latest development in Java. In >particular the work on non-null pointers< We have discussed weeks about nonnull references in D (and I have even discussed about nonnull pointers too), I have done my best. A good percentage of people have said that they like to try this idea. Walter in the end was not interested (I think). But maybe in future the situation will change. There's always hope on this. At the end I have seen no one say "This is a totally wrong idea and will never see the light in D". >and the use of annotations for type-system extensions. Annotations are being >added to D as we speak because it finally became obvious that no amount of >cleverness can make object properties work without additional syntax. So the >patching begins, without a clear vision of the role of annotation in future D.< Annotations in D are just an embryo, and I think they come from no design yet, they are just a bit copied from Java. If you have a vision for their role in D, then it's a very good moment to talk and show such v
Re: Revamped concurrency API
Bartosz Milewski wrote: Nick B Wrote: Nick B wrote: Andrei Alexandrescu wrote: bearophile wrote: Andrei Alexandrescu: Unfortunately, Bartosz has declined to contribute.< I have read a good amount of his interesting blog posts, he has shown me many things I didn't know about. And he was very happy about the idea of helping D concurrency. Do you know why he has changed his mind regarding this? Maybe because Walter has refused the uniqueness/lend ideas? You may want to email Bartosz and ask him. Andrei I will ask him, via his blog, and then will post the link. Nick B. here it is. http://bartoszmilewski.wordpress.com/2009/09/22/ownership-systems-against-data-races/#comment-922 Nick B. It's good to know that my ideas are still circulating in the D community. Thanks, guys! Since I'm now put on the spot, I have to explain my recent detachment from D. I didn't so much "decline to contribute" as hit a wall. I'm a bit of a perfectionist and it's hard for me to subscribe to the "good enough" philosophy (as long as it's better that C++, it's fine for D). My impression is that, as the release of D2 and the publication of Andrei's book are nearing, this attitude is gaining ground. I try to fight this attitude but it's an uphill battle. Or, as Andrei puts it, I start whining and give up ;-). Please stay in touch, we will try to win you back... Of the issues you mention, this one seems the easiest to address: As far as my thread work went, I had an almost working implementation of spawn, except for a nasty compiler bug which resisted all efforts to reduce it to a simple test case. Could you give us _any_ kind of test case (even if it's enormous)?
Re: Revamped concurrency API
bearophile wrote: Bartosz Milewski: It's good to know that my ideas are still circulating in the D community. Thanks, guys!< They will keep circulating! And there's not just D in computer science, future computer languages may use your ideas. Someday I'll design my own C-like language, simpler (and more limited in scope) than D ;-) Since I'm now put on the spot, I have to explain my recent detachment from D.< This makes me (and surely many other persons here) a little sad. Same here, I read your blog with much interest and check it almost daily for updates, your ideas about concurrency are really inspiring! I'm also sure many of us here, me included, did our first steps in GUI programming years ago by following your tutorial on reliable software. D is an engineering project, and it's not small. And the first thing to learn about all sizeable real-world engineering projects (like the creation of a large bridge, or many other things) is that they are first of all politics. They are not the result of rationality, even if often some rationality is required. All large engineering project come out of a large amount of compromises. If you pretend to see one of such systems to come out of rationality alone, you will never see a real one. So every time you have to fight for the rationality to prevail, and you have to be ready to accept some compromises. It's life. I still have faith that Walter and Andrei can filter out all the ideas here no matter how heated up the arguments and come up with a design that will be a decent compromise in the requested features and design changes. it's hard for me to subscribe to the "good enough" philosophy (as long as it's better that C++, it's fine for D).< D language is largely grown, and not designed. D is not Scala or Haskell. I think there is not enough brain mass in this group to design rationally a C++-class language. Probably there's not even brain mass to design a Python-class language. D2 will surely have a big amount of design mistakes, every one here can list you some of them. But one of the best qualities of Walter is that he's not frozen, he sometimes seems to move very slowly, but he never stops moving, and usually he eventually sees the light (and when this happens it's usually a small surprise for everyone else, like the show of a rabbit pulled out of a hat). So I think some of the design mistakes of D2 will be fixed in D3 :-) The brain mass is there, more than enough too. Its the manpower that lacks, we propose way more features than we have manpower to implement in time to either release them or properly test them and see if the design works as good as wanted. A lot of us would like to contribute to the compiler and library but we're already busy with our daily jobs, maybe Walter needs to hire half this newsgroup and build us an office :o) My impression is that, as the release of D2 and the publication of Andrei's book are nearing, this attitude is gaining ground. I try to fight this attitude but it's an uphill battle. Or, as Andrei puts it, I start whining and give up ;-).< Andrei's book comes too much early, forcing rhythms that I think aren't good for D2. On the other hand no software is ever finished, it's just abandoned. And you must not think of D2 as the finished product :-) You may even think of D2 as just a bridge toward D3 :-) D2 is probably just an experiment, to experimentally see what design ideas are good and what to throw away and redesign. And such things are slow. So you may want to come back in few years, to try to improve the concurrency design of D3 :-) I agree here, while I think Andrei's book is a must for the community to grow, it should rush the compiler. I strongly believe D2 needs proper shared and lent semantics, or else we're just creating a stepping stone for D3, leaving D2 in a state that isn't suited for concurrent model implementations in a type-safe manner. I don't mind however getting the spec ready with the features coming in the near future as part of D2 (either in patch releases or minor releases, ie 2.1). The semantics of "shared." I can live with postponing the implementation of the race-free type system, but not with the compiler inserting barriers around all shared reads and writes, even inside synchronized sections.< This is a topic that can be discussed. If enough people think this is a problem, then eventually Walter may change his mind, or he may design something better, etc. If no one discussed problems they can't be fixed/improved. I know that discussing things and accepting compromises is a pain, but it's an inevitable pain. Rationality alone usually never wins, you need politics too. I brought up this topic a number of time without much success. Shared is hard to work with right now without tons of casting. It's also way too easy to accidentally write recursive locks with the current synchronized model, which is a nice fa
Re: Revamped concurrency API
Jeremie Pelletier wrote: I agree here, while I think Andrei's book is a must for the community to grow, it should rush the compiler. I strongly believe D2 needs proper shared and lent semantics, or else we're just creating a stepping stone for D3, leaving D2 in a state that isn't suited for concurrent model implementations in a type-safe manner. s/should rush/shouldn't rush/
Re: Revamped concurrency API
Jeremie Pelletier wrote: Jeremie Pelletier wrote: I agree here, while I think Andrei's book is a must for the community to grow, it should rush the compiler. I strongly believe D2 needs proper shared and lent semantics, or else we're just creating a stepping stone for D3, leaving D2 in a state that isn't suited for concurrent model implementations in a type-safe manner. s/should rush/shouldn't rush/ For the record, I delayed signing for the book for quite a few months. Before I did sign, there was solemn agreement in the D team that we can commit. The book is already late by a few months. If the book is trying to rush anything, it's doing a lousy job at that. Andrei
Re: Revamped concurrency API
Andrei Alexandrescu wrote: For the record, I delayed signing for the book for quite a few months. Before I did sign, there was solemn agreement in the D team that we can commit. [snip] Commit to what, exactly ? Nick B
Re: Revamped concurrency API
Nick B wrote: Andrei Alexandrescu wrote: For the record, I delayed signing for the book for quite a few months. Before I did sign, there was solemn agreement in the D team that we can commit. [snip] Commit to what, exactly ? Nick B Commit to delivering the book and the language matching it. Andrei
Re: Revamped concurrency API
Andrei Alexandrescu wrote: Commit to what, exactly ? Nick B Commit to delivering the book and the language matching it. Andrei To meet a final release of D 2.0 ? Nick B
Re: Revamped concurrency API
Nick B wrote: Andrei Alexandrescu wrote: Commit to what, exactly ? Nick B Commit to delivering the book and the language matching it. Andrei To meet a final release of D 2.0 ? Nick B Yes, that's the plan. D2 final release begets TDPL, and TDPL describes the final release of D2. Andrei
Re: Revamped concurrency API
bearophile Wrote: > > >it's hard for me to subscribe to the "good enough" philosophy (as long as > >it's better that C++, it's fine for D).< > > D language is largely grown, and not designed. D is not Scala or Haskell. > ... > So I think some of the design mistakes of D2 will be fixed in D3 :-) > With every release of D we are narrowing our options. After D2 and TDPL, backward compatibility will become a major thing, so every ad-hoc feature in D2 will have to be carried over. Why is C++ so bad? Not for lack of brainpower. It's because it fell into the compatibility trap (essentially from day one). There are some areas of D, like Andrei's ranges, that are well thought out and theoretically sound (we hope), but there are also hacks upon hacks that make my hair stand on end.
Re: Revamped concurrency API
Don wrote: Bartosz Milewski wrote: [snip] Please stay in touch, we will try to win you back... Of the issues you mention, this one seems the easiest to address: As far as my thread work went, I had an almost working implementation of spawn, except for a nasty compiler bug which resisted all efforts to reduce it to a simple test case. Could you give us _any_ kind of test case (even if it's enormous)? Bartosz - are you able to provide a test case as requested by Don ? Then it might be possible, to get this bug fixed. Nick B.
Re: Revamped concurrency API
Nick B Wrote: > > > Could you give us _any_ kind of test case (even if it's enormous)? > > Bartosz - are you able to provide a test case as requested by Don ? > Then it might be possible, to get this bug fixed. > > Nick B. I can send you the files I have checked out. The problem was in core.thread. I tried to implement a struct Tid (thread ID) with reference-counting semantics and deterministic destruction. It passed all the tests, but when it was used in one particular place in druntime it produced incorrect assembly. Even the slightest change made the bug disappear, so I wasn't able to reproduce it under controlled conditions. Unfortunately, I have undone some of my changes trying to bypass the bug, so at the moment I don't even have the buggy version, but it can be reconstructed. We can discuss it off-line, if you want. Use my email address with -nospam removed.
Re: Revamped concurrency API
On Wed, Oct 14, 2009 at 05:56:10PM -0400, Bartosz Milewski wrote: > The problem was in core.thread. I tried to implement a struct Tid (thread ID) > with reference-counting semantics and deterministic destruction. It passed > all the tests, but when it was used in one particular place in druntime it > produced incorrect assembly. Even the slightest change made the bug > disappear, so I wasn't able to reproduce it under controlled conditions. Was it by any chance a null this pointer making its way into the ebx register somehow, thus causing an access violation on Windows seemingly at random? (My bug didn't happen if I compiled the exact same code on Linux - I figure it must be a Windows codegen problem.) If so, I've been fighting that bug for over two years, similarly unable to track down a small test case. I haven't put it on bugzilla since I can't even really prove it exists to an outsider. I just keep hoping that the next dmd release will magically fix it. -- Adam D. Ruppe http://arsdnet.net
Re: Revamped concurrency API
Bartosz Milewski wrote: Nick B Wrote: Could you give us _any_ kind of test case (even if it's enormous)? Bartosz - are you able to provide a test case as requested by Don ? Then it might be possible, to get this bug fixed. Nick B. I can send you the files I have checked out. The problem was in core.thread. I tried to implement a struct Tid (thread ID) with reference-counting semantics and deterministic destruction. It passed all the tests, but when it was used in one particular place in druntime it produced incorrect assembly. Even the slightest change made the bug disappear, so I wasn't able to reproduce it under controlled conditions. Unfortunately, I have undone some of my changes trying to bypass the bug, so at the moment I don't even have the buggy version, but it can be reconstructed. We can discuss it off-line, if you want. Use my email address with -nospam removed. Bartosz I think that Don is the best person to contact you. I will try to contact him. Nick B
Re: Revamped concurrency API
Bartosz Milewski: >With every release of D we are narrowing our options. After D2 and TDPL, >backward compatibility will become a major thing, so every ad-hoc feature in >D2 will have to be carried over.< D is a bit compatible with the C language, but one of the main selling points of D (D1, D2, D3...) is its newer and cleaned up nature. So I think D3 will break compatibility with frozen-D2 in many places, trying to fix the design errors of D2. D3 will not have a quick release schedule, I think it will try to learn from D2 and D1 in a more thought-out way, because D2 will be an essentially complete language, so the purpose of D3 will be mostly to improve things replacing features with better ones that do similar things :-) For example in D3 the switch may use three ... points as in the GCC extension, and so on. In the meantime people will often use D1 (until LDC becomes a good D2 compiler). Bye, bearophile
Re: Revamped concurrency API
bearophile wrote: Bartosz Milewski: With every release of D we are narrowing our options. After D2 and TDPL, backward compatibility will become a major thing, so every ad-hoc feature in D2 will have to be carried over.< D is a bit compatible with the C language, but one of the main selling points of D (D1, D2, D3...) is its newer and cleaned up nature. So I think D3 will break compatibility with frozen-D2 in many places, trying to fix the design errors of D2. D3 will not have a quick release schedule, I think it will try to learn from D2 and D1 in a more thought-out way, because D2 will be an essentially complete language, so the purpose of D3 will be mostly to improve things replacing features with better ones that do similar things :-) For example in D3 the switch may use three ... points as in the GCC extension, and so on. In the meantime people will often use D1 (until LDC becomes a good D2 compiler). Bye, bearophile Speaking of switch, I have tried to convince Walter to require either a break; or a goto case xxx; at the end of each snippet inside a switch. I was surprised by his answer: "but I use fall through all the time!" :o) I personally think requiring a goto case xxx; is more robust in presence of code maintenance because its semantics is invariant to code moves. Andrei
Re: Revamped concurrency API
bearophile Wrote: > Jeremie Pelletier: > > > I vote for a 'lent' qualifier to be implemented in the compiler. > > You may be right, and Bartosz has explained me why and how to use lent. It > sounds like a nice idea, but it has some costs too (in language complexity, > for example). But I need to use such things in practice, in real (even small) > programs to know/understand if I like and understand them enough. > > So I'd like to have an experimental version of D where such ideas (like lend, > nonnull references, and few other things) can be tried, and if they are > experimentally seen as not good (by me too), they can be discarded and > removed. For me this is true for nonnull references, but such experimental > need is even strong for lend. > > It seem all the long discussion regarding nonnull references has gone > nowhere. That has made me a little sad. Only a bit of flow analysis has being > accepted, to cover certain cases of a specific bug case of uninitialized > objects. > This is where Java shines. Their annotation system is flexible enough to allow this kind of experimenting. In particular they were able to implement const and nonNull using annotations. See the talk by Mike Ernst from the University of Washington: http://www.vimeo.com/4368251 .
Re: Revamped concurrency API
On Thu, Oct 15, 2009 at 12:47 PM, Andrei Alexandrescu wrote: > bearophile wrote: >> >> Bartosz Milewski: >> >>> With every release of D we are narrowing our options. After D2 and TDPL, >>> backward compatibility will become a major thing, so every ad-hoc feature in >>> D2 will have to be carried over.< >> >> D is a bit compatible with the C language, but one of the main selling >> points of D (D1, D2, D3...) is its newer and cleaned up nature. >> So I think D3 will break compatibility with frozen-D2 in many places, >> trying to fix the design errors of D2. D3 will not have a quick release >> schedule, I think it will try to learn from D2 and D1 in a more thought-out >> way, because D2 will be an essentially complete language, so the purpose of >> D3 will be mostly to improve things replacing features with better ones that >> do similar things :-) For example in D3 the switch may use three ... points >> as in the GCC extension, and so on. In the meantime people will often use D1 >> (until LDC becomes a good D2 compiler). >> >> Bye, >> bearophile > > Speaking of switch, I have tried to convince Walter to require either a > break; or a goto case xxx; at the end of each snippet inside a switch. I was > surprised by his answer: "but I use fall through all the time!" :o) But only if there's _some_ statement in between cases, right? case 1: case 3: break; should still be allowed. > I personally think requiring a goto case xxx; is more robust in presence of > code maintenance because its semantics is invariant to code moves. Agreed. --bb
Re: Revamped concurrency API
On 2009-10-15 21:47:05 +0200, Andrei Alexandrescu said: Speaking of switch, I have tried to convince Walter to require either a break; or a goto case xxx; at the end of each snippet inside a switch. I was surprised by his answer: "but I use fall through all the time!" :o) that shows that he unrolls loops ;) basically the only case I know where fall through is really useful Fawzi
Re: Revamped concurrency API
Bill Baxter wrote: case 1: case 3: break; should still be allowed. --bb Or replace with: case 1, 3: break; -- Chris Nicholson-Sauls
Re: Revamped concurrency API
Andrei Alexandrescu wrote: Speaking of switch, I have tried to convince Walter to require either a break; or a goto case xxx; at the end of each snippet inside a switch. I was surprised by his answer: "but I use fall through all the time!" :o) I personally think requiring a goto case xxx; is more robust in presence of code maintenance because its semantics is invariant to code moves. Andrei First of all goto case is without a doubt safer and more robust but please leave as much of D as possible to remain compatible with C. Since D has objects a lot of code can be polymorphic through the classes/interfaces that C didn't have. C's design is trust the programmer, provide full power. It is unsafe agreed but by design. Breaking compatibility between D1, D2 etc may be an issue but if you loose the C then you lose what defines D.
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Nick B wrote: Bartosz Milewski wrote: Nick B Wrote: Could you give us _any_ kind of test case (even if it's enormous)? Bartosz - are you able to provide a test case as requested by Don ? Then it might be possible, to get this bug fixed. Nick B. I can send you the files I have checked out. The problem was in core.thread. I tried to implement a struct Tid (thread ID) with reference-counting semantics and deterministic destruction. It passed all the tests, but when it was used in one particular place in druntime it produced incorrect assembly. Even the slightest change made the bug disappear, so I wasn't able to reproduce it under controlled conditions. Unfortunately, I have undone some of my changes trying to bypass the bug, so at the moment I don't even have the buggy version, but it can be reconstructed. We can discuss it off-line, if you want. Use my email address with -nospam removed. Bartosz I think that Don is the best person to contact you. I will try to contact him. Nick B Don, are you able to contact Bartosz, re the details of this test case. Nick B
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Nick B wrote: Nick B wrote: Bartosz Milewski wrote: Nick B Wrote: Could you give us _any_ kind of test case (even if it's enormous)? Bartosz - are you able to provide a test case as requested by Don ? Then it might be possible, to get this bug fixed. Nick B. I can send you the files I have checked out. The problem was in core.thread. I tried to implement a struct Tid (thread ID) with reference-counting semantics and deterministic destruction. It passed all the tests, but when it was used in one particular place in druntime it produced incorrect assembly. Even the slightest change made the bug disappear, so I wasn't able to reproduce it under controlled conditions. Unfortunately, I have undone some of my changes trying to bypass the bug, so at the moment I don't even have the buggy version, but it can be reconstructed. We can discuss it off-line, if you want. Use my email address with -nospam removed. Bartosz I think that Don is the best person to contact you. I will try to contact him. Nick B Don, are you able to contact Bartosz, re the details of this test case. Nick B Bartosz has sent it to me. I can reproduce the error. It's my top priority, but it'll take a while -- it's nasty.
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Don, el 20 de octubre a las 09:00 me escribiste: > >>>Unfortunately, I have undone some of my changes trying to > >>>bypass the bug, so at the moment I don't even have the buggy > >>>version, but it can be reconstructed. We can discuss it > >>>off-line, if you want. Use my email address with -nospam > >>>removed. > >> > >>Bartosz > >> > >>I think that Don is the best person to contact you. I will try > >>to contact him. > >> > >>Nick B > > > >Don, are you able to contact Bartosz, re the details of this test case. > > > >Nick B > > Bartosz has sent it to me. I can reproduce the error. It's my top > priority, but it'll take a while -- it's nasty. Do you have a bugzilla # so we can keep track of it? Thanks. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ -- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) -- HACIA NEUQUEN: EL JUEVES SALDRA CARAVANA CON PERROS DESDE CAPITAL EN APOYO AL CACHORRO CONDENADO A MUERTE -- Crónica TV
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Don wrote: Don, are you able to contact Bartosz, re the details of this test case. Nick B Bartosz has sent it to me. I can reproduce the error. It's my top priority, but it'll take a while -- it's nasty. Don - thanks for filing this. I did try to contact you, via bugzilla email (which bounced back)and via Skype,(no reply), without success. Nick B
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Nick B wrote: Don wrote: Don, are you able to contact Bartosz, re the details of this test case. Nick B Bartosz has sent it to me. I can reproduce the error. It's my top priority, but it'll take a while -- it's nasty. Don - thanks for filing this. I did try to contact you, via bugzilla email (which bounced back)and via Skype,(no reply), without success. Nick B All my public email addresses are fake. Bugzilla is just spam bait. It clearly comes from a more innocent age. I once made the mistake of submitting a bug to gcc. Although the GCC Bugzilla hides the email in the submitter field, the idiots include the real email address in the 100% useless 'X-Ref' line. Two weeks later, my email address was dead. I've never made that mistake again. I am contactable through my dsource account.
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Don, el 21 de octubre a las 09:46 me escribiste: > Nick B wrote: > >Don wrote: > > > > > >>> > >>>Don, are you able to contact Bartosz, re the details of this test case. > >>> > >>>Nick B > >> > >>Bartosz has sent it to me. I can reproduce the error. It's my > >>top priority, but it'll take a while -- it's nasty. > > > >Don - thanks for filing this. I did try to contact you, via > >bugzilla email (which bounced back)and via Skype,(no reply), > >without success. > > > >Nick B > > > All my public email addresses are fake. Bugzilla is just spam bait. > It clearly comes from a more innocent age. I once made the mistake > of submitting a bug to gcc. Although the GCC Bugzilla hides the > email in the submitter field, the idiots include the real email > address in the 100% useless 'X-Ref' line. Two weeks later, my email > address was dead. I've never made that mistake again. > I am contactable through my dsource account. You don't even use a real account for personal e-mails! That's odd. I couldn't reply your private e-mail about changing the title of the bugzilla report because the From address were fake. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ -- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) -- Hoy estuvimos en el museo de antropología, pero yo voy a volver para estar por lo menos un día ahí adentro... es una locura, como Disney pero de indigenas -- Carla Lucarella (10/2008 contando de su viaje a México)
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Leandro Lucarella wrote: Don, el 21 de octubre a las 09:46 me escribiste: All my public email addresses are fake. Bugzilla is just spam bait. It clearly comes from a more innocent age. I once made the mistake of submitting a bug to gcc. Although the GCC Bugzilla hides the email in the submitter field, the idiots include the real email address in the 100% useless 'X-Ref' line. Two weeks later, my email address was dead. I've never made that mistake again. I am contactable through my dsource account. You don't even use a real account for personal e-mails! That's odd. I couldn't reply your private e-mail about changing the title of the bugzilla report because the From address were fake. Yes, I too use fake personal email accounts, as I do not want my gmail accounts hammered with spam. I could add -nospam to the name, but I am unsure how good the spammers are these days to get around this. Any suggestions/comments, anyone ? Nick B
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Nick B wrote: Leandro Lucarella wrote: Don, el 21 de octubre a las 09:46 me escribiste: All my public email addresses are fake. Bugzilla is just spam bait. It clearly comes from a more innocent age. I once made the mistake of submitting a bug to gcc. Although the GCC Bugzilla hides the email in the submitter field, the idiots include the real email address in the 100% useless 'X-Ref' line. Two weeks later, my email address was dead. I've never made that mistake again. I am contactable through my dsource account. You don't even use a real account for personal e-mails! That's odd. I couldn't reply your private e-mail about changing the title of the bugzilla report because the From address were fake. Yes, I too use fake personal email accounts, as I do not want my gmail accounts hammered with spam. I could add -nospam to the name, but I am unsure how good the spammers are these days to get around this. Any suggestions/comments, anyone ? Nick B If you have a domain, my low-tech approach seems to work pretty well. Andrei
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Nick B, el 22 de octubre a las 19:35 me escribiste: > Leandro Lucarella wrote: > >Don, el 21 de octubre a las 09:46 me escribiste: > >> > >>All my public email addresses are fake. Bugzilla is just spam bait. > >>It clearly comes from a more innocent age. I once made the mistake > >>of submitting a bug to gcc. Although the GCC Bugzilla hides the > >>email in the submitter field, the idiots include the real email > >>address in the 100% useless 'X-Ref' line. Two weeks later, my email > >>address was dead. I've never made that mistake again. > >>I am contactable through my dsource account. > > > >You don't even use a real account for personal e-mails! That's odd. > >I couldn't reply your private e-mail about changing the title of the > >bugzilla report because the From address were fake. > > > Yes, I too use fake personal email accounts, as I do not want my > gmail accounts hammered with spam. I could add -nospam to the name, > but I am unsure how good the spammers are these days to get around > this. > > Any suggestions/comments, anyone ? I just use my real e-mail everywhere and let the spammers eat the world's BW as they please. I just let my anti-spam (bogofilter and gmail in case of the gmail address) to take care of things... -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ -- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) -- Que importante, entonces en estos días de globalización refregar nuestras almas, pasarle el lampazo a nuestros corazones para alcanzar un verdadero estado de babia peperianal. -- Peperino Pómoro
Re: Revamped concurrency API (Don can you contact Bartosz ?)
On Thu, 22 Oct 2009 17:32:00 +0400, Leandro Lucarella wrote: Nick B, el 22 de octubre a las 19:35 me escribiste: Leandro Lucarella wrote: >Don, el 21 de octubre a las 09:46 me escribiste: >> >>All my public email addresses are fake. Bugzilla is just spam bait. >>It clearly comes from a more innocent age. I once made the mistake >>of submitting a bug to gcc. Although the GCC Bugzilla hides the >>email in the submitter field, the idiots include the real email >>address in the 100% useless 'X-Ref' line. Two weeks later, my email >>address was dead. I've never made that mistake again. >>I am contactable through my dsource account. > >You don't even use a real account for personal e-mails! That's odd. >I couldn't reply your private e-mail about changing the title of the >bugzilla report because the From address were fake. > Yes, I too use fake personal email accounts, as I do not want my gmail accounts hammered with spam. I could add -nospam to the name, but I am unsure how good the spammers are these days to get around this. Any suggestions/comments, anyone ? I just use my real e-mail everywhere and let the spammers eat the world's BW as they please. I just let my anti-spam (bogofilter and gmail in case of the gmail address) to take care of things... So do I. And as a result I missed a very important message sent to me about a week ago (I was classified as a spam for some reason) :(
Re: Revamped concurrency API (Don can you contact Bartosz ?)
On Thu, 22 Oct 2009 09:32:00 -0400, Leandro Lucarella wrote: Nick B, el 22 de octubre a las 19:35 me escribiste: Leandro Lucarella wrote: >Don, el 21 de octubre a las 09:46 me escribiste: >> >>All my public email addresses are fake. Bugzilla is just spam bait. >>It clearly comes from a more innocent age. I once made the mistake >>of submitting a bug to gcc. Although the GCC Bugzilla hides the >>email in the submitter field, the idiots include the real email >>address in the 100% useless 'X-Ref' line. Two weeks later, my email >>address was dead. I've never made that mistake again. >>I am contactable through my dsource account. > >You don't even use a real account for personal e-mails! That's odd. >I couldn't reply your private e-mail about changing the title of the >bugzilla report because the From address were fake. > Yes, I too use fake personal email accounts, as I do not want my gmail accounts hammered with spam. I could add -nospam to the name, but I am unsure how good the spammers are these days to get around this. Any suggestions/comments, anyone ? I just use my real e-mail everywhere and let the spammers eat the world's BW as they please. I just let my anti-spam (bogofilter and gmail in case of the gmail address) to take care of things... Been posting on D for over 2 years with my real address. Haven't seen much difference in my spam input (except occasionally when Andrei replies directly to me instead of the newsgroup by accident ;) ) -Steve
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Don wrote: Nick B wrote: Nick B wrote: Bartosz Milewski wrote: Nick B Wrote: Could you give us _any_ kind of test case (even if it's enormous)? Bartosz - are you able to provide a test case as requested by Don ? Then it might be possible, to get this bug fixed. Nick B. I can send you the files I have checked out. The problem was in core.thread. I tried to implement a struct Tid (thread ID) with reference-counting semantics and deterministic destruction. It passed all the tests, but when it was used in one particular place in druntime it produced incorrect assembly. Even the slightest change made the bug disappear, so I wasn't able to reproduce it under controlled conditions. Unfortunately, I have undone some of my changes trying to bypass the bug, so at the moment I don't even have the buggy version, but it can be reconstructed. We can discuss it off-line, if you want. Use my email address with -nospam removed. Bartosz I think that Don is the best person to contact you. I will try to contact him. Nick B Don, are you able to contact Bartosz, re the details of this test case. Nick B Bartosz has sent it to me. I can reproduce the error. It's my top priority, but it'll take a while -- it's nasty. Don - can you advise the Bugzilla number for this ? thanks Nick B
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Nick B wrote: Don wrote: Nick B wrote: Nick B wrote: Bartosz Milewski wrote: Nick B Wrote: Could you give us _any_ kind of test case (even if it's enormous)? Bartosz - are you able to provide a test case as requested by Don ? Then it might be possible, to get this bug fixed. Nick B. I can send you the files I have checked out. The problem was in core.thread. I tried to implement a struct Tid (thread ID) with reference-counting semantics and deterministic destruction. It passed all the tests, but when it was used in one particular place in druntime it produced incorrect assembly. Even the slightest change made the bug disappear, so I wasn't able to reproduce it under controlled conditions. Unfortunately, I have undone some of my changes trying to bypass the bug, so at the moment I don't even have the buggy version, but it can be reconstructed. We can discuss it off-line, if you want. Use my email address with -nospam removed. Bartosz I think that Don is the best person to contact you. I will try to contact him. Nick B Don, are you able to contact Bartosz, re the details of this test case. Nick B Bartosz has sent it to me. I can reproduce the error. It's my top priority, but it'll take a while -- it's nasty. Don - can you advise the Bugzilla number for this ? thanks Nick B 3423. My patch has already been included in the dmd svn, and Bartosz has a compiled DMD with the bug fixed.
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Don wrote: Nick B wrote: Don wrote: Nick B wrote: Nick B wrote: Bartosz Milewski wrote: Nick B Wrote: Could you give us _any_ kind of test case (even if it's enormous)? Bartosz - are you able to provide a test case as requested by Don ? Then it might be possible, to get this bug fixed. Nick B. I can send you the files I have checked out. The problem was in core.thread. I tried to implement a struct Tid (thread ID) with reference-counting semantics and deterministic destruction. It passed all the tests, but when it was used in one particular place in druntime it produced incorrect assembly. Even the slightest change made the bug disappear, so I wasn't able to reproduce it under controlled conditions. Unfortunately, I have undone some of my changes trying to bypass the bug, so at the moment I don't even have the buggy version, but it can be reconstructed. We can discuss it off-line, if you want. Use my email address with -nospam removed. Bartosz I think that Don is the best person to contact you. I will try to contact him. Nick B Don, are you able to contact Bartosz, re the details of this test case. Nick B Bartosz has sent it to me. I can reproduce the error. It's my top priority, but it'll take a while -- it's nasty. Don - can you advise the Bugzilla number for this ? thanks Nick B 3423. My patch has already been included in the dmd svn, and Bartosz has a compiled DMD with the bug fixed. Don Thanks for the quick response. Hopefully Bartosz will be able to get a working implementation of spawn running (see his reply of 13 Oct 2009). Nick B
Re: Revamped concurrency API (Don can you contact Bartosz ?)
Don: > 3423. My patch has already been included in the dmd svn, and Bartosz has > a compiled DMD with the bug fixed. You are my third hero :-) (the first is Richard Feynman). Bye, bearophile
