Bartosz Milewski wrote:
Andrei Alexandrescu Wrote:
Scala doesn't know what to do about threads.
That's my impression too, although Scala's support for actors leaves
D in the dust.
Scala actors are a library.
The trend I'm seeing is that functional languages are getting
increasing attention, and that's exactly because they never share
mutable memory. As far as I can see, languages that are based on
heavy shared mutation are pretty much dead in the water. We have
the chance to not be so.
It's a very sweeping statement. I just looked at the TIOBE index and
couldn't find _any_ functional languages in the top 20.
We can safely ditch Tiobe, but I agree that functional languages aren't
mainstream. There are two trends though. One is that for most of its
existence Haskell has had about 12 users. It has definitely turned an
exponential elbow during the recent years. Similar trends are to be seen
for ML, Ocaml, and friends.
The other trend is that all of today's languages are scrambling to add
support for pure functional programming.
I'm not at all sure the focus must be put on high-level race
avoidance, particularly given that the cost in perceived complexity
is this high.
The complexity argument is tenuous. It might look like my proposal is
complex because I'm dropping the whole system in at once. But it's a
solid, well thought-out system. What I see happening in D is the
creeping complexity resulting from sloppy design. We've been talking
for years about closing some gaping holes in the design of arrays,
slices, immutable, qualifier polymorphism--the list goes on--and
there's little progress. There is no solid semantics for scope and
shared. A solid solution to those issues will look complex too.
What do those holes have to do with the problem at hand? I'm seeing
implementation bugs, not design holes. I'd love them to be fixed as much
as the next guy, but I don't think we're looking at issues that would be
complex (except for scope which sucks; I never claimed there was a
solution to that). All of other features you mention have no holes I
know of in their design.
Why wouldn't we think of making message passing a language feature
in D?
Because we don't have even a tiniest proposal describing it, not to
mention a design.
That doesn't mean we shouldn't think of it.
Why does message passing need erlangization to support message
passing?
Because the strength of the Erlang model is the isolation of
processes. Take away isolation and it's no better than Scala or Java.
Granted, having to explicitly mark objects for sharing in D is a big
help. Here we agree.
So we should be thinking about it, right?
What I discussed was a holistic approach in which language +
standard library provides a trusted computing base.
Have you thought about how to eliminate data races in your holistic
approach? Will "shared" be forbidden in SafeD? Will library-based
message-passing channels (and actors?) only accept simple value types
and immutables?
Shared will be allowed in SafeD but it will be the responsibility of
user code to ensure high-level race elimination. Shared will eliminate
low-level races.
I think it's worth contemplating a scenario in which message passing is
restricted to certain types.
Andrei, you are the grand wizard of squeezing powerful abstractions
out of a kludge of a language that is C++ with its ad-hoc support for
generics. It's impressive and very useful, but it's also hermetic. By
contrast, generic programming in D is relatively easy because of the
right kind of support built into the language (compile-time
interpreter).
Please no ad hominem, flattering or not.
I trust that you could squeeze powerful multithreading abstraction
out of D, even if the language/type system doesn't offer much of a
support. But it will be hermetic. Prove me wrong by implementing a
message queue using the current D2 (plus some things that are still
in the pipeline).
I don't have the time, but I think it's worth looking into what a
message queue implementation should look like.
How is this related to the discussion at hand? You want to put all
concurrency support in the language. That is, you want to put
enough power into the language to be able to typecheck a variety of
concurrent programming primitives and patterns. This approach is
blindsided to the opportunity of defining some of these primitives
in the standard library, in unsafe/unportable D, yet offering safe
primitives to the user. In the process, the user is not hurt
because she still has access to the primitives. What she can't do
is define their own primitives in safe D. But I think that's as
useless a pursuit as adding keywords to C to allow one to implement
malloc() in safe C.
That's a bad analogy. I'm proposing the tightening of the type
system, not the implementation of weak atomics. A better analogy
would be adding immutable/const to the language. Except that I don't
think const-correctness is as important as the safety of
shared-memory concurrency.
It's a good analogy because you want to make possible the implementation
of threading primitives in portable D. I am debating whether that is a
worthy goal. I doubt it is.
Without type-system support, though, such queues would have to be
either unsafe (no guarantee that the client doesn't have
unprotected aliases to messages), or restrict messages to very
simple data structures (pass-by-value and maybe some very clever
unique pointer/array template). The latter approach introduces
huge complexity into the library, essentially making user-defined
extensions impossible (unless the user's name is Andrei ;-) ).
Complexity will be somewhere. The problem is, you want to put much
of it in the language, and that will hit the language user too.
I'm being very careful not to hit the language user. You might have
noticed that my primitive channel, the MVar, is less complex than a
D2 implementation (it doesn't require "synchronized"). And the
compiler will immediately tell you if you use it in an unsafe way.
Semantic checking will always be harder on everyone than a human
being who sits down and implements a provably safe library in ways
that the compiler can't prove.
Be careful with such arguments. Somebody might use them to discredit
immutability.
Let them discredit it and we'll see how strong their argument is.
Let's not forget that right now D is _designed_ to support
shared-memory programming. Every D object has a lock, it supports
synchronized methods, the keyword "shared" is being introduced,
etc. It doesn't look like D is moving away from shared memory. It
looks more like it's adding some window dressing to the
pre-existing mess and bidding its time. I haven't seen a
comprehensive plan for D to tackle concurrency and I'm afraid
that if D doesn't take a solid stance right now, it will miss the
train.
I think we can safely ditch this argument. Walter had no idea what
to do about threads when he defined D, so he put whatever he saw
and understood in Java. He'd be the first to say that - actually,
he wouldn't tell me so, he _told_ me so.
I realize that. Except that the "shared" concept was added very
recently.
To me, adding concurrency capabilities to D is nothing like adding
window dressing on top of whatever crap is there. Java and C++ are
in trouble, and doing what they do doesn't strike me as a good bet.
So far D has been doing exactly what Java and C++ are doing. My
proposal goes way beyond that.
My problem is that I think it goes way beyond that straight in the wrong
directions. It goes on and on and on about how to make deadlock-oriented
programming less susceptible to races. I don't care about
deadlock-oriented programming. I want to stay away from
deadlock-oriented programming. I don't understand why I need a hecatomb
of concepts and notions that help me continue using a programming style
that is unrecommended.
But if you mean D should not support
shared-memory concurrency or give it only lip service, than you
really have to come up with something revolutionary to take its
place. This would obviously not make it into D2 or into your book. So
essentially D2 would doomed in the concurrency department.
Message passing and functional style have been around for a while and
form an increasingly compelling alternative to mutable sharing. We can
support deadlock-oriented programming in addition to these for those who
want it, but I don't think it's an area where we need to pay an arm and
a leg for eliminating high-level races. I just think it's the wrong
problem to work on.
Andrei
