On Tuesday, 20 December 2016 at 10:18:12 UTC, Kelly Sommers wrote:
As an outsider to the D community but someone who has really
wanted to love D the last few years I hope to shed some light
from "outside the bubble" on why I haven't used D and why I use
what I use and what I'm looking for. I hope this can be well
received.
I write a lot of C and Go. But they aren't what I truly want in
"systems" programming languages. I write high performance
infrastructure code like databases or server software that
handles millions of requests per second. One of my projects is
an OSS HTTP server written in C that can serve 10 million
requests/second.
Let's clarify a couple things about C and Go. C's
advantages/disadvantages are:
1. Manual memory management. This could also be seen as a
disadvantage but having the power to tailor memory access is an
advantage for memory usage, allocation and access
optimizafions. Let's face it, high perfowmcne code is all about
memory access.
2. No GC pauses impacting request response times. Java, Go,
.NET etc all have this problem.
3. Harder to write and write well, safely and securely.
4. Few concepts to learn.
5. Composability of libraries is poor. No package system, often
not cross platform.
6. A lot of low level fundamental libraries are written in C
like async I/O libraries or storage engine libraries.
7. Static compilation.
Go's advantages/disadvantages:
1. Static compilation.
2. No manual memory management.
3. Suffers from GC pauses.
4. Great composability of libraries. For example, you can
easily "go get" a Redis protocol library, an ACID memory mapped
persisted b+tree library and build a little database quite
quickly.
5. Few concepts to learn.
6. Performance overhead when calling C libraries like storage
engines.
7: The statement that Go is mostly used in scripting-like
contexts isn't correct. It's very popular in the infrastructure
space like databases and distributed systems implementations.
8. Lack of generics causes a lot of boilerplate code in
comparison to other modern languages.
As an engineer who works on distributed systems in the scale of
thousands of nodes I have to point out that GC'd languages need
to be thought more as in the same category because of how they
operate in production. You have to spend significant effort
keeping these processses happy so that the GC's don't pause too
much hurting response times. I argue all the time that GC'd
languages are not systems languages. There's not enough control
and you can't eliminate GC pauses completely and these GC's
don't scale to modern physical hardware well.
But Go is popular in this category of space despite the
GC/generics because of how composable infrastructure code is in
the Go community. "go get" a Raft library, gossip library,
storage library and you're off and running much faster than in
other languages. However the overhead of calling C libraries
and the GC impact performance. Go knows this weakness of the GC
and has been working hard to eliminate GC pauses as much as
possible. A great effort into sub-millisecond pauses with large
heaps was recently achieved:
https://github.com/golang/proposal/blob/master/design/17503-eliminate-rescan.md
What I really want is what C++ wanted to deliver but it
doesn't. I want something better than writing C but with the
same performance as C and the ability to interface with C
without the performance loss and with easily composable
libraries.
D in my opinion in some ways is close to these goals. It's
simpler to understand and write than C++. It has the problem of
being a GC'd language however and it's unclear to me if the GC
in D is evolving like the Go GC is. I'm not happy about having
to use a GC but if I have to I hope to see the one I'm
investing on evolve because it most certainly impacts
production systems very much.
The things I really want from D to really sway me would be the
following (some already exist):
1. Evolve the GC like Go has.
2. No overhead calling C libraries.
3. Easily composable libraries.
4. Good IDE support.
#2 is really why Go has gained a lot of popularity but #1 is
extremely important on how production systems behave. #2 is
likely something Go cannot solve and why there's an opportunity
for another language to jump in with better performance.
The programming community hasn't found a good modern systems
language yet that aligns with these tradeoffs. I actually think
D and Swift (no GC, ARC, no calling C overhead) are closest to
having the potential for the correct tradeoffs to be systems
languages.
Rust probably is aligned more than anyone with these goals at
the moment but every time I try to learn rust my head hurts and
it's not enjoyable.
I hope this sheds some light on why I really like D but as an
outsider what I'm looking and need in the space I build
software and why I think D could fill a gap others aren't yet.
Thank you for reading my thoughts.
As a complete beginner in D, your post makes a lot of sense to
me. Also, that bit about Rust is spot on. It's nice for fancy
little programs (and I've written quite a few of those), but
babysitting the ownership and lifetimes system detracts away from
the problem I'm trying to solve itself. That's too bad indeed.
I had given Go a try as well, but it does seem to have a very
narrow focus (which Rob Pike has not been shy to claim) plus the
lack of generics is very stultifying (at least for me). I do
believe that Go did well for a few reasons - Rob Pike's strong
hand in carrying out his mandate, a fabulous marketing team, and
the simplistic but consistent nature of the language's design. In
fact, Rust is also "popular" for very much the same marketing
spearheaded by the likes of Steve Klabnik, but the language is
way too complex, and the team in Mozilla just does not care about
that.
I do love C++11 and newer, but I'd rather not use it for any new
projects barring some weekend projects of my own. The type system
is horrendously outdated. If they could make a clean break and
make C++11 the basis, improve error checking, bounds-checking,
and warning messages, and get rid of the C legacy (of course,
that's never going to happen), it would make for a fine language.
I've come to D primarily because I want a modern safe(r) systems
language with an excellent type system that I can use for my
small-to-medium projects without having to spend days figuring
out why the damned things is segfaulting all the time. I have
been reading the threads on this forum, and not everything is
perfect (understandably so), but it's been good going thus far!
