Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
Hi guys, I wanted to share my experiences with Go. I have been using it for the past few months and I have to say I am very impressed. Instead of writing a massive email I created a blog entry: http://goo.gl/g9abOi Hope this helps. - Luis On 09/05/2014 11:44 AM, Jeff Darcy wrote: Does this mean we'll need to learn Go as well as C and Python? As KP points out, the fact that consul is written in Go doesn't mean our code needs to be ... unless we need to contribute code upstream e.g. to add new features. Ditto for etcd also being written in Go, ZooKeeper being written in Java, and so on. It's probably more of an issue that these all require integration into our build/test environments. At least Go, unlike Java, doesn't require any new *run time* support. Python kind of sits in between - it does require runtime support, but it's much less resource-intensive and onerous than Java (no GC-tuning hell). Between that and the fact that it's almost always present already, it just doesn't seem to provoke the same kind of allergic reaction that Java does. However, this is as good a time as any to think about what languages we're going to use for the project going forward. While there are many good reasons for our I/O path to remain in Plain Old C (yes I'm deliberately avoiding the C++ issue), many of those reasons apply only weakly to other parts of the code - not only management code, but also offline processes like self heal and rebalancing. Some people might already be aware that I've used Python for the reconciliation component of NSR, for example, and that version is in almost every way better than the C version it replaces. When we need to interface with code written in other languages, or even interact with communities where other languages are spoken more fluently than C, it's pretty natural to consider using those languages ourselves. Let's look at some of the alternatives. * C++ Code is highly compatible with C, programming styles and idioms less so. Not prominent in most areas we care about. * Java The old standard for a lot of distributed systems - e.g. the entire Hadoop universe, Cassandra, etc. Also a great burden as discussed previously. * Go Definitely the up and comer in distributed systems, for which it was (partly) designed. Easy for C programmers to pick up, and also popular among (former?) Python folks. Light on resources and dependencies. * JavaScript Ubiquitous. Common in HTTP-ish microservice situations, but not so much in true distributed systems. * Ruby Much like JavaScript as far as we're concerned, but less ubiquitous. * Erlang Functional, designed for highly reliable distributed systems, significant use in related areas (e.g. Riak). Obviously, there are many more, but issues of compatibility and talent availability weigh heavier for most than for Erlang (which barely made the list as it is despite its strengths). Of these, the ones without serious drawbacks are JavaScript and Go. As popular as JS is in other specialties, I just don't feel any positive pull to use it in anything we do. As a language it's notoriously loose about many things (e.g. equality comparisons) and prone to the same callback hell from which we already suffer. Go is an entirely different story. We're already bumping up against other projects that use it, and that's no surprise considering how strong the uptake has been among other systems programmers. Language-wise, goroutines might help get us out of callback hell, and it has other features such as channels and defer that might also support a more productive style for our own code. I know that several in the group are already eager to give it a try. While we shouldn't do so for the cool factor alone, for new code that's not in the I/O path the potential productivity benefits make it an option well worth exploring. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
Hi guys, I wanted to share my experiences with Go. I have been using it for the past few months and I have to say I am very impressed. Instead of writing a massive email I created a blog entry: http://goo.gl/g9abOi Fantastic. Thanks, Luis! ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
On 11/09/2014, at 2:51 AM, Luis Pabón wrote: I think the real question is, Why do we depend on core files? What does it provide? If we rethink how we may do debugging, we may realize that we only require core files because we are used to it and it is familiar to us. Now, I am not saying that core files are not useful, but I am saying that we may be able to do most of the necessary debugging by other means. For example, debugging systems running OpenStack Swift which uses Python stack traces has been much easier than analyzing C core files. Just my experience. I would not say that because Go, Java, Ruby, or Python do not create core files, that it would be hard to debug. Instead we need to learn new ways of debugging. Just my $0.02 :-) Yeah, I'm no expert. :) My understanding of our core file usage, is to capture (in a fairly detailed way) the code path of what went wrong when something unexpectedly caused an application crash. We can then copy that file around (with appropriate matching binaries?) to a machine for proper investigation. As long as there's an equivalent kind of approach, it should be ok. If not, then we should probably ping the Go Community and find out their recommendations (or submit a feature request). :) + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
On 09/08/2014 10:29 PM, Krishnan Parthasarathi wrote: While the proposal for Glusterd-2.0 is doing its rounds in the devel/users lists, let me find out how the Go toolchain fares in debugging a live application and a core file, with a dash of go routines and channels for good effect :-) Shouldn't take long. I will share my experience and lets take this discussion from there. Does that make sense? One more thing to explore is Go is not free from data races.(http://dave.cheney.net/2014/06/27/ice-cream-makers-and-data-races) -- regards Aravinda http://aravindavk.in ~KP - Original Message - Two characteristics of a language (tool chain) are important to me, especially when you spend a good part of your time debugging failures/bugs. - Analysing core files. - Ability to reason about space consumption. This becomes important in the case of garbage collected languages. I have written a few toy programs in Go and have been following the language lately. Some of its features like channels and go routines catch my attention as we are aspiring to build reactive and scalable services. Its lack of type-inference and inheritance worries me a little. But, I shouldn't be complaining when our default choice has been C thus far ;) If there's going to be complaining, now's the time. Justin's kind of right that we don't want to be adding languages willy-nilly. If there's something about a language which is likely to preclude its use in certain contexts (e.g. GC languages in the I/O path) or impair our long-term productivity, then that's important to realize. Unfortunately, the list of such drawbacks for C isn't exactly zero-length either. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
[Apologies up front for questionable posting etiquettes]
Two characteristics of a language (tool chain) are important to me, especially
when you spend a good part of your time debugging failures/bugs.
- Analysing core files.
- Ability to reason about space consumption. This becomes important in
the case of garbage collected languages.
I have written a few toy programs in Go and have been following the language
lately. Some of its features like channels and go routines catch my attention
as we are aspiring to build reactive and scalable services. Its lack of
type-inference
and inheritance worries me a little. But, I shouldn't be complaining when
our default choice has been C thus far ;)
~KP
- Original Message -
Digging deeper into Go, I see there is a fascinating discussion in the
language communities comparing Go with C++.
Go has no..
- classes (no inheritance), though it has interfaces (sets of methods) which
remind me of things like gluster's struct xlator_fops {}
- polymorphism
- pointer arithmetic
- generic programming
- etc.
Here is a comparison of C++ with Go from Rob Pike himself (a Go author).
http://commandcenter.blogspot.com/2012/06/less-is-exponentially-more.html
And here are a few counter arguments.
http://lambda-the-ultimate.org/node/4554
Preference for Go seems to come down to how deeply you prefer the C++ object
oriented way of doing things (as Pike calls it, the type-centric focus on
classes). If thats your cup of tea, you may find Go a letdown or step
backwards. Pike implies that coders invest a lot of time to master those
techniques and are reluctant to ditch those skills.
But if you are a C or python programmer, you may see Go as a way to have your
cake (modern stripped down language with lists, maps, packages, interfaces,
no #includes) and eat it too (it compiles to binary, no VM).
As gluster is not beholden in any way to legacy C++, Go seems like a great
fit. I'm looking forward to giving it a spin :)
- Original Message -
From: Dan Lambright dlamb...@redhat.com
To: Jeff Darcy jda...@redhat.com
Cc: Justin Clift jus...@gluster.org, Gluster Devel
gluster-devel@gluster.org
Sent: Friday, September 5, 2014 5:32:05 PM
Subject: Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
One reason to use c++ could be to build components that we wish to share
with
ceph. (Not that I know of any at this time). Also c++0x11 has improved the
language.
But the more I hear about it, the more interesting go sounds..
- Original Message -
From: Jeff Darcy jda...@redhat.com
To: Justin Clift jus...@gluster.org
Cc: Gluster Devel gluster-devel@gluster.org
Sent: Friday, September 5, 2014 11:44:35 AM
Subject: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
Does this mean we'll need to learn Go as well as C and Python?
As KP points out, the fact that consul is written in Go doesn't mean our
code needs to be ... unless we need to contribute code upstream e.g. to
add new features. Ditto for etcd also being written in Go, ZooKeeper
being written in Java, and so on. It's probably more of an issue that
these all require integration into our build/test environments. At
least Go, unlike Java, doesn't require any new *run time* support.
Python kind of sits in between - it does require runtime support, but
it's much less resource-intensive and onerous than Java (no GC-tuning
hell). Between that and the fact that it's almost always present
already, it just doesn't seem to provoke the same kind of allergic
reaction that Java does.
However, this is as good a time as any to think about what languages
we're going to use for the project going forward. While there are many
good reasons for our I/O path to remain in Plain Old C (yes I'm
deliberately avoiding the C++ issue), many of those reasons apply only
weakly to other parts of the code - not only management code, but also
offline processes like self heal and rebalancing. Some people might
already be aware that I've used Python for the reconciliation component
of NSR, for example, and that version is in almost every way better than
the C version it replaces. When we need to interface with code written
in other languages, or even interact with communities where other
languages are spoken more fluently than C, it's pretty natural to
consider using those languages ourselves. Let's look at some of the
alternatives.
* C++
Code is highly compatible with C, programming styles and idioms less
so. Not prominent in most areas we care about.
* Java
The old standard for a lot of distributed systems - e.g. the
entire Hadoop universe, Cassandra, etc. Also a great burden as
discussed previously.
* Go
Definitely the up and comer in distributed systems, for which it
was (partly) designed
Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
Two characteristics of a language (tool chain) are important to me, especially when you spend a good part of your time debugging failures/bugs. - Analysing core files. - Ability to reason about space consumption. This becomes important in the case of garbage collected languages. I have written a few toy programs in Go and have been following the language lately. Some of its features like channels and go routines catch my attention as we are aspiring to build reactive and scalable services. Its lack of type-inference and inheritance worries me a little. But, I shouldn't be complaining when our default choice has been C thus far ;) If there's going to be complaining, now's the time. Justin's kind of right that we don't want to be adding languages willy-nilly. If there's something about a language which is likely to preclude its use in certain contexts (e.g. GC languages in the I/O path) or impair our long-term productivity, then that's important to realize. Unfortunately, the list of such drawbacks for C isn't exactly zero-length either. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
I could see Go used for background type jobs or test harnessing in the beginning, at the discretion of the developer. The question about garbage collection is an unknown and a good point. To me, it makes sense to get experience with Go before using it in the I/O path. Particularly as the language is new. Apparently Go does have a kind of inheritance. It does *not* have virtual functions. Here is a nice blog post. https://geekwentfreak-raviteja.rhcloud.com/blog/2014/03/06/golang-inheritance-by-embedding/ - Original Message - From: Jeff Darcy jda...@redhat.com To: Krishnan Parthasarathi kpart...@redhat.com Cc: Dan Lambright dlamb...@redhat.com, Gluster Devel gluster-devel@gluster.org Sent: Monday, September 8, 2014 8:14:07 AM Subject: Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0) Two characteristics of a language (tool chain) are important to me, especially when you spend a good part of your time debugging failures/bugs. - Analysing core files. - Ability to reason about space consumption. This becomes important in the case of garbage collected languages. I have written a few toy programs in Go and have been following the language lately. Some of its features like channels and go routines catch my attention as we are aspiring to build reactive and scalable services. Its lack of type-inference and inheritance worries me a little. But, I shouldn't be complaining when our default choice has been C thus far ;) If there's going to be complaining, now's the time. Justin's kind of right that we don't want to be adding languages willy-nilly. If there's something about a language which is likely to preclude its use in certain contexts (e.g. GC languages in the I/O path) or impair our long-term productivity, then that's important to realize. Unfortunately, the list of such drawbacks for C isn't exactly zero-length either. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
While the proposal for Glusterd-2.0 is doing its rounds in the devel/users lists, let me find out how the Go toolchain fares in debugging a live application and a core file, with a dash of go routines and channels for good effect :-) Shouldn't take long. I will share my experience and lets take this discussion from there. Does that make sense? ~KP - Original Message - Two characteristics of a language (tool chain) are important to me, especially when you spend a good part of your time debugging failures/bugs. - Analysing core files. - Ability to reason about space consumption. This becomes important in the case of garbage collected languages. I have written a few toy programs in Go and have been following the language lately. Some of its features like channels and go routines catch my attention as we are aspiring to build reactive and scalable services. Its lack of type-inference and inheritance worries me a little. But, I shouldn't be complaining when our default choice has been C thus far ;) If there's going to be complaining, now's the time. Justin's kind of right that we don't want to be adding languages willy-nilly. If there's something about a language which is likely to preclude its use in certain contexts (e.g. GC languages in the I/O path) or impair our long-term productivity, then that's important to realize. Unfortunately, the list of such drawbacks for C isn't exactly zero-length either. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
Digging deeper into Go, I see there is a fascinating discussion in the language
communities comparing Go with C++.
Go has no..
- classes (no inheritance), though it has interfaces (sets of methods) which
remind me of things like gluster's struct xlator_fops {}
- polymorphism
- pointer arithmetic
- generic programming
- etc.
Here is a comparison of C++ with Go from Rob Pike himself (a Go author).
http://commandcenter.blogspot.com/2012/06/less-is-exponentially-more.html
And here are a few counter arguments.
http://lambda-the-ultimate.org/node/4554
Preference for Go seems to come down to how deeply you prefer the C++ object
oriented way of doing things (as Pike calls it, the type-centric focus on
classes). If thats your cup of tea, you may find Go a letdown or step
backwards. Pike implies that coders invest a lot of time to master those
techniques and are reluctant to ditch those skills.
But if you are a C or python programmer, you may see Go as a way to have your
cake (modern stripped down language with lists, maps, packages, interfaces, no
#includes) and eat it too (it compiles to binary, no VM).
As gluster is not beholden in any way to legacy C++, Go seems like a great fit.
I'm looking forward to giving it a spin :)
- Original Message -
From: Dan Lambright dlamb...@redhat.com
To: Jeff Darcy jda...@redhat.com
Cc: Justin Clift jus...@gluster.org, Gluster Devel
gluster-devel@gluster.org
Sent: Friday, September 5, 2014 5:32:05 PM
Subject: Re: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
One reason to use c++ could be to build components that we wish to share with
ceph. (Not that I know of any at this time). Also c++0x11 has improved the
language.
But the more I hear about it, the more interesting go sounds..
- Original Message -
From: Jeff Darcy jda...@redhat.com
To: Justin Clift jus...@gluster.org
Cc: Gluster Devel gluster-devel@gluster.org
Sent: Friday, September 5, 2014 11:44:35 AM
Subject: [Gluster-devel] Languages (was Re: Proposal for GlusterD-2.0)
Does this mean we'll need to learn Go as well as C and Python?
As KP points out, the fact that consul is written in Go doesn't mean our
code needs to be ... unless we need to contribute code upstream e.g. to
add new features. Ditto for etcd also being written in Go, ZooKeeper
being written in Java, and so on. It's probably more of an issue that
these all require integration into our build/test environments. At
least Go, unlike Java, doesn't require any new *run time* support.
Python kind of sits in between - it does require runtime support, but
it's much less resource-intensive and onerous than Java (no GC-tuning
hell). Between that and the fact that it's almost always present
already, it just doesn't seem to provoke the same kind of allergic
reaction that Java does.
However, this is as good a time as any to think about what languages
we're going to use for the project going forward. While there are many
good reasons for our I/O path to remain in Plain Old C (yes I'm
deliberately avoiding the C++ issue), many of those reasons apply only
weakly to other parts of the code - not only management code, but also
offline processes like self heal and rebalancing. Some people might
already be aware that I've used Python for the reconciliation component
of NSR, for example, and that version is in almost every way better than
the C version it replaces. When we need to interface with code written
in other languages, or even interact with communities where other
languages are spoken more fluently than C, it's pretty natural to
consider using those languages ourselves. Let's look at some of the
alternatives.
* C++
Code is highly compatible with C, programming styles and idioms less
so. Not prominent in most areas we care about.
* Java
The old standard for a lot of distributed systems - e.g. the
entire Hadoop universe, Cassandra, etc. Also a great burden as
discussed previously.
* Go
Definitely the up and comer in distributed systems, for which it
was (partly) designed. Easy for C programmers to pick up, and also
popular among (former?) Python folks. Light on resources and
dependencies.
* JavaScript
Ubiquitous. Common in HTTP-ish microservice situations, but not so
much in true distributed systems.
* Ruby
Much like JavaScript as far as we're concerned, but less ubiquitous.
* Erlang
Functional, designed for highly reliable distributed systems,
significant use in related areas (e.g. Riak).
Obviously, there are many more, but issues of compatibility and talent
availability weigh heavier for most than for Erlang (which barely made
the list as it is despite its strengths). Of these, the ones without
serious drawbacks are JavaScript and Go. As popular as JS is in other
specialties, I just don't feel any
