Re: set up spark cluster with heterogeneous hardware

[Akhil Das]

You could also add SPARK_MASTER_IP to bind to a specific host/IP so that it
won't get confused with those hosts in your /etc/hosts file.

Thanks
Best Regards

On Fri, Mar 13, 2015 at 12:00 PM, Du Li l...@yahoo-inc.com.invalid wrote:

 Hi Spark community,

 I searched for a way to configure a heterogeneous cluster because the need
 recently emerged in my project. I didn't find any solution out there. Now I
 have thought out a solution and thought it might be useful to many other
 people with similar needs. Following is a blog post I just wrote. Feedback
 is welcome.

 The capability to accommodate heterogeneous hardware has already been
 there in Spark. We learn how to use it. Oscar goes to those contributors.
 :-)

 Du

 How to set up a Spark cluster with heterogeneous hardware
 http://scala4fun.tumblr.com/post/113486166952






 How to set up a Spark cluster with heterogeneous hardware
 http://scala4fun.tumblr.com/post/113486166952
 Apache Spark (http://spark.apache.org) is a big data processing platform
 that has become widely adopted across the industries in just a few years.
 The strength of Spark is to make use of the main memory that is getting
 larger and cheaper to support low-latency use cases.  In a pra...
 View on scala4fun.tumblr.com
 http://scala4fun.tumblr.com/post/113486166952
 Preview by Yahoo

set up spark cluster with heterogeneous hardware

[Du Li]

Hi Spark community,
I searched for a way to configure a heterogeneous cluster because the need 
recently emerged in my project. I didn't find any solution out there. Now I 
have thought out a solution and thought it might be useful to many other people 
with similar needs. Following is a blog post I just wrote. Feedback is welcome.
The capability to accommodate heterogeneous hardware has already been there in 
Spark. We learn how to use it. Oscar goes to those contributors. :-)
Du
How to set up a Spark cluster with heterogeneous hardware

|   |
|   |   |   |   |   |
| How to set up a Spark cluster with heterogeneous hardwareApache Spark 
(http://spark.apache.org) is a big data processing platform that has become 
widely adopted across the industries in just a few years. The strength of Spark 
is to make use of the main memory that is getting larger and cheaper to support 
low-latency use cases.  In a pra... |
|  |
| View on scala4fun.tumblr.com | Preview by Yahoo |
|  |
|   |