I'm supposing that there's no good solution to having heterogenous hardware
in a cluster. What are the prospects of having something like this in the
future? Am I missing an architectural detail that precludes this
possibility?
Thanks,
Victor
On Fri, Sep 12, 2014 at 12:10 PM, Victor Tso-Guillen
I thought I answered this ... you can easily accomplish this with YARN
by just telling YARN how much memory / CPU each machine has. This can
be configured in groups too rather than per machine. I don't think you
actually want differently-sized executors, and so don't need ratios.
But you can have
Hmm, interesting. I'm using standalone mode but I could consider YARN. I'll
have to simmer on that one. Thanks as always, Sean!
On Wed, Sep 17, 2014 at 12:40 AM, Sean Owen so...@cloudera.com wrote:
I thought I answered this ... you can easily accomplish this with YARN
by just telling YARN how
Ping...
On Thu, Sep 11, 2014 at 5:44 PM, Victor Tso-Guillen v...@paxata.com wrote:
So I have a bunch of hardware with different core and memory setups. Is
there a way to do one of the following:
1. Express a ratio of cores to memory to retain. The spark worker config
would represent all of
So I have a bunch of hardware with different core and memory setups. Is
there a way to do one of the following:
1. Express a ratio of cores to memory to retain. The spark worker config
would represent all of the cores and all of the memory usable for any
application, and the application would
