Hi Susan,
yes, agree with you regarding resource accounting. Imho, in this case
shuffle service must run on node no matter what resources are available(same
as we don't account for resources that "system" takes - mesos agent, OS
itself and any other process that is running on same machine)
One
Hi Igor,
You made a good point about the tradeoffs. I think the main thing you would
get with Marathon is the accounting for resources (the memory and cpus
specified in the config file). That allows Mesos to manage the resources
properly. I don't think the other tools mentioned would reserve
Hi Susan
In general I can get what I need without Marathon, with configuring
external-shuffle-service with puppet/ansible/chef + maybe some alerts for
checks.
I mean in companies that don't have strong Devops teams and want to install
services as simple as possible just by config - Marathon
jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-79976
As the commenter notes, it's not ideal, it's just a workaround.
Susan
On Sat, Jan 20, 2018 at 8:33 AM, igor.berman <igor.ber...@gmail.com> wrote:
> Hi,
> wanted to get some advice regarding managing external shuffle
Hi,
wanted to get some advice regarding managing external shuffle service in
mesos environments
In spark documentation the Marathon is mentioned, however there is very
limited documentation.
I've tried to search for some documentation and it's seems not too difficult
to configure it under
