Vanzin [mailto:van...@cloudera.com]
Sent: Tuesday, September 15, 2015 7:47 PM
To: Adrian Bridgett
Cc: user
Subject: Re: hdfs-ha on mesos - odd bug
On Mon, Sep 14, 2015 at 6:55 AM, Adrian Bridgett <adr...@opensignal.com> wrote:
> 15/09/14 13:00:25 WARN TaskSetManager: Lost task 0.0 in stage
On Mon, Sep 14, 2015 at 6:55 AM, Adrian Bridgett wrote:
> 15/09/14 13:00:25 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0,
> 10.1.200.245): java.lang.IllegalArgumentException:
> java.net.UnknownHostException: nameservice1
> at
>
Hi Sam, in short, no, it's a traditional install as we plan to use spot
instances and didn't want price spikes to kill off HDFS.
We're actually doing a bit of a hybrid, using spot instances for the
mesos slaves, ondemand for the mesos masters. So for the time being,
putting hdfs on the
> On 15 Sep 2015, at 08:55, Adrian Bridgett wrote:
>
> Hi Sam, in short, no, it's a traditional install as we plan to use spot
> instances and didn't want price spikes to kill off HDFS.
>
> We're actually doing a bit of a hybrid, using spot instances for the mesos
>
I've seen similar traces, but couldn't track down the failure completely.
You are using Kerberos for your HDFS cluster, right? AFAIK Kerberos isn't
supported in Mesos deployments.
Can you resolve that host name (nameservice1) from the driver machine (ping
nameservice1)? Can it be resolved from
Thanks Steve - we are already taking the safe route - putting NN and
datanodes on the central mesos-masters which are on demand. Later (much
later!) we _may_ put some datanodes on spot instances (and using several
spot instance types as the spikes seem to only affect one type - worst
case we
I don't know about the broken url. But are you running HDFS as a mesos
framework? If so is it using mesos-dns?
Then you should resolve the namenode via hdfs:///
On Mon, Sep 14, 2015 at 3:55 PM, Adrian Bridgett
wrote:
> I'm hitting an odd issue with running spark on