Spark WebUI link problem in Mesos Master

[Anton Kirillov]

Hi everyone,  

I’m trying to get access to Spark web UI from Mesos Master but with no success: 
the host name displayed properly, but the link is not active, just text. Maybe 
it’s a well-known issue or I misconfigured something, but this problem is 
really annoying.

Jobs are executed in client mode, framework registers successfully and 
execution happens as expected. Spark UI is available on client node but in 
Mesos Master’s interface hostname is just text. When launching in cluster mode 
(dispatcher is already launched) there’s only drivers information available 
with no reference to driver UI.

I’m using Mesos 0.23.0, Spark 1.4.1 with binary executor (I’m using precompiled 
one) deployed to S3. Here’s the contents of spark-env.sh:

export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so
export SPARK_EXECUTOR_URI=https:///spark-1.4.1-bin-hadoop2.6.tgz
export MASTER=mesos://zk:///mesos

export SPARK_PUBLIC_DNS=

Maybe I miss something, but really appreciate any help!  

--  
Anton Kirillov
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

Re: Spark WebUI link problem in Mesos Master

[Tim Chen]

Hi Anton,

Client mode we haven't populated the webui link and only did so for cluster
mode.

If you like you can open a JIRA and it should be a easy ticket for anyone
to work on.

Tim

On Wed, Jul 29, 2015 at 4:27 AM, Anton Kirillov 
wrote:

>  Hi everyone,
>
> I’m trying to get access to Spark web UI from Mesos Master but with no
> success: the host name displayed properly, but the link is not active, just
> text. Maybe it’s a well-known issue or I misconfigured something, but this
> problem is really annoying.
>
> Jobs are executed in client mode, framework registers successfully and
> execution happens as expected. Spark UI is available on client node but in
> Mesos Master’s interface hostname is just text. When launching in cluster
> mode (dispatcher is already launched) there’s only drivers information
> available with no reference to driver UI.
>
> I’m using Mesos 0.23.0, Spark 1.4.1 with binary executor (I’m using
> precompiled one) deployed to S3. Here’s the contents of spark-env.sh:
>
> export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so
> export SPARK_EXECUTOR_URI=https:///spark-1.4.1-bin-hadoop2.6.tgz
> export MASTER=mesos://zk:///mesos
>
> export SPARK_PUBLIC_DNS=
>
> Maybe I miss something, but really appreciate any help!
>
> --
> Anton Kirillov
> Sent with Sparrow 
>
>

Re: Spark WebUI link problem in Mesos Master

[Anton Kirillov]

Hi Tim,

thanks for clarifying this!  

Regarding dispatcher and cluster mode, it seems, that there’s still no way to 
get to exact Spark executor UI but only driver configuration and execution 
stats.  

--  
Anton Kirillov
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Wednesday, July 29, 2015 at 7:48 PM, Tim Chen wrote:

> Hi Anton,
>  
> Client mode we haven't populated the webui link and only did so for cluster 
> mode.
>  
> If you like you can open a JIRA and it should be a easy ticket for anyone to 
> work on.  
>  
> Tim
>  
> On Wed, Jul 29, 2015 at 4:27 AM, Anton Kirillov  (mailto:antonv.kiril...@gmail.com)> wrote:
> > Hi everyone,  
> >  
> > I’m trying to get access to Spark web UI from Mesos Master but with no 
> > success: the host name displayed properly, but the link is not active, just 
> > text. Maybe it’s a well-known issue or I misconfigured something, but this 
> > problem is really annoying.
> >  
> > Jobs are executed in client mode, framework registers successfully and 
> > execution happens as expected. Spark UI is available on client node but in 
> > Mesos Master’s interface hostname is just text. When launching in cluster 
> > mode (dispatcher is already launched) there’s only drivers information 
> > available with no reference to driver UI.
> >  
> > I’m using Mesos 0.23.0, Spark 1.4.1 with binary executor (I’m using 
> > precompiled one) deployed to S3. Here’s the contents of spark-env.sh 
> > (http://spark-env.sh):
> >  
> > export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so
> > export SPARK_EXECUTOR_URI=https:///spark-1.4.1-bin-hadoop2.6.tgz
> > export MASTER=mesos://zk:///mesos
> >  
> > export SPARK_PUBLIC_DNS=
> >  
> > Maybe I miss something, but really appreciate any help!  
> >  
> > --  
> > Anton Kirillov
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >  
>