Hi Tim,
thanks for clarifying this!
Regarding dispatcher and cluster mode, it seems, that there’s still no way to
get to exact Spark executor UI but only driver configuration and execution
stats.
--
Anton Kirillov
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Wednesday, July 29, 2015 at 7:48 PM, Tim Chen wrote:
Hi Anton,
Client mode we haven't populated the webui link and only did so for cluster
mode.
If you like you can open a JIRA and it should be a easy ticket for anyone to
work on.
Tim
On Wed, Jul 29, 2015 at 4:27 AM, Anton Kirillov antonv.kiril...@gmail.com
(mailto:antonv.kiril...@gmail.com) wrote:
Hi everyone,
I’m trying to get access to Spark web UI from Mesos Master but with no
success: the host name displayed properly, but the link is not active, just
text. Maybe it’s a well-known issue or I misconfigured something, but this
problem is really annoying.
Jobs are executed in client mode, framework registers successfully and
execution happens as expected. Spark UI is available on client node but in
Mesos Master’s interface hostname is just text. When launching in cluster
mode (dispatcher is already launched) there’s only drivers information
available with no reference to driver UI.
I’m using Mesos 0.23.0, Spark 1.4.1 with binary executor (I’m using
precompiled one) deployed to S3. Here’s the contents of spark-env.sh
(http://spark-env.sh):
export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so
export SPARK_EXECUTOR_URI=https://S3 Path/spark-1.4.1-bin-hadoop2.6.tgz
export MASTER=mesos://zk://zookeeper hosts/mesos
export SPARK_PUBLIC_DNS=client machine EC2 public DNS name
Maybe I miss something, but really appreciate any help!
--
Anton Kirillov
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)