Re: why does spark web UI keeps changing its port?

2017-01-24 Thread smartzjp
Config your spark master web ui you can set env SPARK_MASTER_WEBUI_PORT= You can running cmd netstat –nao|grep 4040 to check 4040 is in using ——— I am not sure why Spark web UI keeps changing its port every time I restart a cluster? how can I make it run always on one port? I did make

Re: why does spark web UI keeps changing its port?

2017-01-23 Thread Marcelo Vanzin
As I said. Each app gets their own UI. Look at the logs printed to the output. The port will depend on whether they're running on the same host at the same time. This is irrespective of how they are run. On Mon, Jan 23, 2017 at 12:40 PM, kant kodali wrote: > yes I meant

Re: why does spark web UI keeps changing its port?

2017-01-23 Thread kant kodali
yes I meant submitting through spark-submit. so If I do spark-submit A.jar and spark-submit A.jar again. Do I get two UI's or one UI'? and which ports do they run on when using the stand alone mode? On Mon, Jan 23, 2017 at 12:19 PM, Marcelo Vanzin wrote: > Depends on what

Re: why does spark web UI keeps changing its port?

2017-01-23 Thread Marcelo Vanzin
Depends on what you mean by "job". Which is why I prefer "app", which is clearer (something you submit using "spark-submit", for example). But really, I'm not sure what you're asking now. On Mon, Jan 23, 2017 at 12:15 PM, kant kodali wrote: > hmm..I guess in that case my

Re: why does spark web UI keeps changing its port?

2017-01-23 Thread kant kodali
hmm..I guess in that case my assumption of "app" is wrong. I thought the app is a client jar that you submit. no? If so, say I submit multiple jobs then I get two UI'S? On Mon, Jan 23, 2017 at 12:07 PM, Marcelo Vanzin wrote: > No. Each app has its own UI which runs

Re: why does spark web UI keeps changing its port?

2017-01-23 Thread Marcelo Vanzin
No. Each app has its own UI which runs (starting on) port 4040. On Mon, Jan 23, 2017 at 12:05 PM, kant kodali wrote: > I am using standalone mode so wouldn't be 8080 for my app web ui as well? > There is nothing running on 4040 in my cluster. > >

Re: why does spark web UI keeps changing its port?

2017-01-23 Thread kant kodali
I am using standalone mode so wouldn't be 8080 for my app web ui as well? There is nothing running on 4040 in my cluster. http://spark.apache.org/docs/latest/security.html#standalone-mode-only On Mon, Jan 23, 2017 at 11:51 AM, Marcelo Vanzin wrote: > That's the Master,

Re: why does spark web UI keeps changing its port?

2017-01-23 Thread Marcelo Vanzin
That's the Master, whose default port is 8080 (not 4040). The default port for the app's UI is 4040. On Mon, Jan 23, 2017 at 11:47 AM, kant kodali wrote: > I am not sure why Spark web UI keeps changing its port every time I restart > a cluster? how can I make it run always on

why does spark web UI keeps changing its port?

2017-01-23 Thread kant kodali
I am not sure why Spark web UI keeps changing its port every time I restart a cluster? how can I make it run always on one port? I did make sure there is no process running on 4040(spark default web ui port) however it still starts at 8080. any ideas? MasterWebUI: Bound MasterWebUI to 0.0.0.0,