Re: Web UI is not showing up

The web ui is at port 8080. 4040 will show up something when you have a running job or if you have configured history server. On Sep 1, 2015 8:57 PM, "Sunil Rathee" wrote: > > Hi, > > > localhost:4040 is not showing anything on the browser. Do we have to start > some

Re: Web UI is not showing up

localhost:8080 is also not showing anything. Does some application running at the same time? On Tue, Sep 1, 2015 at 9:04 PM, Sonal Goyal wrote: > The web ui is at port 8080. 4040 will show up something when you have a > running job or if you have configured history

Re: Web UI is not showing up

Is your master up? Check the java processes to see if they are running. Best Regards, Sonal Founder, Nube Technologies Reifier covered in YourStory Reifier at Spark Summit 2015

Re: Web UI Links

I figured this out after spelunking the UI code a little. The trick is to set the SPARK_PUBLIC_DNS environmental variable to the public DNS name of each server in the cluster, per node. I'm running in standalone mode, so it was just a matter of adding the setting to spark-env.sh. On Mon, Jul 20,

RE: Web UI vs History Server Bugs

: Web UI vs History Server Bugs well, I'm afraid you've reached the limits of my knowledge ... hopefully someone else can answer On 22 Jun 2015, at 16:37, Jonathon Cai jonathon@yale.edu wrote: No, what I'm seeing is that while the cluster is running, I can't see the app info after

Re: Web UI vs History Server Bugs

No, what I'm seeing is that while the cluster is running, I can't see the app info after the app is completed. That is to say, when I click on the application name on master:8080, no info is shown. However, when I examine the same file on the History Server, the application information opens fine.

Re: Web UI vs History Server Bugs

On 17 Jun 2015, at 19:10, jcai jonathon@yale.edu wrote: Hi, I am running this on Spark stand-alone mode. I find that when I examine the web UI, a couple bugs arise: 1. There is a discrepancy between the number denoting the duration of the application when I run the history server

Re: Web UI vs History Server Bugs

You could possibly open up a JIRA and shoot an email to the dev list. Thanks Best Regards On Wed, Jun 17, 2015 at 11:40 PM, jcai jonathon@yale.edu wrote: Hi, I am running this on Spark stand-alone mode. I find that when I examine the web UI, a couple bugs arise: 1. There is a

RE: Web UI

, September 04, 2014 2:11 PM To: Ruebenacker, Oliver A Cc: Akhil Das; Wonha Ryu; user@spark.apache.org Subject: Re: Web UI Hi all, The JSON version of the web UI is not officially supported; I don't believe this is documented anywhere. The alternative is to set `spark.eventLog.enabled` to true

Re: Web UI

@spark.apache.org *Subject:* Re: Web UI Hi all, The JSON version of the web UI is not officially supported; I don't believe this is documented anywhere. The alternative is to set `spark.eventLog.enabled` to true before running your application. This will create JSON SparkListenerEvents

Re: Web UI

:* Wonha Ryu [mailto:wonha@gmail.com] *Sent:* Wednesday, September 03, 2014 4:27 PM *To:* Ruebenacker, Oliver A *Cc:* user@spark.apache.org *Subject:* Re: Web UI Hey Oliver, IIRC there's no JSON endpoint for application web UI. They only exist for cluster master and worker. - Wonha

RE: Web UI

, 2014 3:09 AM To: Ruebenacker, Oliver A Cc: Wonha Ryu; user@spark.apache.org Subject: Re: Web UI Hi You can see this dochttps://spark.apache.org/docs/latest/spark-standalone.html#configuring-ports-for-network-security for all the available webUI ports. Yes there are ways to get the data metrics

Re: Web UI

and development. Best, Oliver *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com] *Sent:* Thursday, September 04, 2014 3:09 AM *To:* Ruebenacker, Oliver A *Cc:* Wonha Ryu; user@spark.apache.org *Subject:* Re: Web UI Hi You can see this doc https://spark.apache.org/docs/latest

Re: Web UI

Hi Oliver, Spark standalone master and worker support '/json' endpoint in web UI, which returns some of the information in JSON format. I wasn't able to find relevant documentation, though. - Wonha On Wed, Sep 3, 2014 at 12:12 PM, Ruebenacker, Oliver A oliver.ruebenac...@altisource.com wrote:

RE: Web UI

] Sent: Wednesday, September 03, 2014 3:36 PM To: Ruebenacker, Oliver A Cc: user@spark.apache.org Subject: Re: Web UI Hi Oliver, Spark standalone master and worker support '/json' endpoint in web UI, which returns some of the information in JSON format. I wasn't able to find relevant documentation

Re: Web UI

*To:* Ruebenacker, Oliver A *Cc:* user@spark.apache.org *Subject:* Re: Web UI Hi Oliver, Spark standalone master and worker support '/json' endpoint in web UI, which returns some of the information in JSON format. I wasn't able to find relevant documentation, though. - Wonha On Wed, Sep 3

RE: Web UI

. Best, Oliver From: Wonha Ryu [mailto:wonha@gmail.com] Sent: Wednesday, September 03, 2014 4:27 PM To: Ruebenacker, Oliver A Cc: user@spark.apache.org Subject: Re: Web UI Hey Oliver, IIRC there's no JSON endpoint for application web UI. They only exist for cluster master and worker

Re: Web UI doesn't show some stages

The reason is that some operators get pipelined into a single stage. rdd.map(XX).filter(YY) - this executes in a single stage since there is no data movement needed in between these operations. If you call toDeubgString on the final RDD it will give you some information about the exact lineage.

Re: Web UI doesn't show some stages

Try to answer your another question. One sortByKey is triggered by rangePartition which does sample to calculate the range boundaries, which again triggers the first reduceByKey. The second sortByKey is doing the real work to sort based on the partition calculated, which again trigger the