[ https://issues.apache.org/jira/browse/SPARK-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325001#comment-15325001 ]
Andrew Davidson commented on SPARK-15829: ----------------------------------------- Hi Xin I ran netstat on my master. I do not think the port are in use. To submit in cluster mode I use port 6066. If you are using port 7077 you are in client mode. In client mode the application UI will run on the spark master. In cluster mode the application UI runs on which ever slave the driver is running on. If you notice in my original description the url is incorrect. the ip is wrong, the port is correct. Kind regards Andy # bash-4.2# netstat -tulpn Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:8652 0.0.0.0:* LISTEN 3832/gmetad tcp 0 0 0.0.0.0:8787 0.0.0.0:* LISTEN 2584/rserver tcp 0 0 0.0.0.0:36757 0.0.0.0:* LISTEN 2905/java tcp 0 0 0.0.0.0:50070 0.0.0.0:* LISTEN 2905/java tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 2144/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 2095/cupsd tcp 0 0 127.0.0.1:7000 0.0.0.0:* LISTEN 6512/python3.4 tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 2183/sendmail tcp 0 0 0.0.0.0:43813 0.0.0.0:* LISTEN 3093/java tcp 0 0 172.31.22.140:9000 0.0.0.0:* LISTEN 2905/java tcp 0 0 0.0.0.0:8649 0.0.0.0:* LISTEN 3810/gmond tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 3093/java tcp 0 0 0.0.0.0:8651 0.0.0.0:* LISTEN 3832/gmetad tcp 0 0 :::8080 :::* LISTEN 23719/java tcp 0 0 :::8081 :::* LISTEN 5588/java tcp 0 0 ::ffff:172.31.22.140:6066 :::* LISTEN 23719/java tcp 0 0 ::ffff:172.31.22.140:6067 :::* LISTEN 5588/java tcp 0 0 :::22 :::* LISTEN 2144/sshd tcp 0 0 ::1:631 :::* LISTEN 2095/cupsd tcp 0 0 :::19998 :::* LISTEN 3709/java tcp 0 0 :::19999 :::* LISTEN 3709/java tcp 0 0 ::ffff:172.31.22.140:7077 :::* LISTEN 23719/java tcp 0 0 ::ffff:172.31.22.140:7078 :::* LISTEN 5588/java udp 0 0 0.0.0.0:8649 0.0.0.0:* 3810/gmond udp 0 0 0.0.0.0:631 0.0.0.0:* 2095/cupsd udp 0 0 0.0.0.0:38546 0.0.0.0:* 2905/java udp 0 0 0.0.0.0:68 0.0.0.0:* 1142/dhclient udp 0 0 172.31.22.140:123 0.0.0.0:* 2168/ntpd udp 0 0 127.0.0.1:123 0.0.0.0:* 2168/ntpd udp 0 0 0.0.0.0:123 0.0.0.0:* 2168/ntpd bash-4.2# > spark master webpage links to application UI broke when running in cluster > mode > ------------------------------------------------------------------------------- > > Key: SPARK-15829 > URL: https://issues.apache.org/jira/browse/SPARK-15829 > Project: Spark > Issue Type: Bug > Components: EC2 > Affects Versions: 1.6.1 > Environment: AWS ec2 cluster > Reporter: Andrew Davidson > Priority: Critical > > Hi > I created a cluster using the spark-1.6.1-bin-hadoop2.6/ec2/spark-ec2 > I use the stand alone cluster manager. I have a streaming app running in > cluster mode. I notice the master webpage links to the application UI page > are incorrect > It does not look like jira will let my upload images. I'll try and describe > the web pages and the bug > My master is running on > http://ec2-54-215-230-73.us-west-1.compute.amazonaws.com:8080/ > It has a section marked "applications". If I click on one of the running > application ids I am taken to a page showing "Executor Summary". This page > has a link to teh 'application detail UI' the url is > http://ec2-54-215-230-73.us-west-1.compute.amazonaws.com:4041/ > Notice it things the application UI is running on the cluster master. > It is actually running on the same machine as the driver on port 4041. I was > able to reverse engine the url by noticing the private ip address is part of > the worker id . For example worker-20160322041632-172.31.23.201-34909 > next I went on the aws ec2 console to find the public DNS name for this > machine > http://ec2-54-193-104-169.us-west-1.compute.amazonaws.com:4041/streaming/ > Kind regards > Andy -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org