it blocks until 500 Error: HTTP ERROR 500
Problem accessing /proxy/application_1443146594954_0057/. Reason: ???? (TimeOut, I guess) Caused by: java.net.ConnectException: ???? at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at java.net.Socket.<init>(Socket.java:425) at java.net.Socket.<init>(Socket.java:280) Yarn ResourceManager log 2015-10-26 11:30:13,328 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs IP=123.103.21.82 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1443146594954_0057 2015-10-26 11:30:13,328 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Storing application with id application_1443146594954_0057 2015-10-26 11:30:13,328 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1443146594954_0057 State change from NEW to NEW_SAVING 2015-10-26 11:30:13,328 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Storing info for app: application_1443146594954_0057 2015-10-26 11:30:13,355 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1443146594954_0057 State change from NEW_SAVING to SUBMITTED 2015-10-26 11:30:13,356 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler: Accepted application application_1443146594954_0057 from user: hdfs, in queue: default, currently num of applications: 3 2015-10-26 11:30:13,356 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1443146594954_0057 State change from SUBMITTED to ACCEPTED 2015-10-26 11:30:13,356 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: Registering app attempt : appattempt_1443146594954_0057_000001 2015-10-26 11:30:13,356 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1443146594954_0057_000001 State change from NEW to SUBMITTED 2015-10-26 11:30:13,356 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler: Added Application Attempt appattempt_1443146594954_0057_000001 to scheduler from user: hdfs 2015-10-26 11:30:13,357 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1443146594954_0057_000001 State change from SUBMITTED to SCHEDULED 2015-10-26 11:30:13,459 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1443146594954_0057_01_000001 Container Transitioned from NEW to ALLOCATED 2015-10-26 11:30:13,459 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1443146594954_0057 CONTAINERID=container_1443146594954_0057_01_000001 2015-10-26 11:30:13,459 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned container container_1443146594954_0057_01_000001 of capacity <memory:2048, vCores:1> on host d190.mzhen.cn:52604, which has 6 containers, <memory:36864, vCores:6> used and <memory:86016, vCores:10> available after allocation 2015-10-26 11:30:13,460 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Sending NMToken for nodeId : d190.mzhen.cn:52604 for container : container_1443146594954_0057_01_000001 2015-10-26 11:30:13,461 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1443146594954_0057_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 2015-10-26 11:30:13,461 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Clear node set for appattempt_1443146594954_0057_000001 2015-10-26 11:30:13,461 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: Storing attempt: AppId: application_1443146594954_0057 AttemptId: appattempt_1443146594954_0057_000001 MasterContainer: Container: [ContainerId: container_1443146594954_0057_01_000001, NodeId: d190.mzhen.cn:52604, NodeHttpAddress: d190.mzhen.cn:8042, Resource: <memory:2048, vCores:1>, Priority: 0, Token: Token { kind: ContainerToken, service: 123.103.21.88:52604 }, ] 2015-10-26 11:30:13,461 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1443146594954_0057_000001 State change from SCHEDULED to ALLOCATED_SAVING 2015-10-26 11:30:13,505 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1443146594954_0057_000001 State change from ALLOCATED_SAVING to ALLOCATED 2015-10-26 11:30:13,506 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Launching masterappattempt_1443146594954_0057_000001 2015-10-26 11:30:13,507 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Setting up container Container: [ContainerId: container_1443146594954_0057_01_000001, NodeId: d190.mzhen.cn:52604, NodeHttpAddress: d190.mzhen.cn:8042, Resource: <memory:2048, vCores:1>, Priority: 0, Token: Token { kind: ContainerToken, service: 123.103.21.88:52604 }, ] for AM appattempt_1443146594954_0057_000001 2015-10-26 11:30:13,507 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Command to launch container container_1443146594954_0057_01_000001 : {{JAVA_HOME}}/bin/java,-server,-Xmx512m,-Djava.io.tmpdir={{PWD}}/tmp,-Dspark.yarn.app.container.log.dir=<LOG_DIR>,org.apache.spark.deploy.yarn.ExecutorLauncher,--arg,'192.168.3.182:55116',--executor-memory,7168m,--executor-cores,1,--properties-file,{{PWD}}/__spark_conf__/__spark_conf__.properties,1>,<LOG_DIR>/stdout,2>,<LOG_DIR>/stderr 2015-10-26 11:30:13,507 INFO org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager: Create AMRMToken for ApplicationAttempt: appattempt_1443146594954_0057_000001 2015-10-26 11:30:13,507 INFO org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager: Creating password for appattempt_1443146594954_0057_000001 2015-10-26 11:30:13,521 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Done launching container Container: [ContainerId: container_1443146594954_0057_01_000001, NodeId: d190.mzhen.cn:52604, NodeHttpAddress: d190.mzhen.cn:8042, Resource: <memory:2048, vCores:1>, Priority: 0, Token: Token { kind: ContainerToken, service: 123.103.21.88:52604 }, ] for AM appattempt_1443146594954_0057_000001 2015-10-26 11:30:13,521 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1443146594954_0057_000001 State change from ALLOCATED to LAUNCHED 2015-10-26 11:30:14,458 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1443146594954_0057_01_000001 Container Transitioned from ACQUIRED to RUNNING 2015-10-26 11:30:18,561 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for appattempt_1443146594954_0057_000001 (auth:SIMPLE) 2015-10-26 11:30:18,575 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: AM registration appattempt_1443146594954_0057_000001 2015-10-26 11:30:18,575 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs IP=123.103.21.88 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1443146594954_0057 APPATTEMPTID=appattempt_1443146594954_0057_000001 2015-10-26 11:30:18,575 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1443146594954_0057_000001 State change from LAUNCHED to RUNNING 2015-10-26 11:30:18,575 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1443146594954_0057 State change from ACCEPTED to RUNNING 2015-10-26 11:30:19,465 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1443146594954_0057_01_000002 Container Transitioned from NEW to ALLOCATED 2015-10-26 11:30:19,466 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1443146594954_0057 CONTAINERID=container_1443146594954_0057_01_000002 2015-10-26 11:30:19,466 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned container container_1443146594954_0057_01_000002 of capacity <memory:8192, vCores:1> on host d190.mzhen.cn:52604, which has 7 containers, <memory:45056, vCores:7> used and <memory:77824, vCores:9> available after allocation 2015-10-26 11:30:19,555 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1443146594954_0057_01_000003 Container Transitioned from NEW to ALLOCATED 2015-10-26 11:30:19,555 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1443146594954_0057 CONTAINERID=container_1443146594954_0057_01_000003 2015-10-26 11:30:19,555 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned container container_1443146594954_0057_01_000003 of capacity <memory:8192, vCores:1> on host d189.mzhen.cn:61395, which has 6 containers, <memory:43008, vCores:6> used and <memory:79872, vCores:10> available after allocation 2015-10-26 11:30:20,074 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Sending NMToken for nodeId : d190.mzhen.cn:52604 for container : container_1443146594954_0057_01_000002 2015-10-26 11:30:20,260 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1443146594954_0057_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 2015-10-26 11:30:20,261 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Sending NMToken for nodeId : d189.mzhen.cn:61395 for container : container_1443146594954_0057_01_000003 2015-10-26 11:30:20,262 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1443146594954_0057_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 2015-10-26 11:30:20,946 INFO org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is accessing unchecked http://192.168.3.182:29354/ which is the app master GUI of application_1443146594954_0057 owned by hdfs 2015-10-26 11:30:21,469 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1443146594954_0057_01_000002 Container Transitioned from ACQUIRED to RUNNING 2015-10-26 11:30:21,559 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1443146594954_0057_01_000003 Container Transitioned from ACQUIRED to RUNNING 2015-10-26 11:30:23,291 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo: checking for deactivate... 2015-10-26 11:30:37,272 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:30:37,272 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:31:46,427 INFO org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is accessing unchecked http://192.168.3.182:29354/ which is the app master GUI of application_1443146594954_0057 owned by hdfs 2015-10-26 11:32:06,990 INFO org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is accessing unchecked http://192.168.3.182:29354/ which is the app master GUI of application_1443146594954_0057 owned by hdfs 2015-10-26 11:32:08,303 INFO org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is accessing unchecked http://192.168.3.182:29354/ which is the app master GUI of application_1443146594954_0057 owned by hdfs 2015-10-26 11:32:28,120 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:32:28,120 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:32:39,739 INFO org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is accessing unchecked http://192.168.3.182:29354/ which is the app master GUI of application_1443146594954_0057 owned by hdfs 2015-10-26 11:32:44,504 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:32:44,504 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:33:08,084 INFO org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is accessing unchecked http://192.168.3.182:29354/ which is the app master GUI of application_1443146594954_0057 owned by hdfs 2015-10-26 11:33:53,624 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:33:53,624 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:34:14,232 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:34:14,232 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:34:15,640 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:34:15,640 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:34:35,352 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:34:35,352 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:34:47,000 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:34:47,000 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:34:51,736 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:34:51,736 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:35:15,288 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:35:15,288 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:36:00,856 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:36:00,856 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:36:21,464 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:36:21,464 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:36:22,872 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:36:22,872 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:36:42,584 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:36:42,584 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:36:54,232 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:36:54,232 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:36:58,968 WARN org.mortbay.log: /proxy/application_1443146594954_0056/: java.net.ConnectException: 连接超时 2015-10-26 11:37:22,520 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:37:22,520 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:38:08,088 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:38:08,088 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:38:28,696 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:38:28,696 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:38:30,104 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:38:30,104 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:38:49,816 WARN org.mortbay.log: /proxy/application_1443146594954_0057/: java.net.ConnectException: 连接超时 2015-10-26 11:39:01,464 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:39:01,464 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:39:14,935 INFO org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is accessing unchecked http://192.168.3.182:29354/ which is the app master GUI of application_1443146594954_0057 owned by hdfs 2015-10-26 11:39:29,752 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:39:29,752 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:40:15,320 WARN org.mortbay.log: /proxy/application_1443146594954_0057/: java.net.ConnectException: 连接超时 2015-10-26 11:40:35,928 WARN org.mortbay.log: /proxy/application_1443146594954_0057/: java.net.ConnectException: 连接超时 2015-10-26 11:40:37,336 WARN org.mortbay.log: /proxy/application_1443146594954_0057/: java.net.ConnectException: 连接超时 2015-10-26 11:40:49,629 INFO org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is accessing unchecked http://192.168.3.182:29354/ which is the app master GUI of application_1443146594954_0057 owned by hdfs 2015-10-26 11:41:08,696 WARN org.mortbay.log: /proxy/application_1443146594954_0057/: java.net.ConnectException: 连接超时 2015-10-26 11:41:22,136 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:41:22,136 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:41:36,984 WARN org.mortbay.log: /proxy/application_1443146594954_0057/: java.net.ConnectException: 连接超时 2015-10-26 11:42:56,856 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:42:56,856 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:43:29,368 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:43:29,368 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:45:04,088 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:45:04,088 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request 2015-10-26 11:45:36,600 INFO org.apache.commons.httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: 连接超时 2015-10-26 11:45:36,600 INFO org.apache.commons.httpclient.HttpMethodDirector: Retrying request -- Earthson Lu On October 26, 2015 at 15:30:21, Deng Ching-Mallete (och...@apache.org) wrote: Hi Earthson, Unfortunately, attachments aren't allowed in the list so they seemed to have been removed from your email. Anyway, what happens when you click the ApplicationMaster link? Thanks, Deng On Mon, Oct 26, 2015 at 2:21 PM, Earthson <earthson...@gmail.com> wrote: We are using Spark 1.5.1 with `--master yarn`, Yarn RM is running in HA mode. direct visit click ApplicationMaster link YARN RM log -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Yarn-Client-Can-not-access-SparkUI-tp25197.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org