ApplicationMaster link on cluster web page does not work
Hi configuration !-- Site specific YARN configuration properties -- property nameyarn.app.mapreduce.am.staging-dir/name value/user/value /property property nameyarn.nodemanager.aux-services/name valuemapreduce_shuffle/value /property property nameyarn.nodemanager.aux-services.mapreduce_shuffle.class/name valueorg.apache.hadoop.mapred.ShuffleHandler/value /property property nameyarn.application.classpath/name value /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/* /value /property property nameyarn.resourcemanager.scheduler.address/name value0.0.0.0:8030/value /property property nameyarn.web-proxy.address/name valuevm38.dbweb.ee:8089/value /property /configuration when i click job's ApplicationMaster link (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on claster all applications list I will get: HTTP ERROR 404 Problem accessing /proxy/application_1409246753441_0001/mapreduce. Reason: NOT_FOUND /Powered by Jetty:// /I use yarn.web-proxy because without yarn.web-proxy when I click ApplicationMaster link I will have loads of tcp connections with established statuses. In yarn-proxy log there is nothing interesting -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)
Re: ApplicationMaster link on cluster web page does not work
More information after I started resourcemanager [root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start Starting Hadoop resourcemanager: [ OK ] and I open cluster web interface there is some tcp connections to 8088: [root@vm38 ~]# netstat -np | grep 8088 tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:61120ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:64412ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:50139ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:64407ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:58817ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:52250ESTABLISHED 15723/java this is ok Now I started new map reduce job and got tracking url http://server:8088/proxy/application_1409250808355_0001/ now there are loads of connections and tracking url does not response: tcp0 0 :::90.190.106.33:46910 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:35984 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:37559 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:44154 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:45417 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:57294 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:47949 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:55330 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp 432 0 :::90.190.106.33:8088 :::90.190.106.33:45467 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:58405 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:55580 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:51992 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:44578 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:38686 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:39242 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:55916 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:48064 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:35148 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:42638 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:50836 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:44789 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:55051 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:39207 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:53781 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:54180 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:56344 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:52707 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:52274 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:45417 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:46627 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:44583 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:40928 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:47014 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:60939 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:52274 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:38391 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:47321 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:36186 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:39160 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:54352
Re: ApplicationMaster link on cluster web page does not work
Moved resourcemanager to another server and it works. I guess I have some network miss routing there :) Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) On 28/08/14 21:39, Margusja wrote: More information after I started resourcemanager [root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start Starting Hadoop resourcemanager: [ OK ] and I open cluster web interface there is some tcp connections to 8088: [root@vm38 ~]# netstat -np | grep 8088 tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:61120ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:64412ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:50139ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:64407ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:58817ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::84.50.21.39:52250ESTABLISHED 15723/java this is ok Now I started new map reduce job and got tracking url http://server:8088/proxy/application_1409250808355_0001/ now there are loads of connections and tracking url does not response: tcp0 0 :::90.190.106.33:46910 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:35984 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:37559 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:44154 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:45417 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:57294 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:47949 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:55330 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp 432 0 :::90.190.106.33:8088 :::90.190.106.33:45467 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:58405 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:55580 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:51992 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:44578 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:38686 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:39242 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:55916 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:48064 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:35148 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:42638 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:50836 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:44789 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:55051 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:39207 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:53781 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:54180 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:56344 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:52707 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:52274 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:45417 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:46627 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:44583 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:40928 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:47014 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:60939 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:8088 :::90.190.106.33:52274 ESTABLISHED 15723/java tcp0 0 :::90.190.106.33:38391 :::90.190.106.33:8088 ESTABLISHED 15723/java tcp0 0
