Hi Rohit, 1) Are you launching an application using apex-cli? 2) If you are then is it same machine as before which you used WordCount example to launch from? Is it an example from Apache Malhar ? or Hadoop example? 3) Is your yarn-site.xml is same on all the nodes? 4) If possible can you provide yarn-site.xml from all nodes in the email thread?
Thanks & Regards, Vikram On Sat, May 13, 2017 at 11:05 AM, rohit garg <[email protected]> wrote: > Hello Yes HAdoop Componenet Is running fine i tested it with running a > WordCound Jar on Hadoop cluster and it ran perfectly fine > > These are the error ... > > server1 > 6722 ResourceManager > 6438 SecondaryNameNode > 45846 Jps > 6159 NameNode > ------------- > server 2 > 22356 Jps > 8533 DataNode > 8685 NodeManager > -------------------- > server 3 > 42116 NodeManager > 41964 DataNode > 19343 Jps > server4 > ------------------- > server 4 > 42116 NodeManager > 41964 DataNode > 19343 Jps > ------------------ > server 5 > 28181 NodeManager > 28028 DataNode > 5453 Jps > -------------------- > server 6 > 46722 DataNode > 12196 Jps > 46875 NodeManager > > we are using apex-core 3.5.0 > cloudera hadoop > Hadoop 2.6.0-cdh5.11.0 > Subversion http://github.com/cloudera/hadoop -r > 91a488f2c5abb3de0e6ee74080dbc439c7576fb4 > Compiled by jenkins on 2017-04-06T03:07Z > Compiled with protoc 2.5.0 > From source with checksum 1d879599e1ae47be77ed9f8b55ce9dbc > This command was run using /var/data/hadoop/hadoop-2.6.0- > cdh5.11.0/share/hadoop/common/hadoop-common-2.6.0-cdh5.11.0.jar > > java version "1.8.0_92" > Java(TM) SE Runtime Environment (build 1.8.0_92-b14) > Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode) > > ----------------------yarn-site.xml-------------------- > > <configuration> > <property> > <name>yarn.nodemanager.aux-services</name> > <value>mapreduce_shuffle</value> > </property> > <property> > <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> > <value>org.apache.hadoop.mapred.ShuffleHandler</value> > </property> > <property> > <name>yarn.resourcemanager.hostname</name> > <value>ipv4</value> > </property> > <property> > <name>yarn.resourcemanager.address</name> > <value>ipv4:8032</value> > </property> > <property> > <name>yarn.resourcemanager.scheduler.address</name> > <value>ipv4:8030</value> > </property> > <property> > ...... > </configuration> > > > /var/data/yarn/local/usercache/apex/appcache/application_ > 1494584453454_0006/container_1494584453454_0006_02_000001/tmp/chkp3971982956962675237 > as the basepath for checkpointing. > 2017-05-12 17:48:58,725 ERROR [main] stram.StreamingAppMaster > (StreamingAppMaster.java:main(106)) - Exiting Application Master > java.lang.NoClassDefFoundError: org/apache/http/conn/HttpClien > tConnectionManager > at com.datatorrent.stram.StreamingContainerManager.getAppMaster > ContainerInfo(StreamingContainerManager.java:481) > at com.datatorrent.stram.StreamingContainerManager.init(Streami > ngContainerManager.java:448) > at com.datatorrent.stram.StreamingContainerManager.<init>(Strea > mingContainerManager.java:420) > at com.datatorrent.stram.StreamingContainerManager.getInstance( > StreamingContainerManager.java:3065) > at com.datatorrent.stram.StreamingAppMasterService.serviceInit( > StreamingAppMasterService.java:552) > at org.apache.hadoop.service.AbstractService.init(AbstractServi > ce.java:163) > at com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMa > ster.java:102) > Caused by: java.lang.ClassNotFoundException: > org.apache.http.conn.HttpClientConnectionManager > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 7 more > > Also Some Times the error is that it tries to connect to Resource Manager > 0.0.0.0:8030 instead of ip.address:8030 > It does not get to resourcemanager > > > On Fri, May 12, 2017 at 5:37 PM, Mohit Jotwani <[email protected]> > wrote: > >> Dear Rohit, >> >> Can you check whether you have correctly started Yarn on your machine? It >> looks like it is not able to connect to the Resource Manager. >> >> Regards, >> Mohit >> >> On Fri, May 12, 2017 at 5:30 PM, rohit garg <[email protected]> >> wrote: >> >>> >>> ---------- Forwarded message ---------- >>> From: "rohit garg" <[email protected]> >>> Date: 12 May 2017 14:02 >>> Subject: Hdfs + apex-core >>> To: <[email protected]> >>> Cc: >>> >>> I have installed apache apex core but when I submit a app to run on yarn >>> it tries to connect to 0.0.0.0:8032 >>> >>> >>> >> >> >> -- >> >> Regards, >> >> ___________________________________________________ >> >> *Mohit Jotwani* >> >> Product Manager >> >> E: [email protected] | M: +91 97699 62740 >> >> www.datatorrent.com | apex.apache.org >> >> >> > > > -- > > > ---------------RohitGarg. > > > > >
