Hi Akhil, Its running fine when running through Namenode(RM) but fails while running through Gateway, if I add hadoop-core jars to the hadoop directory(/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop/) it works fine.
Its really strange that I am running the job through Spark-Submit and running via NameNode works fine and it fails when running through gateway even when both are having same classpath. Anyone tries running Spark from Gateway? Looking for the quick revert! Thanks, On Wed, Apr 15, 2015 at 12:07 PM, Akhil Das <ak...@sigmoidanalytics.com> wrote: > Make sure your yarn service is running on 8032. > > Thanks > Best Regards > > On Tue, Apr 14, 2015 at 12:35 PM, Vineet Mishra <clearmido...@gmail.com> > wrote: > >> Hi Team, >> >> I am running Spark Word Count example( >> https://github.com/sryza/simplesparkapp), if I go with master as local >> it works fine. >> >> But when I change the master to yarn its end with retries connecting to >> resource manager(stack trace mentioned below), >> >> 15/04/14 11:31:57 INFO RMProxy: Connecting to ResourceManager at / >> 0.0.0.0:8032 >> 15/04/14 11:31:58 INFO Client: Retrying connect to server: >> 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is >> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 >> MILLISECONDS) >> 15/04/14 11:31:59 INFO Client: Retrying connect to server: >> 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is >> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 >> MILLISECONDS) >> >> If I run the same command from Namenode instance it ends with >> ArrayOutofBoundException(Stack trace mentioned below), >> >> 15/04/14 11:38:44 INFO YarnClientSchedulerBackend: SchedulerBackend is >> ready for scheduling beginning after reached minRegisteredResourcesRatio: >> 0.8 >> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1 >> at >> com.cloudera.sparkwordcount.SparkWordCount$.main(SparkWordCount.scala:28) >> at com.cloudera.sparkwordcount.SparkWordCount.main(SparkWordCount.scala) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) >> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) >> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) >> >> Looking forward to get it resolve to work on respective nodes. >> >> Thanks, >> > >
