How did the example spark jobs go? SparkPI etc..? Best Regards, Sonal Founder, Nube Technologies <http://www.nubetech.co> Reifier at Strata Hadoop World <http://strataconf.com/big-data-conference-sg-2015/public/schedule/detail/44606> Reifier at Spark Summit 2015 <https://spark-summit.org/2015/events/real-time-fuzzy-matching-with-spark-and-elastic-search/>
<http://in.linkedin.com/in/sonalgoyal> On Tue, Nov 17, 2015 at 3:24 PM, Kayode Odeyemi <drey...@gmail.com> wrote: > Thanks for the reply Sonal. > > I'm on JDK 7 (/usr/lib/jvm/java-7-oracle) > > My env is a YARN cluster made of 7 nodes (6 datanodes/ > node manager, 1 namenode/resource manager). > > On the namenode, is where I executed the spark-submit job while on one of > the datanodes, I executed 'hadoop fs -put /binstore /user/hadoop-user/' > to dump 1TB of data into all the datanodes. That process is still running > without hassle and it's only using 1.3 GB of 1.7g heap space. > > Initially, I submitted 2 jobs to the YARN cluster which was running for 2 > days and suddenly stops. Nothing in the logs shows the root cause. > > > On Tue, Nov 17, 2015 at 11:42 AM, Sonal Goyal <sonalgoy...@gmail.com> > wrote: > >> Could it be jdk related ? Which version are you on? >> >> Best Regards, >> Sonal >> Founder, Nube Technologies <http://www.nubetech.co> >> Reifier at Strata Hadoop World >> <http://strataconf.com/big-data-conference-sg-2015/public/schedule/detail/44606> >> Reifier at Spark Summit 2015 >> <https://spark-summit.org/2015/events/real-time-fuzzy-matching-with-spark-and-elastic-search/> >> >> <http://in.linkedin.com/in/sonalgoyal> >> >> >> >> On Tue, Nov 17, 2015 at 2:48 PM, Kayode Odeyemi <drey...@gmail.com> >> wrote: >> >>> Anyone experienced this issue as well? >>> >>> On Mon, Nov 16, 2015 at 8:06 PM, Kayode Odeyemi <drey...@gmail.com> >>> wrote: >>> >>>> >>>> Or are you saying that the Java process never even starts? >>>> >>>> >>>> Exactly. >>>> >>>> Here's what I got back from jstack as expected: >>>> >>>> hadoop-user@yks-hadoop-m01:/usr/local/spark/bin$ jstack 31316 >>>> 31316: Unable to open socket file: target process not responding or >>>> HotSpot VM not loaded >>>> The -F option can be used when the target process is not responding >>>> hadoop-user@yks-hadoop-m01:/usr/local/spark/bin$ jstack 31316 -F >>>> Attaching to core -F from executable 31316, please wait... >>>> Error attaching to core file: Can't attach to the core file >>>> >>>> >>>> >>> >>> >>> -- >>> Odeyemi 'Kayode O. >>> http://ng.linkedin.com/in/kayodeodeyemi. t: @charyorde >>> >> >> > > > -- > Odeyemi 'Kayode O. > http://ng.linkedin.com/in/kayodeodeyemi. t: @charyorde >
