Re: JAVA heap space issue

2016-10-24 Thread Sankar Mittapally
rkR.session(master = >>>>>>>> "spark://ip-172-31-6-116:7077",sparkConfig=list(spark.execut >>>>>>>> or.memory="10g",spark.app.name="Testing",spark.driver.memory >>>>>>>> ="14g",sp

Re: JAVA heap space issue

2016-10-24 Thread Mich Talebzadeh
>>>>> /api/contents?type=directory&_=1477289197671 (123.176.38.226) 7.96ms >>>>>>> Exception in thread "broadcast-exchange-0" >>>>>>> java.lang.OutOfMemoryError: Java >>>>>>> heap space >>>>>>> at >>>>>>> org.apache.spark.sql.execution.joins.LongToUnsafeRowMap.appe >>>>>>> nd(HashedRelation.scala:539) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.joins.LongHashedRelation$.app >>>>>>> ly(HashedRelation.scala:803) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.joins.HashedRelation$.apply(H >>>>>>> ashedRelation.scala:105) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.joins.HashedRelationBroadcast >>>>>>> Mode.transform(HashedRelation.scala:816) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.joins.HashedRelationBroadcast >>>>>>> Mode.transform(HashedRelation.scala:812) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.exchange.BroadcastExchangeExe >>>>>>> c$$anonfun$relationFuture$1$$anonfun$apply$1.apply(Broadcast >>>>>>> ExchangeExec. >>>>>>> scala:90) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.exchange.BroadcastExchangeExe >>>>>>> c$$anonfun$relationFuture$1$$anonfun$apply$1.apply(Broadcast >>>>>>> ExchangeExec. >>>>>>> scala:72) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.SQLExecution$.withExecutionId >>>>>>> (SQLExecution.scala:94) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.exchange.BroadcastExchangeExe >>>>>>> c$$anonfun$relationFuture$1.apply(BroadcastExchangeExec.scala:72) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.exchange.BroadcastExchangeExe >>>>>>> c$$anonfun$relationFuture$1.apply(BroadcastExchangeExec.scala:72) >>>>>>> at >>>>>>> scala.concurrent.impl.Future$PromiseCompletingRunnable.lifte >>>>>>> dTree1$1(Future.scala:24) >>>>>>> at >>>>>>> scala.concurrent.impl.Future$PromiseCompletingRunnable.run(F >>>>>>> uture.scala:24) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >>>>>>> Executor.java:1142) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >>>>>>> lExecutor.java:617) >>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> View this message in context: http://apache-spark-user-list. >>>>>>> 1001560.n3.nabble.com/JAVA-heap-space-issue-tp27950.html >>>>>>> Sent from the Apache Spark User List mailing list archive at >>>>>>> Nabble.com. >>>>>>> >>>>>>> >>>>>>> - >>>>>>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >

Re: JAVA heap space issue

2016-10-24 Thread Sankar Mittapally
pace >>>>>> at >>>>>> org.apache.spark.sql.execution.joins.LongToUnsafeRowMap.appe >>>>>> nd(HashedRelation.scala:539) >>>>>> at >>>>>> org.apache.spark.sql.execution.joins.LongHashedRelatio

Re: JAVA heap space issue

2016-10-24 Thread Mich Talebzadeh
cution.joins.HashedRelationBroadcast >>>>> Mode.transform(HashedRelation.scala:816) >>>>> at >>>>> org.apache.spark.sql.execution.joins.HashedRelationBroadcast >>>>> Mode.transform

Re: JAVA heap space issue

2016-10-24 Thread Sankar Mittapally
relationFuture$1$$anonfun$apply$1.apply(Broadcast >>>> ExchangeExec. >>>> scala:90) >>>> at >>>> org.apache.spark.sql.execution.exchange.BroadcastExchangeExe >>>> c$$

Re: JAVA heap space issue

2016-10-24 Thread Mich Talebzadeh
ture$1$$anonfun$apply$1.apply(Broadcast >>> ExchangeExec. >>> scala:72) >>> at >>> org.apache.spark.sql.execution.SQLExecution$.withExecutionId >>> (SQLExecution.scala:94) >>> at >>> org.apache.spark.sql.execution.exchange.Broad

Re: JAVA heap space issue

2016-10-24 Thread Sankar Mittapally
.BroadcastExchangeExe >> c$$anonfun$relationFuture$1.apply(BroadcastExchangeExec.scala:72) >> at >> org.apache.spark.sql.execution.exchange.BroadcastExchangeExe >> c$$anonfun$relationFuture$1.apply(BroadcastExchangeExec.scala:72) >> at >> scala.co

Re: JAVA heap space issue

2016-10-24 Thread Mich Talebzadeh
t.impl.Future$PromiseCompletingRunnable. > liftedTree1$1(Future.scala:24) > at > scala.concurrent.impl.Future$PromiseCompletingRunnable.run( > Future.scala:24) > at > java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at > java.u

JAVA heap space issue

2016-10-24 Thread sankarmittapally
at java.lang.Thread.run(Thread.java:745) -- View this message in context: http://apache-spark-user-list.1