I think I met this problem before, this problem might be due to some race conditions in exit period. The way you mentioned is still valid, this problem only occurs when stopping the application.
Thanks Saisai On Fri, Jan 29, 2016 at 10:22 AM, Nirav Patel <npa...@xactlycorp.com> wrote: > Hi, we were using spark 1.3.1 and launching our spark jobs on yarn-client > mode programmatically via creating a sparkConf and sparkContext object > manually. It was inspired from spark self-contained application example > here: > > > https://spark.apache.org/docs/1.5.2/quick-start.html#self-contained-applications\ > > > > Only additional configuration we would provide would be all related to > yarn like executor instance, cores etc. > > However after upgrading to spark 1.5.2 above application breaks on a line ` > val sparkContext = new SparkContext(sparkConf)` > > 16/01/28 17:38:35 ERROR util.Utils: Uncaught exception in thread main > > java.lang.NullPointerException > > at > org.apache.spark.network.netty.NettyBlockTransferService.close(NettyBlockTransferService.scala:152) > > at org.apache.spark.storage.BlockManager.stop(BlockManager.scala:1228) > > at org.apache.spark.SparkEnv.stop(SparkEnv.scala:100) > > at > org.apache.spark.SparkContext$$anonfun$stop$12.apply$mcV$sp(SparkContext.scala:1749) > > at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1185) > > at org.apache.spark.SparkContext.stop(SparkContext.scala:1748) > > at org.apache.spark.SparkContext.<init>(SparkContext.scala:593) > > > So is this approach still supposed to work? Or do I must use SparkLauncher > class with spark 1.5.2? > > > Thanks > > > > [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> > > <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] > <https://twitter.com/Xactly> [image: Facebook] > <https://www.facebook.com/XactlyCorp> [image: YouTube] > <http://www.youtube.com/xactlycorporation>
