RE: adding shutdownmanagerhook to spark.

2019-05-13 Thread Nasrulla Khan Haris
Thanks Sean Nasrulla -Original Message- From: Sean Owen Sent: Monday, May 13, 2019 4:16 PM To: Nasrulla Khan Haris Cc: dev@spark.apache.org Subject: Re: adding shutdownmanagerhook to spark. Spark just adds a hook to the mechanism that Hadoop exposes. You can do the same. You

Re: adding shutdownmanagerhook to spark.

2019-05-13 Thread Sean Owen
Spark just adds a hook to the mechanism that Hadoop exposes. You can do the same. You shouldn't use Spark's. On Mon, May 13, 2019 at 6:11 PM Nasrulla Khan Haris wrote: > > HI All, > > > > I am trying to add shutdown hook, but looks like shutdown manager object > requires the package to be spark

adding shutdownmanagerhook to spark.

2019-05-13 Thread Nasrulla Khan Haris
HI All, I am trying to add shutdown hook, but looks like shutdown manager object requires the package to be spark only, is there any other API that can help me to do this ? https://github.com/apache/spark/blob/v2.4.0/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala I can see