We cannot have control over RDD going out of scope from memory as it was handled by JVM. Thus I am not sure try and finalize will help. Thus I wanted to use some mechanism to cleanup of some temporary data which is created by RDD immediately as soon as it goes out of scope.
Any ideas ? Thanks, Nasrulla From: Phillip Henry <[email protected]> Sent: Tuesday, June 11, 2019 11:28 PM To: Nasrulla Khan Haris <[email protected]> Cc: Vinoo Ganesh <[email protected]>; [email protected] Subject: Re: Adding Custom finalize method to RDDs. That's not the kind of thing a finalize method was ever supposed to do. Use a try/finally block instead. Phillip On Wed, 12 Jun 2019, 00:01 Nasrulla Khan Haris, <[email protected]<mailto:[email protected]>> wrote: I want to delete some files which I created In my datasource api, as soon as the RDD is cleaned up. Thanks, Nasrulla From: Vinoo Ganesh <[email protected]<mailto:[email protected]>> Sent: Monday, June 10, 2019 1:32 PM To: Nasrulla Khan Haris <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Subject: Re: Adding Custom finalize method to RDDs. Generally overriding the finalize() method is an antipattern (it was in fact deprecated in java 11 https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Object.html#finalize()) . What’s the use case here? From: Nasrulla Khan Haris <[email protected]<mailto:[email protected]>> Date: Monday, June 10, 2019 at 15:44 To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: RE: Adding Custom finalize method to RDDs. Hello Everyone, Is there a way to do it from user-code ? Thanks, Nasrulla From: Nasrulla Khan Haris <[email protected]<mailto:[email protected]>> Sent: Sunday, June 9, 2019 5:30 PM To: [email protected]<mailto:[email protected]> Subject: Adding Custom finalize method to RDDs. Hi All, Is there a way to add custom finalize method to RDD objects to add custom logic when RDDs are destructed by JVM ? Thanks, Nasrulla
