We cannot have control over RDD going out of scope from memory as it was handled by JVM. Thus I am not sure try and finalize will help. Thus I wanted to use some mechanism to cleanup of some temporary data which is created by RDD immediately as soon as it goes out of scope.
Any ideas ? Thanks, Nasrulla From: Phillip Henry <londonjava...@gmail.com> Sent: Tuesday, June 11, 2019 11:28 PM To: Nasrulla Khan Haris <nasrulla.k...@microsoft.com.invalid> Cc: Vinoo Ganesh <vgan...@palantir.com>; dev@spark.apache.org Subject: Re: Adding Custom finalize method to RDDs. That's not the kind of thing a finalize method was ever supposed to do. Use a try/finally block instead. Phillip On Wed, 12 Jun 2019, 00:01 Nasrulla Khan Haris, <nasrulla.k...@microsoft.com.invalid<mailto:nasrulla.k...@microsoft.com.invalid>> wrote: I want to delete some files which I created In my datasource api, as soon as the RDD is cleaned up. Thanks, Nasrulla From: Vinoo Ganesh <vgan...@palantir.com<mailto:vgan...@palantir.com>> Sent: Monday, June 10, 2019 1:32 PM To: Nasrulla Khan Haris <nasrulla.k...@microsoft.com.INVALID<mailto:nasrulla.k...@microsoft.com.INVALID>>; dev@spark.apache.org<mailto:dev@spark.apache.org> Subject: Re: Adding Custom finalize method to RDDs. Generally overriding the finalize() method is an antipattern (it was in fact deprecated in java 11 https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Object.html#finalize()) . What’s the use case here? From: Nasrulla Khan Haris <nasrulla.k...@microsoft.com.INVALID<mailto:nasrulla.k...@microsoft.com.INVALID>> Date: Monday, June 10, 2019 at 15:44 To: "dev@spark.apache.org<mailto:dev@spark.apache.org>" <dev@spark.apache.org<mailto:dev@spark.apache.org>> Subject: RE: Adding Custom finalize method to RDDs. Hello Everyone, Is there a way to do it from user-code ? Thanks, Nasrulla From: Nasrulla Khan Haris <nasrulla.k...@microsoft.com.INVALID<mailto:nasrulla.k...@microsoft.com.INVALID>> Sent: Sunday, June 9, 2019 5:30 PM To: dev@spark.apache.org<mailto:dev@spark.apache.org> Subject: Adding Custom finalize method to RDDs. Hi All, Is there a way to add custom finalize method to RDD objects to add custom logic when RDDs are destructed by JVM ? Thanks, Nasrulla
