Re: Adding Custom finalize method to RDDs.

2019-06-10 Thread Vinoo Ganesh
Generally overriding the finalize() method is an antipattern (it was in fact deprecated in java 11 https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Object.html#finalize()) . What’s the use case here? From: Nasrulla Khan Haris Date: Monday, June 10, 2019 at 15:44 To:

RE: Adding Custom finalize method to RDDs.

2019-06-10 Thread Nasrulla Khan Haris
Hello Everyone, Is there a way to do it from user-code ? Thanks, Nasrulla From: Nasrulla Khan Haris Sent: Sunday, June 9, 2019 5:30 PM To: dev@spark.apache.org Subject: Adding Custom finalize method to RDDs. Hi All, Is there a way to add custom finalize method to RDD objects to add custom

Re: [DISCUSS][SPARK-25299] SPIP: Shuffle storage API

2019-06-10 Thread Imran Rashid
I would be happy to shepherd this. On Wed, Jun 5, 2019 at 7:33 PM Matt Cheah wrote: > Hi everyone, > > > > I wanted to pick this back up again. The discussion has quieted down both > on this thread and on the document. > > > > We made a few revisions to the document to hopefully make it easier

[SS] Why EventTimeStatsAccum for event-time watermark not a named accumulator?

2019-06-10 Thread Jacek Laskowski
Hi, I'm curious why EventTimeStatsAccum is not a named accumulator (not to mention SQLMetric) so the event-time watermark could be monitored in web UI? I've changed the code for EventTimeWatermarkExec physical operator to register EventTimeStatsAccum as a named accumulator and the values are

Re: [DISCUSS][SPARK-25299] SPIP: Shuffle storage API

2019-06-10 Thread Saisai Shao
I'm currently working with MemVerge on the Splash project (one implementation of remote shuffle storage) and followed this ticket for a while. I would like to be a shepherd if no one else volunteered to be. Best regards, Saisai Matt Cheah 于2019年6月6日周四 上午8:33写道: > Hi everyone, > > > > I wanted