+1 -------- Original message --------From: Reynold Xin <[email protected]> Date: 8/30/18 11:11 PM (GMT-08:00) To: Felix Cheung <[email protected]> Cc: dev <[email protected]> Subject: Re: SPIP: Executor Plugin (SPARK-24918) I actually had a similar use case a while ago, but not entirely the same. In my use case, Spark is already up, but I want to make sure all existing (and new) executors run some specific code. Can we update the API to support that? I think that's doable if we split the design into two: one is the ability to do what I just mentioned, and second is the ability to register via config class when Spark starts to run the code.
On Thu, Aug 30, 2018 at 11:01 PM Felix Cheung <[email protected]> wrote: +1 From: Mridul Muralidharan <[email protected]> Sent: Wednesday, August 29, 2018 1:27:27 PM To: [email protected] Subject: Re: SPIP: Executor Plugin (SPARK-24918) +1 I left a couple of comments in NiharS's PR, but this is very useful to have in spark ! Regards, Mridul On Fri, Aug 3, 2018 at 10:00 AM Imran Rashid <[email protected]> wrote: > > I'd like to propose adding a plugin api for Executors, primarily for > instrumentation and debugging > (https://issues.apache.org/jira/browse/SPARK-24918). The changes are small, > but as its adding a new api, it might be spip-worthy. I mentioned it as well in a recent email I sent about memory monitoring > > The spip proposal is here (and attached to the jira as well): https://docs.google.com/document/d/1a20gHGMyRbCM8aicvq4LhWfQmoA5cbHBQtyqIA2hgtc/edit?usp=sharing > > There are already some comments on the jira and pr, and I hope to get more > thoughts and opinions on it. > > thanks, > Imran --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
