code.
>>
>>
>> On Thu, Aug 30, 2018 at 11:01 PM Felix Cheung
>> wrote:
>>
>>> +1
>>> --
>>> *From:* Mridul Muralidharan
>>> *Sent:* Wednesday, August 29, 2018 1:27:27 PM
>>> *To:* dev@spark.apache
Sent:* Wednesday, August 29, 2018 1:27:27 PM
>> *To:* dev@spark.apache.org
>> *Subject:* Re: SPIP: Executor Plugin (SPARK-24918)
>>
>> +1
>> I left a couple of comments in NiharS's PR, but this is very useful to
>> have in spark !
>>
>> Regards,
>>
+1
Original message From: Reynold Xin Date:
8/30/18 11:11 PM (GMT-08:00) To: Felix Cheung Cc:
dev Subject: Re: SPIP: Executor Plugin (SPARK-24918)
I actually had a similar use case a while ago, but not entirely the same. In my
use case, Spark is already up, but I want
ix Cheung
> wrote:
>
>> +1
>> --
>> *From:* Mridul Muralidharan
>> *Sent:* Wednesday, August 29, 2018 1:27:27 PM
>> *To:* dev@spark.apache.org
>> *Subject:* Re: SPIP: Executor Plugin (SPARK-24918)
>>
>> +1
>> I left a couple of comments
8 1:27:27 PM
> *To:* dev@spark.apache.org
> *Subject:* Re: SPIP: Executor Plugin (SPARK-24918)
>
> +1
> I left a couple of comments in NiharS's PR, but this is very useful to
> have in spark !
>
> Regards,
> Mridul
> On Fri, Aug 3, 2018 at 10:00 AM Imran Rashid
&g
+1
From: Mridul Muralidharan
Sent: Wednesday, August 29, 2018 1:27:27 PM
To: dev@spark.apache.org
Subject: Re: SPIP: Executor Plugin (SPARK-24918)
+1
I left a couple of comments in NiharS's PR, but this is very useful to
have in spark !
Regards,
Mridul
On Fri
+1
I left a couple of comments in NiharS's PR, but this is very useful to
have in spark !
Regards,
Mridul
On Fri, Aug 3, 2018 at 10:00 AM Imran Rashid
wrote:
>
> I'd like to propose adding a plugin api for Executors, primarily for
> instrumentation and debugging
>
I get it, the use case here sounds like adding some logic that's not
specific to an app. Adding to every job of every app would be painful then.
Any task can make sure the init has happened, so doesn't matter if later
tasks execute elsewhere. But if the desired init is really for all tasks,
or
The issue is: where do you name the class you want to initialize? In your job?
What if the plugin has nothing to do with your job, e.g. it's some
monitoring tool? Are you going to modify all your jobs so you include
initialization of that particular plugin? Forcing everybody to change
their code,
+1
Class init is not enough because there is nowhere for you to force a
random class to be initialized. This is basically adding that
mechanism, instead of forcing people to add hacks using e.g.
mapPartitions which don't even cover all scenarios.
On Tue, Aug 28, 2018 at 7:09 AM, Sean Owen
Still +0 on the idea, as I am still not sure it does much over simple JVM
mechanisms like a class init. More comments on the JIRA. I can't say it's a
bad idea though, so would not object to it.
On Tue, Aug 28, 2018 at 8:50 AM Imran Rashid
wrote:
> There has been discussion on jira & the PR, all
+1
Tom
On Tue, Aug 28, 2018 at 8:50 AM Imran Rashid
wrote:
> There has been discussion on jira & the PR, all generally positive, so I'd
> like to call a vote for this spip.
>
> I'll start with own +1.
>
> On Fri, Aug 3, 2018 at 11:59 AM Imran Rashid wrote:
>
>> I'd like to propose adding a
There has been discussion on jira & the PR, all generally positive, so I'd
like to call a vote for this spip.
I'll start with own +1.
On Fri, Aug 3, 2018 at 11:59 AM Imran Rashid wrote:
> I'd like to propose adding a plugin api for Executors, primarily for
> instrumentation and debugging (
>
I'd like to propose adding a plugin api for Executors, primarily for
instrumentation and debugging (
https://issues.apache.org/jira/browse/SPARK-24918). The changes are small,
but as its adding a new api, it might be spip-worthy. I mentioned it as
well in a recent email I sent about memory
14 matches
Mail list logo
