Re: Enabling spark_shuffle service without restarting YARN Node Manager

[Vinay Kashyap]

Thanks for your reply guys.

@Alex :  Hope in the future releases we might get a way to this.

@Saisai : The concern regarding the Node Manager restart is that, if in a
shared YARN cluster running other applications as well apart from Spark,
for enabling spark shuffle service, other running applications should not
be disturbed. That was the only concern.


Thanks and regards
Vinay Kashyap



On Wed, Mar 16, 2016 at 4:05 PM, Saisai Shao  wrote:

> If you want to avoid existing job failure while restarting NM, you could
> enable work preserving for NM, in this case, the restart of NM will not
> affect the running containers (containers can still run). That could
> alleviate NM restart problem.
>
> Thanks
> Saisai
>
> On Wed, Mar 16, 2016 at 6:30 PM, Alex Dzhagriev  wrote:
>
>> Hi Vinay,
>>
>> I believe it's not possible as the spark-shuffle code should run in the
>> same JVM process as the Node Manager. I haven't heard anything about on the
>> fly bytecode loading in the Node Manger.
>>
>> Thanks, Alex.
>>
>> On Wed, Mar 16, 2016 at 10:12 AM, Vinay Kashyap 
>> wrote:
>>
>>> Hi all,
>>>
>>> I am using *Spark 1.5.1* in *yarn-client* mode along with *CDH 5.5*
>>>
>>> As per the documentation to enable Dynamic Allocation of Executors in
>>> Spark,
>>> it is required to add the shuffle service jar to YARN Node Manager's
>>> classpath and restart the YARN Node Manager.
>>>
>>> Is there any way to to dynamically supply the shuffle service jar
>>> information from the application itself and avoid disturbing the running
>>> YARN service.
>>>
>>> Tried couple of options by uploading the jar to hdfs and set
>>> *yarn.application.classpath* but did not work. On container launch for
>>> the executor it fails to recognize the shuffle service.
>>>
>>> Any help would be greatly appreciated.
>>>
>>> --
>>> *Thanks and regards*
>>> *Vinay Kashyap*
>>>
>>
>>
>


-- 
*Thanks and regards*
*Vinay Kashyap*

Re: Enabling spark_shuffle service without restarting YARN Node Manager

[Saisai Shao]

If you want to avoid existing job failure while restarting NM, you could
enable work preserving for NM, in this case, the restart of NM will not
affect the running containers (containers can still run). That could
alleviate NM restart problem.

Thanks
Saisai

On Wed, Mar 16, 2016 at 6:30 PM, Alex Dzhagriev  wrote:

> Hi Vinay,
>
> I believe it's not possible as the spark-shuffle code should run in the
> same JVM process as the Node Manager. I haven't heard anything about on the
> fly bytecode loading in the Node Manger.
>
> Thanks, Alex.
>
> On Wed, Mar 16, 2016 at 10:12 AM, Vinay Kashyap 
> wrote:
>
>> Hi all,
>>
>> I am using *Spark 1.5.1* in *yarn-client* mode along with *CDH 5.5*
>>
>> As per the documentation to enable Dynamic Allocation of Executors in
>> Spark,
>> it is required to add the shuffle service jar to YARN Node Manager's
>> classpath and restart the YARN Node Manager.
>>
>> Is there any way to to dynamically supply the shuffle service jar
>> information from the application itself and avoid disturbing the running
>> YARN service.
>>
>> Tried couple of options by uploading the jar to hdfs and set
>> *yarn.application.classpath* but did not work. On container launch for
>> the executor it fails to recognize the shuffle service.
>>
>> Any help would be greatly appreciated.
>>
>> --
>> *Thanks and regards*
>> *Vinay Kashyap*
>>
>
>

Re: Enabling spark_shuffle service without restarting YARN Node Manager

[Alex Dzhagriev]

Hi Vinay,

I believe it's not possible as the spark-shuffle code should run in the
same JVM process as the Node Manager. I haven't heard anything about on the
fly bytecode loading in the Node Manger.

Thanks, Alex.

On Wed, Mar 16, 2016 at 10:12 AM, Vinay Kashyap  wrote:

> Hi all,
>
> I am using *Spark 1.5.1* in *yarn-client* mode along with *CDH 5.5*
>
> As per the documentation to enable Dynamic Allocation of Executors in
> Spark,
> it is required to add the shuffle service jar to YARN Node Manager's
> classpath and restart the YARN Node Manager.
>
> Is there any way to to dynamically supply the shuffle service jar
> information from the application itself and avoid disturbing the running
> YARN service.
>
> Tried couple of options by uploading the jar to hdfs and set
> *yarn.application.classpath* but did not work. On container launch for
> the executor it fails to recognize the shuffle service.
>
> Any help would be greatly appreciated.
>
> --
> *Thanks and regards*
> *Vinay Kashyap*
>

Enabling spark_shuffle service without restarting YARN Node Manager

[Vinay Kashyap]

Hi all,

I am using *Spark 1.5.1* in *yarn-client* mode along with *CDH 5.5*

As per the documentation to enable Dynamic Allocation of Executors in Spark,
it is required to add the shuffle service jar to YARN Node Manager's
classpath and restart the YARN Node Manager.

Is there any way to to dynamically supply the shuffle service jar
information from the application itself and avoid disturbing the running
YARN service.

Tried couple of options by uploading the jar to hdfs and set
*yarn.application.classpath* but did not work. On container launch for the
executor it fails to recognize the shuffle service.

Any help would be greatly appreciated.

-- 
*Thanks and regards*
*Vinay Kashyap*