Re: Stop Spark yarn-client job

[Jeff Zhang]

Could you attach the yarn AM log ?

On Fri, Nov 27, 2015 at 8:10 AM, Jagat Singh  wrote:

> Hi,
>
> What is the correct way to stop fully the Spark job which is running as
> yarn-client using spark-submit.
>
> We are using sc.stop in the code and can see the job still running (in
> yarn resource manager) after final hive insert is complete.
>
> The code flow is
>
> start context
> do somework
> insert to hive
> sc.stop
>
> This is sparkling water job is that matters.
>
> Is there anything else needed ?
>
> Thanks,
>
> J
>
>
>


-- 
Best Regards

Jeff Zhang

Stop Spark yarn-client job

[Jagat Singh]

Hi,

What is the correct way to stop fully the Spark job which is running as
yarn-client using spark-submit.

We are using sc.stop in the code and can see the job still running (in yarn
resource manager) after final hive insert is complete.

The code flow is

start context
do somework
insert to hive
sc.stop

This is sparkling water job is that matters.

Is there anything else needed ?

Thanks,

J