Re: Job can not terminated in Spark 2.0 on Yarn

2016-08-02 Thread dhruve ashar 
Can you provide additional logs.

On Tue, Aug 2, 2016 at 2:13 PM, Liangzhao Zeng 
wrote:

> It is 2.6 and code is very simple. I load data file from Hdfs to create
> rdd then same some samples.
>
>
> Thanks
>
> 发自我的 iPhone
>
> 在 Aug 2, 2016，11:01 AM，Ted Yu  写道：
>
> Which hadoop version are you using ?
>
> Can you show snippet of your code ?
>
> Thanks
>
> On Tue, Aug 2, 2016 at 10:06 AM, Liangzhao Zeng 
> wrote:
>
>> Hi,
>>
>>
>> I migrate my code to Spark 2.0 from 1.6. It finish last stage (and result is 
>> correct) but get following errors then start over.
>>
>>
>> Any idea on what happen?
>>
>>
>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>> already stopped! Dropping event 
>> SparkListenerExecutorMetricsUpdate(2,WrappedArray())
>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>> already stopped! Dropping event 
>> SparkListenerExecutorMetricsUpdate(115,WrappedArray())
>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>> already stopped! Dropping event 
>> SparkListenerExecutorMetricsUpdate(70,WrappedArray())
>> 16/08/02 16:59:33 WARN yarn.YarnAllocator: Expected to find pending 
>> requests, but found none.
>> 16/08/02 16:59:33 WARN netty.Dispatcher: Message 
>> RemoteProcessDisconnected(17.138.53.26:55338) dropped. Could not find 
>> MapOutputTracker.
>>
>>
>>
>> Cheers,
>>
>>
>> LZ
>>
>>
>


-- 
-Dhruve Ashar

Re: Job can not terminated in Spark 2.0 on Yarn

2016-08-02 Thread Liangzhao Zeng 
It is 2.6 and code is very simple. I load data file from Hdfs to create rdd 
then same some samples.


Thanks 

发自我的 iPhone

> 在 Aug 2, 2016，11:01 AM，Ted Yu  写道：
> 
> Which hadoop version are you using ?
> 
> Can you show snippet of your code ?
> 
> Thanks
> 
>> On Tue, Aug 2, 2016 at 10:06 AM, Liangzhao Zeng  
>> wrote:
>> Hi, 
>> 
>> I migrate my code to Spark 2.0 from 1.6. It finish last stage (and result is 
>> correct) but get following errors then start over. 
>> 
>> Any idea on what happen?
>> 
>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>> already stopped! Dropping event 
>> SparkListenerExecutorMetricsUpdate(2,WrappedArray())
>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>> already stopped! Dropping event 
>> SparkListenerExecutorMetricsUpdate(115,WrappedArray())
>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>> already stopped! Dropping event 
>> SparkListenerExecutorMetricsUpdate(70,WrappedArray())
>> 16/08/02 16:59:33 WARN yarn.YarnAllocator: Expected to find pending 
>> requests, but found none.
>> 16/08/02 16:59:33 WARN netty.Dispatcher: Message 
>> RemoteProcessDisconnected(17.138.53.26:55338) dropped. Could not find 
>> MapOutputTracker.
>> 
>> 
>> Cheers,
>> 
>> LZ
>

Re: Job can not terminated in Spark 2.0 on Yarn

2016-08-02 Thread Liangzhao Zeng 
It try to execute the job again, from the first stage. 

发自我的 iPhone

> 在 Aug 2, 2016，11:24 AM，dhruve ashar  写道：
> 
> Hi LZ,
> 
> Getting those error messages in logs is normal behavior. When the job 
> completes, it shuts down the SparkListenerBus as there's no need of relaying 
> any spark events to the interested registered listeners. So trying to add 
> events from executors which are yet to shutdown, logs the error message 
> because the event listener bus has stopped.
> 
> "I migrate my code to Spark 2.0 from 1.6. It finish last stage (and result is 
> correct) but get following errors then start over." 
> 
> Can you elaborate on what do you mean by start over?
> 
> -Dhruve
> 
> 
> 
>> On Tue, Aug 2, 2016 at 1:01 PM, Ted Yu  wrote:
>> Which hadoop version are you using ?
>> 
>> Can you show snippet of your code ?
>> 
>> Thanks
>> 
>>> On Tue, Aug 2, 2016 at 10:06 AM, Liangzhao Zeng  
>>> wrote:
>>> Hi, 
>>> 
>>> I migrate my code to Spark 2.0 from 1.6. It finish last stage (and result 
>>> is correct) but get following errors then start over. 
>>> 
>>> Any idea on what happen?
>>> 
>>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>>> already stopped! Dropping event 
>>> SparkListenerExecutorMetricsUpdate(2,WrappedArray())
>>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>>> already stopped! Dropping event 
>>> SparkListenerExecutorMetricsUpdate(115,WrappedArray())
>>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>>> already stopped! Dropping event 
>>> SparkListenerExecutorMetricsUpdate(70,WrappedArray())
>>> 16/08/02 16:59:33 WARN yarn.YarnAllocator: Expected to find pending 
>>> requests, but found none.
>>> 16/08/02 16:59:33 WARN netty.Dispatcher: Message 
>>> RemoteProcessDisconnected(17.138.53.26:55338) dropped. Could not find 
>>> MapOutputTracker.
>>> 
>>> 
>>> Cheers,
>>> 
>>> LZ
> 
> 
> 
> -- 
> -Dhruve Ashar
>

Re: Job can not terminated in Spark 2.0 on Yarn

2016-08-02 Thread dhruve ashar 
Hi LZ,

Getting those error messages in logs is normal behavior. When the job
completes, it shuts down the SparkListenerBus as there's no need of
relaying any spark events to the interested registered listeners. So trying
to add events from executors which are yet to shutdown, logs the error
message because the event listener bus has stopped.

"I migrate my code to Spark 2.0 from 1.6. It finish last stage (and
result is correct) but get following errors then start over."


Can you elaborate on what do you mean by start over?


-Dhruve




On Tue, Aug 2, 2016 at 1:01 PM, Ted Yu  wrote:

> Which hadoop version are you using ?
>
> Can you show snippet of your code ?
>
> Thanks
>
> On Tue, Aug 2, 2016 at 10:06 AM, Liangzhao Zeng 
> wrote:
>
>> Hi,
>>
>>
>> I migrate my code to Spark 2.0 from 1.6. It finish last stage (and result is 
>> correct) but get following errors then start over.
>>
>>
>> Any idea on what happen?
>>
>>
>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>> already stopped! Dropping event 
>> SparkListenerExecutorMetricsUpdate(2,WrappedArray())
>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>> already stopped! Dropping event 
>> SparkListenerExecutorMetricsUpdate(115,WrappedArray())
>> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
>> already stopped! Dropping event 
>> SparkListenerExecutorMetricsUpdate(70,WrappedArray())
>> 16/08/02 16:59:33 WARN yarn.YarnAllocator: Expected to find pending 
>> requests, but found none.
>> 16/08/02 16:59:33 WARN netty.Dispatcher: Message 
>> RemoteProcessDisconnected(17.138.53.26:55338) dropped. Could not find 
>> MapOutputTracker.
>>
>>
>>
>> Cheers,
>>
>>
>> LZ
>>
>>
>


-- 
-Dhruve Ashar

Re: Job can not terminated in Spark 2.0 on Yarn

2016-08-02 Thread Ted Yu 
Which hadoop version are you using ?

Can you show snippet of your code ?

Thanks

On Tue, Aug 2, 2016 at 10:06 AM, Liangzhao Zeng 
wrote:

> Hi,
>
>
> I migrate my code to Spark 2.0 from 1.6. It finish last stage (and result is 
> correct) but get following errors then start over.
>
>
> Any idea on what happen?
>
>
> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
> already stopped! Dropping event 
> SparkListenerExecutorMetricsUpdate(2,WrappedArray())
> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
> already stopped! Dropping event 
> SparkListenerExecutorMetricsUpdate(115,WrappedArray())
> 16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus has 
> already stopped! Dropping event 
> SparkListenerExecutorMetricsUpdate(70,WrappedArray())
> 16/08/02 16:59:33 WARN yarn.YarnAllocator: Expected to find pending requests, 
> but found none.
> 16/08/02 16:59:33 WARN netty.Dispatcher: Message 
> RemoteProcessDisconnected(17.138.53.26:55338) dropped. Could not find 
> MapOutputTracker.
>
>
>
> Cheers,
>
>
> LZ
>
>

Job can not terminated in Spark 2.0 on Yarn

2016-08-02 Thread Liangzhao Zeng 
Hi,


I migrate my code to Spark 2.0 from 1.6. It finish last stage (and
result is correct) but get following errors then start over.


Any idea on what happen?


16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus
has already stopped! Dropping event
SparkListenerExecutorMetricsUpdate(2,WrappedArray())
16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus
has already stopped! Dropping event
SparkListenerExecutorMetricsUpdate(115,WrappedArray())
16/08/02 16:59:33 ERROR scheduler.LiveListenerBus: SparkListenerBus
has already stopped! Dropping event
SparkListenerExecutorMetricsUpdate(70,WrappedArray())
16/08/02 16:59:33 WARN yarn.YarnAllocator: Expected to find pending
requests, but found none.
16/08/02 16:59:33 WARN netty.Dispatcher: Message
RemoteProcessDisconnected(17.138.53.26:55338) dropped. Could not find
MapOutputTracker.



Cheers,


LZ