Re: Spark big rdd problem

[Eran Witkon]

I run the yarn log command and got the following:
A set of yarnAllocator warnings 'expected to find requests, but found none.'
Then an error:
Akka. ErrorMonitor: associationError ...
But then I still get final app status: Succeeded, exit code 0
What does these errors mean?

On Wed, 16 Dec 2015 at 08:27 Eran Witkon  wrote:

> But what if I don't have more memory?
> On Wed, 16 Dec 2015 at 08:13 Zhan Zhang  wrote:
>
>> There are two cases here. If the container is killed by yarn, you can
>> increase jvm overhead. Otherwise, you have to increase the executor-memory
>> if there is no memory leak happening.
>>
>> Thanks.
>>
>> Zhan Zhang
>>
>> On Dec 15, 2015, at 9:58 PM, Eran Witkon  wrote:
>>
>> If the problem is containers trying to use more memory then they allowed,
>> how do I limit them? I all ready have executor-memory 5G
>> Eran
>> On Tue, 15 Dec 2015 at 23:10 Zhan Zhang  wrote:
>>
>>> You should be able to get the logs from yarn by “yarn logs
>>> -applicationId xxx”, where you can possible find the cause.
>>>
>>> Thanks.
>>>
>>> Zhan Zhang
>>>
>>> On Dec 15, 2015, at 11:50 AM, Eran Witkon  wrote:
>>>
>>> > When running
>>> > val data = sc.wholeTextFile("someDir/*") data.count()
>>> >
>>> > I get numerous warning from yarn till I get aka association exception.
>>> > Can someone explain what happen when spark loads this rdd and can't
>>> fit it all in memory?
>>> > Based on the exception it looks like the server is disconnecting from
>>> yarn and failing... Any idea why? The code is simple but still failing...
>>> > Eran
>>>
>>>
>>

Re: Spark big rdd problem

[Zhan Zhang]

You should be able to get the logs from yarn by “yarn logs -applicationId xxx”, 
where you can possible find the cause.

Thanks.

Zhan Zhang

On Dec 15, 2015, at 11:50 AM, Eran Witkon  wrote:

> When running 
> val data = sc.wholeTextFile("someDir/*") data.count()
> 
> I get numerous warning from yarn till I get aka association exception.
> Can someone explain what happen when spark loads this rdd and can't fit it 
> all in memory?
> Based on the exception it looks like the server is disconnecting from yarn 
> and failing... Any idea why? The code is simple but still failing...
> Eran


-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: Spark big rdd problem

[Eran Witkon]

If the problem is containers trying to use more memory then they allowed,
how do I limit them? I all ready have executor-memory 5G
Eran
On Tue, 15 Dec 2015 at 23:10 Zhan Zhang  wrote:

> You should be able to get the logs from yarn by “yarn logs -applicationId
> xxx”, where you can possible find the cause.
>
> Thanks.
>
> Zhan Zhang
>
> On Dec 15, 2015, at 11:50 AM, Eran Witkon  wrote:
>
> > When running
> > val data = sc.wholeTextFile("someDir/*") data.count()
> >
> > I get numerous warning from yarn till I get aka association exception.
> > Can someone explain what happen when spark loads this rdd and can't fit
> it all in memory?
> > Based on the exception it looks like the server is disconnecting from
> yarn and failing... Any idea why? The code is simple but still failing...
> > Eran
>
>

Re: Spark big rdd problem

[Eran Witkon]

But what if I don't have more memory?
On Wed, 16 Dec 2015 at 08:13 Zhan Zhang  wrote:

> There are two cases here. If the container is killed by yarn, you can
> increase jvm overhead. Otherwise, you have to increase the executor-memory
> if there is no memory leak happening.
>
> Thanks.
>
> Zhan Zhang
>
> On Dec 15, 2015, at 9:58 PM, Eran Witkon  wrote:
>
> If the problem is containers trying to use more memory then they allowed,
> how do I limit them? I all ready have executor-memory 5G
> Eran
> On Tue, 15 Dec 2015 at 23:10 Zhan Zhang  wrote:
>
>> You should be able to get the logs from yarn by “yarn logs -applicationId
>> xxx”, where you can possible find the cause.
>>
>> Thanks.
>>
>> Zhan Zhang
>>
>> On Dec 15, 2015, at 11:50 AM, Eran Witkon  wrote:
>>
>> > When running
>> > val data = sc.wholeTextFile("someDir/*") data.count()
>> >
>> > I get numerous warning from yarn till I get aka association exception.
>> > Can someone explain what happen when spark loads this rdd and can't fit
>> it all in memory?
>> > Based on the exception it looks like the server is disconnecting from
>> yarn and failing... Any idea why? The code is simple but still failing...
>> > Eran
>>
>>
>

Re: Spark big rdd problem

[Zhan Zhang]

There are two cases here. If the container is killed by yarn, you can increase 
jvm overhead. Otherwise, you have to increase the executor-memory if there is 
no memory leak happening.

Thanks.

Zhan Zhang

On Dec 15, 2015, at 9:58 PM, Eran Witkon 
> wrote:

If the problem is containers trying to use more memory then they allowed, how 
do I limit them? I all ready have executor-memory 5G
Eran
On Tue, 15 Dec 2015 at 23:10 Zhan Zhang 
> wrote:
You should be able to get the logs from yarn by “yarn logs -applicationId xxx”, 
where you can possible find the cause.

Thanks.

Zhan Zhang

On Dec 15, 2015, at 11:50 AM, Eran Witkon 
> wrote:

> When running
> val data = sc.wholeTextFile("someDir/*") data.count()
>
> I get numerous warning from yarn till I get aka association exception.
> Can someone explain what happen when spark loads this rdd and can't fit it 
> all in memory?
> Based on the exception it looks like the server is disconnecting from yarn 
> and failing... Any idea why? The code is simple but still failing...
> Eran