Re: Can not allocate executor when running spark on mesos

[Iulian Dragoș]

On Thu, Sep 10, 2015 at 3:35 AM, canan chen  wrote:

> Finally got the answer.  Actually it works fine. The allocation behavior
> on mesos is a little different from yarn/standalone. Seems the executor in
> mesos is lazily allocated (only when job is executed) while executor in
> yarn/standalone is allocated when spark-shell is started.
>

That's in fine-grained mode (the default). You can turn on coarse-grained
mode to acquire executors on startup.

iulian


>
>
>
> On Tue, Sep 8, 2015 at 10:39 PM, canan chen  wrote:
>
>> Yes, I follow the guide in this doc, and run it as mesos client mode
>>
>> On Tue, Sep 8, 2015 at 6:31 PM, Akhil Das 
>> wrote:
>>
>>> In which mode are you submitting your application? (coarse-grained or
>>> fine-grained(default)). Have you gone through this documentation already?
>>> http://spark.apache.org/docs/latest/running-on-mesos.html#using-a-mesos-master-url
>>>
>>> Thanks
>>> Best Regards
>>>
>>> On Tue, Sep 8, 2015 at 12:54 PM, canan chen  wrote:
>>>
 Hi all,

 I try to run spark on mesos, but it looks like I can not allocate
 resources from mesos. I am not expert of mesos, but from the mesos log, it
 seems spark always decline the offer from mesos. Not sure what's wrong,
 maybe need some configuration change. Here's the mesos master log

 I0908 15:08:16.515960 301916160 master.cpp:1767] Received registration
 request for framework 'Spark shell' at
 scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
 I0908 15:08:16.520545 301916160 master.cpp:1834] Registering framework
 20150908-143320-16777343-5050-41965- (Spark shell) at
 scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133 with
 checkpointing disabled and capabilities [  ]
 I0908 15:08:16.522307 300843008 hierarchical.hpp:386] Added framework
 20150908-143320-16777343-5050-41965-
 I0908 15:08:16.525845 301379584 master.cpp:4290] Sending 1 offers to
 framework 20150908-143320-16777343-5050-41965- (Spark shell) at
 scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
 I0908 15:08:16.637677 302452736 master.cpp:2884] Processing DECLINE
 call for offers: [ 20150908-143320-16777343-5050-41965-O0 ] for framework
 20150908-143320-16777343-5050-41965- (Spark shell) at
 scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
 I0908 15:08:16.639197 299233280 hierarchical.hpp:761] Recovered
 cpus(*):8; mem(*):15360; disk(*):470842; ports(*):[31000-32000] (total:
 cpus(*):8; mem(*):15360; disk(*):470842; ports(*):[31000-32000], allocated:
 ) on slave 20150908-143320-16777343-5050-41965-S0 from framework
 20150908-143320-16777343-5050-41965-
 I0908 15:08:21.786932 300306432 master.cpp:4290] Sending 1 offers to
 framework 20150908-143320-16777343-5050-41965- (Spark shell) at
 scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
 I0908 15:08:21.789979 298696704 master.cpp:2884] Processing DECLINE
 call for offers: [ 20150908-143320-16777343-5050-41965-O1 ] for framework
 20150908-143320-16777343-5050-41965- (Spark shell) at
 scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133

>>>
>>>
>>
>


-- 

--
Iulian Dragos

--
Reactive Apps on the JVM
www.typesafe.com

Re: Can not allocate executor when running spark on mesos

[canan chen]

Finally got the answer.  Actually it works fine. The allocation behavior on
mesos is a little different from yarn/standalone. Seems the executor in
mesos is lazily allocated (only when job is executed) while executor in
yarn/standalone is allocated when spark-shell is started.



On Tue, Sep 8, 2015 at 10:39 PM, canan chen  wrote:

> Yes, I follow the guide in this doc, and run it as mesos client mode
>
> On Tue, Sep 8, 2015 at 6:31 PM, Akhil Das 
> wrote:
>
>> In which mode are you submitting your application? (coarse-grained or
>> fine-grained(default)). Have you gone through this documentation already?
>> http://spark.apache.org/docs/latest/running-on-mesos.html#using-a-mesos-master-url
>>
>> Thanks
>> Best Regards
>>
>> On Tue, Sep 8, 2015 at 12:54 PM, canan chen  wrote:
>>
>>> Hi all,
>>>
>>> I try to run spark on mesos, but it looks like I can not allocate
>>> resources from mesos. I am not expert of mesos, but from the mesos log, it
>>> seems spark always decline the offer from mesos. Not sure what's wrong,
>>> maybe need some configuration change. Here's the mesos master log
>>>
>>> I0908 15:08:16.515960 301916160 master.cpp:1767] Received registration
>>> request for framework 'Spark shell' at
>>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>>> I0908 15:08:16.520545 301916160 master.cpp:1834] Registering framework
>>> 20150908-143320-16777343-5050-41965- (Spark shell) at
>>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133 with
>>> checkpointing disabled and capabilities [  ]
>>> I0908 15:08:16.522307 300843008 hierarchical.hpp:386] Added framework
>>> 20150908-143320-16777343-5050-41965-
>>> I0908 15:08:16.525845 301379584 master.cpp:4290] Sending 1 offers to
>>> framework 20150908-143320-16777343-5050-41965- (Spark shell) at
>>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>>> I0908 15:08:16.637677 302452736 master.cpp:2884] Processing DECLINE call
>>> for offers: [ 20150908-143320-16777343-5050-41965-O0 ] for framework
>>> 20150908-143320-16777343-5050-41965- (Spark shell) at
>>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>>> I0908 15:08:16.639197 299233280 hierarchical.hpp:761] Recovered
>>> cpus(*):8; mem(*):15360; disk(*):470842; ports(*):[31000-32000] (total:
>>> cpus(*):8; mem(*):15360; disk(*):470842; ports(*):[31000-32000], allocated:
>>> ) on slave 20150908-143320-16777343-5050-41965-S0 from framework
>>> 20150908-143320-16777343-5050-41965-
>>> I0908 15:08:21.786932 300306432 master.cpp:4290] Sending 1 offers to
>>> framework 20150908-143320-16777343-5050-41965- (Spark shell) at
>>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>>> I0908 15:08:21.789979 298696704 master.cpp:2884] Processing DECLINE call
>>> for offers: [ 20150908-143320-16777343-5050-41965-O1 ] for framework
>>> 20150908-143320-16777343-5050-41965- (Spark shell) at
>>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>>>
>>
>>
>

Can not allocate executor when running spark on mesos

[canan chen]

Hi all,

I try to run spark on mesos, but it looks like I can not allocate resources
from mesos. I am not expert of mesos, but from the mesos log, it seems
spark always decline the offer from mesos. Not sure what's wrong, maybe
need some configuration change. Here's the mesos master log

I0908 15:08:16.515960 301916160 master.cpp:1767] Received registration
request for framework 'Spark shell' at
scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
I0908 15:08:16.520545 301916160 master.cpp:1834] Registering framework
20150908-143320-16777343-5050-41965- (Spark shell) at
scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133 with
checkpointing disabled and capabilities [  ]
I0908 15:08:16.522307 300843008 hierarchical.hpp:386] Added framework
20150908-143320-16777343-5050-41965-
I0908 15:08:16.525845 301379584 master.cpp:4290] Sending 1 offers to
framework 20150908-143320-16777343-5050-41965- (Spark shell) at
scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
I0908 15:08:16.637677 302452736 master.cpp:2884] Processing DECLINE call
for offers: [ 20150908-143320-16777343-5050-41965-O0 ] for framework
20150908-143320-16777343-5050-41965- (Spark shell) at
scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
I0908 15:08:16.639197 299233280 hierarchical.hpp:761] Recovered cpus(*):8;
mem(*):15360; disk(*):470842; ports(*):[31000-32000] (total: cpus(*):8;
mem(*):15360; disk(*):470842; ports(*):[31000-32000], allocated: ) on slave
20150908-143320-16777343-5050-41965-S0 from framework
20150908-143320-16777343-5050-41965-
I0908 15:08:21.786932 300306432 master.cpp:4290] Sending 1 offers to
framework 20150908-143320-16777343-5050-41965- (Spark shell) at
scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
I0908 15:08:21.789979 298696704 master.cpp:2884] Processing DECLINE call
for offers: [ 20150908-143320-16777343-5050-41965-O1 ] for framework
20150908-143320-16777343-5050-41965- (Spark shell) at
scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133

Re: Can not allocate executor when running spark on mesos

[Akhil Das]

In which mode are you submitting your application? (coarse-grained or
fine-grained(default)). Have you gone through this documentation already?
http://spark.apache.org/docs/latest/running-on-mesos.html#using-a-mesos-master-url

Thanks
Best Regards

On Tue, Sep 8, 2015 at 12:54 PM, canan chen  wrote:

> Hi all,
>
> I try to run spark on mesos, but it looks like I can not allocate
> resources from mesos. I am not expert of mesos, but from the mesos log, it
> seems spark always decline the offer from mesos. Not sure what's wrong,
> maybe need some configuration change. Here's the mesos master log
>
> I0908 15:08:16.515960 301916160 master.cpp:1767] Received registration
> request for framework 'Spark shell' at
> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
> I0908 15:08:16.520545 301916160 master.cpp:1834] Registering framework
> 20150908-143320-16777343-5050-41965- (Spark shell) at
> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133 with
> checkpointing disabled and capabilities [  ]
> I0908 15:08:16.522307 300843008 hierarchical.hpp:386] Added framework
> 20150908-143320-16777343-5050-41965-
> I0908 15:08:16.525845 301379584 master.cpp:4290] Sending 1 offers to
> framework 20150908-143320-16777343-5050-41965- (Spark shell) at
> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
> I0908 15:08:16.637677 302452736 master.cpp:2884] Processing DECLINE call
> for offers: [ 20150908-143320-16777343-5050-41965-O0 ] for framework
> 20150908-143320-16777343-5050-41965- (Spark shell) at
> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
> I0908 15:08:16.639197 299233280 hierarchical.hpp:761] Recovered cpus(*):8;
> mem(*):15360; disk(*):470842; ports(*):[31000-32000] (total: cpus(*):8;
> mem(*):15360; disk(*):470842; ports(*):[31000-32000], allocated: ) on slave
> 20150908-143320-16777343-5050-41965-S0 from framework
> 20150908-143320-16777343-5050-41965-
> I0908 15:08:21.786932 300306432 master.cpp:4290] Sending 1 offers to
> framework 20150908-143320-16777343-5050-41965- (Spark shell) at
> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
> I0908 15:08:21.789979 298696704 master.cpp:2884] Processing DECLINE call
> for offers: [ 20150908-143320-16777343-5050-41965-O1 ] for framework
> 20150908-143320-16777343-5050-41965- (Spark shell) at
> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>

Re: Can not allocate executor when running spark on mesos

[canan chen]

Yes, I follow the guide in this doc, and run it as mesos client mode

On Tue, Sep 8, 2015 at 6:31 PM, Akhil Das 
wrote:

> In which mode are you submitting your application? (coarse-grained or
> fine-grained(default)). Have you gone through this documentation already?
> http://spark.apache.org/docs/latest/running-on-mesos.html#using-a-mesos-master-url
>
> Thanks
> Best Regards
>
> On Tue, Sep 8, 2015 at 12:54 PM, canan chen  wrote:
>
>> Hi all,
>>
>> I try to run spark on mesos, but it looks like I can not allocate
>> resources from mesos. I am not expert of mesos, but from the mesos log, it
>> seems spark always decline the offer from mesos. Not sure what's wrong,
>> maybe need some configuration change. Here's the mesos master log
>>
>> I0908 15:08:16.515960 301916160 master.cpp:1767] Received registration
>> request for framework 'Spark shell' at
>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>> I0908 15:08:16.520545 301916160 master.cpp:1834] Registering framework
>> 20150908-143320-16777343-5050-41965- (Spark shell) at
>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133 with
>> checkpointing disabled and capabilities [  ]
>> I0908 15:08:16.522307 300843008 hierarchical.hpp:386] Added framework
>> 20150908-143320-16777343-5050-41965-
>> I0908 15:08:16.525845 301379584 master.cpp:4290] Sending 1 offers to
>> framework 20150908-143320-16777343-5050-41965- (Spark shell) at
>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>> I0908 15:08:16.637677 302452736 master.cpp:2884] Processing DECLINE call
>> for offers: [ 20150908-143320-16777343-5050-41965-O0 ] for framework
>> 20150908-143320-16777343-5050-41965- (Spark shell) at
>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>> I0908 15:08:16.639197 299233280 hierarchical.hpp:761] Recovered
>> cpus(*):8; mem(*):15360; disk(*):470842; ports(*):[31000-32000] (total:
>> cpus(*):8; mem(*):15360; disk(*):470842; ports(*):[31000-32000], allocated:
>> ) on slave 20150908-143320-16777343-5050-41965-S0 from framework
>> 20150908-143320-16777343-5050-41965-
>> I0908 15:08:21.786932 300306432 master.cpp:4290] Sending 1 offers to
>> framework 20150908-143320-16777343-5050-41965- (Spark shell) at
>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>> I0908 15:08:21.789979 298696704 master.cpp:2884] Processing DECLINE call
>> for offers: [ 20150908-143320-16777343-5050-41965-O1 ] for framework
>> 20150908-143320-16777343-5050-41965- (Spark shell) at
>> scheduler-1ea1c85b-68bd-40b4-8c7c-ddccfd56f82b@192.168.3.3:57133
>>
>
>