Yes, it seems Hadoop framework did not consume all offered resources: if
framework launch task (1 CPUs) on offer (10 CPUs), the other 9 CPUs will
return back to master (recoverResouces).
Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
Platform OpenSource Technology, STG, IBM GCG
Thanks everyone!
Stephan - There's a couple of useful points there, will definitely give it
a read.
Klaus - Thanks, we're running a bunch of different frameworks, in that list
there's Hadoop MRv1, Apache Spark, Marathon and a couple of home grown
frameworks we have. In this particular case the
Guangya - Nope, there's no outstanding offers for any frameworks, the ones
that are getting offers are responding properly.
Klaus - This was just a sample of logs for a single agent, the cluster has
at least ~40 agents at any one time.
On 21 January 2016 at 15:20, Guangya Liu
Can you please help check if some outstanding offers in cluster which does
not accept by any framework? You can check this via the endpoint of
/master/state.json endpoint.
If there are some outstanding offers, you can start the master with a
offer_timeout flag to let master rescind some offers if
Do you mean the only one slave is offered to some framework but the others
are starving?
Mesos allocator (DRF) offer resources by host; so if there's only one host,
the other framework can not get resources. We're have several JIRAs on how
to balance resources between frameworks.
Da
Hey all, I am trying to come up with a process that I can say "I am running
as "prod" principal, I connect to the reserve endpoint, and I make a
request for X CPU, Y Mem,, for the "dev role and usable by the "dev"
principal.
I feel like that I should be able to reserve that out, i.e. as a prod
Hi John,
I believe what you're attempting to do should be supported. Try
reserving the resources with "principal = prod" and "role = dev". That
will mean that the dev role will be allowed to use the resources, but
only principals that are allowed to unreserve prod's resources (as
configured by
dev-sandbox-mesos-slave1 mesos-slave[9527]: 2016-01-21
17:54:46,517:9527(0x7f18d63e1700):ZOO_INFO@zookeeper_init@786: Initiating
client connection, host=dev-sandbox-mesos-zk1.nyc.dev.yodle.com:2181
sessionTimeout=1 watcher=0x7f18dfac6610 sessionId=0 sessionPassw
d= context=0x7f18b8002180 f
527]: 2016-01-21
> 17:54:46,517:9527(0x7f18d63e1700):ZOO_INFO@log_env@753: Client
> environment:user.dir=/
> Jan 21 17:54:46 dev-sandbox-mesos-slave1 mesos-slave[9527]: 2016-01-21
> 17:54:46,517:9527(0x7f18d63e1700):ZOO_INFO@zookeeper_init@786: Initiating
> client connection, host=d
9 matches
Mail list logo
