Thanks for your reply. I checked the UI, but I'm not sure where to find this info. I can see the number of CPUs or memory, but nothing about it's dominant resource or share... (Mesos 0.21.1). I guess I can compute the share by looking at the total cores and memory, though :)
It'd be great if there was a more direct way to understand how the allocator works, and why it skips a certain framework. I checked the sources but there doesn't seem to be any logging in that part of the code... iulian On Wed, Aug 19, 2015 at 2:34 PM, Hans van den Bogert <hansbog...@gmail.com> wrote: > Have you inspected the framework page/tab in the Mesos master web UI? > Perhaps, as you already suspect, DRF is only handing out resources to > frameworks which have a lower dominant resource. So you could check if your > spark instance has a high dominant resource due to the executors taking up > a lot of memory. > > I’m having alike problems in a, albeit contrived, environment where there > are 4 long running spark instances, where the first instance (only first > by a small time value) gets offered all mesos-slaves and runs the executor. > The next instances have a lower chance of getting the same amount of > memory, but as their dominant resource is lower (memory) they more often > get CPU resources compared to that first instance. Counter intuitively, the > first instance finishes last. > > On 19 Aug 2015, at 14:07, Iulian Dragoș <iulian.dra...@typesafe.com> > wrote: > > I am facing a problem with a framework not getting any resource offers for > 15-20 minutes, while other frameworks (8-9 of them) continuously get offers. > > The framework is Spark (running in fine-grained mode), and is launched > with Chronos. After a few tasks successfully executed, it stops getting > offers, though looking at the master logs we see other frameworks getting > offers every few seconds. For some reason, the Spark one isn't getting them > for a very long period of time. > > Can such behaviour be explained by the DRF algorithm? How could I debug > this? > > thanks, > iulian > > -- > > -- > Iulian Dragos > > ------ > Reactive Apps on the JVM > www.typesafe.com > er > > > -- -- Iulian Dragos ------ Reactive Apps on the JVM www.typesafe.com
