Re: How to explain SchedulerBackend.reviveOffers()?

2016-06-20 Thread Matei Zaharia
Hi Jacek, This applies to all schedulers actually -- it just tells Spark to re-check the available nodes and possibly launch tasks on them, because a new stage was submitted. Then when any node is available, the scheduler will call the TaskSetManager with an "offer" for the node. Matei > On

How to explain SchedulerBackend.reviveOffers()?

2016-06-20 Thread Jacek Laskowski
Hi, Whenever I see `backend.reviveOffers()` I'm struggling myself with properly explaining what it does. My understanding is that it requests a SchedulerBackend (that's responsible for talking to a cluster manager) to...that's the moment I'm not sure about. How would you explain