Github user Tagar commented on the issue: https://github.com/apache/spark/pull/19046 @tgravescs, here's quote from Wilfred Spiegelenburg - hope it answers both of your questions. > The behaviour I discussed earlier around the Spark AM reservations is not optimal. It turns out that the AM is releasing and then acquiring the reservations again and again until it has enough to run all tasks that it needs. This seems to be triggered by getting a container assigned to the application by the scheduler. Due to the sustained backlog trigger doubling the request size each time it floods the scheduler with requests. This issue will be logged as an internal jira at first. The next steps will be to discuss that behaviour with the Spark team with the goal of making it behave better on the cluster. The MR AM does behave better in this respect as it takes into account the available resources for the application via what is called "headroom". The Spark AM does not do this. thanks.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org