Ngone51 commented on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-675220711
> In that case, this 20% can be helpful. The problem is why you would like to set it to 20% in first place? Why not 10% or 30%? If one exactly know he/she would expect 20% more space, I think he/she would/could also set it to 36,000, especially when he/she has no background of this PR. Instead of just increasing the queue size(percentage or fixed number), I'd prefer to improve the utilization of the current queue. The idea of “pool” might be a good inspiration. And as @tgravescs mentioned above, if we worry about some queues, we could set the minimum size to avoid aggressive "sharing". But I think the most important thing here is that we need to investigate the statistics of the queues of common applications and see if the "pool" could really help to ease the problem. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
