GitHub user mukulmurthy opened a pull request: https://github.com/apache/spark/pull/22176
[SPARK-25181][CORE] Limit Thread Pool size in BlockManager Master and Slave endpoints ## What changes were proposed in this pull request? Limit Thread Pool size in BlockManager Master and Slave endpoints. Currently, BlockManagerMasterEndpoint and BlockManagerSlaveEndpoint both have thread pools with nearly unbounded (Integer.MAX_VALUE) numbers of threads. In certain cases, this can lead to driver OOM errors. This change limits the thread pools to 100 threads; this should not break any existing behavior because any tasks beyond that number will get queued. ## How was this patch tested? Manual testing Please review http://spark.apache.org/contributing.html before opening a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mukulmurthy/oss-spark 25181-threads Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22176.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22176 ---- commit c2223c3862619d2191ea787f3a2ee3c0d8d67ff2 Author: Mukul Murthy <mukul.murthy@...> Date: 2018-08-21T23:16:11Z Limit thread pools in BlockManager Master and Slave endpoints ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org