GitHub user carsonwang opened a pull request:
https://github.com/apache/spark/pull/11713
[SPARK-13889][YARN] Fix integer overflow when calculating the max number of
executor failure
## What changes were proposed in this pull request?
The max number of executor failure before failing the application is
default to twice the maximum number of executors if dynamic allocation is
enabled. The default value for "spark.dynamicAllocation.maxExecutors" is
Int.MaxValue. So this causes an integer overflow and a wrong result. The
calculated value of the default max number of executor failure is 3. This PR
adds a check to avoid the overflow.
## How was this patch tested?
It tests if the value is greater that Int.MaxValue / 2 to avoid the
overflow when it multiplies 2.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/carsonwang/spark IntOverflow
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11713.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11713
----
commit 8469148f32be6ed7a1ccb5178c5ab2d421d71dc4
Author: Carson Wang <carson.w...@intel.com>
Date: 2016-03-15T04:31:14Z
Fix integer overflow when calculating max executor failure number
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
