[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18322 LGTM, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18322 LGTM! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/18322 cc @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18322 @jiangxb1987 can you please help to review this PR? This is a simple code improvement to avoid some unnecessary code execution when left cores is not enough for one executor. I don't

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/18322 @jerryshao, I have added a unit test in MasterSuite, would you like to review it again, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18322 @eatoncys can you please add a unit test in `MasterSuite` to verify your new code if possible? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/18322 @jerryshao I have added warning logs in SparkSubmit , would you like to review it again, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/18322 @jerryshao Ok, I will add warning logs in SparkSubmit, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18322 I see, I understand your changes now. IMO, because user specifically request for 3 cores per executor (as an example), it is not so good to allocate 1 executor with only 1 core, this may

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/18322 @jerryshao I have modified the "app.coresLeft>0" to "app.coresLeft >= coresPerExecutor.getOrElse(1)". And another question is : is it will be better to allocate another executor with 1 core

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/18322 @jerryshao The problem is: If we start an app with the param --total-executor-cores=4 and spark.executor.cores=3, the code "app.coresLeft>0" is always true in

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18322 If you don't see any issue here, what's problem you met? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/18322 @jerryshao I have not see any issue here, and I have tested this again using the latest Master code, the problem also exists. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18322 According to my test, current code of `Master` could handle this situation correctly, did you see any issue here without your fix? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18322 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this