[GitHub] spark issue #19845: [SPARK-22651][PYTHON][ML] Prevent initiating multiple Hi...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19845 **[Test build #84342 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84342/testReport)** for PR 19845 at commit

[GitHub] spark issue #19797: [SPARK-22570][SQL] Avoid to create a lot of global varia...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19797 **[Test build #84341 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84341/testReport)** for PR 19797 at commit

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84345/testReport)** for PR 19811 at commit

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #84340 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84340/testReport)** for PR 19222 at commit

[GitHub] spark pull request #19854: SPARK-22660:Use position() and limit() to fix amb...

2017-11-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19854#discussion_r154154221 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformationExec.scala --- @@ -412,7 +412,9 @@ case class HiveScriptIOSchema

[GitHub] spark pull request #19854: SPARK-22660:Use position() and limit() to fix amb...

2017-11-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19854#discussion_r154154076 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java --- @@ -169,8 +169,10 @@ public static ByteBuffer allocateDirectBuffer(int

[GitHub] spark issue #19797: [SPARK-22570][SQL] Avoid to create a lot of global varia...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19797 **[Test build #84337 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84337/testReport)** for PR 19797 at commit

[GitHub] spark issue #19797: [SPARK-22570][SQL] Avoid to create a lot of global varia...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19797 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19844: [SPARK-22635][SQL][ORC] FileNotFoundException whi...

2017-11-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19844 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19854: SPARK-22660:Use position() and limit() to fix ambiguity ...

2017-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19854 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19826: [SPARK-22428][DOC] Add spark application garbage collect...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19826 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19826: [SPARK-22428][DOC] Add spark application garbage collect...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19826 **[Test build #84343 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84343/testReport)** for PR 19826 at commit

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154153331 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/LoggingPodStatusWatcher.scala --- @@ -0,0 +1,184 @@

[GitHub] spark issue #19844: [SPARK-22635][SQL][ORC] FileNotFoundException while read...

2017-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19844 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19845: [SPARK-22651][PYTHON][ML] Prevent initiating multiple Hi...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19845 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19845: [SPARK-22651][PYTHON][ML] Prevent initiating multiple Hi...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19845 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84342/ Test PASSed. ---

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154134373 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverKubernetesCredentialsStep.scala --- @@ -0,0

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154134483 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverKubernetesCredentialsStep.scala --- @@ -0,0

[GitHub] spark issue #19854: SPARK-22660:Use position() and limit() to fix ambiguity ...

2017-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19854 Let's fix the title to `[SPARK-22660][BUILD] Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9` and fill up the PR description BTW. ---

[GitHub] spark pull request #19854: SPARK-22660:Use position() and limit() to fix amb...

2017-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19854#discussion_r154138361 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -99,7 +99,7 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request #19826: [SPARK-22428][DOC] Add spark application garbage ...

2017-11-30 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19826#discussion_r154156746 --- Diff: docs/configuration.md --- @@ -2306,7 +2346,6 @@ showDF(properties, numRows = 200, truncate = FALSE) - --- End

[GitHub] spark issue #19828: [SPARK-22614] Dataset API: repartitionByRange(...)

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19828 **[Test build #84338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84338/testReport)** for PR 19828 at commit

[GitHub] spark issue #19828: [SPARK-22614] Dataset API: repartitionByRange(...)

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19828 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19828: [SPARK-22614] Dataset API: repartitionByRange(...)

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19828 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84338/ Test PASSed. ---

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-30 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r154133181 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -129,34 +152,119 @@ final class QuantileDiscretizer

[GitHub] spark issue #19826: [SPARK-22428][DOC] Add spark application garbage collect...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19826 **[Test build #84343 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84343/testReport)** for PR 19826 at commit

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154143906 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/Client.scala --- @@ -0,0 +1,234 @@ +/* + *

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-11-30 Thread bdrillard
Github user bdrillard commented on the issue: https://github.com/apache/spark/pull/19811 As some context, I had initially found array initializations necessary because the number of `init` methods created to do line-by-line var initializations for large test cases was still

[GitHub] spark issue #19797: [SPARK-22570][SQL] Avoid to create a lot of global varia...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19797 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84341/ Test PASSed. ---

[GitHub] spark issue #19797: [SPARK-22570][SQL] Avoid to create a lot of global varia...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19797 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84340/ Test PASSed. ---

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19851: [SPARK-22654][TESTS] Retry Spark tarball download if fai...

2017-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19851 Merged to master and branch-2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154134088 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesClientFactory.scala --- @@ -0,0 +1,104 @@ +/*

[GitHub] spark issue #19841: [SPARK-22642][SQL] the createdTempDir will not be delete...

2017-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19841 Could we minimise the diff? It seems hard to follow to me too. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19826: [SPARK-22428][DOC] Add spark application garbage ...

2017-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19826#discussion_r154134827 --- Diff: docs/configuration.md --- @@ -2306,7 +2346,6 @@ showDF(properties, numRows = 200, truncate = FALSE) - --- End diff

[GitHub] spark issue #19854: SPARK-22660:Use position() and limit() to fix ambiguity ...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19854 **[Test build #84344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84344/testReport)** for PR 19854 at commit

[GitHub] spark issue #19826: [SPARK-22428][DOC] Add spark application garbage collect...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19826 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84343/ Test PASSed. ---

[GitHub] spark issue #19797: [SPARK-22570][SQL] Avoid to create a lot of global varia...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19797 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84337/ Test PASSed. ---

[GitHub] spark pull request #19852: [SPARK-22655][PySpark] Throw exception rather tha...

2017-11-30 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19852#discussion_r154122767 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -319,7 +319,7 @@ private[spark] abstract class BasePythonRunner[IN,

[GitHub] spark issue #19845: [SPARK-22651][PYTHON][ML] Prevent initiating multiple Hi...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19845 **[Test build #84342 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84342/testReport)** for PR 19845 at commit

[GitHub] spark pull request #19851: [SPARK-22654][TESTS] Retry Spark tarball download...

2017-11-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19851 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19826: [SPARK-22428][DOC] Add spark application garbage collect...

2017-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19826 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19854: SPARK-22660:Use position() and limit() to fix ambiguity ...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19854 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19854: SPARK-22660:Use position() and limit() to fix ambiguity ...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19854 **[Test build #84344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84344/testReport)** for PR 19854 at commit

[GitHub] spark issue #19854: SPARK-22660:Use position() and limit() to fix ambiguity ...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19854 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84344/ Test FAILed. ---

[GitHub] spark pull request #19854: SPARK-22660:Use position() and limit() to fix amb...

2017-11-30 Thread kellyzly
GitHub user kellyzly opened a pull request: https://github.com/apache/spark/pull/19854 SPARK-22660:Use position() and limit() to fix ambiguity issue in scal… …a-2.12 and JDK9 ## What changes were proposed in this pull request? (Please fill in changes proposed

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154012613 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -177,11 +190,67 @@ class CodegenContext {

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154012714 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -177,11 +190,67 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154014248 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -177,11 +190,67 @@ class CodegenContext

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154010888 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -168,6 +166,21 @@ class CodegenContext {

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154019031 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -177,11 +190,67 @@ class CodegenContext {

[GitHub] spark issue #19805: [SPARK-22649][PYTHON][SQL] Adding localCheckpoint to Dat...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19805 **[Test build #84334 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84334/testReport)** for PR 19805 at commit

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154029577 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverKubernetesCredentialsStep.scala --- @@ -0,0

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154022151 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/Client.scala --- @@ -0,0 +1,234 @@ +/* + * Licensed

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-11-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19811 I think we should do optimization incrementally. From ``` private int a; private int b; private int c; ... a = 3; b = 1; c = 5; ... ``` to ```

[GitHub] spark issue #19854: SPARK-22660:Use position() and limit() to fix ambiguity ...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19854 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154016387 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -177,11 +190,67 @@ class CodegenContext {

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154018089 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -177,11 +190,67 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154019399 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -168,6 +166,21 @@ class CodegenContext {

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154019257 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -168,6 +166,21 @@ class CodegenContext {

[GitHub] spark issue #19840: [SPARK-22640][PYSPARK][YARN]switch python exec on execut...

2017-11-30 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19840 cc @cloud-fan @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19840: [SPARK-22640][PYSPARK][YARN]switch python exec on...

2017-11-30 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19840#discussion_r154028953 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -57,9 +57,14 @@ private[spark] abstract class BasePythonRunner[IN,

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154016042 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesClientFactory.scala --- @@ -0,0 +1,104 @@ +/* + *

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154029412 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverKubernetesCredentialsStep.scala --- @@ -0,0

[GitHub] spark issue #19855: [SPARK-22662] [SQL] Failed to prune columns after rewrit...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19855 **[Test build #84333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84333/testReport)** for PR 19855 at commit

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154020585 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -168,6 +166,21 @@ class CodegenContext {

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-11-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r154030837 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -177,11 +190,67 @@ class CodegenContext {

[GitHub] spark pull request #19851: [SPARK-22654][TESTS] Retry Spark tarball download...

2017-11-30 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19851#discussion_r154027755 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -50,14 +52,24 @@ class

[GitHub] spark pull request #19851: [SPARK-22654][TESTS] Retry Spark tarball download...

2017-11-30 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19851#discussion_r154029237 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -50,14 +52,24 @@ class

[GitHub] spark pull request #19851: [SPARK-22654][TESTS] Retry Spark tarball download...

2017-11-30 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19851#discussion_r154027519 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -50,14 +52,24 @@ class

[GitHub] spark issue #19852: [SPARK-22655][PySpark] Throw exception rather than exit ...

2017-11-30 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19852 Please update the title `[SPARK-22655][PySpark] Throw exception rather than exit silently in PythonRunner when Spark session is stopped` ---

[GitHub] spark pull request #19852: [SPARK-22655][PySpark] Throw exception rather tha...

2017-11-30 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19852#discussion_r154035525 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -319,7 +319,7 @@ private[spark] abstract class BasePythonRunner[IN,

[GitHub] spark pull request #19792: [SPARK-22566][PYTHON] Better error message for `_...

2017-11-30 Thread gberger
Github user gberger commented on a diff in the pull request: https://github.com/apache/spark/pull/19792#discussion_r154036405 --- Diff: python/pyspark/sql/types.py --- @@ -1108,19 +1109,33 @@ def _has_nulltype(dt): return isinstance(dt, NullType) -def

[GitHub] spark pull request #19828: [SPARK-22614] Dataset API: repartitionByRange(......

2017-11-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19828#discussion_r154149190 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -838,6 +839,30 @@ case class

[GitHub] spark pull request #19828: [SPARK-22614] Dataset API: repartitionByRange(......

2017-11-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19828#discussion_r154158225 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -839,8 +839,6 @@ case class

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18692 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18692 **[Test build #84349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84349/testReport)** for PR 18692 at commit

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18692 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84349/ Test FAILed. ---

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19717 **[Test build #84354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84354/testReport)** for PR 19717 at commit

[GitHub] spark pull request #19828: [SPARK-22614] Dataset API: repartitionByRange(......

2017-11-30 Thread adrian-ionescu
Github user adrian-ionescu commented on a diff in the pull request: https://github.com/apache/spark/pull/19828#discussion_r154159546 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -839,8 +839,6 @@ case class

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19717 **[Test build #84346 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84346/testReport)** for PR 19717 at commit

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-11-30 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r154186642 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -55,8 +55,20 @@ import

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-11-30 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r154192687 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -115,9 +116,35 @@ abstract class Expression

[GitHub] spark pull request #19848: [SPARK-22162] Executors and the driver should use...

2017-11-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19848#discussion_r154158671 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -524,6 +525,13 @@ class PairRDDFunctionsSuite extends SparkFunSuite

[GitHub] spark pull request #19848: [SPARK-22162] Executors and the driver should use...

2017-11-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19848#discussion_r154156234 --- Diff: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala --- @@ -106,6 +106,12 @@ abstract class FileCommitProtocol { */

[GitHub] spark pull request #19848: [SPARK-22162] Executors and the driver should use...

2017-11-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19848#discussion_r154157813 --- Diff: core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala --- @@ -70,7 +79,14 @@ object SparkHadoopMapRedUtil extends Logging {

[GitHub] spark pull request #19848: [SPARK-22162] Executors and the driver should use...

2017-11-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19848#discussion_r154156530 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -524,6 +525,13 @@ class PairRDDFunctionsSuite extends SparkFunSuite

[GitHub] spark pull request #19848: [SPARK-22162] Executors and the driver should use...

2017-11-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19848#discussion_r154156477 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -524,6 +525,13 @@ class PairRDDFunctionsSuite extends SparkFunSuite

[GitHub] spark pull request #19848: [SPARK-22162] Executors and the driver should use...

2017-11-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19848#discussion_r154137939 --- Diff: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala --- @@ -106,6 +106,12 @@ abstract class FileCommitProtocol { */

[GitHub] spark issue #19828: [SPARK-22614] Dataset API: repartitionByRange(...)

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19828 **[Test build #84347 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84347/testReport)** for PR 19828 at commit

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19717 **[Test build #84346 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84346/testReport)** for PR 19717 at commit

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19717 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19717 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19717 **[Test build #84348 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84348/testReport)** for PR 19717 at commit

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19717 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84348/ Test FAILed. ---

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-11-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r154168292 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/BaseDriverConfigurationStep.scala --- @@ -0,0

[GitHub] spark pull request #19848: [SPARK-22162] Executors and the driver should use...

2017-11-30 Thread rezasafi
Github user rezasafi commented on a diff in the pull request: https://github.com/apache/spark/pull/19848#discussion_r154168397 --- Diff: core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala --- @@ -70,7 +79,14 @@ object SparkHadoopMapRedUtil extends Logging {

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19717 **[Test build #84350 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84350/testReport)** for PR 19717 at commit

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84345/ Test FAILed. ---

  1   2   3   4   >