[GitHub] spark issue #21427: [SPARK-24324][PYTHON] Pandas Grouped Map UDF should assi...

2018-05-30 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/21427 @gatorsmile @ueshin Thanks for joining the discussion! I wonder if you agree that at least for case (1) here https://github.com/apache/spark/pull/21427#issuecomment-392070950, we should

[GitHub] spark issue #21454: [SPARK-24337][Core] Improve error messages for Spark con...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21454 **[Test build #91316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91316/testReport)** for PR 21454 at commit

[GitHub] spark issue #21427: [SPARK-24324][PYTHON] Pandas Grouped Map UDF should assi...

2018-05-30 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21427 Here are some examples that currently work, but would no longer work under the proposed fix. These are all cases where columns are named with strings, but the names do not match the schema (let

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-30 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/21260 Will review this asap. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21427: [SPARK-24324][PYTHON] Pandas Grouped Map UDF should assi...

2018-05-30 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21427 Thanks @ueshin and @gatorsmile for taking a look, I agree the proposed fix changes some behavior, but I think that behavior is either error-prone or doesn't make much sense. Let me put up some

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r191948799 --- Diff: docs/running-on-kubernetes.md --- @@ -121,8 +121,8 @@ This URI is the location of the example jar that is already in the Docker image.

[GitHub] spark pull request #21454: [SPARK-24337][Core] Improve error messages for Sp...

2018-05-30 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21454#discussion_r191944805 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -448,6 +473,22 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request #21454: [SPARK-24337][Core] Improve error messages for Sp...

2018-05-30 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21454#discussion_r191944797 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -448,6 +473,22 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request #21454: [SPARK-24337][Core] Improve error messages for Sp...

2018-05-30 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21454#discussion_r191945182 --- Diff: core/src/test/scala/org/apache/spark/SparkConfSuite.scala --- @@ -339,6 +340,38 @@ class SparkConfSuite extends SparkFunSuite with

[GitHub] spark issue #21454: [SPARK-24337][Core] Improve error messages for Spark con...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21454 **[Test build #91315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91315/testReport)** for PR 21454 at commit

[GitHub] spark pull request #21331: [SPARK-24276][SQL] Order of literals in IN should...

2018-05-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21331 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21362: [SPARK-24197][SparkR][FOLLOWUP] Fixing failing tests for...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21362 Will do more investigation. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21331 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-05-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r191941962 --- Diff: common/network-common/src/main/java/org/apache/spark/network/server/StreamData.java --- @@ -0,0 +1,96 @@ +/* + * Licensed to the

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-05-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r191941503 --- Diff: common/network-common/src/main/java/org/apache/spark/network/server/RpcHandler.java --- @@ -38,15 +38,24 @@ * * This method

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-05-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r191940304 --- Diff: common/network-common/src/main/java/org/apache/spark/network/protocol/UploadStream.java --- @@ -0,0 +1,107 @@ +/* + * Licensed to the

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r191940083 --- Diff: docs/running-on-kubernetes.md --- @@ -121,8 +121,8 @@ This URI is the location of the example jar that is already in the Docker image.

[GitHub] spark issue #21464: [WEBUI] Avoid possibility of script in query param keys

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21464 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21464: [WEBUI] Avoid possibility of script in query param keys

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21464 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3711/

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-05-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r191939431 --- Diff: common/network-common/src/main/java/org/apache/spark/network/protocol/UploadStream.java --- @@ -0,0 +1,107 @@ +/* + * Licensed to the

[GitHub] spark issue #21464: [WEBUI] Avoid possibility of script in query param keys

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21464 **[Test build #91314 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91314/testReport)** for PR 21464 at commit

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-05-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r191938203 --- Diff: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java --- @@ -141,26 +141,14 @@ public void fetchChunk(

[GitHub] spark issue #21452: [MINOR][CORE] Log committer class used by HadoopMapRedCo...

2018-05-30 Thread ejono
Github user ejono commented on the issue: https://github.com/apache/spark/pull/21452 Seems like the test build failed for an unrelated reason, right? This is my first contribution in a long time; have the tests been flaky lately? ---

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r191936265 --- Diff: docs/running-on-kubernetes.md --- @@ -121,8 +121,8 @@ This URI is the location of the example jar that is already in the Docker image.

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-05-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r191935821 --- Diff: common/network-common/src/main/java/org/apache/spark/network/client/StreamInterceptor.java --- @@ -50,16 +52,22 @@ @Override

[GitHub] spark issue #21346: [SPARK-6237][NETWORK] Network-layer changes to allow str...

2018-05-30 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/21346 I'm going to be starting a more detailed review pass on this now and will be getting caught back up with the discussion that's happened so far. One high-level point I'd like to keep in

[GitHub] spark issue #21464: [WEBUI] Avoid possibility of script in query param keys

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21464 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3710/

[GitHub] spark issue #21464: [WEBUI] Avoid possibility of script in query param keys

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21464 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19456: [SPARK] [Scheduler] Configurable default scheduling mode

2018-05-30 Thread varuvish
Github user varuvish commented on the issue: https://github.com/apache/spark/pull/19456 Can you guarantee that your users share an even workload? With this setup you could potentially have a user with a high workload using 1/#users resources. Wouldn't using the queue feature be

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r191930265 --- Diff: docs/running-on-kubernetes.md --- @@ -121,8 +121,8 @@ This URI is the location of the example jar that is already in the Docker image.

[GitHub] spark pull request #21464: [WEBUI] Avoid possibility of script in query para...

2018-05-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21464#discussion_r191933176 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -206,7 +206,9 @@ private[ui] class AllJobsPage(parent: JobsTab, store:

[GitHub] spark issue #21464: [WEBUI] Avoid possibility of script in query param keys

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21464 **[Test build #91313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91313/testReport)** for PR 21464 at commit

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-05-30 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21459 @rxin Mostly for Java 9+. [ASM 6.x](https://mvnrepository.com/artifact/org.ow2.asm/asm/6.0/usages) has been proven in many projects such as FB Presto, Google Guice Core Library, CGLIB,

[GitHub] spark pull request #21464: [WEBUI] Avoid possibility of script in query para...

2018-05-30 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/21464 [WEBUI] Avoid possibility of script in query param keys As discussed separately, this avoids the possibility of XSS on certain request param keys. CC @vanzin You can merge this pull

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21462 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3576/ ---

[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...

2018-05-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21450#discussion_r191929581 --- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java --- @@ -190,6 +194,23 @@ public void testSparkRShell()

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21462 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3576/ ---

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3709/

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21462 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...

2018-05-30 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21450#discussion_r191928359 --- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java --- @@ -190,6 +194,23 @@ public void testSparkRShell()

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21462 **[Test build #91312 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91312/testReport)** for PR 21462 at commit

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21462 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91312/ Test PASSed. ---

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21462 **[Test build #91312 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91312/testReport)** for PR 21462 at commit

[GitHub] spark issue #21400: [SPARK-24351][SS]offsetLog/commitLog purge thresholdBatc...

2018-05-30 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/21400 LGTM other than the one comment. Sorry it took me so long to get to this. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21400: [SPARK-24351][SS]offsetLog/commitLog purge thresh...

2018-05-30 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21400#discussion_r191278428 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -34,7 +34,8 @@ class ContinuousSuiteBase

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/21462 jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21400: [SPARK-24351][SS]offsetLog/commitLog purge thresh...

2018-05-30 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21400#discussion_r191278390 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -225,6 +226,40 @@ class ContinuousSuite

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21331 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21331 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91309/ Test PASSed. ---

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21331 **[Test build #91309 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91309/testReport)** for PR 21331 at commit

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 @foxish pls review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191919610 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala --- @@ -0,0 +1,120 @@

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91308/ Test PASSed. ---

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191918908 --- Diff: pom.xml --- @@ -760,6 +760,12 @@ 1.10.19 test + --- End diff -- I don't think this

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #91308 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91308/testReport)** for PR 19691 at commit

[GitHub] spark issue #21333: [SPARK-23778][CORE] Avoid unneeded shuffle when union ge...

2018-05-30 Thread varuvish
Github user varuvish commented on the issue: https://github.com/apache/spark/pull/21333 Nice change! I tested this out as well and verified that the shuffle doesn't happen. I did notice that this change wasn't reflected in the dataset API. Is that something that should be addressed

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191914872 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleEventHandler.scala --- @@ -0,0

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191914601 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventQueue.scala --- @@ -0,0 +1,29 @@

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191914114 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala --- @@ -0,0 +1,120 @@

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191913958 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -154,6 +154,24 @@ private[spark] object Config

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21068 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91307/ Test PASSed. ---

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21068 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21068 **[Test build #91307 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91307/testReport)** for PR 21068 at commit

[GitHub] spark pull request #21454: [SPARK-24337][Core] Improve error messages for Sp...

2018-05-30 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21454#discussion_r191883353 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -448,6 +473,20 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request #21454: [SPARK-24337][Core] Improve error messages for Sp...

2018-05-30 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21454#discussion_r191899959 --- Diff: core/src/test/scala/org/apache/spark/SparkConfSuite.scala --- @@ -339,6 +341,38 @@ class SparkConfSuite extends SparkFunSuite with

[GitHub] spark issue #21362: [SPARK-24197][SparkR][FOLLOWUP] Fixing failing tests for...

2018-05-30 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21362 @mn-mikke Actually, yes. But unfortunately I'm not familiar with R and the environment I heard about it, so I'm not able to figure out the reason for now. ---

[GitHub] spark issue #21362: [SPARK-24197][SparkR][FOLLOWUP] Fixing failing tests for...

2018-05-30 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21362 We haven’t seen it fail? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21362: [SPARK-24197][SparkR][FOLLOWUP] Fixing failing tests for...

2018-05-30 Thread mn-mikke
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21362 @ueshin Have you experienced the same problem with the failing tests? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21437: [SPARK-24397][PYSPARK] Added TaskContext.getLocal...

2018-05-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21437#discussion_r191881226 --- Diff: python/pyspark/taskcontext.py --- @@ -88,3 +89,9 @@ def taskAttemptId(self): TaskAttemptID. """ return

[GitHub] spark issue #21437: [SPARK-24397][PYSPARK] Added TaskContext.getLocalPropert...

2018-05-30 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/21437 @HyukjinKwon @BryanCutler @JoshRosen @ueshin is this good to go? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21437: [SPARK-24397][PYSPARK] Added TaskContext.getLocal...

2018-05-30 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21437#discussion_r191879730 --- Diff: python/pyspark/taskcontext.py --- @@ -88,3 +89,9 @@ def taskAttemptId(self): TaskAttemptID. """ return

[GitHub] spark pull request #21437: [SPARK-24397][PYSPARK] Added TaskContext.getLocal...

2018-05-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21437#discussion_r191877900 --- Diff: python/pyspark/taskcontext.py --- @@ -88,3 +89,9 @@ def taskAttemptId(self): TaskAttemptID. """ return

[GitHub] spark issue #21448: [SPARK-24408][SQL][DOC] Move abs, bitwiseNOT, isnan, nan...

2018-05-30 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21448 I'd only move abs and nothing else. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21362: [SPARK-24197][SparkR][FOLLOWUP] Fixing failing tests for...

2018-05-30 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21362 Hi, any updates on this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19959: [SPARK-22766] Install R linter package in spark lib dire...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19959 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91311/ Test FAILed. ---

[GitHub] spark issue #19959: [SPARK-22766] Install R linter package in spark lib dire...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19959 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19959: [SPARK-22766] Install R linter package in spark lib dire...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19959 **[Test build #91311 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91311/testReport)** for PR 19959 at commit

[GitHub] spark issue #19959: [SPARK-22766] Install R linter package in spark lib dire...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19959 **[Test build #91311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91311/testReport)** for PR 19959 at commit

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21246 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191865387 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -154,6 +154,24 @@ private[spark] object Config

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191868513 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventQueue.scala --- @@ -0,0 +1,29 @@ +/*

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191866216 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala --- @@ -0,0 +1,120 @@ +/*

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191869618 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleEventHandler.scala --- @@ -0,0

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191871593 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala --- @@ -0,0 +1,120 @@ +/*

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-30 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191865834 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala --- @@ -0,0 +1,120 @@ +/*

[GitHub] spark issue #21246: [SPARK-23901][SQL] Add masking functions

2018-05-30 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21246 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21413: [SPARK-23161][PYSPARK][ML]Add missing APIs to Python GBT...

2018-05-30 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/21413 Thanks a lot @BryanCutler for your help! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191870129 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +352,62 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191869090 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +352,62 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191854612 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +352,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191854114 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +352,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191854703 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +352,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191854585 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +352,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r191853613 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -291,37 +289,57 @@ class Dataset[T] private[sql]( } }

[GitHub] spark pull request #21413: [SPARK-23161][PYSPARK][ML]Add missing APIs to Pyt...

2018-05-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21413 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21413: [SPARK-23161][PYSPARK][ML]Add missing APIs to Python GBT...

2018-05-30 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21413 merged to master, thanks @huaxingao ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21448: [SPARK-24408][SQL][DOC] Move abs, bitwiseNOT, isnan, nan...

2018-05-30 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21448 If it’s ok I’ll take a quick look at this on Friday as discussed with Jacek. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3575/ ---

<    1   2   3   4   5   >