[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-05-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17077 Also cc @cloud-fan who is the original PR author who implemented bucketBy. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-05-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17077 @zero323 Could you also update the [SQL document](http://spark.apache.org/docs/latest/sql-programming-guide.html)?

[GitHub] spark pull request #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for...

2017-05-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17077#discussion_r115134650 --- Diff: python/pyspark/sql/readwriter.py --- @@ -563,6 +563,60 @@ def partitionBy(self, *cols): self._jwrite =

[GitHub] spark issue #17882: [WIP][SPARK-20079][try 2][yarn] Re registration of AM ha...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17882 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76533/ Test PASSed. ---

[GitHub] spark issue #17882: [WIP][SPARK-20079][try 2][yarn] Re registration of AM ha...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17882 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17882: [WIP][SPARK-20079][try 2][yarn] Re registration of AM ha...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17882 **[Test build #76533 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76533/testReport)** for PR 17882 at commit

[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17770 **[Test build #76541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76541/testReport)** for PR 17770 at commit

[GitHub] spark issue #17865: [SPARK-20456][Docs] Add examples for functions collectio...

2017-05-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17865 Could you please check the documents we did in Scala APIs? It sounds like we forgot to update the Python function descriptions when we did the change in the Scala APIs. --- If your project is

[GitHub] spark pull request #17865: [SPARK-20456][Docs] Add examples for functions co...

2017-05-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17865#discussion_r115134581 --- Diff: python/pyspark/sql/functions.py --- @@ -409,7 +432,7 @@ def isnan(col): @since(1.6) def isnull(col): -"""An expression

[GitHub] spark pull request #17831: [SPARK-18777][PYTHON][SQL] Return UDF from udf.re...

2017-05-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17831 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17831: [SPARK-18777][PYTHON][SQL] Return UDF from udf.register

2017-05-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17831 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17835: [SPARK-20557] [SQL] Support JDBC data type Time w...

2017-05-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17835 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17835: [SPARK-20557] [SQL] Support JDBC data type Time with Tim...

2017-05-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17835 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17887 **[Test build #76540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76540/testReport)** for PR 17887 at commit

[GitHub] spark pull request #17736: [SPARK-20399][SQL] Can't use same regex pattern b...

2017-05-06 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/17736 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...

2017-05-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17887 cc @dbtsai @cloud-fan @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17887: [SPARK-20399][SQL][WIP] Add a config to fallback ...

2017-05-06 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/17887 [SPARK-20399][SQL][WIP] Add a config to fallback string literal parsing consistent with old sql parser behavior ## What changes were proposed in this pull request? Follow the discussion in

[GitHub] spark issue #17886: [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can not ge...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17886 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17886: [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can not ge...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17886 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76539/ Test FAILed. ---

[GitHub] spark issue #17886: [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can not ge...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17886 **[Test build #76539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76539/testReport)** for PR 17886 at commit

[GitHub] spark issue #17886: [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can not ge...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17886 **[Test build #76539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76539/testReport)** for PR 17886 at commit

[GitHub] spark issue #17878: [SPARK-20543][SPARKR][FOLLOWUP] Don't skip tests on AppV...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17878 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17878: [SPARK-20543][SPARKR][FOLLOWUP] Don't skip tests on AppV...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17878 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76532/ Test PASSed. ---

[GitHub] spark issue #17878: [SPARK-20543][SPARKR][FOLLOWUP] Don't skip tests on AppV...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17878 **[Test build #76532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76532/testReport)** for PR 17878 at commit

[GitHub] spark issue #17881: [SPARK-20621][deploy]Delete deprecated config parameter ...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17881 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76531/ Test PASSed. ---

[GitHub] spark issue #17886: [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can not ge...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17886 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17886: [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can not ge...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17886 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76537/ Test FAILed. ---

[GitHub] spark issue #17881: [SPARK-20621][deploy]Delete deprecated config parameter ...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17881 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17886: [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can not ge...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17886 **[Test build #76537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76537/testReport)** for PR 17886 at commit

[GitHub] spark issue #17881: [SPARK-20621][deploy]Delete deprecated config parameter ...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17881 **[Test build #76531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76531/testReport)** for PR 17881 at commit

[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17770 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17770 **[Test build #76536 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76536/testReport)** for PR 17770 at commit

[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76536/ Test FAILed. ---

[GitHub] spark pull request #15466: [SPARK-13983][SQL] HiveThriftServer2 can not get ...

2017-05-06 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/15466 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17874: [SPARK-20612][SQL][WIP] Throw exception when there is un...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17874 **[Test build #76538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76538/testReport)** for PR 17874 at commit

[GitHub] spark issue #17880: [SPARK-20620][TEST]Add some unit tests into NullExpressi...

2017-05-06 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/17880 @gatorsmile thanks,l will do it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17886: [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can not ge...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17886 **[Test build #76537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76537/testReport)** for PR 17886 at commit

[GitHub] spark issue #17874: [SPARK-20612][SQL][WIP] Throw exception when there is un...

2017-05-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17874 @cloud-fan This rule could make the query work: Seq(1).toDF("c1").createOrReplaceTempView("onerow") sql( """ | select 1 |from (select 1

[GitHub] spark pull request #17886: [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can...

2017-05-06 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/17886 [SPARK-13983][SQL][WIP] Fix HiveThriftServer2 can not get "--hiveconf" and ''--hivevar" variables since 2.x ## What changes were proposed in this pull request? Fix HiveThriftServer2 can

[GitHub] spark issue #17884: [SparkR][Doc] fix typo in vignettes

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17884 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17884: [SparkR][Doc] fix typo in vignettes

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17884 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76534/ Test PASSed. ---

[GitHub] spark issue #17884: [SparkR][Doc] fix typo in vignettes

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17884 **[Test build #76534 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76534/testReport)** for PR 17884 at commit

[GitHub] spark pull request #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for...

2017-05-06 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17077#discussion_r115133626 --- Diff: python/pyspark/sql/readwriter.py --- @@ -563,6 +563,60 @@ def partitionBy(self, *cols): self._jwrite =

[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17770 **[Test build #76536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76536/testReport)** for PR 17770 at commit

[GitHub] spark issue #17885: [SPARK-20627][PYSPARK] Drop the hadoop distirbution name...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17885 **[Test build #76535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76535/testReport)** for PR 17885 at commit

[GitHub] spark issue #17885: [SPARK-20627][PYSPARK] Drop the hadoop distirbution name...

2017-05-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17885 I'll target this for master, branch-2.2, branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #17885: [SPARK-20627][PYSPARK] Drop the hadoop distirbuti...

2017-05-06 Thread holdenk
GitHub user holdenk opened a pull request: https://github.com/apache/spark/pull/17885 [SPARK-20627][PYSPARK] Drop the hadoop distirbution name from the Python version ## What changes were proposed in this pull request? Drop the hadoop distirbution name from the Python

[GitHub] spark issue #17884: [SparkR][Doc] fix typo in vignettes

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17884 **[Test build #76534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76534/testReport)** for PR 17884 at commit

[GitHub] spark issue #17884: [SparkR][Doc] fix typo in vignettes

2017-05-06 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/17884 @felixcheung I ran a quick QA on the vignettes and fixed some additional typos and styles. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #17697: [SPARK-20414][MLLIB] avoid creating only 16 reduc...

2017-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17697#discussion_r115133315 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/rdd/MLPairRDDFunctionsSuite.scala --- @@ -22,9 +22,13 @@ import

[GitHub] spark issue #17697: [SPARK-20414][MLLIB] avoid creating only 16 reducers whe...

2017-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17697 I left some comments here though, I think I am not confident enough for a sign-off. Please let me defer to @srowen and @tejasapatil --- If your project is set up for it, you can reply to this

[GitHub] spark pull request #17697: [SPARK-20414][MLLIB] avoid creating only 16 reduc...

2017-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17697#discussion_r115133197 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/rdd/MLPairRDDFunctionsSuite.scala --- @@ -22,9 +22,13 @@ import

[GitHub] spark pull request #17697: [SPARK-20414][MLLIB] avoid creating only 16 reduc...

2017-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17697#discussion_r115133107 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/rdd/MLPairRDDFunctions.scala --- @@ -49,6 +53,7 @@ class MLPairRDDFunctions[K: ClassTag, V:

[GitHub] spark pull request #17697: [SPARK-20414][MLLIB] avoid creating only 16 reduc...

2017-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17697#discussion_r115133103 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/rdd/MLPairRDDFunctions.scala --- @@ -40,7 +40,11 @@ class MLPairRDDFunctions[K: ClassTag, V:

[GitHub] spark pull request #17697: [SPARK-20414][MLLIB] avoid creating only 16 reduc...

2017-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17697#discussion_r115133174 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/rdd/MLPairRDDFunctions.scala --- @@ -40,7 +40,11 @@ class MLPairRDDFunctions[K: ClassTag, V:

[GitHub] spark issue #17882: [WIP][SPARK-20079][try 2][yarn] Re registration of AM ha...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17882 **[Test build #76533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76533/testReport)** for PR 17882 at commit

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS as optimizer for LinearSV...

2017-05-06 Thread debasish83
Github user debasish83 commented on the issue: https://github.com/apache/spark/pull/17862 @hhbyyh can we smooth the hinge-loss using soft-max (variant of ReLU) and then use LBFGS ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #17697: [SPARK-20414][MLLIB] avoid creating only 16 reducers whe...

2017-05-06 Thread yangyangyyy
Github user yangyangyyy commented on the issue: https://github.com/apache/spark/pull/17697 @HyukjinKwon @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17831: [SPARK-18777][PYTHON][SQL] Return UDF from udf.register

2017-05-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17831 This change LGTM. I go to check #17848. It seems to me that the PR simply adds two flags into ScalaUDF. It appears that there is not API change regarding with existing UDF registration. I

[GitHub] spark pull request #17801: [MINOR][SQL][DOCS] Improve unix_timestamp's scala...

2017-05-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17801#discussion_r115132676 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2657,22 +2661,27 @@ object functions { /** *

[GitHub] spark issue #17878: [SPARK-20543][SPARKR][FOLLOWUP] Don't skip tests on AppV...

2017-05-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17878 Thanks, @HyukjinKwon AppVeyor looks good, waiting for Jenkins again (although, it has nothing to do with it..) --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #17878: [SPARK-20543][SPARKR][FOLLOWUP] Don't skip tests on AppV...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17878 **[Test build #76532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76532/testReport)** for PR 17878 at commit

[GitHub] spark issue #17881: [SPARK-20621][deploy]Delete deprecated config parameter ...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17881 **[Test build #76531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76531/testReport)** for PR 17881 at commit

[GitHub] spark issue #17878: [SPARK-20543][SPARKR][FOLLOWUP] Don't skip tests on AppV...

2017-05-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17878 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17881: [SPARK-20621][deploy]Delete deprecated config parameter ...

2017-05-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17881 Jenkins, ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17884: [SparkR][Doc] fix typo in vignettes

2017-05-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17884 @actuaryzhang thanks - would you have a chance to run a quick QA check on the rest of the vignettes, if you haven't already? --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #17884: [SparkR][Doc] fix typo in vignettes

2017-05-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17884 This test seems flaky on AppVeyor, not sure why ``` Failed - 1. Error: spark.glm and predict

[GitHub] spark issue #17854: [SPARK-20564][Deploy] Reduce massive executor failures w...

2017-05-06 Thread mariahualiu
Github user mariahualiu commented on the issue: https://github.com/apache/spark/pull/17854 Now I can comfortably use 2500 executors. But when I pushed the executor count to 3000, I saw a lot of heartbeat timeout errors. It is something else we can improve, probably another jira.

[GitHub] spark issue #17854: [SPARK-20564][Deploy] Reduce massive executor failures w...

2017-05-06 Thread mariahualiu
Github user mariahualiu commented on the issue: https://github.com/apache/spark/pull/17854 I re-ran the same application adding these configurations "--conf spark.yarn.scheduler.heartbeat.interval-ms=15000 --conf spark.yarn.launchContainer.count.simultaneously=50". Though it took 50

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17298 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17298 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76530/ Test FAILed. ---

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17298 **[Test build #76530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76530/testReport)** for PR 17298 at commit

[GitHub] spark issue #17854: [SPARK-20564][Deploy] Reduce massive executor failures w...

2017-05-06 Thread mariahualiu
Github user mariahualiu commented on the issue: https://github.com/apache/spark/pull/17854 Let me describe what I've seen when using 2500 executors. 1. In the first a few (2~3) requests, AM received all (in this case 2500) containers from Yarn. 2. In a few seconds, 2500

[GitHub] spark issue #17644: [SPARK-17729] [SQL] Enable creating hive bucketed tables

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17644 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76529/ Test PASSed. ---

[GitHub] spark issue #17644: [SPARK-17729] [SQL] Enable creating hive bucketed tables

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17644 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17644: [SPARK-17729] [SQL] Enable creating hive bucketed tables

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17644 **[Test build #76529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76529/testReport)** for PR 17644 at commit

[GitHub] spark issue #17854: [SPARK-20564][Deploy] Reduce massive executor failures w...

2017-05-06 Thread mariahualiu
Github user mariahualiu commented on the issue: https://github.com/apache/spark/pull/17854 @squito yes, I capped the number of resources in updateResourceRequests so that YarnAllocator asks for less number of resources in each iteration. When allocation fails one iteration, the

[GitHub] spark issue #17884: [SparkR][Doc] fix typo in vignettes

2017-05-06 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/17884 @HyukjinKwon Thanks for pointing this out. I will keep this in mind next time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #17801: [MINOR][SQL][DOCS] Improve unix_timestamp's scaladoc (an...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17801 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76528/ Test PASSed. ---

[GitHub] spark issue #17801: [MINOR][SQL][DOCS] Improve unix_timestamp's scaladoc (an...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17801 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17801: [MINOR][SQL][DOCS] Improve unix_timestamp's scaladoc (an...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17801 **[Test build #76528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76528/testReport)** for PR 17801 at commit

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17298 **[Test build #76530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76530/testReport)** for PR 17298 at commit

[GitHub] spark issue #17451: [SPARK-19866][ML][PySpark] Add local version of Word2Vec...

2017-05-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17451 Great, let me know if there is any questions @keypointt :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #17831: [SPARK-18777][PYTHON][SQL] Return UDF from udf.register

2017-05-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17831 @gatorsmile want to know if you're ok with this going into master or if you still have concerns about this if its targeted to 2.3? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for...

2017-05-06 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/17077#discussion_r115129876 --- Diff: python/pyspark/sql/readwriter.py --- @@ -563,6 +563,60 @@ def partitionBy(self, *cols): self._jwrite =

[GitHub] spark pull request #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for...

2017-05-06 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/17077#discussion_r115129682 --- Diff: python/pyspark/sql/readwriter.py --- @@ -563,6 +563,60 @@ def partitionBy(self, *cols): self._jwrite =

[GitHub] spark pull request #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for...

2017-05-06 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/17077#discussion_r115129884 --- Diff: python/pyspark/sql/readwriter.py --- @@ -563,6 +563,60 @@ def partitionBy(self, *cols): self._jwrite =

[GitHub] spark pull request #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Pa...

2017-05-06 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/17849#discussion_r115129786 --- Diff: python/pyspark/ml/tests.py --- @@ -1355,7 +1370,7 @@ def test_java_params(self): for name, cls in inspect.getmembers(module,

[GitHub] spark pull request #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Pa...

2017-05-06 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/17849#discussion_r115129846 --- Diff: python/pyspark/ml/wrapper.py --- @@ -263,7 +282,14 @@ def _fit_java(self, dataset): def _fit(self, dataset):

[GitHub] spark issue #17644: [SPARK-17729] [SQL] Enable creating hive bucketed tables

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17644 **[Test build #76529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76529/testReport)** for PR 17644 at commit

[GitHub] spark issue #17644: [SPARK-17729] [SQL] Enable creating hive bucketed tables

2017-05-06 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/17644 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16966: [SPARK-18409][ML]LSH approxNearestNeighbors should use a...

2017-05-06 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/16966 @MLnick @jkbradley @sethah Could you take a review? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #17092: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-05-06 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/17092 @MLnick @jkbradley @sethah Could you take a review? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #17801: [MINOR][SQL][DOCS] Improve unix_timestamp's scala...

2017-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17801#discussion_r115128695 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2647,7 +2647,11 @@ object functions { } /** - *

[GitHub] spark pull request #17801: [MINOR][SQL][DOCS] Improve unix_timestamp's scala...

2017-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17801#discussion_r115128766 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2657,22 +2661,27 @@ object functions { /** *

[GitHub] spark issue #17884: [SparkR][Doc] fix typo in vignettes

2017-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17884 I know it is legitimate but It would be worth double checking other typos too. Usually, single typo PR is not encouraged up to my knowledge given reviwing, building and merging costs. --- If

[GitHub] spark issue #17835: [SPARK-20557] [SQL] Support JDBC data type Time with Tim...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17835 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17835: [SPARK-20557] [SQL] Support JDBC data type Time with Tim...

2017-05-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17835 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76526/ Test PASSed. ---

[GitHub] spark issue #17835: [SPARK-20557] [SQL] Support JDBC data type Time with Tim...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17835 **[Test build #76526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76526/testReport)** for PR 17835 at commit

[GitHub] spark issue #17801: [MINOR][SQL][DOCS] Improve unix_timestamp's scaladoc (an...

2017-05-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17801 **[Test build #76528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76528/testReport)** for PR 17801 at commit

  1   2   3   >