[GitHub] spark pull request: [SPARK-13432][SQL] add the source file name an...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11301#issuecomment-216761109 **[Test build #57730 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57730/consoleFull)** for PR 11301 at commit [`6ecc00e`](https://gi

[GitHub] spark pull request: [SPARK-3767] [CORE] Support wildcard in Spark ...

2016-05-03 Thread devaraj-kavali
Github user devaraj-kavali commented on the pull request: https://github.com/apache/spark/pull/12753#issuecomment-216760992 @rxin, Please have a look into this and let me know any thing needs to be done here. About @, M/R also uses @ for the taskid wild card in java opts and there is

[GitHub] spark pull request: [SPARK-15113][PySpark][ML] Add missing num fea...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12889#issuecomment-216760706 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-15113][PySpark][ML] Add missing num fea...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12889#issuecomment-216760705 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-15113][PySpark][ML] Add missing num fea...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12889#issuecomment-216760656 **[Test build #57726 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57726/consoleFull)** for PR 12889 at commit [`c1961ae`](https://g

[GitHub] spark pull request: [SPARK-14489][SPARK-14153][ML][PYSPARK] Suppor...

2016-05-03 Thread MLnick
Github user MLnick closed the pull request at: https://github.com/apache/spark/pull/12577 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-3767] [CORE] Support wildcard in Spark ...

2016-05-03 Thread devaraj-kavali
Github user devaraj-kavali commented on a diff in the pull request: https://github.com/apache/spark/pull/12753#discussion_r61995355 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala --- @@ -166,14 +166,15 @@ private[spark] cla

[GitHub] spark pull request: [SPARK-14139][SQL] RowEncoder should preserve ...

2016-05-03 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12364#issuecomment-216760181 cc @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216760048 @jkbradley Thanks for your comments. I want to make sure that you said we will freeze the pyspark.mllib.linalg APIs, so do I need to change the current PySpark ml/mllib

[GitHub] spark pull request: [SPARK-14127][SQL] Native "DESC [EXTENDED | FO...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12844#issuecomment-216759826 **[Test build #57729 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57729/consoleFull)** for PR 12844 at commit [`b5dbc15`](https://gi

[GitHub] spark pull request: [SPARK-14772][ML,PySpark]Python ML Params.copy...

2016-05-03 Thread holdenk
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/12888#issuecomment-216759643 Thanks for tackling this issue :) For a better understanding - is there a reason why adding a flag for this behaviour instead of just changing it (since it is a bug) -

[GitHub] spark pull request: [SPARK-13001] [CORE] [MESOS] Prevent getting o...

2016-05-03 Thread dragos
Github user dragos commented on the pull request: https://github.com/apache/spark/pull/10924#issuecomment-216759444 > On 4 mai 2016, at 02:22, Sebastien Rainville wrote: > > @dragos I finally did the change. Sorry for the delay > Excellent, thanks! I won't be ab

[GitHub] spark pull request: [SPARK-14772][ML,PySpark]Python ML Params.copy...

2016-05-03 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12888#discussion_r61994856 --- Diff: python/pyspark/ml/param/__init__.py --- @@ -357,7 +357,7 @@ def getOrDefault(self, param): return self._defaultParamMap[param]

[GitHub] spark pull request: [SPARK-15113][PySpark][ML] Add missing num fea...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12889#issuecomment-216759233 **[Test build #57726 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57726/consoleFull)** for PR 12889 at commit [`c1961ae`](https://gi

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216759232 **[Test build #57728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57728/consoleFull)** for PR 12870 at commit [`7df46dc`](https://gi

[GitHub] spark pull request: [SPARK-15106][PYSPARK][ML] Add PySpark package...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12883#issuecomment-216759235 **[Test build #57727 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57727/consoleFull)** for PR 12883 at commit [`51c9ae9`](https://gi

[GitHub] spark pull request: [SPARK-15113][PySpark][ML] Add missing num fea...

2016-05-03 Thread holdenk
GitHub user holdenk opened a pull request: https://github.com/apache/spark/pull/12889 [SPARK-15113][PySpark][ML] Add missing num features num classes ## What changes were proposed in this pull request? Add missing `numFeatures` and `numClasses` to the wrapped Java models in

[GitHub] spark pull request: [SPARK-15108] [SQL] Describe Permanent UDTF

2016-05-03 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/12885#issuecomment-216758934 : ) Partially right. Even if the function is loaded, we might not be able to find it. The reason is we might not specify the database name in `Describe Function`.

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216758801 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216758796 **[Test build #57725 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57725/consoleFull)** for PR 12870 at commit [`731`](https://g

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216758799 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216758602 **[Test build #57725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57725/consoleFull)** for PR 12870 at commit [`731`](https://gi

[GitHub] spark pull request: [SPARK-15106][PYSPARK][ML] Add PySpark package...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12883#issuecomment-216758603 **[Test build #57724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57724/consoleFull)** for PR 12883 at commit [`ba4ce9d`](https://gi

[GitHub] spark pull request: [SPARK-14139][SQL] RowEncoder should preserve ...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12364#issuecomment-216758272 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14139][SQL] RowEncoder should preserve ...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12364#issuecomment-216758270 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14139][SQL] RowEncoder should preserve ...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12364#issuecomment-216758163 **[Test build #57716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57716/consoleFull)** for PR 12364 at commit [`8870650`](https://g

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216757717 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216757713 **[Test build #57723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57723/consoleFull)** for PR 12870 at commit [`b504a60`](https://g

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216757716 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14237][SQL] De-duplicate partition valu...

2016-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12866 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14906][ML][WIP] Move VectorUDT and Matr...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12870#issuecomment-216757529 **[Test build #57723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57723/consoleFull)** for PR 12870 at commit [`b504a60`](https://gi

[GitHub] spark pull request: [SPARK-15073][SQL] Hide SparkSession construct...

2016-05-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12873#discussion_r61993910 --- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/Main.scala --- @@ -94,10 +94,10 @@ object Main extends Logging { def createSpar

[GitHub] spark pull request: [SPARK-14237][SQL] De-duplicate partition valu...

2016-05-03 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/12866#issuecomment-216755741 Thanks for the review! Merged this to master and branch-2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-14772][ML,PySpark]Python ML Params.copy...

2016-05-03 Thread hujy
GitHub user hujy opened a pull request: https://github.com/apache/spark/pull/12888 [SPARK-14772][ML,PySpark]Python ML Params.copy treats uid, paramMaps … [SPARK-14772](https://issues.apache.org/jira/browse/SPARK-14772#) ##What changes were proposed in this pull request?

[GitHub] spark pull request: [SPARK-14772][ML,PySpark]Python ML Params.copy...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12888#issuecomment-216755365 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-15103][SQL] Refactored FileCatalog clas...

2016-05-03 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/12879#issuecomment-216753341 ping @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-14427][SQL] Support persisting partitio...

2016-05-03 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/12204#issuecomment-216753358 ping @cloud-fan @yhuai Any more comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11395][SPARKR] Support over and window ...

2016-05-03 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/10094#discussion_r61993129 --- Diff: R/pkg/R/window.R --- @@ -0,0 +1,88 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license a

[GitHub] spark pull request: [SPARK-15110][SparkR] Implement repartitionByC...

2016-05-03 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/12887#discussion_r61992959 --- Diff: R/pkg/R/DataFrame.R --- @@ -594,6 +594,35 @@ setMethod("repartition", dataFrame(sdf) }) +#' RepartitionBy

[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12810#issuecomment-216752641 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12810#issuecomment-216752640 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12810#issuecomment-216752537 **[Test build #57713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57713/consoleFull)** for PR 12810 at commit [`2e949c8`](https://g

[GitHub] spark pull request: [SPARK-14124] [SQL] [FOLLOWUP] Implement Datab...

2016-05-03 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/12081#issuecomment-216752333 @andrewor14 I also think about creating directory in implementation of create database. Metastore APIs created the directory for us no matter whether the pr

[GitHub] spark pull request: [SPARK-15107][SQL] Allow varying # iterations ...

2016-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12884 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-15107][SQL] Allow varying # iterations ...

2016-05-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12884#issuecomment-216751557 Thanks - merging in master / branch-2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-15107][SQL] Allow varying # iterations ...

2016-05-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12884#issuecomment-216751414 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-5929][PYSPARK] Context addPyPackage and...

2016-05-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12398#issuecomment-216751245 @buckhx These API seems useful, could you also add an argument for bin/spark-submit (only for requirement file) ? --- If your project is set up for it, you can reply t

[GitHub] spark pull request: [SPARK-15107][SQL] Allow varying # iterations ...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12884#issuecomment-216751198 **[Test build #57721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57721/consoleFull)** for PR 12884 at commit [`69c483e`](https://gi

[GitHub] spark pull request: [SPARK-14124] [SQL] [FOLLOWUP] Implement Datab...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12081#issuecomment-216751211 **[Test build #57722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57722/consoleFull)** for PR 12081 at commit [`b885f7b`](https://gi

[GitHub] spark pull request: [SPARK-5929][PYSPARK] Context addPyPackage and...

2016-05-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12398#discussion_r61992444 --- Diff: python/pyspark/context.py --- @@ -814,6 +817,40 @@ def addPyFile(self, path): import importlib importlib.invalidate

[GitHub] spark pull request: [SPARK-14127][SQL] Native "DESC [EXTENDED | FO...

2016-05-03 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/12844#discussion_r61992398 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -290,29 +291,88 @@ case class DescribeTableCommand(table: Tab

[GitHub] spark pull request: [SPARK-14127][SQL] Native "DESC [EXTENDED | FO...

2016-05-03 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/12844#discussion_r61992394 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -290,29 +291,88 @@ case class DescribeTableCommand(table: Tab

[GitHub] spark pull request: [SPARK-15107][SQL] Allow varying # iterations ...

2016-05-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12884#issuecomment-216750804 I reset the changes. They are taking too long to run. In the future, we should revisit this file and break it down into multiple files grouped by category (e.g. jo

[GitHub] spark pull request: [SPARK-12922][SparkR][WIP] Implement gapply() ...

2016-05-03 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/12836#issuecomment-216750713 Ok - if the behavior we get from `dapply(repartition(df, cols))` is the same as `groupByKey().flatMap` then I'm fine with going with the simpler implementation.

[GitHub] spark pull request: [SPARK-5929][PYSPARK] Context addPyPackage and...

2016-05-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12398#discussion_r61992299 --- Diff: python/pyspark/context.py --- @@ -814,6 +817,40 @@ def addPyFile(self, path): import importlib importlib.invalidate

[GitHub] spark pull request: [SPARK-5929][PYSPARK] Context addPyPackage and...

2016-05-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12398#discussion_r61992184 --- Diff: python/pyspark/tests.py --- @@ -1947,6 +1947,33 @@ def test_with_stop(self): sc.stop() self.assertEqual(SparkContext._a

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216750238 **[Test build #57720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57720/consoleFull)** for PR 12818 at commit [`3b289d9`](https://gi

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216750176 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216749947 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216749940 **[Test build #57718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57718/consoleFull)** for PR 12818 at commit [`3b289d9`](https://g

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216749945 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14124] [SQL] [FOLLOWUP] Implement Datab...

2016-05-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12081#discussion_r61991923 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -148,9 +148,17 @@ class SessionCatalog(

[GitHub] spark pull request: [SPARK-14237][SQL] De-duplicate partition valu...

2016-05-03 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12866#issuecomment-216749798 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark pull request: [SPARK-14993] [SQL] Fix Partition Discovery In...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12828#issuecomment-216749648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14993] [SQL] Fix Partition Discovery In...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12828#issuecomment-216749645 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14993] [SQL] Fix Partition Discovery In...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12828#issuecomment-216749556 **[Test build #57711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57711/consoleFull)** for PR 12828 at commit [`252065c`](https://g

[GitHub] spark pull request: [SPARK-5929][PYSPARK] Context addPyPackage and...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12398#issuecomment-216749580 **[Test build #2967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2967/consoleFull)** for PR 12398 at commit [`ce9966e`](https://

[GitHub] spark pull request: [SPARK-5929][PYSPARK] Context addPyPackage and...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12398#issuecomment-216749512 **[Test build #2967 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2967/consoleFull)** for PR 12398 at commit [`ce9966e`](https://g

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216749347 **[Test build #57718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57718/consoleFull)** for PR 12818 at commit [`3b289d9`](https://gi

[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12810#issuecomment-216749350 **[Test build #57719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57719/consoleFull)** for PR 12810 at commit [`041b7fe`](https://gi

[GitHub] spark pull request: [SPARK-12922][SparkR][WIP] Implement gapply() ...

2016-05-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12836#issuecomment-216749219 The number of partition could be specified in repartitioning() (or 200 as default). `KeyValueGroupedDataset.flatMapGroups()` accept a function as `(K, Iterator[

[GitHub] spark pull request: [SPARK-15110][SparkR] Implement repartitionByC...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12887#issuecomment-216749047 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [Spark-14933][SQL] Failed to create view out o...

2016-05-03 Thread xwu0226
Github user xwu0226 commented on the pull request: https://github.com/apache/spark/pull/12716#issuecomment-216749038 @yhuai @liancheng @liancheng @hvanhovell Can any of you help taking a quick look at this change? Thank you very much! --- If your project is set up for it, you can rep

[GitHub] spark pull request: [SPARK-15110][SparkR] Implement repartitionByC...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12887#issuecomment-216749048 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-15110][SparkR] Implement repartitionByC...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12887#issuecomment-216749012 **[Test build #57715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57715/consoleFull)** for PR 12887 at commit [`3ee277a`](https://g

[GitHub] spark pull request: [SPARK-15025][SQL] fix duplicate of PATH key i...

2016-05-03 Thread xwu0226
Github user xwu0226 commented on the pull request: https://github.com/apache/spark/pull/12804#issuecomment-216748933 @yhuai @liancheng @liancheng @hvanhovell Can any of you help take a quick look at this change? Thank you very much! --- If your project is set up for it, you can reply

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216748780 Oh, I misunderstood your first comment.I think I should not take out `setLineSeparator()` here but maybe I should open another issue ticket to set `normalizeLineEnd

[GitHub] spark pull request: [SPARK-14124] [SQL] [FOLLOWUP] Implement Datab...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12081#issuecomment-216748843 **[Test build #57717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57717/consoleFull)** for PR 12081 at commit [`59a9805`](https://gi

[GitHub] spark pull request: [SPARK-6339][SQL] Supports create CREATE TEMPO...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12872#issuecomment-216748484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-15107][SQL] Allow varying # iterations ...

2016-05-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12884#issuecomment-216748498 We changed the number of rows in same benchmark, but did not update the results, should we also update the result? --- If your project is set up for it, you can reply t

[GitHub] spark pull request: [SPARK-6339][SQL] Supports create CREATE TEMPO...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12872#issuecomment-216748483 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-6339][SQL] Supports create CREATE TEMPO...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12872#issuecomment-216748405 **[Test build #57710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57710/consoleFull)** for PR 12872 at commit [`1e20bb0`](https://g

[GitHub] spark pull request: [SPARK-15107][SQL] Allow varying # iterations ...

2016-05-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12884#discussion_r61991224 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/BenchmarkWholeStageCodegen.scala --- @@ -346,7 +373,7 @@ class BenchmarkWholeStageCodegen ex

[GitHub] spark pull request: [SPARK-14124] [SQL] [FOLLOWUP] Implement Datab...

2016-05-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12081#discussion_r61991201 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -348,4 +364,99 @@ class HiveDDLSuite extends QueryTest wi

[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...

2016-05-03 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12810#discussion_r61991132 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -41,19 +41,17 @@ import org.apache.spark.sql.types._

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread jbax
Github user jbax commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216747285 Foo and bar are part of the same value, they just happen to have a line ending in between. And yes `setLineSeparator()` it is related to the values themselves when writing

[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...

2016-05-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12810#discussion_r61991082 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -41,19 +41,17 @@ import org.apache.spark.sql.types._

[GitHub] spark pull request: [SPARK-15110][SparkR] Implement repartitionByC...

2016-05-03 Thread NarineK
Github user NarineK commented on a diff in the pull request: https://github.com/apache/spark/pull/12887#discussion_r61991037 --- Diff: R/pkg/R/DataFrame.R --- @@ -594,6 +594,35 @@ setMethod("repartition", dataFrame(sdf) }) +#' RepartitionByC

[GitHub] spark pull request: [SPARK-14684] [SPARK-15026] [SQL] Disallow Dro...

2016-05-03 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/12801#issuecomment-216745696 But, this PR does not allow it for ensuring the atomicity of the command. https://github.com/gatorsmile/spark/blob/banDropMultiPart/sql/core/src/main/scala/org/ap

[GitHub] spark pull request: [SPARK-13902][SCHEDULER] Make DAGScheduler.get...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12655#issuecomment-216745078 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-15110][SparkR] Implement repartitionByC...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12887#issuecomment-216745101 **[Test build #57715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57715/consoleFull)** for PR 12887 at commit [`3ee277a`](https://gi

[GitHub] spark pull request: [SPARK-13902][SCHEDULER] Make DAGScheduler.get...

2016-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12655#issuecomment-216745080 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14139][SQL] RowEncoder should preserve ...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12364#issuecomment-216745065 **[Test build #57716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57716/consoleFull)** for PR 12364 at commit [`8870650`](https://gi

[GitHub] spark pull request: [SPARK-14684] [SPARK-15026] [SQL] Disallow Dro...

2016-05-03 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/12801#issuecomment-216744413 ```scala case class AlterTableDropPartition( tableName: TableIdentifier, specs: Seq[TablePartitionSpec], ifExists: Boolean) ``` The

[GitHub] spark pull request: [SPARK-13902][SCHEDULER] Make DAGScheduler.get...

2016-05-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12655#issuecomment-216744662 **[Test build #57709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57709/consoleFull)** for PR 12655 at commit [`ab92488`](https://g

[GitHub] spark pull request: [SPARK-15110][SparkR] Implement repartitionByC...

2016-05-03 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/12887#discussion_r61990844 --- Diff: R/pkg/R/DataFrame.R --- @@ -594,6 +594,35 @@ setMethod("repartition", dataFrame(sdf) }) +#' Repartitio

[GitHub] spark pull request: [SPARK-14684] [SPARK-15026] [SQL] Disallow Dro...

2016-05-03 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12801#issuecomment-216743828 what if the partition spec of the table is a, b, and we do `ALTER TABLE tbl DROP PARTITION (a=1, b=1) PARTITION (a=1,b=2)`? --- If your project is set up for it, you

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216743678 @jbax Ah, I guess `foo` and `bar` are separate rows, right? `stripLineEnd` will be applied for each row. If I got you wrong and `setLineSeparator()` is rela

[GitHub] spark pull request: [SPARK-15095] [SQL] remove HiveSessionHook fro...

2016-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12881 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [MINOR][SQL] Remove not affected settings for ...

2016-05-03 Thread jbax
Github user jbax commented on the pull request: https://github.com/apache/spark/pull/12818#issuecomment-216743260 What happens if you do this: ``` scala> "foo\r\nbar\r\n".stripLineEnd ``` Shouldn't the result be this? ``` res0: String = foo\r\n bar ```

  1   2   3   4   5   6   7   8   9   10   >