[GitHub] spark issue #13371: [SPARK-15639][SQL] Try to push down filter at RowGroups ...

2016-06-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13371 @yhuai Your step 3 may not work. We are going to filter the row groups for each parquet file to read in `VectorizedParquetRecordReader`. I think we don't do anything regarding creating splits? ---

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-09 Thread frreiss
Github user frreiss commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66564793 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1695,176 @@ object

[GitHub] spark pull request #13585: [SPARK-15859][SQL] Optimize the partition pruning...

2016-06-09 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13585#discussion_r66564745 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala --- @@ -65,4 +69,95 @@ class QueryPartitionSuite extends

[GitHub] spark pull request #13413: [SPARK-15663][SQL] SparkSession.catalog.listFunct...

2016-06-09 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/13413#discussion_r66564698 --- Diff: python/pyspark/sql/tests.py --- @@ -1481,17 +1481,7 @@ def test_list_functions(self): spark.sql("CREATE DATABASE some_db")

[GitHub] spark pull request #13436: [SPARK-15696][SQL] Improve `crosstab` to have a c...

2016-06-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13436 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #13415: [SPARK-15676] [SQL] Disallow Column Names as Partition C...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13415 **[Test build #60275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60275/consoleFull)** for PR 13415 at commit

[GitHub] spark issue #13436: [SPARK-15696][SQL] Improve `crosstab` to have a consiste...

2016-06-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13436 LGTM - merging in master/2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13415: [SPARK-15676] [SQL] Disallow Column Names as Partition C...

2016-06-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13415 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13371: [SPARK-15639][SQL] Try to push down filter at RowGroups ...

2016-06-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13371 @yhuai Parquet also does this filtering at ParquetRecordReader

[GitHub] spark pull request #13585: [SPARK-15859][SQL] Optimize the partition pruning...

2016-06-09 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13585#discussion_r66563744 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -92,6 +92,36 @@ object PhysicalOperation extends

[GitHub] spark pull request #13576: [SPARK-15840][SQL] Add two missing options in doc...

2016-06-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13576#discussion_r66563599 --- Diff: python/pyspark/sql/readwriter.py --- @@ -327,6 +334,8 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark pull request #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13569 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #13576: [SPARK-15840][SQL] Add two missing options in doc...

2016-06-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13576#discussion_r66563370 --- Diff: python/pyspark/sql/readwriter.py --- @@ -207,9 +207,10 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

[GitHub] spark issue #13496: [SPARK-15753][SQL] Move Analyzer stuff to Analyzer from ...

2016-06-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13496 try to ping @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #13413: [SPARK-15663][SQL] SparkSession.catalog.listFunct...

2016-06-09 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13413#discussion_r66563299 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -58,15 +60,39 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13569 Merging in master/2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13569#discussion_r66563276 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -54,6 +54,10 @@ class SubquerySuite extends QueryTest with SharedSQLContext

[GitHub] spark pull request #13413: [SPARK-15663][SQL] SparkSession.catalog.listFunct...

2016-06-09 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13413#discussion_r66563267 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -855,7 +855,8 @@ class SessionCatalog(

[GitHub] spark pull request #13413: [SPARK-15663][SQL] SparkSession.catalog.listFunct...

2016-06-09 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13413#discussion_r66563161 --- Diff: python/pyspark/sql/tests.py --- @@ -1481,17 +1481,7 @@ def test_list_functions(self): spark.sql("CREATE DATABASE some_db")

[GitHub] spark issue #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to allow t...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13342 **[Test build #60274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60274/consoleFull)** for PR 13342 at commit

[GitHub] spark pull request #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to ...

2016-06-09 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/13342#discussion_r66562319 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -232,7 +234,7 @@ final class DataFrameWriter private[sql](df: DataFrame)

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13571 **[Test build #60273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60273/consoleFull)** for PR 13571 at commit

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-09 Thread frreiss
Github user frreiss commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66561815 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1695,176 @@ object

[GitHub] spark issue #13589: [SPARK-15822][SPARK-15825][SQL] Fix SMJ Segfault/Invalid...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13589 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13589: [SPARK-15822][SPARK-15825][SQL] Fix SMJ Segfault/Invalid...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13589 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60268/ Test PASSed. ---

[GitHub] spark issue #13589: [SPARK-15822][SPARK-15825][SQL] Fix SMJ Segfault/Invalid...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13589 **[Test build #60268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60268/consoleFull)** for PR 13589 at commit

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13571 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60269/ Test FAILed. ---

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13571 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13571 **[Test build #60269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60269/consoleFull)** for PR 13571 at commit

[GitHub] spark pull request #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13569#discussion_r66561018 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -54,6 +54,10 @@ class SubquerySuite extends QueryTest with

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-09 Thread frreiss
Github user frreiss commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66561017 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1695,176 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-09 Thread frreiss
Github user frreiss commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66560947 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1695,176 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-09 Thread frreiss
Github user frreiss commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66560868 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1695,176 @@ object

[GitHub] spark issue #13371: [SPARK-15639][SQL] Try to push down filter at RowGroups ...

2016-06-09 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/13371 @viirya I took a look at parquet's code. Seems parquet only evaluate row group level filters when generating splits

[GitHub] spark issue #13591: [Minor] Replace all occurrences of None: Option[X] with ...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13591 **[Test build #60272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60272/consoleFull)** for PR 13591 at commit

[GitHub] spark pull request #13591: [Minor] Replace all occurrences of None: Option[X...

2016-06-09 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/13591 [Minor] Replace all occurrences of None: Option[X] with Option.empty[X] ## What changes were proposed in this pull request? Replace all occurrences of None: Option[X] with Option.empty[X]

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #3073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3073/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13586: [SPARK-15860] Metrics for codegen size and perf

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13586 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13586: [SPARK-15860] Metrics for codegen size and perf

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13586 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60267/ Test FAILed. ---

[GitHub] spark issue #13586: [SPARK-15860] Metrics for codegen size and perf

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13586 **[Test build #60267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60267/consoleFull)** for PR 13586 at commit

[GitHub] spark issue #13572: [SPARK-15838] [SQL] Better Error Message When Having Dat...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13572 **[Test build #60271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60271/consoleFull)** for PR 13572 at commit

[GitHub] spark issue #13541: [SPARK-15803][PYSPARK] Support with statement syntax for...

2016-06-09 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/13541 It's weird that I can pass the python test in local, but fails on github jenkins. And the error message not clear to me. @davies Could you help take a look at it ? Thanks. Here's the output from

[GitHub] spark issue #13323: [SPARK-15555] [Mesos] Driver with --supervise option can...

2016-06-09 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13323 Thanks @devaraj-kavali, this LGTM. @andrewor14 can you take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #13541: [SPARK-15803][PYSPARK] Support with statement syntax for...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13541 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13541: [SPARK-15803][PYSPARK] Support with statement syntax for...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13541 **[Test build #60270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60270/consoleFull)** for PR 13541 at commit

[GitHub] spark issue #13541: [SPARK-15803][PYSPARK] Support with statement syntax for...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13541 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60270/ Test FAILed. ---

[GitHub] spark pull request #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to ...

2016-06-09 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/13342#discussion_r66558301 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -232,7 +234,7 @@ final class DataFrameWriter private[sql](df: DataFrame) {

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-09 Thread frreiss
Github user frreiss commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66558119 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1695,176 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-09 Thread frreiss
Github user frreiss commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66558125 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1695,176 @@ object

[GitHub] spark issue #13541: [SPARK-15803][PYSPARK] Support with statement syntax for...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13541 **[Test build #60270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60270/consoleFull)** for PR 13541 at commit

[GitHub] spark issue #13576: [SPARK-15840][SQL] Add two missing options in documentat...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13576 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60265/ Test PASSed. ---

[GitHub] spark issue #13576: [SPARK-15840][SQL] Add two missing options in documentat...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13576 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13576: [SPARK-15840][SQL] Add two missing options in documentat...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13576 **[Test build #60265 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60265/consoleFull)** for PR 13576 at commit

[GitHub] spark issue #13590: SPARK-15858: Fix calculating error by tree stack over fl...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13590 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #13590: SPARK-15858: Fix calculating error by tree stack ...

2016-06-09 Thread mhmoudr
GitHub user mhmoudr opened a pull request: https://github.com/apache/spark/pull/13590 SPARK-15858: Fix calculating error by tree stack over flow problem an… ## What changes were proposed in this pull request? Improving evaluateEachIteration function in mllib as it fails

[GitHub] spark issue #13572: [SPARK-15838] [SQL] CACHE TABLE AS SELECT should not rep...

2016-06-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13572 Yeah, in Spark 1.6, we also silently drop the temporary table if the names are the same. Let me remove the related changes and update the title and JIRA --- If your project is set up for it,

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13571 **[Test build #60269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60269/consoleFull)** for PR 13571 at commit

[GitHub] spark issue #13588: SPARK-15858: Fix calculating error by tree stack over fl...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13588 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13589: [SPARK-15822][SPARK-15825][SQL] Fix SMJ Segfault/Invalid...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13589 **[Test build #60268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60268/consoleFull)** for PR 13589 at commit

[GitHub] spark issue #13576: [SPARK-15840][SQL] Add two missing options in documentat...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13576 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13576: [SPARK-15840][SQL] Add two missing options in documentat...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13576 **[Test build #60264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60264/consoleFull)** for PR 13576 at commit

[GitHub] spark issue #13576: [SPARK-15840][SQL] Add two missing options in documentat...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13576 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60264/ Test PASSed. ---

[GitHub] spark pull request #13589: [SPARK-15822][SPARK-15825][SQL] Fix SMJ Segfault/...

2016-06-09 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13589 [SPARK-15822][SPARK-15825][SQL] Fix SMJ Segfault/Invalid results ## What changes were proposed in this pull request? I'll add desc later ## How was this patch tested? TBD

[GitHub] spark pull request #13588: SPARK-15858: Fix calculating error by tree stack ...

2016-06-09 Thread mhmoudr
GitHub user mhmoudr opened a pull request: https://github.com/apache/spark/pull/13588 SPARK-15858: Fix calculating error by tree stack over flow problem an… ## What changes were proposed in this pull request? Improving evaluateEachIteration function in mllib as it fails

[GitHub] spark issue #13587: [Documentation] fixed groupby aggregation example for py...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13587 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #13587: [Documentation] fixed groupby aggregation example...

2016-06-09 Thread mortada
GitHub user mortada opened a pull request: https://github.com/apache/spark/pull/13587 [Documentation] fixed groupby aggregation example for pyspark ## What changes were proposed in this pull request? fixing documentation for the groupby/agg example in python ## How

[GitHub] spark issue #13576: [SPARK-15840][SQL] Add two missing options in documentat...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13576 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60259/ Test PASSed. ---

[GitHub] spark issue #13576: [SPARK-15840][SQL] Add two missing options in documentat...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13576 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13586: [SPARK-15860] Metrics for codegen size and perf

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13586 **[Test build #60267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60267/consoleFull)** for PR 13586 at commit

[GitHub] spark issue #13576: [SPARK-15840][SQL] Add two missing options in documentat...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13576 **[Test build #60259 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60259/consoleFull)** for PR 13576 at commit

[GitHub] spark pull request #13586: [SPARK-15860] Metrics for codegen size and perf

2016-06-09 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/13586 [SPARK-15860] Metrics for codegen size and perf ## What changes were proposed in this pull request? Adds codahale metrics for the codegen source text size and how long it takes to compile.

[GitHub] spark issue #13585: [SPARK-15859][SQL] Optimize the partition pruning within...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13585 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13585: [SPARK-15859][SQL] Optimize the partition pruning within...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13585 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60263/ Test FAILed. ---

[GitHub] spark issue #13585: [SPARK-15859][SQL] Optimize the partition pruning within...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13585 **[Test build #60263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60263/consoleFull)** for PR 13585 at commit

[GitHub] spark issue #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to allow t...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13342 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60260/ Test PASSed. ---

[GitHub] spark issue #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to allow t...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13342 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to allow t...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13342 **[Test build #60260 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60260/consoleFull)** for PR 13342 at commit

[GitHub] spark issue #13558: [SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13558 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13558: [SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13558 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60266/ Test FAILed. ---

[GitHub] spark issue #13558: [SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13558 **[Test build #60266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60266/consoleFull)** for PR 13558 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #3073 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3073/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to allow t...

2016-06-09 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/13342 LGTM, except a few minor nits. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13580: Revert "[SPARK-14485][CORE] ignore task finished for exe...

2016-06-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13580 LGTM -- seems like the lower risk option here, especially for 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to ...

2016-06-09 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/13342#discussion_r66552040 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ForeachWriter.scala --- @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #13582: [SPARK-15850][SQL] Remove function grouping in Sp...

2016-06-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13582 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #13577: [Minor][Doc] Improve SQLContext Documentation and...

2016-06-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13577 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #13582: [SPARK-15850][SQL] Remove function grouping in SparkSess...

2016-06-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13582 LGTM - merging to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13558: [SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13558 **[Test build #60266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60266/consoleFull)** for PR 13558 at commit

[GitHub] spark pull request #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to ...

2016-06-09 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/13342#discussion_r66551851 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ForeachWriter.scala --- @@ -68,9 +71,9 @@ import org.apache.spark.sql.streaming.ContinuousQuery

[GitHub] spark issue #13558: [SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into...

2016-06-09 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/13558 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13371: [SPARK-15639][SQL] Try to push down filter at RowGroups ...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13371 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13371: [SPARK-15639][SQL] Try to push down filter at RowGroups ...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13371 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60256/ Test PASSed. ---

[GitHub] spark issue #13371: [SPARK-15639][SQL] Try to push down filter at RowGroups ...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13371 **[Test build #60256 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60256/consoleFull)** for PR 13371 at commit

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13584 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60261/ Test PASSed. ---

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13584 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13584 **[Test build #60261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60261/consoleFull)** for PR 13584 at commit

[GitHub] spark pull request #13583: [SPARK-15853][SQL]HDFSMetadataLog.get should clos...

2016-06-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13583 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to allow t...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13342 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to allow t...

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13342 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60254/ Test PASSed. ---

[GitHub] spark issue #13342: [SPARK-15593][SQL]Add DataFrameWriter.foreach to allow t...

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13342 **[Test build #60254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60254/consoleFull)** for PR 13342 at commit

  1   2   3   4   5   >