[GitHub] spark issue #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/13989 What do you mean by both positive and negative cases? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69075298 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -166,8 +166,8 @@ private[sql] class SessionState(sparkSession:

[GitHub] spark issue #13603: [SPARK-15865][CORE] Blacklist should not result in job h...

2016-06-29 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/13603 LGTM! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #13971: [SPARK-16289][SQL] Implement posexplode table gen...

2016-06-29 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13971#discussion_r69075261 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/GeneratorExpressionSuite.scala --- @@ -0,0 +1,71 @@ +/* + *

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69075247 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -85,5 +85,10 @@ case class LogicalRelation(

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69075198 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -265,6 +265,11 @@ abstract class LogicalPlan

[GitHub] spark pull request #13972: [SPARK-16294][SQL] Labelling support for the incl...

2016-06-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13972 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69074558 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -265,6 +265,11 @@ abstract class LogicalPlan

[GitHub] spark issue #13972: [SPARK-16294][SQL] Labelling support for the include_exa...

2016-06-29 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13972 @yinxusen Do you have time to consolidate example files for `mllib-data-types.md`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #13972: [SPARK-16294][SQL] Labelling support for the include_exa...

2016-06-29 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13972 LGTM2. Merged into master and branch-2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69074411 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -85,5 +85,10 @@ case class LogicalRelation(

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69074328 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -265,6 +265,11 @@ abstract class LogicalPlan

[GitHub] spark pull request #13971: [SPARK-16289][SQL] Implement posexplode table gen...

2016-06-29 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13971#discussion_r69074335 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/GeneratorExpressionSuite.scala --- @@ -0,0 +1,71 @@ +/* + *

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69074265 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -265,6 +265,11 @@ abstract class LogicalPlan

[GitHub] spark issue #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13989 Test cases are not enough to cover the metadata refreshing. The current metadata cache is only used for data source tables. We still could convert Hive tables to data source tables. For example,

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69074253 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2307,6 +2307,19 @@ class Dataset[T] private[sql]( def distinct():

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69074131 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -85,5 +85,10 @@ case class LogicalRelation(

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69074039 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -265,6 +265,11 @@ abstract class LogicalPlan

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69073906 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -166,8 +166,8 @@ private[sql] class SessionState(sparkSession:

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69073454 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -85,5 +85,10 @@ case class LogicalRelation(

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69073383 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -265,6 +265,11 @@ abstract class LogicalPlan

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69073191 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -139,18 +139,6 @@ private[hive] class

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69072136 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2307,6 +2307,19 @@ class Dataset[T] private[sql]( def distinct():

[GitHub] spark issue #13988: [WIP][SPARK-16101][SQL] Refactoring CSV data source to b...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13988 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13988: [WIP][SPARK-16101][SQL] Refactoring CSV data source to b...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13988 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61523/ Test PASSed. ---

[GitHub] spark issue #13988: [WIP][SPARK-16101][SQL] Refactoring CSV data source to b...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13988 **[Test build #61523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61523/consoleFull)** for PR 13988 at commit

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69071788 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -85,5 +85,10 @@ case class LogicalRelation(

[GitHub] spark issue #13978: [SPARK-16256][DOCS] Minor fixes on the Structured Stream...

2016-06-29 Thread ScrapCodes
Github user ScrapCodes commented on the issue: https://github.com/apache/spark/pull/13978 Looks good ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69071622 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2307,6 +2307,19 @@ class Dataset[T] private[sql]( def distinct():

[GitHub] spark issue #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13969 **[Test build #3152 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3152/consoleFull)** for PR 13969 at commit

[GitHub] spark issue #13966: [SPARK-16276][SQL] Implement elt SQL function

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13966 **[Test build #3153 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3153/consoleFull)** for PR 13966 at commit

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13989#discussion_r69071525 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2307,6 +2307,19 @@ class Dataset[T] private[sql]( def distinct():

[GitHub] spark pull request #13966: [SPARK-16276][SQL] Implement elt SQL function

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13966#discussion_r69070865 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -162,6 +163,46 @@ case class

[GitHub] spark issue #13987: [SPARK-16313][SQL] Spark should not silently drop except...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13987 **[Test build #61528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61528/consoleFull)** for PR 13987 at commit

[GitHub] spark pull request #13966: [SPARK-16276][SQL] Implement elt SQL function

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13966#discussion_r69070679 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -162,6 +163,46 @@ case class

[GitHub] spark issue #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13989 Before, I tried to merge `invalidateTable` and `refreshTable`. @yhuai left the following comment: https://github.com/apache/spark/pull/13156#discussion_r63729506 I think maybe we

[GitHub] spark issue #13982: [SPARK-16304] LinkageError should not crash Spark execut...

2016-06-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13982 cc @JoshRosen and @ericl --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13767: [MINOR][SQL] Not dropping all necessary tables

2016-06-29 Thread techaddict
Github user techaddict commented on the issue: https://github.com/apache/spark/pull/13767 cc: @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #13990: [SPARK-16287][SQL][WIP] Implement str_to_map SQL functio...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13990 **[Test build #61525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61525/consoleFull)** for PR 13990 at commit

[GitHub] spark issue #13987: [SPARK-16313][SQL] Spark should not silently drop except...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13987 **[Test build #61526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61526/consoleFull)** for PR 13987 at commit

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13926 **[Test build #61527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61527/consoleFull)** for PR 13926 at commit

[GitHub] spark pull request #13990: [SPARK-16287][SQL][WIP] Implement str_to_map SQL ...

2016-06-29 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/13990 [SPARK-16287][SQL][WIP] Implement str_to_map SQL function ## What changes were proposed in this pull request? This PR adds `str_to_map` SQL function in order to remove Hive fallback.

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13926 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13989 **[Test build #61524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61524/consoleFull)** for PR 13989 at commit

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13926 ping @hvanhovell Could you please take a look at this again? : ) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #13886: [SPARK-16185] [SQL] Better Error Messages When Creating ...

2016-06-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13886 Could you please review this PR again? @cloud-fan Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13989 cc @cloud-fan / @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/13989 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #13989: [SPARK-16311][SQL] Improve metadata refresh

2016-06-29 Thread petermaxlee
GitHub user petermaxlee opened a pull request: https://github.com/apache/spark/pull/13989 [SPARK-16311][SQL] Improve metadata refresh ## What changes were proposed in this pull request? This patch implements the 3 things specified in SPARK-16311: (1) Append a message to

[GitHub] spark issue #13979: [SPARK-SPARK-16302] [SQL] Set the right number of partit...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13979 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61520/ Test PASSed. ---

[GitHub] spark issue #13979: [SPARK-SPARK-16302] [SQL] Set the right number of partit...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13979 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13979: [SPARK-SPARK-16302] [SQL] Set the right number of partit...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13979 **[Test build #61520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61520/consoleFull)** for PR 13979 at commit

[GitHub] spark pull request #13987: [SPARK-16313][SQL] Spark should not silently drop...

2016-06-29 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/13987#discussion_r69067474 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala --- @@ -58,10 +56,16 @@ class ListingFileCatalog(

[GitHub] spark issue #13987: [SPARK-16313][SQL] Spark should not silently drop except...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13987 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61521/ Test FAILed. ---

[GitHub] spark issue #13987: [SPARK-16313][SQL] Spark should not silently drop except...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13987 **[Test build #61521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61521/consoleFull)** for PR 13987 at commit

[GitHub] spark issue #13987: [SPARK-16313][SQL] Spark should not silently drop except...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13987 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimizer

2016-06-29 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13906 @cloud-fan Yea, that's a good point. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13988: [WIP][SPARK-16101][SQL] Refactoring CSV data source to b...

2016-06-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/13988 I still need to correct some nits and check the consistency with JSON data source but I opened this just to check if it breaks anything. I will submit some more commits soon. --- If your

[GitHub] spark issue #13988: [WIP][SPARK-16101][SQL] Refactoring CSV data source to b...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13988 **[Test build #61523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61523/consoleFull)** for PR 13988 at commit

[GitHub] spark pull request #13988: [WIP][SPARK-16101][SQL] Refactoring CSV data sour...

2016-06-29 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/13988 [WIP][SPARK-16101][SQL] Refactoring CSV data source to be consistent with JSON data source ## What changes were proposed in this pull request? This PR refactors CSV data source to be

[GitHub] spark issue #13829: [SPARK-16071][SQL] Checks size limit when doubling the a...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13829 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13829: [SPARK-16071][SQL] Checks size limit when doubling the a...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13829 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61517/ Test PASSed. ---

[GitHub] spark issue #13829: [SPARK-16071][SQL] Checks size limit when doubling the a...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13829 **[Test build #61517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61517/consoleFull)** for PR 13829 at commit

[GitHub] spark issue #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimizer

2016-06-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13906 @liancheng , I think we still need to keep some simple rules for unary node, which also helps the binary cases, as the empty relation is propagated up. --- If your project is set up for it, you

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r69065541 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CollapseEmptyPlan.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r69065425 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CollapseEmptyPlan.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to

[GitHub] spark issue #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimizer

2016-06-29 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13906 My feeling is that, this optimization rule is mostly useful for binary plan nodes like inner join and intersection, where we can avoid scanning output of the non-empty side. On the other

[GitHub] spark issue #13829: [SPARK-16071][SQL] Checks size limit when doubling the a...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13829 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13829: [SPARK-16071][SQL] Checks size limit when doubling the a...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13829 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61515/ Test PASSed. ---

[GitHub] spark issue #13829: [SPARK-16071][SQL] Checks size limit when doubling the a...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13829 **[Test build #61515 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61515/consoleFull)** for PR 13829 at commit

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r69065025 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CollapseEmptyPlan.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r69064885 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CollapseEmptyPlan.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to

[GitHub] spark issue #13978: [SPARK-16256][DOCS] Minor fixes on the Structured Stream...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13978 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61522/ Test PASSed. ---

[GitHub] spark issue #13978: [SPARK-16256][DOCS] Minor fixes on the Structured Stream...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13978 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13978: [SPARK-16256][DOCS] Minor fixes on the Structured Stream...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13978 **[Test build #61522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61522/consoleFull)** for PR 13978 at commit

[GitHub] spark issue #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStreams to ...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11863 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61513/ Test PASSed. ---

[GitHub] spark issue #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStreams to ...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11863 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStreams to ...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11863 **[Test build #61513 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61513/consoleFull)** for PR 11863 at commit

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r69064054 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CollapseEmptyPlan.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to

[GitHub] spark issue #13829: [SPARK-16071][SQL] Checks size limit when doubling the a...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13829 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13829: [SPARK-16071][SQL] Checks size limit when doubling the a...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13829 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61514/ Test FAILed. ---

[GitHub] spark issue #13829: [SPARK-16071][SQL] Checks size limit when doubling the a...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13829 **[Test build #61514 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61514/consoleFull)** for PR 13829 at commit

[GitHub] spark issue #13978: [SPARK-16256][DOCS] Minor fixes on the Structured Stream...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13978 **[Test build #61522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61522/consoleFull)** for PR 13978 at commit

[GitHub] spark issue #13987: [SPARK-16313][SQL] Spark should not silently drop except...

2016-06-29 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/13987 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #13987: [SPARK-16313][SQL] Spark should not silently drop except...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13987 **[Test build #61521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61521/consoleFull)** for PR 13987 at commit

[GitHub] spark pull request #13987: [SPARK-16313][SQL] Spark should not silently drop...

2016-06-29 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/13987 [SPARK-16313][SQL] Spark should not silently drop exceptions in file listing ## What changes were proposed in this pull request? Spark silently drops exceptions during file listing. This is a very

[GitHub] spark issue #13972: [SPARK-16294][SQL] Labelling support for the include_exa...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13972 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61519/ Test PASSed. ---

[GitHub] spark issue #13972: [SPARK-16294][SQL] Labelling support for the include_exa...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13972 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13972: [SPARK-16294][SQL] Labelling support for the include_exa...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13972 **[Test build #61519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61519/consoleFull)** for PR 13972 at commit

[GitHub] spark issue #13979: [SPARK-SPARK-16302] [SQL] Set the right number of partit...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13979 **[Test build #61520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61520/consoleFull)** for PR 13979 at commit

[GitHub] spark issue #12384: [SPARK-14608] [ML] transformSchema needs better document...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12384 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61518/ Test PASSed. ---

[GitHub] spark issue #12384: [SPARK-14608] [ML] transformSchema needs better document...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12384 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12384: [SPARK-14608] [ML] transformSchema needs better document...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12384 **[Test build #61518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61518/consoleFull)** for PR 12384 at commit

[GitHub] spark issue #13941: [SPARK-16249][ML] Change visibility of Object ml.cluster...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13941 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13941: [SPARK-16249][ML] Change visibility of Object ml.cluster...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13941 **[Test build #61516 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61516/consoleFull)** for PR 13941 at commit

[GitHub] spark issue #13941: [SPARK-16249][ML] Change visibility of Object ml.cluster...

2016-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13941 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61516/ Test PASSed. ---

[GitHub] spark issue #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStreams to ...

2016-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11863 **[Test build #3150 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3150/consoleFull)** for PR 11863 at commit

[GitHub] spark issue #13972: [SPARK-16294][SQL] Labelling support for the include_exa...

2016-06-29 Thread yinxusen
Github user yinxusen commented on the issue: https://github.com/apache/spark/pull/13972 @mengxr With this PR merged, I think we can also fix the [SPARK-13015 (mllib-data-types.md )](https://issues.apache.org/jira/browse/SPARK-13015) with a consolidated example file. --- If your

[GitHub] spark issue #13972: [SPARK-16294][SQL] Labelling support for the include_exa...

2016-06-29 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13972 @yinxusen Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #13972: [SPARK-16294][SQL] Labelling support for the include_exa...

2016-06-29 Thread yinxusen
Github user yinxusen commented on the issue: https://github.com/apache/spark/pull/13972 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

  1   2   3   4   5   6   7   8   >