[GitHub] spark issue #13901: [SPARK-16199][SQL] Add a method to list the referenced c...

[gatorsmile]

Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/13901
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13900: [SPARK-16173][SQL] Can't join describe() of DataFrame in...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/13900
  
Oh, great! I didn't notice that is mergeable for 1.5 branch.
Thank you!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce additonal implementation wi...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13680
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13900: [SPARK-16173][SQL] Can't join describe() of DataFrame in...

[davies]

Github user davies commented on the issue:

https://github.com/apache/spark/pull/13900
  
@dongjoon-hyun had also merged into 1.5 branch


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce additonal implementation wi...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13680
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61222/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce additonal implementation wi...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13680
  
**[Test build #61222 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61222/consoleFull)**
 for PR 13680 at commit 
[`bf12e72`](https://github.com/apache/spark/commit/bf12e72b6ab0e6adb208fbdd60417e148905433b).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[davies]

Github user davies commented on the issue:

https://github.com/apache/spark/pull/13887
  
Merged into master, thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13902: [SPARK-16173] [SQL] Can't join describe() of DataFrame i...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/13902
  
Thank you, @davies !


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13902: [SPARK-16173] [SQL] Can't join describe() of DataFrame i...

[davies]

Github user davies commented on the issue:

https://github.com/apache/spark/pull/13902
  
Merged into 1.6


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13663: [SPARK-15950][SQL] Eliminate unreachable code at project...

[kiszk]

Github user kiszk commented on the issue:

https://github.com/apache/spark/pull/13663
  
@cloud-fan , @davies , I would appreciate it if you would look at this 
again.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13758
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61221/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13758
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13758
  
**[Test build #61221 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61221/consoleFull)**
 for PR 13758 at commit 
[`5be9873`](https://github.com/apache/spark/commit/5be987369073f93bb0965ae4523521af5db96fb0).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce additonal implementation wi...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13680
  
**[Test build #61222 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61222/consoleFull)**
 for PR 13680 at commit 
[`bf12e72`](https://github.com/apache/spark/commit/bf12e72b6ab0e6adb208fbdd60417e148905433b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/13892


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[hvanhovell]

Github user hvanhovell commented on the issue:

https://github.com/apache/spark/pull/13892
  
Merging to master/2.0. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61217/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61217 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61217/consoleFull)**
 for PR 13892 at commit 
[`1a445c8`](https://github.com/apache/spark/commit/1a445c887ebf0982186d2ac8b62aac627509eee3).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13868: [SPARK-15899] [SQL] Fix the construction of the file pat...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13868
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61219/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13868: [SPARK-15899] [SQL] Fix the construction of the file pat...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13868
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13902: [SPARK-16173] [SQL] Can't join describe() of DataFrame i...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13902
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61220/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13868: [SPARK-15899] [SQL] Fix the construction of the file pat...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13868
  
**[Test build #61219 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61219/consoleFull)**
 for PR 13868 at commit 
[`fb50965`](https://github.com/apache/spark/commit/fb5096504553f9a1c7378b7ddbfc8680d6add04b).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13902: [SPARK-16173] [SQL] Can't join describe() of DataFrame i...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13902
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13902: [SPARK-16173] [SQL] Can't join describe() of DataFrame i...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13902
  
**[Test build #61220 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61220/consoleFull)**
 for PR 13902 at commit 
[`f114f08`](https://github.com/apache/spark/commit/f114f08a628b98f7e713c692fa4f7d4c10d253c0).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61216/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61216 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61216/consoleFull)**
 for PR 13892 at commit 
[`4924984`](https://github.com/apache/spark/commit/4924984ec46d264d70598f578d8c82a7e1a35c37).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13852: [SPARK-16200][SQL] Rename AggregateFunction#supportsPart...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13852
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61215/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13852: [SPARK-16200][SQL] Rename AggregateFunction#supportsPart...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13852
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13893: [SPARK-14172][SQL] Hive table partition predicate...

[jiangxb1987]

Github user jiangxb1987 commented on a diff in the pull request:

https://github.com/apache/spark/pull/13893#discussion_r68483394
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala
 ---
@@ -64,10 +64,12 @@ object PhysicalOperation extends PredicateHelper {
 val substitutedFields = 
fields.map(substitute(aliases)).asInstanceOf[Seq[NamedExpression]]
 (Some(substitutedFields), filters, other, 
collectAliases(substitutedFields))
 
-  case Filter(condition, child) if condition.deterministic =>
+  case Filter(condition, child) =>
+val (deterministicFilters, nondeterministicFilters) = 
splitConjunctivePredicates(condition)
+  .partition(_.deterministic)
 val (fields, filters, other, aliases) = 
collectProjectsAndFilters(child)
-val substitutedCondition = substitute(aliases)(condition)
-(fields, filters ++ 
splitConjunctivePredicates(substitutedCondition), other, aliases)
+val substitutedFilters = 
deterministicFilters.map(substitute(aliases)(_))
+(fields, filters ++ deterministicFilters, other, aliases)
--- End diff --

@AlekseiS U r right! I have fixed this, thank you!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13852: [SPARK-16200][SQL] Rename AggregateFunction#supportsPart...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13852
  
**[Test build #61215 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61215/consoleFull)**
 for PR 13852 at commit 
[`00aac26`](https://github.com/apache/spark/commit/00aac26c6a71f6b09019649f44192977b0873093).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13758
  
**[Test build #61221 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61221/consoleFull)**
 for PR 13758 at commit 
[`5be9873`](https://github.com/apache/spark/commit/5be987369073f93bb0965ae4523521af5db96fb0).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61213/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61213 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61213/consoleFull)**
 for PR 13892 at commit 
[`dd7e233`](https://github.com/apache/spark/commit/dd7e233d2df97a68eeac4570647ac9f93e044d70).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13868: [SPARK-15899] [SQL] Fix the construction of the file pat...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13868
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61214/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13868: [SPARK-15899] [SQL] Fix the construction of the file pat...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13868
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13868: [SPARK-15899] [SQL] Fix the construction of the file pat...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13868
  
**[Test build #61214 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61214/consoleFull)**
 for PR 13868 at commit 
[`a797fbe`](https://github.com/apache/spark/commit/a797fbe641766bca03bd48e8addc81a68fc30f04).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61212/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61212 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61212/consoleFull)**
 for PR 13892 at commit 
[`681b993`](https://github.com/apache/spark/commit/681b993d5d018d88281f45f95b28607965b38ba2).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce additonal implementation wi...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13680
  
**[Test build #61218 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61218/consoleFull)**
 for PR 13680 at commit 
[`517aa72`](https://github.com/apache/spark/commit/517aa72b364df19bd2c4443cce678cbf8f86ec96).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce additonal implementation wi...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13680
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61218/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce additonal implementation wi...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13680
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13023: [SPARK-15177] [SparkR] [ML] SparkR 2.0 QA: New R ...

[yanboliang]

Github user yanboliang closed the pull request at:

https://github.com/apache/spark/pull/13023


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13896: [SPARK-16193] [TESTS] Address flaky ExternalAppendOnlyMa...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13896
  
**[Test build #3132 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3132/consoleFull)**
 for PR 13896 at commit 
[`bcc0297`](https://github.com/apache/spark/commit/bcc0297e8af639132bebb0703eb7df8c8fea15f9).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13867: [SPARK-16161][SQL] Ambiguous error message for un...

[hvanhovell]

Github user hvanhovell commented on a diff in the pull request:

https://github.com/apache/spark/pull/13867#discussion_r68482646
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
 ---
@@ -911,19 +911,30 @@ class Analyzer(
  * Resolve the correlated expressions in a subquery by using the an 
outer plans' references. All
  * resolved outer references are wrapped in an [[OuterReference]]
  */
-private def resolveOuterReferences(plan: LogicalPlan, outer: 
LogicalPlan): LogicalPlan = {
+private def resolveOuterReferences(
+plan: LogicalPlan,
+outers: Seq[LogicalPlan]): LogicalPlan = {
+  var throwError = false
   plan transformDown {
 case q: LogicalPlan if q.childrenResolved && !q.resolved =>
   q transformExpressions {
 case u @ UnresolvedAttribute(nameParts) =>
   withPosition(u) {
 try {
-  outer.resolve(nameParts, resolver) match {
+  val outer = outers.iterator
--- End diff --

Is there are way to simplify the code below? Two try's seem a bit hacky? We 
could move try catch into the loop and throw an exception if we could not 
resolve the expr


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13902: [SPARK-16173] [SQL] Can't join describe() of DataFrame i...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13902
  
**[Test build #61220 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61220/consoleFull)**
 for PR 13902 at commit 
[`f114f08`](https://github.com/apache/spark/commit/f114f08a628b98f7e713c692fa4f7d4c10d253c0).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13902: [SPARK-16173] [SQL] Can't join describe() of DataFrame i...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/13902
  
cc @davies .
This is a PR for branch 1.6.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13902: [SPARK-16173] [SQL] Can't join describe() of Data...

[dongjoon-hyun]

GitHub user dongjoon-hyun opened a pull request:

https://github.com/apache/spark/pull/13902

[SPARK-16173] [SQL] Can't join describe() of DataFrame in Scala 2.10

## What changes were proposed in this pull request?

This PR fixes `DataFrame.describe()` by forcing materialization to make the 
`Seq` serializable. Currently, `describe()` of `DataFrame` throws `Task not 
serializable` Spark exceptions when joining in Scala 2.10.

## How was this patch tested?

Manual. (After building with Scala 2.10, test on bin/spark-shell and 
bin/pyspark.)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dongjoon-hyun/spark SPARK-16173-branch-1.6

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/13902.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #13902


commit f114f08a628b98f7e713c692fa4f7d4c10d253c0
Author: Dongjoon Hyun 
Date:   2016-06-25T01:50:29Z

[SPARK-16173] [SQL] Can't join describe() of DataFrame in Scala 2.10




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13868: [SPARK-15899] [SQL] Fix the construction of the file pat...

[avulanov]

Github user avulanov commented on the issue:

https://github.com/apache/spark/pull/13868
  
It seems that the root cause is that hadoop `Path.toUri` adds only `/` to 
the beginning of the path but does not add `file:`. So, here and there in the 
code we observe in-place concatenations of `file:` and `/`. Java File `toURI` 
adds `file:/` to the beginning and `/` to the end of the path. In particular, 
Java `File.toURI.getPath` equals to Hadoop `Path.toUri.toString`. Also, some 
tests did not handle Windows paths with back slashes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13867: [SPARK-16161][SQL] Ambiguous error message for un...

[hvanhovell]

Github user hvanhovell commented on a diff in the pull request:

https://github.com/apache/spark/pull/13867#discussion_r68482400
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
 ---
@@ -911,19 +911,30 @@ class Analyzer(
  * Resolve the correlated expressions in a subquery by using the an 
outer plans' references. All
  * resolved outer references are wrapped in an [[OuterReference]]
  */
-private def resolveOuterReferences(plan: LogicalPlan, outer: 
LogicalPlan): LogicalPlan = {
+private def resolveOuterReferences(
+plan: LogicalPlan,
+outers: Seq[LogicalPlan]): LogicalPlan = {
+  var throwError = false
   plan transformDown {
 case q: LogicalPlan if q.childrenResolved && !q.resolved =>
   q transformExpressions {
 case u @ UnresolvedAttribute(nameParts) =>
   withPosition(u) {
 try {
-  outer.resolve(nameParts, resolver) match {
+  val outer = outers.iterator
+  var expr: Option[NamedExpression] = None
--- End diff --

`var expo = Option.empty[NamedExpression]`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13868: [SPARK-15899] [SQL] Fix the construction of the file pat...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13868
  
**[Test build #61219 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61219/consoleFull)**
 for PR 13868 at commit 
[`fb50965`](https://github.com/apache/spark/commit/fb5096504553f9a1c7378b7ddbfc8680d6add04b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[hvanhovell]

Github user hvanhovell commented on the issue:

https://github.com/apache/spark/pull/13892
  
LGTM - pending jenkins


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61211/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13892
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61211 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61211/consoleFull)**
 for PR 13892 at commit 
[`73dc6e7`](https://github.com/apache/spark/commit/73dc6e74711ce88b57ec5b8f6f9a0a7b07e06ad5).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61217 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61217/consoleFull)**
 for PR 13892 at commit 
[`1a445c8`](https://github.com/apache/spark/commit/1a445c887ebf0982186d2ac8b62aac627509eee3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce additonal implementation wi...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13680
  
**[Test build #61218 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61218/consoleFull)**
 for PR 13680 at commit 
[`517aa72`](https://github.com/apache/spark/commit/517aa72b364df19bd2c4443cce678cbf8f86ec96).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61216 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61216/consoleFull)**
 for PR 13892 at commit 
[`4924984`](https://github.com/apache/spark/commit/4924984ec46d264d70598f578d8c82a7e1a35c37).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[maropu]

Github user maropu commented on a diff in the pull request:

https://github.com/apache/spark/pull/13892#discussion_r68482138
  
--- Diff: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala
 ---
@@ -435,6 +435,23 @@ class AnalysisErrorSuite extends AnalysisTest {
 "another aggregate function." :: Nil)
   }
 
+  test("we should fail analysis when we find map type data in 
collect_set") {
--- End diff --

okay


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13892: [SPARK-16192][SQL] Add type checks in CollectSet

[hvanhovell]

Github user hvanhovell commented on a diff in the pull request:

https://github.com/apache/spark/pull/13892#discussion_r68482123
  
--- Diff: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala
 ---
@@ -435,6 +435,23 @@ class AnalysisErrorSuite extends AnalysisTest {
 "another aggregate function." :: Nil)
   }
 
+  test("we should fail analysis when we find map type data in 
collect_set") {
--- End diff --

Lets remove this test.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/13887
  
Hi, @davies .
Now, it passed.
If there is anything for me to do, please let me know.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13760: [SPARK-16012][SparkR] implement gapplyCollect which will...

[shivaram]

Github user shivaram commented on the issue:

https://github.com/apache/spark/pull/13760
  
@felixcheung Any other comments on this ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13887
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61210/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13887
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13887
  
**[Test build #61210 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61210/consoleFull)**
 for PR 13887 at commit 
[`9d550e3`](https://github.com/apache/spark/commit/9d550e3b7885daacdb01e3e54d01f5157af20791).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13852: [SQL][DOC] Update a description for AggregateFunction#su...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13852
  
**[Test build #61215 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61215/consoleFull)**
 for PR 13852 at commit 
[`00aac26`](https://github.com/apache/spark/commit/00aac26c6a71f6b09019649f44192977b0873093).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13685: [SPARK-15963][CORE] Catch `TaskKilledException` correctl...

[lw-lin]

Github user lw-lin commented on the issue:

https://github.com/apache/spark/pull/13685
  
Thanks, @squito @markhamstra !


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13868: [SPARK-15899] [SQL] Fix the construction of the file pat...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13868
  
**[Test build #61214 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61214/consoleFull)**
 for PR 13868 at commit 
[`a797fbe`](https://github.com/apache/spark/commit/a797fbe641766bca03bd48e8addc81a68fc30f04).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CheckAnalysis

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61213 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61213/consoleFull)**
 for PR 13892 at commit 
[`dd7e233`](https://github.com/apache/spark/commit/dd7e233d2df97a68eeac4570647ac9f93e044d70).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CheckAnalysis

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61212 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61212/consoleFull)**
 for PR 13892 at commit 
[`681b993`](https://github.com/apache/spark/commit/681b993d5d018d88281f45f95b28607965b38ba2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13892: [SPARK-16192][SQL] Add type checks in CheckAnalys...

[maropu]

Github user maropu commented on a diff in the pull request:

https://github.com/apache/spark/pull/13892#discussion_r68481381
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
 ---
@@ -73,9 +73,12 @@ trait CheckAnalysis extends PredicateHelper {
   s"invalid cast from ${c.child.dataType.simpleString} to 
${c.dataType.simpleString}")
 
   case g: Grouping =>
-failAnalysis(s"grouping() can only be used with 
GroupingSets/Cube/Rollup")
+failAnalysis("grouping() can only be used with 
GroupingSets/Cube/Rollup")
   case g: GroupingID =>
-failAnalysis(s"grouping_id() can only be used with 
GroupingSets/Cube/Rollup")
+failAnalysis("grouping_id() can only be used with 
GroupingSets/Cube/Rollup")
+
+  case c: CollectSet if 
!c.child.dataType.existsRecursively(_.isInstanceOf[MapType]) =>
--- End diff --

okay


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13892: [SPARK-16192][SQL] Add type checks in CheckAnalys...

[hvanhovell]

Github user hvanhovell commented on a diff in the pull request:

https://github.com/apache/spark/pull/13892#discussion_r68481285
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
 ---
@@ -73,9 +73,12 @@ trait CheckAnalysis extends PredicateHelper {
   s"invalid cast from ${c.child.dataType.simpleString} to 
${c.dataType.simpleString}")
 
   case g: Grouping =>
-failAnalysis(s"grouping() can only be used with 
GroupingSets/Cube/Rollup")
+failAnalysis("grouping() can only be used with 
GroupingSets/Cube/Rollup")
   case g: GroupingID =>
-failAnalysis(s"grouping_id() can only be used with 
GroupingSets/Cube/Rollup")
+failAnalysis("grouping_id() can only be used with 
GroupingSets/Cube/Rollup")
+
+  case c: CollectSet if 
!c.child.dataType.existsRecursively(_.isInstanceOf[MapType]) =>
--- End diff --

Please do not use `CheckAnalysis` for this; move this check into the 
`CollectSet` expression by overriding `Expression.checkInputDataTypes`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13896: [SPARK-16193] [TESTS] Address flaky ExternalAppendOnlyMa...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13896
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61195/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13896: [SPARK-16193] [TESTS] Address flaky ExternalAppendOnlyMa...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13896
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13896: [SPARK-16193] [TESTS] Address flaky ExternalAppendOnlyMa...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13896
  
**[Test build #61195 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61195/consoleFull)**
 for PR 13896 at commit 
[`bcc0297`](https://github.com/apache/spark/commit/bcc0297e8af639132bebb0703eb7df8c8fea15f9).
 * This patch **fails from timeout after a configured wait of \`250m\`**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13892: [SPARK-16192][SQL] Add type checks in CheckAnalysis

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13892
  
**[Test build #61211 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61211/consoleFull)**
 for PR 13892 at commit 
[`73dc6e7`](https://github.com/apache/spark/commit/73dc6e74711ce88b57ec5b8f6f9a0a7b07e06ad5).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13248: [SPARK-15194] [ML] Add Python ML API for MultivariateGau...

[praveendareddy21]

Github user praveendareddy21 commented on the issue:

https://github.com/apache/spark/pull/13248
  
@vectorijk I will be pushing the changes in few days.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13847: [SPARK-16135][SQL] Remove hashCode and euqals in ArrayBa...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13847
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13847: [SPARK-16135][SQL] Remove hashCode and euqals in ArrayBa...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13847
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61208/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13847: [SPARK-16135][SQL] Remove hashCode and euqals in ArrayBa...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13847
  
**[Test build #61208 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61208/consoleFull)**
 for PR 13847 at commit 
[`e4b1384`](https://github.com/apache/spark/commit/e4b138403859ff1662a50a88279dd653a7258c62).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13896: [SPARK-16193] [TESTS] Address flaky ExternalAppendOnlyMa...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13896
  
**[Test build #3132 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3132/consoleFull)**
 for PR 13896 at commit 
[`bcc0297`](https://github.com/apache/spark/commit/bcc0297e8af639132bebb0703eb7df8c8fea15f9).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13839: [SPARK-16128][SQL] Allow setting length of charac...

[shivaram]

Github user shivaram commented on a diff in the pull request:

https://github.com/apache/spark/pull/13839#discussion_r68480005
  
--- Diff: R/pkg/R/DataFrame.R ---
@@ -194,7 +194,12 @@ setMethod("isLocal",
 setMethod("showDF",
   signature(x = "SparkDataFrame"),
   function(x, numRows = 20, truncate = TRUE) {
-s <- callJMethod(x@sdf, "showString", numToInt(numRows), 
truncate)
+truncate2 <- as.numeric(truncate)
+if (truncate2 == 1) {
--- End diff --

Related to the other comment you can use `is.logical(truncate)` to see if 
`truncate` is a boolean in R


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13900: [SPARK-16173][SQL] Can't join describe() of DataFrame in...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/13900
  
Thank you for merging.
Sure. I will make a patch for 1.6. 
Should I make a patch for 1.5, too?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/13887
  
Anyway, thank you in advance, @davies . :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13897: [SPARK-16195][SQL] Allow users to specify empty o...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/13897


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13900: [SPARK-16173][SQL] Can't join describe() of DataF...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/13900


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13897: [SPARK-16195][SQL] Allow users to specify empty over cla...

[hvanhovell]

Github user hvanhovell commented on the issue:

https://github.com/apache/spark/pull/13897
  
LGTM - merging to master/2.0


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13900: [SPARK-16173][SQL] Can't join describe() of DataFrame in...

[davies]

Github user davies commented on the issue:

https://github.com/apache/spark/pull/13900
  
@dongjoon-hyun Could you send a patch for 1.6?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13839: [SPARK-16128][SQL] Allow setting length of charac...

[ScrapCodes]

Github user ScrapCodes commented on a diff in the pull request:

https://github.com/apache/spark/pull/13839#discussion_r68479735
  
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -282,8 +284,18 @@ def show(self, n=20, truncate=True):
 |  2|Alice|
 |  5|  Bob|
 +---+-+
-"""
-print(self._jdf.showString(n, truncate))
+>>> df.show(truncate=3)
++---++
+|age|name|
++---++
+|  2| Ali|
+|  5| Bob|
++---++
+"""
+if truncate == 1:
--- End diff --

Yes, I did it because, in python 1 and True are indistinguishable.
```python
Type "help", "copyright", "credits" or "license" for more information.
>>> a=True
>>> isinstance(a, bool)
True
>>> isinstance(a, int)
True
>>> a == 1
True
>>> a == True
True
>>> 

```

In both python 2 and 3. 
But,
```python
>>> a=1
>>> isinstance(a, bool)
False
```
Might help here. I can go ahead and change it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/13887
  
Maybe, you are confused with https://github.com/apache/spark/pull/13900 .
It passed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/13887
  
Oh, right. It's pending Jenkins.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[davies]

Github user davies commented on the issue:

https://github.com/apache/spark/pull/13887
  
Reverted, will merge this again once it passed jenkins.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13887: [SPARK-16186][SQL] Support partition batch prunin...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/13887


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13887: [SPARK-16186][SQL] Support partition batch pruning with ...

[davies]

Github user davies commented on the issue:

https://github.com/apache/spark/pull/13887
  
Sorry, the jenkins has not finished ...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13900: [SPARK-16173][SQL] Can't join describe() of DataFrame in...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13900
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61207/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13900: [SPARK-16173][SQL] Can't join describe() of DataFrame in...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13900
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13900: [SPARK-16173][SQL] Can't join describe() of DataFrame in...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13900
  
**[Test build #61207 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61207/consoleFull)**
 for PR 13900 at commit 
[`c258e2c`](https://github.com/apache/spark/commit/c258e2c559506880c5ac6582a9a7bd5aa41d1242).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org