[GitHub] spark issue #21939: [SPARK-23874][SQL][PYTHON] Upgrade Apache Arrow to 0.10....

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21939 **[Test build #93853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93853/testReport)** for PR 21939 at commit

[GitHub] spark issue #21939: [SPARK-23874][SQL][PYTHON] Upgrade Apache Arrow to 0.10....

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21939 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21928: [SPARK-24976][PYTHON] Allow None for Decimal type conver...

2018-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21928 Thank you @felixcheung, @icexelloss and @BryanCutler. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21939: [SPARK-23874][SQL][PYTHON] Upgrade Apache Arrow to 0.10....

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21939 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21939: [SPARK-23874][SQL][PYTHON] Upgrade Apache Arrow to 0.10....

2018-07-31 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21939 This is a WIP, Arrow 0.10.0 hasn't been released yet but I wanted to get this up since the 2.4.0 code freeze is coming up and there might not be to much time in case some things need

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-07-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/21889 > Where does that leave both of these PRs? Do we still want this one with the code refactoring or to go back to the original? Are there any comments for this PR that would block merging? I've set

[GitHub] spark pull request #21895: [SPARK-24948][SHS] Delegate check access permissi...

2018-07-31 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21895#discussion_r206726059 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -973,6 +978,42 @@ private[history] object FsHistoryProvider

[GitHub] spark issue #21939: [SPARK-23874][SQL][PYTHON] Upgrade Apache Arrow to 0.10....

2018-07-31 Thread holdensmagicalunicorn
Github user holdensmagicalunicorn commented on the issue: https://github.com/apache/spark/pull/21939 @BryanCutler, thanks! I am a bot who has found some folks who might be able to help with the review:@HyukjinKwon, @cloud-fan and @vanzin ---

[GitHub] spark pull request #21939: [SPARK-23874][SQL][PYTHON] Upgrade Apache Arrow t...

2018-07-31 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/21939 [SPARK-23874][SQL][PYTHON] Upgrade Apache Arrow to 0.10.0 ## What changes were proposed in this pull request? Upgrade Apache Arrow to 0.10.0 Version 0.10.0 will allow for the

[GitHub] spark pull request #21895: [SPARK-24948][SHS] Delegate check access permissi...

2018-07-31 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21895#discussion_r206725814 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -973,6 +978,42 @@ private[history] object FsHistoryProvider

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19449 **[Test build #93852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93852/testReport)** for PR 19449 at commit

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21103 **[Test build #93851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93851/testReport)** for PR 21103 at commit

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19449 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19449 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19449 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21928: [SPARK-24976][PYTHON] Allow None for Decimal type...

2018-07-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21928 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21928: [SPARK-24976][PYTHON] Allow None for Decimal type conver...

2018-07-31 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21928 merged to master and branch-2.3, thanks @HyukjinKwon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21938: [SPARK-24982][SQL] UDAF resolution should not throw Asse...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21938 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21938: [SPARK-24982][SQL] UDAF resolution should not throw Asse...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21938 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93843/ Test FAILed. ---

[GitHub] spark issue #21938: [SPARK-24982][SQL] UDAF resolution should not throw Asse...

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21938 **[Test build #93843 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93843/testReport)** for PR 21938 at commit

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206709783 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206716473 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContextImpl.scala --- @@ -39,8 +44,53 @@ private[spark] class BarrierTaskContextImpl(

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206628185 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206708836 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206718449 --- Diff: core/src/test/scala/org/apache/spark/scheduler/BarrierTaskContextSuite.scala --- @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206709039 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206649993 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206627185 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206718064 --- Diff: core/src/test/scala/org/apache/spark/scheduler/BarrierTaskContextSuite.scala --- @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206710204 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -27,6 +27,33 @@ trait BarrierTaskContext extends TaskContext { * Sets

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206707616 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206716080 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContextImpl.scala --- @@ -39,8 +44,53 @@ private[spark] class BarrierTaskContextImpl(

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206716701 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -567,4 +567,14 @@ package object config { .intConf

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206709349 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206707253 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206628140 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206707063 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206707800 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206650166 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206717197 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala --- @@ -61,6 +61,9 @@ private[spark] trait TaskScheduler { */

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206716437 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContextImpl.scala --- @@ -39,8 +44,51 @@ private[spark] class BarrierTaskContextImpl(

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206717347 --- Diff: core/src/test/scala/org/apache/spark/scheduler/BarrierTaskContextSuite.scala --- @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206709985 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -27,6 +27,33 @@ trait BarrierTaskContext extends TaskContext { * Sets

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/21854 LGTM pending tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21854 **[Test build #93850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93850/testReport)** for PR 21854 at commit

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21854 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21854 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #21854: [SPARK-24896][SQL] Uuid should produce different ...

2018-07-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21854#discussion_r206714766 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1446,3 +1448,17 @@ object

[GitHub] spark pull request #21854: [SPARK-24896][SQL] Uuid should produce different ...

2018-07-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21854#discussion_r206711618 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2124,6 +2124,9 @@ class Analyzer( override def

[GitHub] spark issue #21937: [WIP][SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21937 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21937: [WIP][SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21937 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93840/ Test PASSed. ---

[GitHub] spark issue #21937: [WIP][SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21937 **[Test build #93840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93840/testReport)** for PR 21937 at commit

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19449 **[Test build #4225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4225/testReport)** for PR 19449 at commit

[GitHub] spark issue #21936: [SPARK-24981][Core] ShutdownHook timeout causes job to f...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21936 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93839/ Test PASSed. ---

[GitHub] spark issue #21936: [SPARK-24981][Core] ShutdownHook timeout causes job to f...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21936 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21936: [SPARK-24981][Core] ShutdownHook timeout causes job to f...

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21936 **[Test build #93839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93839/testReport)** for PR 21936 at commit

[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce Hint for SQL Queries

2018-07-31 Thread jzhuge
Github user jzhuge commented on the issue: https://github.com/apache/spark/pull/21911 Exploring a suggestion brought to me offline: make this hint a SELECT hint, not an INSERT hint. This way, the hint can be applied to subqueries. ---

[GitHub] spark issue #21934: [SPARK-24951][SQL] Table valued functions should throw A...

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21934 **[Test build #93849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93849/testReport)** for PR 21934 at commit

[GitHub] spark issue #21934: [SPARK-24951][SQL] Table valued functions should throw A...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21934 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21934: [SPARK-24951][SQL] Table valued functions should throw A...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21934 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21934: [SPARK-24951][SQL] Table valued functions should throw A...

2018-07-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21934 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93835/ Test PASSed. ---

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21102 **[Test build #93835 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93835/testReport)** for PR 21102 at commit

[GitHub] spark issue #21913: [WIP][SPARK-24005][CORE] Remove usage of Scala’s paral...

2018-07-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/21913 Yeah, let's also fix other instances. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce Hint for SQL Queries

2018-07-31 Thread jzhuge
Github user jzhuge commented on the issue: https://github.com/apache/spark/pull/21911 @gatorsmile Spark SQL Hint models after Hive SQL Hint. Impala also has hints: https://www.cloudera.com/documentation/enterprise/5-13-x/topics/impala_hints.html#hints. ---

[GitHub] spark issue #21913: [WIP][SPARK-24005][CORE] Remove usage of Scala’s paral...

2018-07-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21913 OK, got it. Yes that's worth mentioning in the doc too. I'll have to go read up to get why this works differently, but obviously accept that it does. There are other instances of `.par`, like

[GitHub] spark pull request #21913: [WIP][SPARK-24005][CORE] Remove usage of Scala’...

2018-07-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21913#discussion_r206704908 --- Diff: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala --- @@ -254,4 +254,49 @@ private[spark] object ThreadUtils {

[GitHub] spark pull request #21913: [WIP][SPARK-24005][CORE] Remove usage of Scala’...

2018-07-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21913#discussion_r206704241 --- Diff: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala --- @@ -254,4 +254,49 @@ private[spark] object ThreadUtils {

[GitHub] spark pull request #21913: [WIP][SPARK-24005][CORE] Remove usage of Scala’...

2018-07-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21913#discussion_r206704548 --- Diff: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala --- @@ -254,4 +254,49 @@ private[spark] object ThreadUtils {

[GitHub] spark pull request #21913: [WIP][SPARK-24005][CORE] Remove usage of Scala’...

2018-07-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21913#discussion_r206704050 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/FileBasedWriteAheadLog.scala --- @@ -313,9 +313,7 @@ private[streaming] object

[GitHub] spark issue #21913: [WIP][SPARK-24005][CORE] Remove usage of Scala’s paral...

2018-07-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/21913 > What problem does this solve? @srowen `readParquetFootersInParallel` is called in executors. When a task is cancelled, it will still keeping run. If it reads lots of files, it will take

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-07-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r206702526 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -308,10 +319,10 @@ object

[GitHub] spark issue #21852: [SPARK-24893] [SQL] Remove the entire CaseWhen if all th...

2018-07-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21852 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21610: Updates to LICENSE and NOTICE

2018-07-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21610 BTW I think this can be closed. I separately overhauled the LICENSE/NOTICE. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21722: Spark-24742: Fix NullPointerexception in Field Metadata

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21722 **[Test build #4227 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4227/testReport)** for PR 21722 at commit

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93836/ Test FAILed. ---

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21505: [SPARK-24457][SQL] Improving performance of strin...

2018-07-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21505#discussion_r206699663 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -111,6 +113,22 @@ object DateTimeUtils {

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21103 **[Test build #93836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93836/testReport)** for PR 21103 at commit

[GitHub] spark pull request #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO suppo...

2018-07-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21847#discussion_r206699403 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala --- @@ -165,16 +186,117 @@ class AvroSerializer(rootCatalystType:

[GitHub] spark pull request #21624: [SPARK-24639][DOC] Add three config in the doc

2018-07-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21624#discussion_r206698219 --- Diff: docs/configuration.md --- @@ -456,6 +456,40 @@ Apart from these, the following properties are also available, and may be useful from JVM

[GitHub] spark pull request #21624: [SPARK-24639][DOC] Add three config in the doc

2018-07-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21624#discussion_r206698447 --- Diff: docs/configuration.md --- @@ -541,6 +575,13 @@ Apart from these, the following properties are also available, and may be useful Python

[GitHub] spark pull request #21884: [SPARK-24960][K8S] explicitly expose ports on dri...

2018-07-31 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21884#discussion_r206699070 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala --- @@ -87,6 +87,10 @@

[GitHub] spark pull request #21639: [SPARK-24653][tests] Avoid cross-job pollution in...

2018-07-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21639#discussion_r206697727 --- Diff: core/src/main/scala/org/apache/spark/TestUtils.scala --- @@ -233,6 +233,21 @@ private[spark] object TestUtils { } } +

[GitHub] spark issue #21639: [SPARK-24653][tests] Avoid cross-job pollution in TestUt...

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21639 **[Test build #4226 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4226/testReport)** for PR 21639 at commit

[GitHub] spark issue #21884: [SPARK-24960][K8S] explicitly expose ports on driver con...

2018-07-31 Thread adelbertc
Github user adelbertc commented on the issue: https://github.com/apache/spark/pull/21884 @mccheah Done and done --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r206696830 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -35,18 +36,22 @@ import

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r206696107 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -275,9 +287,13 @@ private[yarn] class

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-31 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r206695469 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -156,6 +160,14 @@ private[yarn] class

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21699 **[Test build #93848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93848/testReport)** for PR 21699 at commit

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-31 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21699 > it would be nice to support it and test it in DataFrame pivot too. @maryannxue Supported and tested. Please, have a look at:

[GitHub] spark issue #21852: [SPARK-24893] [SQL] Remove the entire CaseWhen if all th...

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21852 **[Test build #93847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93847/testReport)** for PR 21852 at commit

[GitHub] spark issue #21852: [SPARK-24893] [SQL] Remove the entire CaseWhen if all th...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21852 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21852: [SPARK-24893] [SQL] Remove the entire CaseWhen if all th...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21852 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...

2018-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #93846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93846/testReport)** for PR 21847 at commit

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...

2018-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93846/ Test PASSed. ---

<    1   2   3   4   5   6   7   >