[GitHub] spark issue #19939: [SPARK-20557] [SQL] Only support TIMESTAMP WITH TIME ZON...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19939 **[Test build #84692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84692/testReport)** for PR 19939 at commit

[GitHub] spark issue #19939: [SPARK-20557] [SQL] Only support TIMESTAMP WITH TIME ZON...

2017-12-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19939 cc @cloud-fan @ueshin @srowen @JannikArndt --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19939: [SPARK-20557] [SQL] Only support TIMESTAMP WITH T...

2017-12-09 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/19939 [SPARK-20557] [SQL] Only support TIMESTAMP WITH TIME ZONE for Oracle Dialect ## What changes were proposed in this pull request? In the previous PRs,

[GitHub] spark issue #19938: [SPARK-22747][SQL] Localize lifetime of mutable states i...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19938 **[Test build #84691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84691/testReport)** for PR 19938 at commit

[GitHub] spark pull request #19938: [SPARK-22747][SQL] Localize lifetime of mutable s...

2017-12-09 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/19938 [SPARK-22747][SQL] Localize lifetime of mutable states in HashAggregateExec ## What changes were proposed in this pull request? This PR localizes lifetime of mutable states, which are used

[GitHub] spark pull request #19918: [SPARK-22726] [TEST] Basic tests for Binary Compa...

2017-12-09 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19918#discussion_r155939348 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/binaryComparison.sql --- @@ -0,0 +1,287 @@ +-- +-- Licensed to the

[GitHub] spark issue #19933: [SPARK-22744][CORE] Add a configuration to show the appl...

2017-12-09 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/19933 I think you miss my point. Please image that I am an infra team member in a company, and spark users are almost using spark sql or just offering an application jar. What I want is get the submit

[GitHub] spark issue #19764: [SPARK-22539][SQL] Add second order for rangepartitioner...

2017-12-09 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19764 Ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19932 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84689/ Test PASSed. ---

[GitHub] spark issue #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19932 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19932 **[Test build #84689 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84689/testReport)** for PR 19932 at commit

[GitHub] spark issue #19865: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19865 I confirmed that this failure does not occur after merging #19937 in my environment. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19928: [SPARK-22267][SQL][TEST] Spark SQL incorrectly reads ORC...

2017-12-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19928 Thank you for review and approval, @HyukjinKwon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84690/ Test PASSed. ---

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #84690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84690/testReport)** for PR 19527 at commit

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19594 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #84690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84690/testReport)** for PR 19527 at commit

[GitHub] spark issue #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19932 **[Test build #84689 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84689/testReport)** for PR 19932 at commit

[GitHub] spark pull request #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19932#discussion_r155936167 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -353,15 +374,6 @@ class StatisticsSuite extends

[GitHub] spark pull request #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19932#discussion_r155936087 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -213,6 +213,29 @@ class StatisticsSuite extends

[GitHub] spark issue #19754: [BUILD] update release scripts

2017-12-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19754 (The package build failure, which began Nov 26, is:) ``` gpg: skipped

[GitHub] spark issue #19754: [BUILD] update release scripts

2017-12-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19754 The package build was failing for a while, but the docs build has a probably-related new error: ``` ATTENTION! Your password for authentication realm:

[GitHub] spark issue #19934: [SPARK-3685][CORE] Prints explicit warnings when configu...

2017-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19934 Hi @liancheng, @steveloughran and @srowen. What do you think about the current approach? The JIRA seems targeting "Spark's local dir should accept only local paths" and it's my best

[GitHub] spark pull request #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19932#discussion_r155935430 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -213,6 +213,29 @@ class StatisticsSuite extends

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-12-09 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r155935022 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark-base/Dockerfile --- @@ -0,0 +1,43 @@ +# +# Licensed to the Apache

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-12-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r155934917 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark-base/Dockerfile --- @@ -0,0 +1,43 @@ +# +# Licensed to the Apache

[GitHub] spark pull request #19937: [SPARK-22746][SQL] Avoid the generation of useles...

2017-12-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19937#discussion_r155934663 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -617,6 +619,7 @@ case class SortMergeJoinExec(

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19717 sounds like a good to go from @vanzin and a couple of others too. any other comment? --- - To unsubscribe, e-mail:

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-12-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16578 yes! sorry about the delay, I think there's a lot of interests in this PR. @gatorsmile @viirya ? --- - To

[GitHub] spark issue #19754: [BUILD] update release scripts

2017-12-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19754 yes, specifically this changes the output location for https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-master-docs/

[GitHub] spark pull request #19805: [SPARK-22649][PYTHON][SQL] Adding localCheckpoint...

2017-12-09 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19805#discussion_r155934026 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -537,9 +537,48 @@ class Dataset[T] private[sql]( */ @Experimental

[GitHub] spark issue #19865: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19865 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84688/ Test FAILed. ---

[GitHub] spark issue #19865: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19865 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19865: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19865 **[Test build #84688 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84688/testReport)** for PR 19865 at commit

[GitHub] spark issue #19865: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19865 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84687/ Test FAILed. ---

[GitHub] spark issue #19865: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19865 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19865: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19865 **[Test build #84687 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84687/testReport)** for PR 19865 at commit

[GitHub] spark pull request #19676: [SPARK-14516][FOLLOWUP] Adding ClusteringEvaluato...

2017-12-09 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19676#discussion_r155929522 --- Diff: examples/src/main/java/org/apache/spark/examples/ml/JavaKMeansExample.java --- @@ -51,9 +52,14 @@ public static void main(String[] args) {

[GitHub] spark pull request #19676: [SPARK-14516][FOLLOWUP] Adding ClusteringEvaluato...

2017-12-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19676#discussion_r155928871 --- Diff: examples/src/main/java/org/apache/spark/examples/ml/JavaKMeansExample.java --- @@ -51,9 +52,14 @@ public static void main(String[] args) {

[GitHub] spark issue #19937: [SPARK-22746][SQL] Avoid the generation of useless mutab...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19937 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84685/ Test PASSed. ---

[GitHub] spark issue #19937: [SPARK-22746][SQL] Avoid the generation of useless mutab...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19937 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19937: [SPARK-22746][SQL] Avoid the generation of useless mutab...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19937 **[Test build #84685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84685/testReport)** for PR 19937 at commit

[GitHub] spark issue #19525: [SPARK-22289] [ML] Add JSON support for Matrix parameter...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19525 **[Test build #84686 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84686/testReport)** for PR 19525 at commit

[GitHub] spark issue #19525: [SPARK-22289] [ML] Add JSON support for Matrix parameter...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84686/ Test PASSed. ---

[GitHub] spark issue #19525: [SPARK-22289] [ML] Add JSON support for Matrix parameter...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19525 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19937: [SPARK-22746][SQL] Avoid the generation of useles...

2017-12-09 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19937#discussion_r155927167 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -617,6 +619,7 @@ case class SortMergeJoinExec(

[GitHub] spark issue #19911: [SPARK-22729][SQL] Add getTruncateQuery to JdbcDialect

2017-12-09 Thread danielvdende
Github user danielvdende commented on the issue: https://github.com/apache/spark/pull/19911 @dongjoon-hyun @gatorsmile As @gatorsmile pointed out, the `isCascadingTruncateTable` is a method in the public API, so we can't just drop it. I've made changes again, now the truncate query

[GitHub] spark issue #19865: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19865 **[Test build #84688 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84688/testReport)** for PR 19865 at commit

[GitHub] spark pull request #19937: [SPARK-22746][SQL] Avoid the generation of useles...

2017-12-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19937#discussion_r155927018 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -617,6 +619,7 @@ case class SortMergeJoinExec(

[GitHub] spark pull request #19937: [SPARK-22746][SQL] Avoid the generation of useles...

2017-12-09 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19937#discussion_r155926973 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -617,6 +619,7 @@ case class SortMergeJoinExec(

[GitHub] spark issue #19865: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19865 **[Test build #84687 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84687/testReport)** for PR 19865 at commit

[GitHub] spark pull request #19937: [SPARK-22746][SQL] Avoid the generation of useles...

2017-12-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19937#discussion_r155926812 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -617,6 +619,7 @@ case class SortMergeJoinExec(

[GitHub] spark issue #19934: [SPARK-3685][CORE] Prints explicit warnings when configu...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19934 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19934: [SPARK-3685][CORE] Prints explicit warnings when configu...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19934 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84684/ Test PASSed. ---

[GitHub] spark issue #19934: [SPARK-3685][CORE] Prints explicit warnings when configu...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19934 **[Test build #84684 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84684/testReport)** for PR 19934 at commit

[GitHub] spark pull request #19937: [SPARK-22746][SQL] Avoid the generation of useles...

2017-12-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19937#discussion_r155926316 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -617,6 +619,7 @@ case class SortMergeJoinExec(

[GitHub] spark issue #19936: Branch 0.5

2017-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19936 @khanm002, could you close this please? If you are going to propose a change, please follow this linke - https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark. ---

[GitHub] spark issue #19931: [SPARK-22672][SQL][TEST][FOLLOWUP] Fix to use `spark.con...

2017-12-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19931 Oh, thank you so much, @HyukjinKwon . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19525: [SPARK-22289] [ML] Add JSON support for Matrix parameter...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19525 **[Test build #84686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84686/testReport)** for PR 19525 at commit

[GitHub] spark pull request #19937: [SPARK-22746][SQL] Avoid the generation of useles...

2017-12-09 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19937#discussion_r155925941 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -617,6 +619,7 @@ case class SortMergeJoinExec(

[GitHub] spark issue #19865: [SPARK-22668][SQL] Assert to ensure no global variables ...

2017-12-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19865 @cloud-fan unfortunately, #19878 did not fix this issue. #19937 will fix this issue. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19937: [SPARK-22746][SQL] Avoid the generation of useless mutab...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19937 **[Test build #84685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84685/testReport)** for PR 19937 at commit

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19527 Ok, I understood. In other words, the extra category is added as the last category and `dropLast` option works as before. It makes sense to me. ---

[GitHub] spark pull request #19937: [SPARK-22746][SQL] Avoid the generation of useles...

2017-12-09 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/19937 [SPARK-22746][SQL] Avoid the generation of useless mutable states by SortMergeJoin ## What changes were proposed in this pull request? This PR reduce the number of global mutable variables

[GitHub] spark pull request #19754: [BUILD] update release scripts

2017-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19754 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19754: [BUILD] update release scripts

2017-12-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19754 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19935: Branch 0.6

2017-12-09 Thread khanm002
Github user khanm002 closed the pull request at: https://github.com/apache/spark/pull/19935 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19933: [SPARK-22744][CORE] Add a configuration to show the appl...

2017-12-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19933 You're just using this as a write-only config to pass info, and it's for your own app-specific purposes. It's easier to just pass this info or log it in your app. ---

[GitHub] spark issue #19935: Branch 0.6

2017-12-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19935 @khanm002 close this --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19936: Branch 0.5

2017-12-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19936 @khanm002 close this --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19936: Branch 0.5

2017-12-09 Thread khanm002
Github user khanm002 commented on a diff in the pull request: https://github.com/apache/spark/pull/19936#discussion_r155924835 --- Diff: repl/src/main/scala/spark/repl/SparkILoop.scala --- @@ -200,7 +200,7 @@ class SparkILoop(in0: Option[BufferedReader], val out: PrintWriter, val

[GitHub] spark issue #19936: Branch 0.5

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19936 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19935: Branch 0.6

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19935 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19936: Branch 0.5

2017-12-09 Thread khanm002
GitHub user khanm002 opened a pull request: https://github.com/apache/spark/pull/19936 Branch 0.5 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this patch

[GitHub] spark pull request #19935: Branch 0.6

2017-12-09 Thread khanm002
GitHub user khanm002 opened a pull request: https://github.com/apache/spark/pull/19935 Branch 0.6 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this patch

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19594 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84683/ Test PASSed. ---

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19594 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19594 **[Test build #84683 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84683/testReport)** for PR 19594 at commit

[GitHub] spark issue #19934: [SPARK-3685][CORE] Prints explicit warnings when configu...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19934 **[Test build #84684 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84684/testReport)** for PR 19934 at commit

[GitHub] spark pull request #19934: [SPARK-3685][CORE] Prints explicit warnings when ...

2017-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19934#discussion_r155922544 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -829,7 +829,18 @@ private[spark] object Utils extends Logging { }

[GitHub] spark pull request #19934: [SPARK-3685][CORE] Prints explicit warnings when ...

2017-12-09 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/19934 [SPARK-3685][CORE] Prints explicit warnings when configured local directories are set to URIs ## What changes were proposed in this pull request? This PR proposes to print warnings

[GitHub] spark issue #19933: [SPARK-22744][CORE] Add a configuration to show the appl...

2017-12-09 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/19933 How about change it to system property and it can show in Spark UI. It really can help user (platform team) to trace, control and support application. ---

[GitHub] spark pull request #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19932#discussion_r155921370 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -413,32 +413,7 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19594 **[Test build #84683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84683/testReport)** for PR 19594 at commit

[GitHub] spark issue #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19932 cc @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19594 retest this please.. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19594 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84682/ Test FAILed. ---

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19594 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19594 **[Test build #84682 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84682/testReport)** for PR 19594 at commit

[GitHub] spark pull request #19931: [SPARK-22672][SQL][TEST][FOLLOWUP] Fix to use `sp...

2017-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19931 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19931: [SPARK-22672][SQL][TEST][FOLLOWUP] Fix to use `spark.con...

2017-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19931 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19931: [SPARK-22672][SQL][TEST][FOLLOWUP] Fix to use `spark.con...

2017-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19931 @dongjoon-hyun mind double checking if there is anything we should do as a followup before we go merging this? --- - To

[GitHub] spark issue #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19932 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84680/ Test PASSed. ---

[GitHub] spark issue #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19932 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19932 **[Test build #84680 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84680/testReport)** for PR 19932 at commit

[GitHub] spark issue #19933: [SPARK-22744][CORE] Add a configuration to show the appl...

2017-12-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19933 Your app can already log what it likes. This isn't a config, just an attempt to pass a piece of info. I think this should be closed. ---

[GitHub] spark issue #19676: [SPARK-14516][FOLLOWUP] Adding ClusteringEvaluator to ex...

2017-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19676 **[Test build #84681 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84681/testReport)** for PR 19676 at commit

[GitHub] spark issue #19676: [SPARK-14516][FOLLOWUP] Adding ClusteringEvaluator to ex...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19676 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84681/ Test PASSed. ---

[GitHub] spark issue #19676: [SPARK-14516][FOLLOWUP] Adding ClusteringEvaluator to ex...

2017-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19676 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   >