[GitHub] spark issue #19259: [BACKPORT-2.1][SPARK-19318][SPARK-22041][SQL] Docker tes...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19259 LGTM pending test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17819 **[Test build #81867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81867/testReport)** for PR 17819 at commit

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17819 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17819 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17819 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81864/ Test FAILed. ---

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17819 **[Test build #81864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81864/testReport)** for PR 17819 at commit

[GitHub] spark issue #19259: [BACKPORT-2.1][SPARK-19318][SPARK-22041][SQL] Docker tes...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19259 **[Test build #81866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81866/testReport)** for PR 19259 at commit

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12646 **[Test build #81865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81865/testReport)** for PR 12646 at commit

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-17 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/19256 Thanks @gatorsmile Does the following logic look good to you? ``` if(any dialect's isCascadingTruncateTable returns true) return Some(true) else if (any dialect's

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r139337054 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/AggregateFieldExtractionPushdown.scala --- @@ -0,0 +1,77 @@ +/* + *

[GitHub] spark issue #19234: [SPARK-22010][PySpark] Change fromInternal method of Tim...

2017-09-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19234 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r139336399 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/JoinFieldExtractionPushdown.scala --- @@ -0,0 +1,66 @@ +/* + *

[GitHub] spark pull request #19260: [SPARK-22043][PYTHON] Improves error message for ...

2017-09-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19260 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19260: [SPARK-22043][PYTHON] Improves error message for show_pr...

2017-09-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19260 Merged to master, branch-2.2 and branch-2.1. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19260: [SPARK-22043][PYTHON] Improves error message for show_pr...

2017-09-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19260 Thanks @maver1ck, @felixcheung and @viirya. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19249: [SPARK-22032][PySpark] Speed up StructType conversion

2017-09-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19249 Thanks for double checking @ueshin. Yes, I noticed that too while reviewing it. I just decided to merge it as is because I am quite sure of this one given struct type is the root type

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15544 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81863/ Test FAILed. ---

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15544 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15544 **[Test build #81863 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81863/testReport)** for PR 15544 at commit

[GitHub] spark issue #19249: [SPARK-22032][PySpark] Speed up StructType conversion

2017-09-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19249 A late LGTM. Btw, can we use the same idea for `MapType`? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17819 **[Test build #81864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81864/testReport)** for PR 17819 at commit

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19230 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15544 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81862/ Test FAILed. ---

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15544 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15544 **[Test build #81862 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81862/testReport)** for PR 15544 at commit

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19230 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81861/ Test FAILed. ---

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19230 **[Test build #81861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81861/testReport)** for PR 19230 at commit

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19230 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19259: [BACKPORT-2.1][SPARK-19318][SPARK-22041][SQL] Doc...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19259#discussion_r139333133 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -913,6 +913,25 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r139333125 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -0,0 +1,130 @@ +/* + *

[GitHub] spark pull request #19259: [BACKPORT-2.1][SPARK-19318][SPARK-22041][SQL] Doc...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19259#discussion_r139333048 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -63,15 +63,40 @@ class

[GitHub] spark pull request #19259: [BACKPORT-2.1][SPARK-19318][SPARK-22041][SQL] Doc...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19259#discussion_r139333045 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -82,7 +107,7 @@ class

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19196 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19196 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81860/ Test FAILed. ---

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19196 **[Test build #81860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81860/testReport)** for PR 19196 at commit

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r139331293 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -63,9 +74,22 @@ private[parquet] class

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r139331198 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -63,9 +74,22 @@ private[parquet] class

[GitHub] spark pull request #19164: [SPARK-21953] Show both memory and disk bytes spi...

2017-09-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19164 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-17 Thread aoxiangcao
Github user aoxiangcao commented on the issue: https://github.com/apache/spark/pull/19221 i'm sorry if it right to ask here .i have some question when use 'insert overwrite directory /user/appUser/test'. first, i start the thrift by 'hdfs', and login into beeline with user 'appUser'.

[GitHub] spark issue #19164: [SPARK-21953] Show both memory and disk bytes spilled if...

2017-09-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19164 LGTM, merging to master/2.2/2.1! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15544 **[Test build #81863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81863/testReport)** for PR 15544 at commit

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15544 **[Test build #81862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81862/testReport)** for PR 15544 at commit

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19230 **[Test build #81861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81861/testReport)** for PR 19230 at commit

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19230 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19230: [SPARK-22003][SQL] support array column in vector...

2017-09-17 Thread liufengdb
Github user liufengdb commented on a diff in the pull request: https://github.com/apache/spark/pull/19230#discussion_r139329522 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java --- @@ -16,6 +16,7 @@ */ package

[GitHub] spark pull request #19230: [SPARK-22003][SQL] support array column in vector...

2017-09-17 Thread liufengdb
Github user liufengdb commented on a diff in the pull request: https://github.com/apache/spark/pull/19230#discussion_r139329523 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnVectorSuite.scala --- @@ -0,0 +1,201 @@ +/* + * Licensed to

[GitHub] spark issue #19260: [SPARK-22043][PYTHON] Improves error message for show_pr...

2017-09-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19260 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19196 **[Test build #81860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81860/testReport)** for PR 19196 at commit

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2017-09-17 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/15544#discussion_r139327658 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproxCountDistinctForIntervals.scala --- @@ -0,0 +1,248 @@

[GitHub] spark issue #19252: [SPARK-21969][SQL] CommandUtils.updateTableStats should ...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19252 This is not a bug. We just follow [the behavior of Hive's dynamic partition insert](https://docs.databricks.com/spark/latest/spark-sql/language-manual/insert.html#dynamic-partition-inserts).

[GitHub] spark pull request #19258: add MockNetCat

2017-09-17 Thread bluejoe2008
Github user bluejoe2008 closed the pull request at: https://github.com/apache/spark/pull/19258 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19230: [SPARK-22003][SQL] support array column in vector...

2017-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19230#discussion_r139324826 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnVectorSuite.scala --- @@ -0,0 +1,201 @@ +/* + * Licensed to the

[GitHub] spark issue #19259: [BACKPORT-2.1][SPARK-19318][SPARK-22041][SQL] Docker tes...

2017-09-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19259 @gatorsmile Yes, Docker integration tests passed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19263: Optionally add block updates to log

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19263 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19263: Optionally add block updates to log

2017-09-17 Thread michaelmior
GitHub user michaelmior opened a pull request: https://github.com/apache/spark/pull/19263 Optionally add block updates to log I see that block updates are not logged to the event log. This makes sense as a default for performance reasons. However, I find it helpful when

[GitHub] spark issue #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2017-09-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19218 Sorry, guys. I've been away from keyboard since last Friday night. I'll be back on next Tuesday (PST). --- - To

[GitHub] spark issue #19252: [SPARK-21969][SQL] CommandUtils.updateTableStats should ...

2017-09-17 Thread aokolnychyi
Github user aokolnychyi commented on the issue: https://github.com/apache/spark/pull/19252 @gatorsmile thanks for the feedback. I also covered ``TruncateTableCommand`` with additional tests. However, I see a bit strange behavior while creating a test for

[GitHub] spark issue #19243: [SPARK-21780][R] Simpler Dataset.sample API in R

2017-09-17 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19243 thinking about this, I wonder if it is more common in R to skip param with default values and the rest of param by names, like `sample(df, fraction=1.0)` ---

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19256 Let me correct what I said above. The logics should be > If any dialect's `isCascadingTruncateTable` returns `true`, we should return `true`. ---

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-17 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/19256 Thanks @gatorsmile I will change both the implementation and the PR title. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19256 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81859/ Test FAILed. ---

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19256 **[Test build #81859 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81859/testReport)** for PR 19256 at commit

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19256 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19252: [SPARK-21969][SQL] CommandUtils.updateTableStats should ...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19252 Actually, the right fix should add `refreshTable(identifier)` to the SessionCatalog's [alterTableStats

[GitHub] spark pull request #19252: [SPARK-21969][SQL] CommandUtils.updateTableStats ...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19252#discussion_r139317292 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala --- @@ -44,6 +44,7 @@ object CommandUtils extends Logging {

[GitHub] spark issue #19260: [SPARK-22043][PYTHON] Improves error message for show_pr...

2017-09-17 Thread maver1ck
Github user maver1ck commented on the issue: https://github.com/apache/spark/pull/19260 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19252: [SPARK-21969][SQL] CommandUtils.updateTableStats should ...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19252 `TruncateTableCommand` also has similar issues. Could you also fix it in this PR? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19260: [SPARK-22043][PYTHON] Improves error message for show_pr...

2017-09-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19260 cc @maver1ck and @viirya who I believe recently ran Python profile. Could you take a look please when you have some time? ---

[GitHub] spark issue #19226: [SPARK-21985][PySpark] PairDeserializer is broken for do...

2017-09-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19226 Merged to master, branch-2.2 and branch-2.1. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19226: [SPARK-21985][PySpark] PairDeserializer is broken...

2017-09-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19226 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19249: [SPARK-22032][PySpark] Speed up StructType conver...

2017-09-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19249 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19249: [SPARK-22032][PySpark] Speed up StructType conversion

2017-09-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19249 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19230: [SPARK-22003][SQL] support array column in vector...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19230#discussion_r139316092 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java --- @@ -16,6 +16,7 @@ */ package

[GitHub] spark issue #19259: [BACKPORT-2.1][SPARK-19318][SPARK-22041][SQL] Docker tes...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19259 Have you run the docker test? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19261: [SPARK-22040] Add current_date function with timezone id

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19261 Any other database has such an interface? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19256 BTW, could you update the PR title? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19256: [SPARK-21338][SQL]implement isCascadingTruncateTa...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19256#discussion_r139315660 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala --- @@ -41,4 +41,8 @@ private class AggregatedDialect(dialects:

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19256 **[Test build #81859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81859/testReport)** for PR 19256 at commit

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19256 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19204 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81858/ Test PASSed. ---

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19204 **[Test build #81858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81858/testReport)** for PR 19204 at commit

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19204 **[Test build #81858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81858/testReport)** for PR 19204 at commit

[GitHub] spark issue #19226: [SPARK-21985][PySpark] PairDeserializer is broken for do...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19226: [SPARK-21985][PySpark] PairDeserializer is broken for do...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81856/ Test PASSed. ---

[GitHub] spark issue #19226: [SPARK-21985][PySpark] PairDeserializer is broken for do...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19226 **[Test build #81856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81856/testReport)** for PR 19226 at commit

[GitHub] spark issue #19262: [MINOR][ML] Remove unnecessary default value setting for...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19262 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19262: [MINOR][ML] Remove unnecessary default value setting for...

2017-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19262 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81857/ Test PASSed. ---

[GitHub] spark issue #19262: [MINOR][ML] Remove unnecessary default value setting for...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19262 **[Test build #81857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81857/testReport)** for PR 19262 at commit

[GitHub] spark pull request #19256: [SPARK-21338][SQL]implement isCascadingTruncateTa...

2017-09-17 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19256#discussion_r139313120 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala --- @@ -41,4 +41,8 @@ private class AggregatedDialect(dialects:

[GitHub] spark pull request #18193: [SPARK-15616] [SQL] CatalogRelation should fallba...

2017-09-17 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/18193#discussion_r139312866 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -139,6 +138,54 @@ class DetermineTableStats(session: SparkSession)

[GitHub] spark issue #19262: [MINOR][ML] Remove unnecessary default value setting for...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19262 **[Test build #81857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81857/testReport)** for PR 19262 at commit

[GitHub] spark pull request #19204: [SPARK-21981][PYTHON][ML] Added Python interface ...

2017-09-17 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19204#discussion_r139312695 --- Diff: python/pyspark/ml/evaluation.py --- @@ -328,6 +329,86 @@ def setParams(self, predictionCol="prediction", labelCol="label", kwargs

[GitHub] spark issue #19226: [SPARK-21985][PySpark] PairDeserializer is broken for do...

2017-09-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19226 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r139312657 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala --- @@ -77,20 +77,21 @@ trait

[GitHub] spark issue #19226: [SPARK-21985][PySpark] PairDeserializer is broken for do...

2017-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19226 **[Test build #81856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81856/testReport)** for PR 19226 at commit

[GitHub] spark pull request #19262: [MINOR][ML] Remove unnecessary default value sett...

2017-09-17 Thread yanboliang
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/19262 [MINOR][ML] Remove unnecessary default value setting for evaluators. ## What changes were proposed in this pull request? Remove unnecessary default value setting for all evaluators, as we

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-17 Thread vkhristenko
Github user vkhristenko commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r139312613 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -0,0 +1,130 @@ +/* + *

[GitHub] spark pull request #19204: [SPARK-21981][PYTHON][ML] Added Python interface ...

2017-09-17 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19204#discussion_r139312388 --- Diff: python/pyspark/ml/evaluation.py --- @@ -328,6 +329,86 @@ def setParams(self, predictionCol="prediction", labelCol="label", kwargs

[GitHub] spark pull request #19204: [SPARK-21981][PYTHON][ML] Added Python interface ...

2017-09-17 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19204#discussion_r139312199 --- Diff: python/pyspark/ml/evaluation.py --- @@ -328,6 +329,86 @@ def setParams(self, predictionCol="prediction", labelCol="label", kwargs

  1   2   >