[GitHub] spark pull request #22647: [SPARK-25655] [BUILD] Add Pspark-ganglia-lgpl to ...

2018-10-05 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22647 [SPARK-25655] [BUILD] Add Pspark-ganglia-lgpl to the scala style check. ## What changes were proposed in this pull request? Our lint failed due to the following errors: ``` [INFO

[GitHub] spark pull request #22622: [SPARK-25635][SQL][BUILD] Support selective direc...

2018-10-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22622#discussion_r222911645 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -115,6 +116,69 @@ abstract class OrcSuite

[GitHub] spark pull request #22622: [SPARK-25635][SQL][BUILD] Support selective direc...

2018-10-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22622#discussion_r222911529 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -115,6 +116,69 @@ abstract class OrcSuite

[GitHub] spark issue #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22399 It sounds like the build is broken after merging this PR --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22399 https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/job/spark-master-compile-maven-hadoop-2.6/8432

[GitHub] spark issue #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22399 https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/job/spark-master-compile-maven-hadoop-2.7/8521

[GitHub] spark pull request #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22605#discussion_r222877184 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/BloomFilterBenchmark.scala --- @@ -0,0 +1,87 @@ +/* + * Licensed

[GitHub] spark pull request #22622: [SPARK-25635][SQL][BUILD] Support selective direc...

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22622#discussion_r222876905 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -115,6 +116,69 @@ abstract class OrcSuite

[GitHub] spark pull request #22594: [MINOR][SQL] When batch reading, the number of by...

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22594#discussion_r222875551 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala --- @@ -104,12 +104,14 @@ class FileScanRDD

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r222874417 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CsvExpressionsSuite.scala --- @@ -0,0 +1,145

[GitHub] spark pull request #22622: [SPARK-25635][SQL][BUILD] Support selective direc...

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22622#discussion_r222855990 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -115,6 +116,69 @@ abstract class OrcSuite

[GitHub] spark pull request #22466: [SPARK-25464][SQL]On dropping the Database it wil...

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22466#discussion_r222855301 --- Diff: python/pyspark/sql/tests.py --- @@ -351,7 +351,7 @@ def tearDown(self): super(SQLTests, self).tearDown() # tear

[GitHub] spark pull request #22466: [SPARK-25464][SQL]On dropping the Database it wil...

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22466#discussion_r222855237 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -407,6 +407,7 @@ abstract class DDLSuite extends

[GitHub] spark issue #22466: [SPARK-25464][SQL]On dropping the Database it will drop ...

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22466 @sandeep-katta Can you update the PR title? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r222777373 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r222776820 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r222773963 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r222774540 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r222774199 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r222772770 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CsvExpressionsSuite.scala --- @@ -0,0 +1,145

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r222771917 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed

[GitHub] spark issue #22621: [SPARK-25602][SQL] SparkPlan.getByteArrayRdd should not ...

2018-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22621 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-10-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21688 cc @dbkerkela @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22620: [SPARK-25601][PYTHON] Register Grouped aggregate ...

2018-10-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22620#discussion_r222487117 --- Diff: python/pyspark/sql/udf.py --- @@ -310,9 +319,11 @@ def register(self, name, f, returnType=None): "Invalid retur

[GitHub] spark pull request #22480: [SPARK-25473][PYTHON][SS][TEST] ForeachWriter tes...

2018-10-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22480#discussion_r222486276 --- Diff: python/pyspark/sql/tests.py --- @@ -1962,6 +1962,9 @@ def __getstate__(self): def __setstate__(self, state

[GitHub] spark pull request #22620: [SPARK-25601][PYTHON] Register Grouped aggregate ...

2018-10-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22620#discussion_r222452544 --- Diff: python/pyspark/sql/udf.py --- @@ -310,9 +319,11 @@ def register(self, name, f, returnType=None): "Invalid retur

[GitHub] spark issue #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilter shou...

2018-10-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22614 Yes. Let us add a conf for controlling the fallback. Please also add the test cases for verifying it. Thanks

[GitHub] spark issue #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilter shou...

2018-10-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22614 Let us add a conf to control it? Failing fast is better than hanging. If users want to get all partitions, they can change the conf by themselves

[GitHub] spark issue #22619: [SQL][MINOR] Make use of TypeCoercion.findTightestCommon...

2018-10-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22619 Any behavior change? Test cases? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222190138 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #22047: [SPARK-19851] Add support for EVERY and ANY (SOME) aggre...

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22047 Let me post something I wrote recently. Could you add test cases to ensure that we do not break the "Ignore NULLs" policy > All the set/aggregate functions ignore NULLs

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222134301 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222122774 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222122400 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22606 Yes. Let us wait for the release of 2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22606 @dongjoon-hyun Could you help me fix the website? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22606 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

2018-10-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22606 cc @yhuai @rxin @cloud-fan @srowen @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

2018-10-01 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22606 [SPARK-25592] Setting version to 3.0.0-SNAPSHOT ## What changes were proposed in this pull request? This patch is to bump the master branch version to 3.0.0-SNAPSHOT. ## How

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

2018-10-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22602 @mgaido91 Could you change the title to [WIP] before you add the test case? Also cc @hvanhovell @kiszk who are the best person to review these code

[GitHub] spark issue #22461: [SPARK-25453][SQL][TEST] OracleIntegrationSuite IllegalA...

2018-09-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22461 Thanks! Merged to master/2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22573: [SPARK-25558][SQL] Pushdown predicates for nested fields...

2018-09-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22573 Updating `Filter` APIs sounds reasonable to me. This should be part of our data source API v2. cc @cloud-fan @rxin @rdblue

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-09-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21221 @mccheah When you merged the code, could you also leave the comments about which branches you did the merge

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-09-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21221 For the other reviewers, this was merged to master (not 2.4) --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r221449056 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -520,7 +520,10 @@ object FunctionRegistry

[GitHub] spark issue #22586: [SPARK-25568][Core]Continue to update the remaining accu...

2018-09-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22586 Thanks! Merged to master/2.4/2.3/2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22528 We should stop returning a wrong result. Please fix it. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22586: [SPARK-25568][Core]Continue to update the remaining accu...

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22586 This was introduced by AccumulatorV2. It might be a blocker issue for Spark 2.4, since this could return an incorrect result

[GitHub] spark issue #22586: [SPARK-25568][Core]Continue to update the remaining accu...

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22586 LGTM cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22586: [SPARK-25568][Core]Continue to update the remaini...

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22586#discussion_r221415820 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -1880,6 +1880,26 @@ class DAGSchedulerSuite extends

[GitHub] spark issue #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANALYZE TA...

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22566 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22420: [SPARK-25429][SQL]Use Set improve SparkListenerBus effic...

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22420 @wangyum Could you take it over and submit a new PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22060: [DO NOT MERGE][TEST ONLY] Add once-policy rule check

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22060 @maropu Are you willing to take this over? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22519: [SPARK-25505][SQL] The output order of grouping columns ...

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22519 Thanks! Merged to 2.4 and master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANALYZE TA...

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22566 LGTM except a few minor comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r221158906 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -653,6 +653,49 @@ class StatisticsSuite extends

[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r221158556 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -48,9 +52,13 @@ case class

[GitHub] spark issue #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANALYZE TA...

2018-09-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22566 @dilipbiswal Thanks for working on this! Also cc @juliuszsompolski --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r220979068 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -33,11 +33,13 @@ import

[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r220978815 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -50,7 +52,26 @@ case class

[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r220978327 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -70,25 +91,9 @@ case class

[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r220978087 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -50,7 +52,26 @@ case class

[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r220977575 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -50,7 +52,26 @@ case class

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-09-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r220973430 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -1245,3 +1245,27 @@ case class BRound

[GitHub] spark issue #22560: [SPARK-25547][Spark Core] Pluggable JDBC connection fact...

2018-09-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22560 Could you please post the example you deleted in the PR description? If we merge it, it can be part of our commit message. We can easily understand the context of this PR. Let me think

[GitHub] spark issue #22361: Revert [SPARK-10399] [SPARK-23879] [SPARK-23762] [SPARK-...

2018-09-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22361 Nico will give a talk in the upcoming spark summit: https://databricks.com/session/a-framework-for-evaluating-the-performance-and-the-correctness-of-the-spark-sql-engine If possible

[GitHub] spark pull request #22466: [SPARK-25464][SQL]On dropping the Database it wil...

2018-09-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22466#discussion_r220795973 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -66,6 +66,19 @@ case class CreateDatabaseCommand

[GitHub] spark pull request #22466: [SPARK-25464][SQL]On dropping the Database it wil...

2018-09-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22466#discussion_r220795446 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -66,6 +66,19 @@ case class CreateDatabaseCommand

[GitHub] spark pull request #22466: [SPARK-25464][SQL]On dropping the Database it wil...

2018-09-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22466#discussion_r220795176 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -66,6 +66,19 @@ case class CreateDatabaseCommand

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-09-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22198 cc @maryannxue --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22528 Generally, this is useful for reading the zip archives containing a single CSV. The extra support JSON file is not very useful. The major limitation is we have to read the zipped CSV

[GitHub] spark issue #22545: [SPARK-25525][SQL][PYSPARK] Do not update conf for exist...

2018-09-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22545 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22540: [SPARK-24324] [PYTHON] [FOLLOW-UP] Rename the Con...

2018-09-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22540#discussion_r220363870 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala --- @@ -131,11 +131,8 @@ object ArrowUtils { } else

[GitHub] spark issue #21816: [SPARK-24794][CORE] Driver launched through rest should ...

2018-09-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21816 cc @srinathshankar @ericl --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22540: [SPARK-24324] [PYTHON] [FOLLOW-UP] Rename the Con...

2018-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22540#discussion_r220065589 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala --- @@ -131,11 +131,8 @@ object ArrowUtils { } else

[GitHub] spark pull request #22540: [SPARK-24324] [PYTHON] [FOLLOW-UP] Rename the Con...

2018-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22540#discussion_r220065308 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1295,15 +1295,15 @@ object SQLConf { .booleanConf

[GitHub] spark issue #22455: [SPARK-24572][SPARKR] "eager execution" for R shell, IDE

2018-09-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22455 Great feature! cc @falaki --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22540: [SPARK-24324] [PYTHON] [FOLLOW-UP] Add legacy pre...

2018-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22540#discussion_r219973059 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala --- @@ -131,11 +131,8 @@ object ArrowUtils { } else

[GitHub] spark pull request #22540: [SPARK-24324] [PYTHON] [FOLLOW-UP] Add legacy pre...

2018-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22540#discussion_r219972870 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1915,8 +1915,8 @@ class SQLConf extends Serializable

[GitHub] spark pull request #22540: [SPARK-24324] [PYTHON] [FOLLOW-UP] Add legacy pre...

2018-09-24 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22540 [SPARK-24324] [PYTHON] [FOLLOW-UP] Add legacy prefix in spark.sql.execution.pandas.groupedMap.assignColumnsByPosition ## What changes were proposed in this pull request? Add the legacy

[GitHub] spark pull request #22366: [SPARK-25384][SQL] Removing of spark.sql.fromJson...

2018-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22366#discussion_r219943808 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -517,12 +517,10 @@ case class

[GitHub] spark issue #22511: [SPARK-25422][CORE] Don't memory map blocks streamed to ...

2018-09-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22511 Also cc @zsxwing @JoshRosen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22429: [SPARK-25440][SQL] Dumping query execution info t...

2018-09-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22429#discussion_r219729921 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala --- @@ -250,5 +265,22 @@ class QueryExecution(val sparkSession

[GitHub] spark pull request #22429: [SPARK-25440][SQL] Dumping query execution info t...

2018-09-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22429#discussion_r219729889 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala --- @@ -250,5 +265,22 @@ class QueryExecution(val sparkSession

[GitHub] spark pull request #22429: [SPARK-25440][SQL] Dumping query execution info t...

2018-09-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22429#discussion_r219729210 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -668,11 +670,19 @@ case class WholeStageCodegenExec

[GitHub] spark pull request #22458: [SPARK-25459] Add viewOriginalText back to Catalo...

2018-09-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22458#discussion_r219727730 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -2348,4 +2348,17 @@ class HiveDDLSuite

[GitHub] spark issue #21999: [WIP][SQL] Flattening nested structures

2018-09-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21999 Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22523: [MINOR] Always Close the tempFile in _serialize_to_jvm

2018-09-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22523 cc @ueshin @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22523: [MINOR] Always Close the tempFile in _serialize_t...

2018-09-21 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22523 [MINOR] Always Close the tempFile in _serialize_to_jvm ## What changes were proposed in this pull request? Always close the tempFile after `serializer.dump_stream(data, tempFile

[GitHub] spark pull request #22461: [SPARK-25453][SQL][TEST] OracleIntegrationSuite I...

2018-09-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22461#discussion_r219546151 --- Diff: docs/sql-programming-guide.md --- @@ -1489,7 +1489,7 @@ See the [Apache Avro Data Source Guide](avro-data-source-guide.html

[GitHub] spark pull request #22461: [SPARK-25453][SQL][TEST] OracleIntegrationSuite I...

2018-09-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22461#discussion_r219542061 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -462,6 +464,9 @@ class

[GitHub] spark pull request #22461: [SPARK-25453][SQL][TEST] OracleIntegrationSuite I...

2018-09-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22461#discussion_r219392779 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -462,6 +464,9 @@ class

[GitHub] spark issue #22506: [SPARK-25494][SQL] Upgrade Spark's use of Janino to 3.0....

2018-09-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22506 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22461: [SPARK-25453][SQL][TEST] OracleIntegrationSuite I...

2018-09-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22461#discussion_r219359831 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -442,6 +442,8 @@ class

[GitHub] spark pull request #22461: [SPARK-25453][SQL][TEST] OracleIntegrationSuite I...

2018-09-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22461#discussion_r219359638 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -442,6 +442,8 @@ class

[GitHub] spark pull request #22467: [SPARK-25465][TEST] Refactor Parquet test suites ...

2018-09-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22467#discussion_r219356330 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSourceSuite.scala --- @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #22505: Revert "[SPARK-23715][SQL] the input of to/from_utc_time...

2018-09-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22505 cc @rxin @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22505: Revert "[SPARK-23715][SQL] the input of to/from_u...

2018-09-20 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22505 Revert "[SPARK-23715][SQL] the input of to/from_utc_timestamp can not have timezone ## What changes were proposed in this pull request? This reverts c

[GitHub] spark issue #22458: [SPARK-25459] Add viewOriginalText back to CatalogTable

2018-09-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22458 This is reasonable to me. cc @jiangxb1987 @cloud-fan @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22458: [SPARK-25459] Add viewOriginalText back to CatalogTable

2018-09-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22458 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

<    1   2   3   4   5   6   7   8   9   10   >