[GitHub] spark issue #23005: [SPARK-26005] [SQL] Upgrade ANTRL from 4.7 to 4.7.1

2018-11-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/23005 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22962: [SPARK-25921][PySpark] Fix barrier task run without Barr...

2018-11-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22962 cc @jiangxb1987 @MrBago --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #23005: [SPARK-26005] [SQL] Upgrade ANTRL from 4.7 to 4.7...

2018-11-11 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/23005 [SPARK-26005] [SQL] Upgrade ANTRL from 4.7 to 4.7.1 ## What changes were proposed in this pull request? Based on the release description of ANTRL 4.7.1., https://github.com/antlr/antlr4

[GitHub] spark issue #21766: [SPARK-24803][SQL] add support for numeric

2018-11-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21766 @wangtao605 Do you mind documenting our behavior in our Spark SQL doc? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #17648: [SPARK-19851] Add support for EVERY and ANY (SOME) aggre...

2018-11-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17648 @ptkool Thanks for your contribution! This feature will be available in the next release. Spark 3.0 --- - To unsubscribe, e

[GitHub] spark issue #21679: [SPARK-24695] [SQL]: To add support to return Calendar i...

2018-11-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21679 For your information, @cloud-fan and I are discussing how we expose calendar interval data types. Will post the proposal later

[GitHub] spark issue #22932: [SPARK-25102][SQL] Write Spark version to ORC/Parquet fi...

2018-11-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22932 LGTM. Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22932: [SPARK-25102][SQL] Write Spark version to ORC/Par...

2018-11-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22932#discussion_r232443802 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcOutputWriter.scala --- @@ -36,11 +37,17 @@ private[orc] class

[GitHub] spark pull request #22759: [MINOR][SQL][DOC] Correct parquet nullability doc...

2018-11-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22759#discussion_r232441937 --- Diff: docs/sql-programming-guide.md --- @@ -706,7 +706,7 @@ data across a fixed number of buckets and can be used when a number of unique va

[GitHub] spark issue #22759: [MINOR][SQL][DOC] Correct parquet nullability documentat...

2018-11-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22759 LGTM Could you do us a favor to add the test cases for ensuring that the generated parquet files have a correct nullability value

[GitHub] spark pull request #22932: [SPARK-25102][SQL] Write Spark version to ORC/Par...

2018-11-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22932#discussion_r232424657 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -274,6 +278,15 @@ private[orc] class OrcOutputWriter

[GitHub] spark pull request #22932: [SPARK-25102][SQL] Write Spark version to ORC/Par...

2018-11-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22932#discussion_r232424626 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcOutputWriter.scala --- @@ -36,11 +41,17 @@ private[orc] class

[GitHub] spark issue #22987: [SPARK-25979][SQL] Window function: allow parentheses ar...

2018-11-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22987 LGTM Thanks! Merged to master. I accidentally merged it to 2.4. Feel free to revert it if anybody has a concern about this fix

[GitHub] spark issue #22990: [SPARK-25988] [SQL] Keep names unchanged when deduplicat...

2018-11-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22990 Thanks! Merged to master/2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22990: [SPARK-25988] [SQL] Keep names unchanged when deduplicat...

2018-11-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22990 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22990: [SPARK-25988] [SQL] Keep names unchanged when ded...

2018-11-08 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22990 [SPARK-25988] [SQL] Keep names unchanged when deduplicating the column names in Analyzer ## What changes were proposed in this pull request? When the queries do not use the column names

[GitHub] spark issue #22987: [SPARK-25979][SQL] Window function: allow parentheses ar...

2018-11-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22987 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22987: [SPARK-25979][SQL] Window function: allow parenth...

2018-11-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22987#discussion_r232136609 --- Diff: sql/core/src/test/resources/sql-tests/inputs/window.sql --- @@ -109,3 +109,9 @@ last_value(false, false) OVER w AS last_value_contain_null

[GitHub] spark pull request #22969: [SPARK-22827][SQL][FOLLOW-UP] Throw `SparkOutOfMe...

2018-11-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22969#discussion_r232112326 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -787,7 +789,7 @@ case class HashAggregateExec

[GitHub] spark pull request #21602: [SPARK-24613][SQL] Cache with UDF could not be ma...

2018-11-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21602#discussion_r232107563 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -142,7 +142,7 @@ class CacheManager extends Logging

[GitHub] spark issue #22949: [minor] update known_translations

2018-11-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22949 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22947: [SPARK-24913][SQL] Make AssertNotNull and AssertT...

2018-11-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22947#discussion_r231230211 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -66,6 +66,8 @@ case class AssertTrue(child

[GitHub] spark issue #21596: [SPARK-24601] Update Jackson to 2.9.6

2018-11-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21596 It sounds like Jackson have an important behavior change that might impact us. ```Scala import com.fasterxml.jackson.annotation.JsonInclude.Include import

[GitHub] spark pull request #22949: [minor] update known_translations

2018-11-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22949#discussion_r230858019 --- Diff: dev/create-release/known_translations --- @@ -203,3 +203,61 @@ shenh062326 - Shen Hong aokolnychyi - Anton Okolnychyi linbojin

[GitHub] spark pull request #22949: [minor] update known_translations

2018-11-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22949#discussion_r230856955 --- Diff: dev/create-release/known_translations --- @@ -203,3 +203,61 @@ shenh062326 - Shen Hong aokolnychyi - Anton Okolnychyi linbojin

[GitHub] spark issue #22928: [SPARK-25926][CORE] Move config entries in core module t...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22928 Keeping them in separate source files is also fine to me. I think we should put them in the same package

[GitHub] spark pull request #22889: [SPARK-25882][SQL] Added a function to join two d...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22889#discussion_r230614726 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -883,6 +883,31 @@ class Dataset[T] private[sql]( join(right, Seq

[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22913 Also cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #22693: [SPARK-25701][SQL] Supports calculation of table ...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22693#discussion_r230609824 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -115,26 +116,45 @@ class ResolveHiveSerdeTable(session

[GitHub] spark pull request #19796: [SPARK-22581][SQL] Catalog api does not allow to ...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19796#discussion_r230609716 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala --- @@ -411,7 +410,29 @@ abstract class Catalog { tableName

[GitHub] spark issue #22089: [SPARK-25098][SQL]‘Cast’ will return NULL when input...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22089 @wangyum Could you please take it over? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22089: [SPARK-25098][SQL]‘Cast’ will return NULL whe...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22089#discussion_r230609486 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala --- @@ -98,6 +98,7 @@ class CastSuite extends

[GitHub] spark issue #22941: [SPARK-25936][SQL] Fix InsertIntoDataSourceCommand does ...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22941 I think this is not a bug, although the plan is confusing. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22930: [SPARK-24869][SQL] Fix SaveIntoDataSourceCommand'...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22930#discussion_r230609078 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/SaveIntoDataSourceCommand.scala --- @@ -37,13 +37,12 @@ case class

[GitHub] spark pull request #22941: [SPARK-25936][SQL] Fix InsertIntoDataSourceComman...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22941#discussion_r230609046 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala --- @@ -589,4 +590,33 @@ class InsertSuite extends DataSourceTest

[GitHub] spark pull request #22941: [SPARK-25936][SQL] Fix InsertIntoDataSourceComman...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22941#discussion_r230608937 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoDataSourceCommand.scala --- @@ -30,14 +30,13 @@ import

[GitHub] spark issue #22936: [SPARK-19799] Support WITH clause (CTE) in subqueries

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22936 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22932: [SPARK-25102][SQL] Write Spark version to ORC/Parquet fi...

2018-11-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22932 Will take a look this week. Thanks for your work! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22932: [SPARK-25102][SQL] Write Spark version to ORC/Par...

2018-11-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22932#discussion_r230563752 --- Diff: sql/core/src/test/resources/sql-tests/results/describe-part-after-analyze.sql.out --- @@ -93,7 +93,7 @@ Partition Values [ds=2017-08

[GitHub] spark issue #22255: [SPARK-25102][Spark Core] Write Spark version informatio...

2018-11-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22255 Just to confirm it. `created_by` is set to `parquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a

[GitHub] spark issue #22921: [SPARK-25908][CORE][SQL] Remove old deprecated items in ...

2018-11-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22921 LGTM except a minor comment about the test case. Also we need to fix the PySpark test failure --- - To unsubscribe, e-mail

[GitHub] spark pull request #22921: [SPARK-25908][CORE][SQL] Remove old deprecated it...

2018-11-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22921#discussion_r230283948 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -279,27 +254,6 @@ class DataFrameSuite extends QueryTest

[GitHub] spark issue #22255: [SPARK-25102][Spark Core] Write Spark version informatio...

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22255 Also cc @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22255: [SPARK-25102][Spark Core] Write Spark version informatio...

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22255 @dongjoon-hyun Do you want to take this over? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22255: [SPARK-25102][Spark Core] Write Spark version informatio...

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22255 https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L902 @rdblue Can we use created_by? ``` /** String for application that wrote

[GitHub] spark issue #15599: [SPARK-18022][SQL] java.lang.NullPointerException instea...

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15599 @shatestest Your problem is different from the issue this PR tries to resolve. If you can provide a test case to reproduce it, feel free to open a JIRA

[GitHub] spark pull request #22927: [SPARK-25918][SQL] LOAD DATA LOCAL INPATH should ...

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22927#discussion_r230217147 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -393,7 +393,7 @@ object LoadDataCommand

[GitHub] spark pull request #22927: [SPARK-25918][SQL] LOAD DATA LOCAL INPATH should ...

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22927#discussion_r230216125 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -1987,6 +1987,13 @@ class SQLQuerySuite extends

[GitHub] spark issue #22924: [SPARK-25891][PYTHON] Upgrade to Py4J 0.10.8.1

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22924 I would be more conservative in these cases. The new release of Py4J might introduce new regressions. Not all the changes are clearly documented in their release notes. It is kind of aggressive

[GitHub] spark issue #22905: [SPARK-25894][SQL] Add a ColumnarFileFormat type which r...

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22905 @mallman Could you run the EXPLAIN with this new changes and post it in the PR description? --- - To unsubscribe, e-mail

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21688 cc @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22924: [SPARK-25891][PYTHON] Upgrade to Py4J 0.10.8.1

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22924 I am against this change in 2.4 branch unless the bugs are critical for our end users. --- - To unsubscribe, e-mail

[GitHub] spark issue #22788: [SPARK-25769][SQL]make UnresolvedAttribute.sql escape ne...

2018-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22788 This sounds a safe change. cc @liancheng --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22910: [SPARK-25899][TESTS]Fix flaky CoarseGrainedSchedulerBack...

2018-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22910 Thanks! Merged to master/2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22847: [SPARK-25850][SQL] Make the split threshold for t...

2018-10-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22847#discussion_r229879855 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -812,6 +812,17 @@ object SQLConf { .intConf

[GitHub] spark issue #22778: [SPARK-25784][SQL] Infer filters from constraints after ...

2018-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22778 Let us hold this PR and try to fix https://github.com/apache/spark/pull/17520 instead. --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #22890: [SPARK-25883][SQL][Minor] Override method `prettyName` i...

2018-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22890 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22847: [SPARK-25850][SQL] Make the split threshold for t...

2018-10-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22847#discussion_r229572426 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -812,6 +812,18 @@ object SQLConf { .intConf

[GitHub] spark issue #22870: [SPARK-25862][SQL] Remove rangeBetween APIs introduced i...

2018-10-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22870 LGTM Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2018-10-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17520 @dilipbiswal Could you take this over? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22778: [SPARK-25784][SQL] Infer filters from constraints...

2018-10-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22778#discussion_r229361802 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -171,10 +171,13 @@ abstract class Optimizer

[GitHub] spark pull request #22778: [SPARK-25784][SQL] Infer filters from constraints...

2018-10-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22778#discussion_r229178084 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -171,10 +171,13 @@ abstract class Optimizer

[GitHub] spark issue #22879: [SPARK-25872][SQL][TEST] Add an optimizer tracker for TP...

2018-10-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22879 For generating the golden files, I would not generate the whole plan. Instead, we can keep the node name. That should be good enough

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22857 Please be really careful in null handling. It could easily introduce the correctness bugs like what we recently fixed

[GitHub] spark issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

2018-10-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22878 cc @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22870: [SPARK-25862][SQL] Remove rangeBetween APIs intro...

2018-10-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22870#discussion_r228776577 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -206,7 +206,7 @@ case class

[GitHub] spark issue #22514: [SPARK-25271][SQL] Hive ctas commands should use data so...

2018-10-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22514 This requires more careful review. I do not think we can make it in RC5 --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22817: [SPARK-25816][SQL] Fix attribute resolution in nested ex...

2018-10-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22817 LGTM Thanks! Merged to master/2.4/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22817: [SPARK-25816][SQL] Fix attribute resolution in nested ex...

2018-10-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22817 The fix looks fine to me. cc @cloud-fan @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22817: [SPARK-25816][SQL] Fix attribute resolution in ne...

2018-10-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22817#discussion_r228729920 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2578,4 +2578,12 @@ class DataFrameSuite extends QueryTest

[GitHub] spark issue #22817: [SPARK-25816][SQL] Fix attribute resolution in nested ex...

2018-10-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22817 Will take a look at this tonight. Thanks for reporting this! --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22827: [SPARK-25832][SQL][BRANCH-2.4] Revert newly added...

2018-10-26 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/22827 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22827: [SPARK-25832][SQL] Revert newly added map related...

2018-10-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22827#discussion_r228267525 --- Diff: sql/core/src/test/resources/sql-tests/inputs/higher-order-functions.sql --- @@ -60,8 +60,3 @@ select zip_with(array('a', 'b', 'c'), array('d

[GitHub] spark issue #22827: [SPARK-25832][SQL] Revert newly added map related functi...

2018-10-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22827 @dongjoon-hyun I think that can be kept. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22827: [SPARK-25832][SQL] Revert newly added map related functi...

2018-10-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22827 @felixcheung @mengxr The last commit https://github.com/apache/spark/pull/22827/commits/f1f3d0b09527d28fd8a1f1a2c8e2358c10da175b is for R related revert. Please take a look

[GitHub] spark issue #22827: [SPARK-25832][SQL] Revert newly added map related functi...

2018-10-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22827 @cloud-fan @dongjoon-hyun @rxin @felixcheung --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22827: [SPARK-25832][SQL] Revert newly added map related...

2018-10-25 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22827 [SPARK-25832][SQL] Revert newly added map related functions ## What changes were proposed in this pull request? - Revert [SPARK-23935][SQL] Adding map_entries function: https

[GitHub] spark issue #22821: [SPARK-25832][SQL] remove newly added map related functi...

2018-10-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22821 I will submit a PR to revert all these changes. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22773 Yes. The goal of migration guide is for helping end users upgrade their Spark. --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22773 All the things that could break the existing user applications should be documented in the migration guide. This will simplify the system upgrade of our end users

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22773 This is an external change. This needs a migration guide update. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22817: [SPARK-25816][SQL] ResolveReferences should work bottom-...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22817 This would not work, since we need both bottom-up and top-down. You would hit the test failure. --- - To unsubscribe, e-mail

[GitHub] spark pull request #22794: [SPARK-24570][SQL] Implement Spark own GetTablesO...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22794#discussion_r227872504 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkGetTablesOperation.scala --- @@ -0,0 +1,121

[GitHub] spark pull request #22794: [SPARK-24570][SQL] Implement Spark own GetTablesO...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22794#discussion_r227872346 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkGetTablesOperation.scala --- @@ -0,0 +1,121

[GitHub] spark pull request #22794: [SPARK-24570][SQL] Implement Spark own GetTablesO...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22794#discussion_r227869218 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala --- @@ -644,6 +644,47 @@ class

[GitHub] spark issue #22794: [SPARK-24570][SQL] Implement Spark own GetTablesOperatio...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22794 cc @srinathshankar --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22144: [SPARK-24935][SQL] : Problem with Executing Hive UDF's f...

2018-10-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22144 I think this is different from the blocker tickets we opened before. We should try our best to avoid accidentally dropping the existing support. Please encourage more people in the community

[GitHub] spark issue #22772: [SPARK-24499][SQL][DOC][Followup] Fix some broken links

2018-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22772 LGTM Let us merge this PR first. If we find more, feel free to open a new follow-up PR --- - To unsubscribe, e-mail

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22750 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needs...

2018-10-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22750#discussion_r226818662 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -168,10 +168,11 @@ case class FileSourceScanExec

[GitHub] spark pull request #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needs...

2018-10-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22750#discussion_r226806646 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -164,12 +162,11 @@ private[sql] trait ColumnarBatchScan

[GitHub] spark issue #22756: [SPARK-25758][ML] Deprecate computeCost on BisectingKMea...

2018-10-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22756 Done --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22756: [SPARK-25758][ML] Deprecate computeCost on BisectingKMea...

2018-10-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22756 Let me revert it. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22756: [SPARK-25758][ML] Deprecate computeCost on BisectingKMea...

2018-10-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22756 cc @mengxr WDYT? It does not sound a blocker to me. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22732: [SPARK-25044][FOLLOW-UP] Change ScalaUDF constructor sig...

2018-10-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22732 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22732: [SPARK-25044][FOLLOW-UP] Change ScalaUDF constructor sig...

2018-10-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22732 LGTM pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22732: [SPARK-25044][FOLLOW-UP] Change ScalaUDF constructor sig...

2018-10-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22732 For Scala 2.11, we should not introduce any behavior change and also keep binary and source compatibility. ``` scala> import org.apache.spark.sql.types.DataTypes imp

[GitHub] spark pull request #22769: [SPARK-24499][DOC][FOLLOW-UP] Split the page of s...

2018-10-18 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/22769 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22769: [SPARK-24499][DOC][FOLLOW-UP] Split the page of s...

2018-10-18 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22769 [SPARK-24499][DOC][FOLLOW-UP] Split the page of sql-programming-guide.html to multiple separate pages ## What changes were proposed in this pull request? Forgot to clean remove the link

[GitHub] spark issue #22769: [SPARK-24499][DOC][FOLLOW-UP] Split the page of sql-prog...

2018-10-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22769 cc @dongjoon-hyun @xuanyuanking --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

<    1   2   3   4   5   6   7   8   9   10   >