[GitHub] spark issue #23214: [SPARK-26155] Optimizing the performance of LongToUnsafe...

2018-12-03 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/23214 maybe add some detailed test result in description and explain the reason for this in code comment? --- - To unsubscribe, e

[GitHub] spark issue #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of `Colum...

2018-12-02 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/23152 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of `Colum...

2018-12-02 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/23152 @juliuszsompolski I have updated it accordingly, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of...

2018-11-30 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/23152#discussion_r237808846 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2276,4 +2276,16 @@ class SQLQuerySuite extends

[GitHub] spark pull request #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of...

2018-11-30 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/23152#discussion_r237808676 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2276,4 +2276,16 @@ class SQLQuerySuite extends

[GitHub] spark pull request #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of...

2018-11-29 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/23152#discussion_r237735460 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -879,13 +879,13

[GitHub] spark pull request #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of...

2018-11-28 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/23152#discussion_r237380128 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -821,6 +822,32 @@ class

[GitHub] spark pull request #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of...

2018-11-28 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/23152#discussion_r237359357 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -821,6 +822,32 @@ class

[GitHub] spark issue #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of `Colum...

2018-11-28 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/23152 @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of `Colum...

2018-11-27 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/23152 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of...

2018-11-26 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/23152 [SPARK-26181][SQL] the `hasMinMaxStats` method of `ColumnStatsMap` is not correct ## What changes were proposed in this pull request? For now the `hasMinMaxStats` will return the same

[GitHub] spark pull request #21282: [SPARK-23934][SQL] Adding map_from_entries functi...

2018-05-10 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/21282#discussion_r187291692 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -118,6 +120,229 @@ case class

[GitHub] spark pull request #21227: Backport [SPARK-24133][SQL] Check for integer ove...

2018-05-03 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/21227#discussion_r185801594 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java --- @@ -81,7 +81,9 @@ public void close

[GitHub] spark pull request #21227: [SPARK-24133][SQL] Check for integer overflows wh...

2018-05-03 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/21227#discussion_r185768453 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java --- @@ -81,7 +81,9 @@ public void close

[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

2017-01-05 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r94798265 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +341,91 @@ object

[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

2017-01-05 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r94797482 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +341,91 @@ object

[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

2017-01-05 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r94796481 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +341,91 @@ object

[GitHub] spark pull request #12391: [SPARK-14631][SQL]drop database cascade should dr...

2016-12-12 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/12391#discussion_r92030896 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogSuite.scala --- @@ -778,6 +778,24 @@ abstract class

[GitHub] spark pull request #12391: [SPARK-14631][SQL]drop database cascade should dr...

2016-12-12 Thread adrian-wang
Github user adrian-wang closed the pull request at: https://github.com/apache/spark/pull/12391 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #12391: [SPARK-14631][SQL]drop database cascade should drop func...

2016-12-12 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/12391 I'll update this ASAP --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15011: [SPARK-17122][SQL]support drop current database

2016-11-02 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/15011 Thanks for your kind review @cloud-fan @gatorsmile , this helps our enabling for BigBench on Spark 2.x --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15011: [SPARK-17122][SQL]support drop current database

2016-09-12 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/15011 @hvanhovell I have checked with Hive and MySQL, they all support dropping current database. By asking user to switch to another database before drop the current one is not enough though

[GitHub] spark issue #15011: [SPARK-17122][SQL]support drop current database

2016-09-08 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/15011 @jameszhouyi @chenghao-intel --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15011: [SPARK-17122][SQL]support drop current database

2016-09-08 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/15011 [SPARK-17122][SQL]support drop current database ## What changes were proposed in this pull request? In Spark 1.6 and earlier, we can drop the database we are using. In Spark 2.0

[GitHub] spark pull request #14991: [SPARK-17427][SQL] function SIZE should return -1...

2016-09-06 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/14991 [SPARK-17427][SQL] function SIZE should return -1 when parameter is null ## What changes were proposed in this pull request? `select size(null)` returns -1 in Hive. In order

[GitHub] spark issue #14366: [SPARK-16732][SQL] Remove unused codes in subexpressionE...

2016-07-26 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/14366 good catch! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #14280: [SPARK-16515][SQL][FOLLOW-UP] Fix test `script` o...

2016-07-20 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/14280#discussion_r71477889 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -64,14 +67,19 @@ class SQLQuerySuite extends QueryTest

[GitHub] spark issue #14169: [SPARK-16515][SQL]set default record reader and writer f...

2016-07-15 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/14169 @rxin Only those script transformation cases which use LazySimpleSerde would be affected. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #14169: [SPARK-16515][SQL]set default record reader and writer f...

2016-07-14 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/14169 @rxin In Spark 2.0, those conf values start with "hive.", which have default value in HiveConf, cannot get the default value now. --- If your project is set up for it, you

[GitHub] spark issue #14169: [WIP][SPARK-16515][SQL]set default record reader and wri...

2016-07-13 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/14169 I have updated my code and switch to use bash as test case. Hope it will work for Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #14169: [WIP][SPARK-16515][SQL]set default record reader and wri...

2016-07-12 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/14169 This is strange because I can pass the specific test on my local. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #14169: [SPARK-16515][SQL]set default record reader and w...

2016-07-12 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/14169 [SPARK-16515][SQL]set default record reader and writer for script transformation ## What changes were proposed in this pull request? In `ScriptInputOutputSchema`, we read default

[GitHub] spark pull request #14089: [SPARK-16415][SQL] fix catalog string error

2016-07-07 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/14089 [SPARK-16415][SQL] fix catalog string error ## What changes were proposed in this pull request? In #13537 we truncate `simpleString` if it is a long `StructType`. But sometimes we

[GitHub] spark pull request #13783: [HOTFIX][SPARK-15613][SQL]Set test runtime timezo...

2016-06-26 Thread adrian-wang
Github user adrian-wang closed the pull request at: https://github.com/apache/spark/pull/13783 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #13783: [HOTFIX][SPARK-15613][SQL]Set test runtime timezone for ...

2016-06-26 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/13783 Closing, thx! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #13784: [SPARK-16078] [SQL] from_utc_timestamp/to_utc_tim...

2016-06-20 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/13784#discussion_r67795207 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -886,23 +886,45 @@ object DateTimeUtils

[GitHub] spark pull request #13783: [HOTFIX][SPARK-15613][SQL]Set test runtime timezo...

2016-06-20 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/13783 [HOTFIX][SPARK-15613][SQL]Set test runtime timezone for DateTimeUtilSuite ## What changes were proposed in this pull request? With not default timezone setting in DateTimeUtilsSuite

[GitHub] spark issue #13652: [SPARK-15613] [SQL] Fix incorrect days to millis convers...

2016-06-20 Thread adrian-wang
Github user adrian-wang commented on the issue: https://github.com/apache/spark/pull/13652 @lw-lin @robbinspg I submit a PR to fix the failure: #13783 . Thanks for your information! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #13652: [SPARK-15613] [SQL] Fix incorrect days to millis ...

2016-06-15 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/13652#discussion_r67289086 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -851,6 +851,29 @@ object DateTimeUtils

[GitHub] spark pull request: [SPARK-15397] [SQL] fix string udf locate as h...

2016-05-19 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/13186#issuecomment-220278544 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-15397] [SQL] fix string udf locate as h...

2016-05-19 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/13186#issuecomment-220250927 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-15397] [SQL] fix string udf locate as h...

2016-05-18 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/13186 [SPARK-15397] [SQL] fix string udf locate as hive ## What changes were proposed in this pull request? in hive, `locate("aa", "aaa", 0)` would yield 0, `locate(&qu

[GitHub] spark pull request: [SPARK-14021][SQL] custom context support for ...

2016-05-09 Thread adrian-wang
Github user adrian-wang closed the pull request at: https://github.com/apache/spark/pull/11843 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-14786] Remove hive-cli dependency from ...

2016-04-20 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/12551#issuecomment-212696187 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-14631][SQL]drop database cascade should...

2016-04-15 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/12391#discussion_r59838034 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala --- @@ -17,20 +17,22 @@ package

[GitHub] spark pull request: [SPARK-14631][SQL][WIP]drop database cascade s...

2016-04-14 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/12391 [SPARK-14631][SQL][WIP]drop database cascade should drop functions for HiveExternalCatalog ## What changes were proposed in this pull request? as HIVE-12304, drop database cascade

[GitHub] spark pull request: [SPARK-14631][SQL][WIP]drop database cascade s...

2016-04-14 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/12391#issuecomment-209895482 @chenghao-intel --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14021][SQL][WIP] custom context support...

2016-03-19 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/11843 [SPARK-14021][SQL][WIP] custom context support for SparkSQLEnv ## What changes were proposed in this pull request? This is to create a custom context for command `bin/spark-sql

[GitHub] spark pull request: [SPARK-12789] [SQL] Support Order By Ordinal i...

2016-03-19 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11815#discussion_r56745951 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -435,6 +435,11 @@ object SQLConf { defaultValue = Some

[GitHub] spark pull request: [SPARK-12789] [SQL] Support Order By Ordinal i...

2016-03-19 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11815#discussion_r56745738 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -435,6 +435,11 @@ object SQLConf { defaultValue = Some

[GitHub] spark pull request: [SPARK-12855][MINOR] remove spark.sql.dialect ...

2016-03-16 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/11758 [SPARK-12855][MINOR] remove spark.sql.dialect from doc and test ## What changes were proposed in this pull request? Since developer API of plug-able parser has been removed in #10801

[GitHub] spark pull request: [MINOR][TEST][SQL] Remove wrong "expected" par...

2016-03-15 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/11718#issuecomment-196712224 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-13427][SQL] Support USING clause in JOI...

2016-03-14 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11297#discussion_r56101451 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1169,6 +1170,7 @@ object PushPredicateThroughJoin

[GitHub] spark pull request: [SPARK-13648] Add Hive Cli to classes for isol...

2016-03-08 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/11495#issuecomment-193670192 Why we need `hive-cli` here? This is not version specific. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-11624][SPARK-11972][SQL]fix commands th...

2016-02-19 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/9589#discussion_r53545337 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -104,29 +104,39 @@ private[hive] class HiveClientImpl

[GitHub] spark pull request: [SPARK-11624][SPARK-11972][SQL]fix commands th...

2016-02-17 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/9589#issuecomment-185523881 @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-13332][SQL] Decimal datatype suppo...

2016-02-16 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11212#discussion_r53115148 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -170,6 +170,7 @@ public void write

[GitHub] spark pull request: [WIP][SPARK-13332][SQL] Decimal datatype suppo...

2016-02-15 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11212#discussion_r52968764 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -523,11 +523,45 @@ case class Atan2(left

[GitHub] spark pull request: [WIP][SPARK-13332][SQL] Decimal datatype suppo...

2016-02-15 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11212#discussion_r52968376 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -170,6 +170,7 @@ public void write

[GitHub] spark pull request: [WIP][SPARK-13332][SQL] Decimal datatype suppo...

2016-02-15 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11212#discussion_r52968287 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathFunctionsSuite.scala --- @@ -351,6 +350,20 @@ class

[GitHub] spark pull request: [WIP][SQL] Decimal datatype support for pow

2016-02-15 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11212#discussion_r52968214 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathFunctionsSuite.scala --- @@ -103,8 +103,7 @@ class

[GitHub] spark pull request: [SPARK-11624][SPARK-11972][SQL]fix commands th...

2016-02-13 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/9589#issuecomment-183811059 @marmbrus We have instantiated and started a instance of `CliSessionState`, and when we init `SparkSQLEnv`, we will create a `SessionState`. --- If your project

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-13 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-183791219 @srowen you are right. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11071#discussion_r51851087 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -55,10 +56,19 @@ object DateTimeUtils

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11071#discussion_r51851099 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -55,10 +56,19 @@ object DateTimeUtils

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11071#discussion_r51852765 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -55,10 +56,19 @@ object DateTimeUtils

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/11071#discussion_r51853631 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -55,10 +56,19 @@ object DateTimeUtils

[GitHub] spark pull request: [SPARK-13185][SQL] Improve the performance of ...

2016-02-04 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/11071#issuecomment-180201265 The map key could be like "UTC+01:00". "American/Los Angeles", "PST", etc., they are already cached in `getTimeZone`, but the method

[GitHub] spark pull request: [SPARK-12828][SQL] Natural join follow-up

2016-02-03 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/11070#issuecomment-179664171 LGTM, thx! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-02-03 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/10762#issuecomment-179642503 @cloud-fan any more comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13056][SQL] map column would throw NPE ...

2016-02-01 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10964#discussion_r51392316 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2046,6 +2046,15 @@ class SQLQuerySuite extends QueryTest

[GitHub] spark pull request: [SPARK-13056][SQL] map column would throw NPE ...

2016-02-01 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/10964#issuecomment-177877914 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13056][SQL] map column would throw NPE ...

2016-01-31 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10964#discussion_r51381272 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2056,4 +2056,11 @@ class SQLQuerySuite extends QueryTest

[GitHub] spark pull request: [SPARK-13056][SQL] map column would throw NPE ...

2016-01-29 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10964#discussion_r51240308 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -1463,4 +1463,11 @@ class SQLQuerySuite extends

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-29 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r51295569 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -171,13 +171,21 @@ case class Join

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-28 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r51090849 --- Diff: sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SparkSqlLexer.g --- @@ -328,6 +328,8 @@ KW_WEEK: 'WEEK'|'WEEKS

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-28 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r51094651 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -104,6 +105,9 @@ trait CheckAnalysis

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-28 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r51106564 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -919,6 +919,7 @@ object PushPredicateThroughJoin

[GitHub] spark pull request: [SPARK-13056][SQL] map column would throw NPE ...

2016-01-28 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10964#discussion_r51226292 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeExtractors.scala --- @@ -218,7 +218,7 @@ case class

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-28 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/10762#issuecomment-176532100 Acutally MySQL and Oracle does not support normal full outer join either. PostgreSQL does support natural full outer join: http://www.postgresql.org/docs/9.1

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-28 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r51226827 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala --- @@ -248,4 +249,29 @@ class AnalysisSuite extends

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-27 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/10762#issuecomment-176006912 Sure, I'll update this pr today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-13056][SQL] map column would throw NPE ...

2016-01-27 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/10964 [SPARK-13056][SQL] map column would throw NPE if value is null Jira: https://issues.apache.org/jira/browse/SPARK-13056 Create a map like { "a": "somestr

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-21 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/10762#issuecomment-173804613 When the parser calls the constructor, how can we get the schema of tables? We need schema to build project list and conditions. --- If your project is set up

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-20 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/10762#issuecomment-173465114 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12789]Support order by index

2016-01-18 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10731#discussion_r50063958 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -445,6 +445,26 @@ class Analyzer( val

[GitHub] spark pull request: [SPARK-12789]Support order by index

2016-01-17 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/10731#issuecomment-172424340 order by 2 should be the second column, I think --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-14 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/10762 [SPARK-12828][SQL]add natural join support Jira: https://issues.apache.org/jira/browse/SPARK-12828 You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-14 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r49813610 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala --- @@ -179,10 +180,15 @@ object SqlParser extends

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-14 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r49825914 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2067,4 +2067,24 @@ class SQLQuerySuite extends QueryTest

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-14 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r49825919 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala --- @@ -179,10 +180,15 @@ object SqlParser extends

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-14 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r49828096 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -144,6 +147,35 @@ case class Join

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-14 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10762#discussion_r49827571 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -144,6 +147,35 @@ case class Join

[GitHub] spark pull request: [SPARK-12828][SQL]add natural join support

2016-01-14 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/10762#issuecomment-171895087 @rxin, Thanks for you time, I'll draft another version accordingly. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-12789]Support order by index

2016-01-12 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10731#discussion_r49539497 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -22,8 +22,9 @@ import java.sql.Timestamp import

[GitHub] spark pull request: [SPARK-12789]Support order by index

2016-01-12 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10731#discussion_r49539533 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -441,7 +442,7 @@ class Analyzer

[GitHub] spark pull request: [SPARK-12789]Support order by index

2016-01-12 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10731#discussion_r49539463 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -22,8 +22,9 @@ import java.sql.Timestamp import

[GitHub] spark pull request: [SPARK-12789]Support order by index

2016-01-12 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10731#discussion_r49540540 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -22,8 +22,9 @@ import java.sql.Timestamp import

[GitHub] spark pull request: [SPARK-11624][SPARK-11972][SQL]fix commands th...

2016-01-05 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/9589#discussion_r48829056 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/ClientWrapper.scala --- @@ -151,29 +152,34 @@ private[hive] class ClientWrapper

[GitHub] spark pull request: [Spark-12260][wip][Streaming]Graceful Shutdown...

2015-12-22 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/10252#discussion_r48232238 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala --- @@ -330,6 +330,20 @@ abstract class DStream[T: ClassTag

  1   2   3   4   5   6   7   8   >