[GitHub] spark pull request #13585: [SPARK-15859][SQL] Optimize the partition pruning...

2016-06-10 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13585#discussion_r66575743 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -92,6 +92,36 @@ object PhysicalOperation extends

[GitHub] spark pull request #13585: [SPARK-15859][SQL] Optimize the partition pruning...

2016-06-09 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13585#discussion_r66564745 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala --- @@ -65,4 +69,95 @@ class QueryPartitionSuite extends

[GitHub] spark pull request #13585: [SPARK-15859][SQL] Optimize the partition pruning...

2016-06-09 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13585#discussion_r66563744 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -92,6 +92,36 @@ object PhysicalOperation extends

[GitHub] spark pull request #13522: [SPARK-14321][SQL] Reduce date format cost and st...

2016-06-06 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13522#discussion_r65899067 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -435,20 +437,23 @@ abstract class

[GitHub] spark pull request #13522: [SPARK-14321][SQL] Reduce date format cost and st...

2016-06-06 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13522#discussion_r65898385 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -554,14 +561,19 @@ case class

[GitHub] spark pull request #13486: [SPARK-15743][SQL] Prevent saving with all-column...

2016-06-04 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13486#discussion_r65799660 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -350,6 +350,10 @@ private[sql] object

[GitHub] spark pull request #13486: [SPARK-15743][SQL] Prevent saving with all-column...

2016-06-04 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13486#discussion_r65799422 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -350,6 +350,10 @@ private[sql] object

[GitHub] spark pull request: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTI...

2016-05-24 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13177#issuecomment-221457800 Thanks @rxin . Added it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTI...

2016-05-23 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13177#discussion_r64335588 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -480,11 +480,21 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTI...

2016-05-23 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13177#discussion_r64334408 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -480,11 +480,21 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTI...

2016-05-23 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13177#issuecomment-221147177 Hi @andrewor14, I have checked out the CausedBy source code, I think it will return the root cause of the Exception being thrown not the first Exception. I

[GitHub] spark pull request: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTI...

2016-05-22 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13177#issuecomment-220880826 Hi @andrewor14, sorry to bother you, but does this pr need to be further refined, or it is ready to merge? Could you please give me some instructions? Thanks

[GitHub] spark pull request: [SPARK-15379][SQL] check special invalid date

2016-05-22 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13169#issuecomment-220834946 @cloud-fan Failed on some unrelated cases too, can you help me retest it again? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-15379][SQL] check special invalid date

2016-05-22 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13169#discussion_r64150440 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala --- @@ -353,6 +353,20 @@ class DateTimeUtilsSuite

[GitHub] spark pull request: [SPARK-15379][SQL] check special invalid date

2016-05-20 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13169#issuecomment-220556056 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-15379][SQL] check special invalid date

2016-05-20 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13169#issuecomment-220549615 seems like a irrelevant fail. retest it please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-15379][SQL] check special invalid date

2016-05-19 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13169#issuecomment-220531075 Fixed scala style, retest it please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-15379][SQL] check special invalid date

2016-05-19 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13169#issuecomment-220523060 Addressed your comments. @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-15379][SQL] check special invalid date

2016-05-19 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13169#issuecomment-220512727 @cloud-fan Could you please help me look at this some time? A simple fix. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTI...

2016-05-19 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/13177#issuecomment-220501798 @andrewor14 Thanks :-). Do I still need to modify the code? Frankly, I don't really understand your comment above. ("this won't actually work beca

[GitHub] spark pull request: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTI...

2016-05-18 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13177#discussion_r63815687 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -480,7 +480,7 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request: [SPARK-15379][SQL] check special invalid date

2016-05-18 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13169#discussion_r63814998 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -58,6 +58,7 @@ object DateTimeUtils { final

[GitHub] spark pull request: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTI...

2016-05-18 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/13177#discussion_r63773414 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -480,7 +480,7 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTI...

2016-05-18 Thread wangyang1992
GitHub user wangyang1992 opened a pull request: https://github.com/apache/spark/pull/13177 [SPARK-15388][SQL] Fix spark sql CREATE FUNCTION using hive 1.2.1 ## What changes were proposed in this pull request? spark.sql("CREATE FUNCTION myfu

[GitHub] spark pull request: [SPARK-14414][SQL] Make DDL exceptions more co...

2016-05-18 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/12853#discussion_r63763111 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -616,7 +619,8 @@ private[hive] class HiveClientImpl

[GitHub] spark pull request: [SPARK-15379][SQL] check special invalid date

2016-05-18 Thread wangyang1992
GitHub user wangyang1992 opened a pull request: https://github.com/apache/spark/pull/13169 [SPARK-15379][SQL] check special invalid date ## What changes were proposed in this pull request? When invalid date string like "2015-02-29 00:00:00" are cast as date or

[GitHub] spark pull request: [SPARK-13934][SQL] fixed table identifier

2016-03-24 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/11929#issuecomment-200703478 In my case, my application processes lots of auto-generated table identifiers. Some of them use backticks and some of them do not. If we upgrade to 1.6.1 without

[GitHub] spark pull request: [SPARK-13934][SQL] fixed table identifier

2016-03-23 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/11929#issuecomment-200694207 BTW, I cannot reproduce this problem in master. I pushed this pr in case there will be another release in this branch. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-13934][SQL] fixed table identifier

2016-03-23 Thread wangyang1992
GitHub user wangyang1992 opened a pull request: https://github.com/apache/spark/pull/11929 [SPARK-13934][SQL] fixed table identifier ## What changes were proposed in this pull request? Table identifier that starts in a form of scientific notation (like 1e34) will throw an

[GitHub] spark pull request: [SPARK-13934][SQL] Fixed table name parsing

2016-03-20 Thread wangyang1992
Github user wangyang1992 commented on a diff in the pull request: https://github.com/apache/spark/pull/11762#discussion_r56349547 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/CatalystQlSuite.scala --- @@ -171,6 +171,7 @@ class CatalystQlSuite extends

[GitHub] spark pull request: [SPARK-13934][SQL] Fixed table name parsing

2016-03-19 Thread wangyang1992
GitHub user wangyang1992 opened a pull request: https://github.com/apache/spark/pull/11762 [SPARK-13934][SQL] Fixed table name parsing ## What changes were proposed in this pull request? val tableName = "1e34abcd" hc.sql("select 123").registe

[GitHub] spark pull request: [SPARK-13934][SQL] Fixed table name parsing

2016-03-19 Thread wangyang1992
Github user wangyang1992 closed the pull request at: https://github.com/apache/spark/pull/11762 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-13100] [SQL] improving the performance ...

2016-01-30 Thread wangyang1992
Github user wangyang1992 commented on the pull request: https://github.com/apache/spark/pull/10994#issuecomment-177151828 @srowen No, just that one in this file. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13100] [SQL] improving the performance ...

2016-01-30 Thread wangyang1992
GitHub user wangyang1992 opened a pull request: https://github.com/apache/spark/pull/10994 [SPARK-13100] [SQL] improving the performance of stringToDate method in DateTimeUtils.scala Using an instance variable to hold an GMT TimeZone object instead of instantiate it every time