[GitHub] spark pull request #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBe...

2018-07-05 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21677#discussion_r200260115 --- Diff: sql/core/benchmarks/FilterPushdownBenchmark-results.txt --- @@ -0,0 +1,556 @@ +[ Pushdown for many distinct value

[GitHub] spark pull request #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBe...

2018-07-05 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21677#discussion_r200246934 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala --- @@ -427,16 +245,122 @@ object

[GitHub] spark pull request #21556: [SPARK-24549][SQL] Support Decimal type push down...

2018-07-05 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21556#discussion_r200246162 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -82,6 +120,30 @@ private[parquet] class

[GitHub] spark issue #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBenchmark

2018-07-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21677 @HyukjinKwon Can you merge this to master first? I would like to update the [Benchmark results](https://github.com/apache/spark/pull/21677/files#diff-c5c0bfc86983d5779269cf75da8ed645) of several

[GitHub] spark pull request #21556: [SPARK-24549][SQL] Support Decimal type push down...

2018-07-04 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21556#discussion_r200170975 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -82,6 +120,30 @@ private[parquet] class

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21696 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-03 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21696#discussion_r200011264 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -19,187 +19,200 @@ package

[GitHub] spark pull request #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-03 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21696#discussion_r19002 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -19,166 +19,186 @@ package

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-03 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r12294 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -42,6 +42,10 @@ private[parquet] class

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-03 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r12316 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -93,6 +101,10 @@ private[parquet] class

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-03 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r11024 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -42,6 +42,14 @@ private[parquet] class

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-03 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r199986187 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -42,6 +42,14 @@ private[parquet] class

[GitHub] spark pull request #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21696#discussion_r199672805 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -19,166 +19,186 @@ package

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21696 cc @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21556: [SPARK-24549][SQL] Support Decimal type push down...

2018-07-02 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21556#discussion_r199442189 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -359,6 +369,70 @@ class

[GitHub] spark pull request #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21696 [SPARK-24716][SQL] Refactor ParquetFilters ## What changes were proposed in this pull request? Replace DataFrame schema to Parquet file schema when create `ParquetFilters`. more

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-06-30 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21682 [SPARK-24706][SQL] ByteType and ShortType support pushdown to parquet ## What changes were proposed in this pull request? `ByteType` and `ShortType` support pushdown to parquet data source

[GitHub] spark issue #21681: Pin tag 210

2018-06-30 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21681 @zhangchj1990 Looks mistakenly open. Mind closing this please? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-06-29 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21603 Benchmark result: ``` ##[ Pushdown benchmark for InSet -> InFilters ]## Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac O

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-29 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21623 Benchmark result: ``` ###[ Pushdown benchmark for StringStartsWith ]### Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X

[GitHub] spark issue #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBenchmark

2018-06-29 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21677 cc @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-06-29 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21556 Benchmark results: ``` ###[ Pushdown benchmark for Decimal ] Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X

[GitHub] spark pull request #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBe...

2018-06-29 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21677 [SPARK-24692][TESTS] Improvement FilterPushdownBenchmark ## What changes were proposed in this pull request? 1. Write the result to `benchmarks/FilterPushdownBenchmark-results.txt

[GitHub] spark pull request #21623: [SPARK-24638][SQL] StringStartsWith support push ...

2018-06-29 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21623#discussion_r199116993 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -660,6 +688,62 @@ class

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-29 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21623 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21623: [SPARK-24638][SQL] StringStartsWith support push ...

2018-06-28 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21623#discussion_r199043411 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -660,6 +661,56 @@ class

[GitHub] spark pull request #21623: [SPARK-24638][SQL] StringStartsWith support push ...

2018-06-28 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21623#discussion_r199043210 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -22,16 +22,23 @@ import java.sql.Date

[GitHub] spark pull request #21547: [SPARK-24538][SQL] ByteArrayDecimalType support p...

2018-06-28 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/21547 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21547: [SPARK-24538][SQL] ByteArrayDecimalType support push dow...

2018-06-28 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21547 Close it because I have implement it in [SPARK-24549](https://issues.apache.org/jira/browse/SPARK-24549). --- - To unsubscribe

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-06-28 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21556 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-26 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21603#discussion_r198146352 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -270,6 +270,11 @@ private[parquet] class

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-06-26 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21603 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-26 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21603#discussion_r198124578 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -270,6 +270,11 @@ private[parquet] class

[GitHub] spark pull request #21641: [SPARK-24658][SQL] Remove workaround for ANTLR bu...

2018-06-25 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21641 [SPARK-24658][SQL] Remove workaround for ANTLR bug ## What changes were proposed in this pull request? Issue antlr/antlr4#781 has already been fixed, so the workaround of extracting

[GitHub] spark pull request #21623: [SPARK-24638][SQL] StringStartsWith support push ...

2018-06-25 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21623#discussion_r197992151 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -660,6 +660,30 @@ class

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-25 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21623 cc @gszadovszky @nandorKollar --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21623: [SPARK-24638][SQL] StringStartsWith support push ...

2018-06-23 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21623 [SPARK-24638][SQL] StringStartsWith support push down ## What changes were proposed in this pull request? `StringStartsWith` support push down. About 50% savings in compute time

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-22 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21603#discussion_r197603396 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -378,6 +378,17 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-21 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21603#discussion_r197338867 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -270,6 +270,11 @@ private[parquet] class

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-06-21 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21603 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21603#discussion_r197011649 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -270,6 +270,11 @@ private[parquet] class

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-20 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21603 [SPARK-17091][SQL] Add rule to convert IN predicate to equivalent Parquet filter ## What changes were proposed in this pull request? Add a new optimizer rule to convert an IN predicate

[GitHub] spark issue #21556: [SPARK-24549][SQL] 32BitDecimalType and 64BitDecimalType...

2018-06-19 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21556 cc @gatorsmile @rdblue --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2018-06-19 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/18424 @ptkool Are you still working on? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21556: [SPARK-24549][SQL] 32BitDecimalType and 64BitDecimalType...

2018-06-15 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21556 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21556: [SPARK-24549][SQL] 32BitDecimalType and 64BitDecimalType...

2018-06-15 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21556 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21556: [SPARK-24549][SQL] 32BitDecimalType and 64BitDecimalType...

2018-06-14 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21556 Another performance test: https://user-images.githubusercontent.com/5399861/41448622-437d029a-708e-11e8-9c18-5d9f17cd1edf.png

[GitHub] spark issue #21547: [SPARK-24538][SQL] ByteArrayDecimalType support push dow...

2018-06-14 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21547 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21547: [SPARK-24538][SQL] ByteArrayDecimalType support push dow...

2018-06-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21547 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21547: [SPARK-24538][SQL] ByteArrayDecimalType support p...

2018-06-13 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21547#discussion_r195289284 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -37,6 +39,23 @@ private[parquet] class

[GitHub] spark pull request #21556: [SPARK-24549][SQL] 32BitDecimalType and 64BitDeci...

2018-06-13 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21556#discussion_r195283330 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -62,6 +62,16 @@ private[parquet] class

[GitHub] spark pull request #21556: [SPARK-24549][SQL] 32BitDecimalType and 64BitDeci...

2018-06-13 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21556 [SPARK-24549][SQL] 32BitDecimalType and 64BitDecimalType support push down ## What changes were proposed in this pull request? [32BitDecimalType](https://github.com/apache/spark/blob

[GitHub] spark issue #21547: [SPARK-24538][SQL] ByteArrayDecimalType support push dow...

2018-06-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21547 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21547: [SPARK-24538][SQL] ByteArrayDecimalType support p...

2018-06-12 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21547 [SPARK-24538][SQL] ByteArrayDecimalType support push down to the data sources ## What changes were proposed in this pull request? [ByteArrayDecimalType](https://github.com/apache/spark

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21479 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-01 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21479 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-01 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21479 [SPARK-23903][SQL] Add support for date extract ## What changes were proposed in this pull request? Add support for date `extract`, supported field same as [Hive](https://github.com

[GitHub] spark issue #21010: [SPARK-23900][SQL] format_number support user specifed f...

2018-05-31 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21010 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21010: [SPARK-23900][SQL] format_number support user specifed f...

2018-05-30 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21010 > Basically LGTM, but I'm wondering what if the expr2 is not like a format string? The same as Hive: ```sql spark-sql> SELECT format_number(12332.123456, 'abc'); ab

[GitHub] spark pull request #21460: [SPARK-23442][SQL] Improvement reading from parti...

2018-05-30 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21460 [SPARK-23442][SQL] Improvement reading from partitioned and bucketed table. ## What changes were proposed in this pull request? For a partitioned and bucketed table. With the increasing

[GitHub] spark issue #21010: [SPARK-23900][SQL] format_number support user specifed f...

2018-05-28 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21010 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21010: [SPARK-23900][SQL] format_number support user specifed f...

2018-05-28 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21010 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompress...

2018-05-26 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21431 Yes. I have tested with `—conf spark.io.compression.codec=zstd`. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortC...

2018-05-25 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21431 [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompressionCodecNames to configuration. ## What changes were proposed in this pull request? Spark provides three codecs: `lz4`, `lzf

[GitHub] spark pull request #21423: [SPARK-24378][SQL] Fix date_trunc function incorr...

2018-05-24 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21423 [SPARK-24378][SQL] Fix date_trunc function incorrect examples ## What changes were proposed in this pull request? Fix `date_trunc` function incorrect examples. ## How

[GitHub] spark issue #21404: [SPARK-24360][SQL] Support Hive 3.0 metastore

2018-05-23 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21404 Can we remove the old hive support? such as 0.12, 0.13 and 0.14. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-05-22 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20274 @srowen I have updated. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21375: [HOT-FIX][SQL] Fix: SQLConf.scala:1757: not found...

2018-05-19 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/21375 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21375: [HOT-FIX][SQL] Fix: SQLConf.scala:1757: not found...

2018-05-19 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21375 [HOT-FIX][SQL] Fix: SQLConf.scala:1757: not found: value Utils ## What changes were proposed in this pull request? Fix: `SQLConf.scala:1757: not found: value Utils` ## How

[GitHub] spark issue #21343: [SPARK-24292][SQL] Proxy user cannot connect to HiveMeta...

2018-05-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21343 This problem seems to have been fixed, can you try [v2.3.1-rc1](https://github.com/apache/spark/releases/tag/v2.3.1-rc1

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-05-16 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/18853 **Spark vs Teradata**: https://user-images.githubusercontent.com/5399861/40102134-43a138e2-591c-11e8-8bf1-00fb9b72e026.png;> https://user-images.githubusercontent.com/5399861/40102

[GitHub] spark pull request #21328: Ci

2018-05-15 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/21328 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21328: Ci

2018-05-15 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21328 Ci ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this patch

[GitHub] spark issue #21010: [SPARK-23900][SQL] format_number support user specifed f...

2018-05-02 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21010 retest please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21189: [SPARK-24117][SQL] Unified the getSizePerRow

2018-04-28 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21189 [SPARK-24117][SQL] Unified the getSizePerRow ## What changes were proposed in this pull request? This pr unified the `getSizePerRow` because `getSizePerRow` is used in many places

[GitHub] spark issue #21170: [SPARK-22732][SS][FOLLOW-UP] Fix memoryV2.scala toString...

2018-04-27 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21170 cc @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21170: [SPARK-22732][SS][FOLLOW-UP] Fix memoryV2.scala t...

2018-04-26 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21170 [SPARK-22732][SS][FOLLOW-UP] Fix memoryV2.scala toString error ## What changes were proposed in this pull request? Fix `memoryV2.scala` toString error ## How was this patch tested

[GitHub] spark pull request #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-04-23 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/20659 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21077: [SPARK-21033][CORE][FOLLOW-UP] Update Spillable

2018-04-16 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21077 [SPARK-21033][CORE][FOLLOW-UP] Update Spillable ## What changes were proposed in this pull request? Update ```scala SparkEnv.get.conf.getLong

[GitHub] spark pull request #21010: [SPARK-23900][SQL] format_number support user spe...

2018-04-09 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21010 [SPARK-23900][SQL] format_number support user specifed format as argument ## What changes were proposed in this pull request? `format_number` support user specifed format as argument

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-04-09 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20944 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-04-07 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20944 cc @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20944: [SPARK-23831][SQL] Add org.apache.derby to Isolat...

2018-04-04 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20944#discussion_r179158019 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala --- @@ -188,6 +188,9 @@ private[hive] class

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/18853 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20944: [SPARK-23831][SQL] Add org.apache.derby to Isolat...

2018-03-30 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20944 [SPARK-23831][SQL] Add org.apache.derby to IsolatedClientLoader ## What changes were proposed in this pull request? Add `org.apache.derby` to `IsolatedClientLoader`, otherwise it may throw

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-29 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20785 Ping @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20898: [SPARK-23789][SQL] Shouldn't set hive.metastore.u...

2018-03-27 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/20898 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20898: [SPARK-23789][SQL] Shouldn't set hive.metastore.uris bef...

2018-03-27 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20898 It looks like --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible...

2018-03-24 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20866#discussion_r176906868 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -92,8 +93,8 @@ private[security] class

[GitHub] spark pull request #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible...

2018-03-24 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20866#discussion_r176906522 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -92,8 +93,8 @@ private[security] class

[GitHub] spark pull request #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible...

2018-03-24 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20866#discussion_r176906474 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -92,8 +94,9 @@ private[security] class

[GitHub] spark issue #20898: [SPARK-23789][SQL] Shouldn't set hive.metastore.uris bef...

2018-03-24 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20898 Yes, it's proxy user: ``` export HADOOP_PROXY_USER=user spark-sql --master yarn ``` --- - To unsubscribe, e-mail

[GitHub] spark issue #20898: [SPARK-23789][SQL] Shouldn't set hive.metastore.uris bef...

2018-03-24 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20898 cc @yaooqinn @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20898: [SPARK-23789][SQL] Shouldn't set hive.metastore.u...

2018-03-24 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20898 [SPARK-23789][SQL] Shouldn't set hive.metastore.uris before invoking HiveDelegationTokenProvider ## What changes were proposed in this pull request? `spark-sql` can't connect to metastore

[GitHub] spark pull request #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible...

2018-03-24 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20866#discussion_r176902474 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -92,8 +93,8 @@ private[security] class

[GitHub] spark issue #20867: Spark 23759

2018-03-20 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20867 Please update the title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible...

2018-03-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20866#discussion_r175859248 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -908,11 +912,39 @@ private[hive] object HiveClientImpl

[GitHub] spark pull request #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible...

2018-03-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20866#discussion_r175858111 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -908,11 +912,39 @@ private[hive] object HiveClientImpl

[GitHub] spark pull request #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible...

2018-03-20 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20866 [SPARK-23749][SQL] Avoid Hive.get() to compatible with different Hive metastore ## What changes were proposed in this pull request? Avoid `Hive.get()` to compatible with different Hive

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20659 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

<    1   2   3   4   5   6   7   8   9   10   >