Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21603
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202506240
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -803,6 +804,67 @@ class ParquetFil
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202505997
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -803,6 +804,67 @@ class ParquetFil
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202505899
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -803,6 +804,67 @@ class ParquetFil
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202500542
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -747,6 +748,66 @@ class ParquetFilterS
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202418834
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -222,6 +225,14 @@ private[parquet] clas
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202418683
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -222,6 +225,14 @@ private[parquet] clas
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202418582
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -222,6 +225,14 @@ private[parquet] clas
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202418387
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala
---
@@ -376,7 +374,8 @@ class ParquetFileFo
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202302865
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -222,6 +225,14 @@ private[parquet] class P
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202286983
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -386,6 +386,17 @@ object SQLConf {
.booleanConf
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202286636
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -386,6 +386,17 @@ object SQLConf {
.booleanConf
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202283085
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -386,6 +386,17 @@ object SQLConf {
.booleanConf
.
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202255923
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -386,6 +386,17 @@ object SQLConf {
.booleanConf
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202256677
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -747,6 +748,66 @@ class ParquetFil
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r202256602
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -747,6 +748,66 @@ class ParquetFil
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r200559860
--- Diff: sql/core/benchmarks/FilterPushdownBenchmark-results.txt ---
@@ -417,120 +417,120 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
InSet -> In
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r198146352
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +270,11 @@ private[parquet] class P
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r198124578
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +270,11 @@ private[parquet] class P
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197686539
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -378,6 +378,17 @@ object SQLConf {
.booleanConf
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197685463
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +270,11 @@ private[parquet] cla
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197681414
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -378,6 +378,17 @@ object SQLConf {
.booleanConf
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197603396
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -378,6 +378,17 @@ object SQLConf {
.booleanConf
.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197523544
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -378,6 +378,17 @@ object SQLConf {
.booleanConf
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197386773
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -378,6 +378,17 @@ object SQLConf {
.booleanConf
.c
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197382691
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -378,6 +378,17 @@ object SQLConf {
.booleanConf
.c
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197381611
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -31,7 +31,7 @@ import org.apache.spark.sq
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197380232
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -685,8 +695,27 @@ class ParquetFilter
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197381574
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -378,6 +378,17 @@ object SQLConf {
.booleanConf
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197338867
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +270,11 @@ private[parquet] class P
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197336527
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +270,11 @@ private[parquet] cla
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197191390
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +270,11 @@ private[parquet] clas
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197083827
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +270,11 @@ private[parquet] class
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197083425
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +270,11 @@ private[parquet] class
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21603#discussion_r197011649
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +270,11 @@ private[parquet] class P
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/21603
[SPARK-17091][SQL] Add rule to convert IN predicate to equivalent Parquet
filter
## What changes were proposed in this pull request?
Add a new optimizer rule to convert an IN predicate to
36 matches
Mail list logo
