Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21070
I looked over the code and I fond this issue could be fix by;
https://github.com/apache/spark/compare/master...maropu:FixFilterPushdown
```
scala> sql("select * from parquetTable where value = 0").explain
== Physical Plan ==
*(1) Project [c1#35, c2#36, c3#37, c4#38, c5#39, c6#40, value#41]
+- *(1) Filter (isnotnull(value#41) && (cast(value#41 as int) = 0))
+- *(1) FileScan parquet [c1#35,c2#36,c3#37,c4#38,c5#39,c6#40,value#41]
Batched: true, Format: Parquet, Location:
InMemoryFileIndex[file:/Users/maropu/Desktop/parquet-test], PartitionFilters:
[], PushedFilters: [IsNotNull(value), EqualTo(value,0)], ReadSchema:
struct<c1:string,c2:string,c3:string,c4:string,c5:string,c6:string,value:string>
```
But, I don't know this is a known issue and this fix is a right approach.
cc: @HyukjinKwon
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
