liukun4515 opened a new issue, #3421: URL: https://github.com/apache/arrow-datafusion/issues/3421
**Describe the bug**
After the type coercion moved to logical optimizer.
The order of optimizer rule is the `filter_push_down` is before the
`TypeCoercion`, but we can't get the right logical plan
For example
```
❯ \d test
+---------------+--------------+------------+-------------+-----------------------------+-------------+
| table_catalog | table_schema | table_name | column_name | data_type
| is_nullable |
+---------------+--------------+------------+-------------+-----------------------------+-------------+
| datafusion | public | test | 0 |
Timestamp(Nanosecond, None) | YES |
| datafusion | public | test | 1 | Date32
| YES |
| datafusion | public | test | 2 | Int64
| YES |
| datafusion | public | test | 3 | Int64
| YES |
| datafusion | public | test | 7 | Decimal128(9, 0)
| YES |
| datafusion | public | test | 8 | Float32
| YES |
| datafusion | public | test | 11 | Utf8
| YES |
| datafusion | public | test | 16 | Utf8
| YES |
| datafusion | public | test | 100000 | Int64
| YES |
| datafusion | public | test | 100001 | Int64
| YES |
| datafusion | public | test | 100002 | Int64
| YES |
+---------------+--------------+------------+-------------+-----------------------------+-------------+
```
```
❯ explain select "1" from test where "8" = 1;
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type | plan
|
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan | Projection: #test.1
|
| | Filter: #test.8 = CAST(Int64(1) AS Float32)
|
| | TableScan: test projection=[1, 8],
partial_filters=[#test.8 = Int64(1)]
|
```
```
❯ explain verbose select "1" from test where "8" = 1;
+-------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type | plan
|
+-------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| initial_logical_plan | Projection:
#test.1
|
| | Filter: #test.8
= Int64(1)
|
| | TableScan:
test
|
| logical_plan after simplify_expressions | SAME TEXT AS ABOVE
|
| logical_plan after pre_cast_lit_in_comparison | SAME TEXT AS ABOVE
|
| logical_plan after decorrelate_where_exists | SAME TEXT AS ABOVE
|
| logical_plan after decorrelate_where_in | SAME TEXT AS ABOVE
|
| logical_plan after scalar_subquery_to_join | SAME TEXT AS ABOVE
|
| logical_plan after subquery_filter_to_join | SAME TEXT AS ABOVE
|
| logical_plan after eliminate_filter | SAME TEXT AS ABOVE
|
| logical_plan after common_sub_expression_eliminate | SAME TEXT AS ABOVE
|
| logical_plan after eliminate_limit | SAME TEXT AS ABOVE
|
| logical_plan after projection_push_down | Projection:
#test.1
|
| | Filter: #test.8
= Int64(1)
|
| | TableScan:
test projection=[1, 8]
|
| logical_plan after rewrite_disjunctive_predicate | SAME TEXT AS ABOVE
|
| logical_plan after reduce_outer_join | SAME TEXT AS ABOVE
|
| logical_plan after filter_push_down | Projection:
#test.1
|
| | Filter: #test.8
= Int64(1)
|
| | TableScan:
test projection=[1, 8], partial_filters=[#test.8 = Int64(1)]
|
| logical_plan after TypeCoercion | Projection:
#test.1
|
| | Filter: #test.8
= CAST(Int64(1) AS Float32)
|
| | TableScan:
test projection=[1, 8], partial_filters=[#test.8 = Int64(1)]
```
The `partial_filters` was not casted to the right type, it will break the
pruning for the parquet.
Why we assign this order for the optimizer rule?
@andygrove
**To Reproduce**
Steps to reproduce the behavior:
**Expected behavior**
A clear and concise description of what you expected to happen.
**Additional context**
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
