daijy opened a new pull request, #55257:
URL: https://github.com/apache/spark/pull/55257
### What changes were proposed in this pull request?
Improve the existing UnwrapCastInBinaryComparison to unwrap string to date,
ie. cast(date_key_string as date)=date_literal_constant =>
date_key_string=cast(date_literal_constant, string)
### Why are the changes needed?
Spark fail to push date_key_string=date_literal_constant condition to
iceberg table, this works fine for parquet table (date_key is a string
partition column):
```
explain select date_key, count(*) from
roku.fact_device_log_amoeba_events_hourly where date_key=date_add(current_date,
-10) group by date_key;
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=false
+- HashAggregate(keys=[date_key#25], functions=[count(1)])
+- Exchange hashpartitioning(date_key#25, 200), ENSURE_REQUIREMENTS,
[plan_id=16]
+- HashAggregate(keys=[date_key#25], functions=[partial_count(1)])
+- Filter (cast(date_key#25 as date) = 2026-03-28)
+- BatchScan
spark_catalog.roku.fact_device_log_amoeba_events_hourly[date_key#25]
spark_catalog.roku.fact_device_log_amoeba_events_hourly (branch=null)
[filters=date_key IS NOT NULL, groupedBy=] RuntimeFilters: []
```
The reason is Spark add a cast on the left side to cast string to date. The
cast expression pushed to iceberg data source which is not able to process. A
workaround is to add cast on right side, ie. cast date to string.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Unit test
### Was this patch authored or co-authored using generative AI tooling?
claude-4.5-sonnet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
