Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21070 Yup, https://github.com/apache/spark/pull/21070#issuecomment-386793202, just for clarification, the given attribute and literal is castable but they are not being as so, right? I believe this is a known issue and there were several tries: One approach was directly casting always and it was reverted (roughly 2 years ago?). Another approach was constant folding at optimizer level but it was rejected as it's too messy. Another approach was directly casting and comparing both values but it was also rejected since it sounded unsafe. It was a long old story so probably worth double checking the history but I feel sure that I remember this story correctly. The key point IIUC was that `translateFilter` should be super conservative and sounds we better need to check every possibility.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org