frankli created SPARK-37051: ------------------------------- Summary: The filter operator gets wrong results in ORC char/varchar types Key: SPARK-37051 URL: https://issues.apache.org/jira/browse/SPARK-37051 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.2 Environment: Spark 3.1.2
Scala 2.12 / Java 1.8 Reporter: frankli When I try the following sample SQL on the TPCDS data, the filter operator returns an empty row set (shown in web ui). _select * from item where i_category = 'Music' limit 100;_ The table is in ORC format, and i_category is char(50) type. I guest that the char(50) type will remains redundant blanks after the actual word. It will affect the boolean value of "x.equals(Y)", and results in wrong results. By the way, Spark's tests should add more cases on ORC format. !image-2021-10-19-11-01-55-597.png|width=1085,height=499! -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org