uros-b commented on code in PR #56422:
URL: https://github.com/apache/spark/pull/56422#discussion_r3903275001
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala:
##########
@@ -167,6 +167,11 @@ class ParquetFilters(
parquetSchemaType.logicalTypeAnnotation,
parquetSchemaType.primitiveTypeName)
}
Review Comment:
PTAL at:
[localTimeToMicros](https://github.com/apache/spark/blob/7efda02f75cb8b2cd49043deb17c277a10e06102/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala#L198-L200)
floors nanosecond literals. For example, t < 12:00:00.000000001 is pushed as t
< 12:00:00.000000, incorrectly pruning a row at exactly 12:00:00. NotEq has the
same false-negative problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
