wgtmac commented on code in PR #1023: URL: https://github.com/apache/parquet-mr/pull/1023#discussion_r1114020234
########## parquet-hadoop/src/main/java/org/apache/parquet/filter2/statisticslevel/StatisticsFilter.java: ########## @@ -289,8 +320,14 @@ public <T extends Comparable<T>> Boolean visit(Lt<T> lt) { T value = lt.getValue(); - // drop if value <= min - return stats.compareMinToValue(value) >= 0; + // we are looking for records where v < someValue + if (stats.compareMinToValue(value) >= 0) { + // drop if value <= min + return BLOCK_CANNOT_MATCH; + } else { + // if value > min, we must take it + return BLOCK_MUST_MATCH; Review Comment: @gszadovszky Maybe I didn't fully understand your point. Are you concerned about the performance or correctness? The `DictionaryFilter` is already decoding the dictionary to evaluate predicates like `lt` to return `BLOCK_CANNOT_MATCH`. This patch does not introduce more overhead but supports returning `BLOCK_MUST_MATCH` at the same time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org