guiyanakuang commented on issue #1075: URL: https://github.com/apache/orc/issues/1075#issuecomment-1087526984
Thanks to capturing these failed tests, @williamhyun I conducted some investigation. Prior to #1055, it was just coincidence that these tests would pass, after #1055 was fixed so that the push-down judgement could continue to be executed. 1. `testIsNaN[format = orc]` This test is no longer working. When orc is not written to Double.NaN, and DoubleStatistics sum is a finite value, we can correctly push down `no_nans = NaN` and skip this stripe. 2. `testNotNaN[format = orc]` is still a valid test. The reason this test doesn't pass is that the statistics check for the Float type is missing. https://github.com/apache/orc/blob/70c504cbd759342476d2fb2c9cc75a5025cb7c11/java/core/src/java/org/apache/orc/impl/RecordReaderImpl.java#L696-L703 `category == TypeDescription.Category.DOUBLE || category == TypeDescription.Category.FLOAT` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@orc.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
