dwsmith1983 commented on PR #6040: URL: https://github.com/apache/datafusion-comet/pull/6040#issuecomment-5743409681
> As a non-blocking validation improvement, please run the single test with only `with_row_selection_enabled(false)` changed and record whether the assertion fails. Ran it, and it did not fail: with only that flag changed the old fixture read 855,375 bytes of 20,887,912, still under the bound, because the residual becomes a parquet row filter that decodes every `id` page and then skips the `payload` pages by row selection, and the compressed sorted `id` column was only about 300 KB. 590a91965 stores the table uncompressed and makes `id` dominate the file. With row selection disabled the range read is now 8,555,451 bytes of 20,567,500 and the test fails; enabled, it is 617,063 bytes and passes with about a 6x margin. Both measurements are in the test comment and the description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
