Re: [I] Enable `split_file_groups_by_statistics` by default [datafusion]

2024-07-29 Thread via GitHub
alamb commented on issue #10336: URL: https://github.com/apache/datafusion/issues/10336#issuecomment-2256432095 Sorry for the delay @leoyvens and thank you for this analysis > https://github.com/apache/datafusion/issues/11170 I would personally love to take this approach

Re: [I] Enable `split_file_groups_by_statistics` by default [datafusion]

2024-07-23 Thread via GitHub
leoyvens commented on issue #10336: URL: https://github.com/apache/datafusion/issues/10336#issuecomment-2246022064 One thing I've noticed is that after DataFusion 40 this actually works in my use case, likely thanks to the statistics code getting fixed, so good news there! It does require a

Re: [I] Enable `split_file_groups_by_statistics` by default [datafusion]

2024-05-04 Thread via GitHub
alamb commented on issue #10336: URL: https://github.com/apache/datafusion/issues/10336#issuecomment-2094127979 THank you @yyy1000 🙏 I think a good place to start would be to write some sqllogic level tests to cover the important cases Perhaos for the first test: 1. Create

Re: [I] Enable `split_file_groups_by_statistics` by default [datafusion]

2024-05-03 Thread via GitHub
yyy1000 commented on issue #10336: URL: https://github.com/apache/datafusion/issues/10336#issuecomment-2093968410 I'd like to help it. 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] Enable `split_file_groups_by_statistics` by default [datafusion]

2024-05-01 Thread via GitHub
alamb commented on issue #10336: URL: https://github.com/apache/datafusion/issues/10336#issuecomment-2089121776 Example test coverage we should add I think: https://github.com/apache/datafusion/pull/9593#discussion_r1585517605 -- This is an automated message from the Apache Git Service. T

[I] Enable `split_file_groups_by_statistics` by default [datafusion]

2024-05-01 Thread via GitHub
alamb opened a new issue, #10336: URL: https://github.com/apache/datafusion/issues/10336 ### Is your feature request related to a problem or challenge? Part of https://github.com/apache/datafusion/issues/10313 In https://github.com/apache/datafusion/pull/9593, @suremarc added a