rluvaton commented on issue #16979: URL: https://github.com/apache/datafusion/issues/16979#issuecomment-3135784332
Do we know where it fails? is it in the `MultiLevelMergeBuilder`? we should also add some context in the error below when we can't reserve memory even for the minimum required spill files to merge: https://github.com/apache/datafusion/blob/0183244a426e5e1fda28c7b6e6961f70f04c0744/datafusion/physical-plan/src/sorts/multi_level_merge.rs#L381 we know it's not in as we would see in the context: https://github.com/apache/datafusion/blob/0183244a426e5e1fda28c7b6e6961f70f04c0744/datafusion/physical-plan/src/sorts/sort.rs#L767-L777 but it can also be in: https://github.com/apache/datafusion/blob/0183244a426e5e1fda28c7b6e6961f70f04c0744/datafusion/physical-plan/src/sorts/sort.rs#L684-L692 which the `SortPreservingMergeStream` also allocate memory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
