Weijun-H commented on issue #25183: URL: https://github.com/apache/datafusion/issues/25183#issuecomment-5759499452
We have a downstream sorter-local prototype that selects a byte-bounded output prefix and reserves its construction budget before `BatchBuilder` calls `interleave`. Multi-row reservation failures retry with a smaller prefix; a single row may exceed the byte target but must still obtain its reservation. This raises the compatibility question in this issue: failing that final reservation could reject queries that currently continue spilling. We would like feedback on that policy before presenting it as unchanged behavior. The byte target is not a hard resident-memory cap; unsupported types, including view arrays until shared-buffer retention is handled, would retain the current path. We plan a small PR against current main with focused regression tests, without performance claims. Is this issue the right place to discuss the policy, or would a separate sorter-output issue be preferable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
