alamb commented on issue #18381: URL: https://github.com/apache/datafusion/issues/18381#issuecomment-3516581958
> This way, after calling df.collect_but_not_consume and receiving Poll::Ready(None), the stream would still be alive and available for further operations I don't think most DataFusion operators are implemented with this use case in mind and some non trivial changes might be made I am not sure what the use case is, but it sounds a little like "streaming" execution where some data is fed into the plan and output is produced before the input is entirely consumed. Streaming only works with certain queries and plans (look for "unbounded"), such as https://docs.rs/datafusion/latest/datafusion/physical_plan/execution_plan/enum.Boundedness.html#variant.Unbounded Hopefully that is helpful -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
