zhuqi-lucas commented on PR #14766:
URL: https://github.com/apache/datafusion/pull/14766#issuecomment-2667860443
Thank you @2010YOUY01 for review, good suggestion, if i make sense right, so
we will control two extra things:
1. --stop-after-max-rows, default: false , we can use this to control the
end of the streaming.
```rust
if results.len() >= max_rows && stop-after-max-rows{
break;
}
```
2. --retain-full-results, default: false , we can use this to control the
memory usage:
Maybe we can get more smart for memory usage for keeping the result and
print? For example, we add a option that peek-memory-for-batch-result.
Also we can register the peek-memory-for-batch-result to memory-pool?
```
current_total_usage
if current_total_usage + batch >= memory-for-batch-result {
1. // we should print it
2. // we should free the current_total_usage
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
