yjshen commented on code in PR #3386:
URL: https://github.com/apache/arrow-datafusion/pull/3386#discussion_r966573456
##########
datafusion/core/src/physical_plan/sorts/sort_preserving_merge.rs:
##########
@@ -388,22 +389,23 @@ impl SortPreservingMergeStream {
}
Some(Ok(batch)) => {
if batch.num_rows() > 0 {
- let cursor = match SortKeyCursor::new(
+ let cols = self
+ .column_expressions
+ .iter()
+ .map(|expr| {
+
Ok(expr.evaluate(&batch)?.into_array(batch.num_rows()))
+ })
+ .collect::<Result<Vec<_>>>()?;
+
+ let rows = self.row_converter.convert(&cols);
+
+ let cursor = SortKeyCursor::new(
Review Comment:
We need to track the total memory used by all cursors since the cursor now
holds `Rows`. We could do this as follow-ups but note here as it came to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
