seddonm1 commented on a change in pull request #8982:
URL: https://github.com/apache/arrow/pull/8982#discussion_r547476612
##########
File path: rust/datafusion/src/physical_plan/mod.rs
##########
@@ -107,6 +107,13 @@ pub async fn collect(plan: Arc<dyn ExecutionPlan>) ->
Result<Vec<RecordBatch>> {
/// Partitioning schemes supported by operators.
#[derive(Debug, Clone)]
pub enum Partitioning {
+ /// Allocate batches using a round-robin algorithm
+ RoundRobinBatch(usize),
+ /// Allocate rows using a round-robin algorithm. This provides
finer-grained partitioning
+ /// than `RoundRobinBatch` but also has much more overhead.
+ RoundRobinRow(usize),
Review comment:
Agree. I was trying to implement the `RoundRobinRow` functionality
independently and was going down a route similar to the `StructBuilder` vector
of builders route:
https://github.com/apache/arrow/blob/master/rust/arrow/src/array/builder.rs#L1600.
Staying at the `RecordBatch` level is much more sensible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
