tustvold commented on code in PR #7800:
URL: https://github.com/apache/arrow-datafusion/pull/7800#discussion_r1356542214
##########
datafusion/physical-plan/src/stream.rs:
##########
@@ -251,12 +208,95 @@ impl Stream for RecordBatchReceiverStream {
}
}
-impl RecordBatchStream for RecordBatchReceiverStream {
+impl<O: Send + Sync> ReceiverStream<O> {
+ /// Create a new [`ReceiverStream`] from the provided schema and stream
+ pub fn new(schema: SchemaRef, inner: BoxStream<'static, Result<O>>) ->
Self {
+ Self { schema, inner }
+ }
+}
+
+impl RecordBatchStream for ReceiverStream<RecordBatch> {
fn schema(&self) -> SchemaRef {
self.schema.clone()
}
}
+#[async_trait::async_trait]
+pub(crate) trait StreamAdaptor: Send + Sync {
+ type Item: Send + Sync;
+
+ async fn call(
Review Comment:
I wonder if this trait might be more `DRY` if instead of receiving a
`Sender` it instead had a signature like
```
fn execute(&self, partition: usize, context: Arc<TaskContext>) ->
Result<BoxStream<'static, Result<RecordBatch>>>;
```
This would avoid implementations needing to duplicate the receiver
management logic, and would allow using the standard
https://docs.rs/futures/latest/futures/prelude/stream/trait.TryStreamExt.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
