ion-elgreco commented on code in PR #1241:
URL:
https://github.com/apache/datafusion-python/pull/1241#discussion_r2354427053
##########
python/datafusion/dataframe.py:
##########
@@ -370,6 +375,39 @@ def schema(self) -> pa.Schema:
Describing schema of the DataFrame
"""
return self.df.schema()
+
+ def to_batches(self) -> list[pa.RecordBatch]:
+ """Convert DataFrame to list of RecordBatches."""
+ return self.collect() # delegate to existing method
Review Comment:
> I'm not saying it should even return arro3 RecordBatches... because that's
still an external dependency that datafusion is requiring on users. Datafusion
could return a minimal batch object that just holds the RecordBatch pointer and
then the user transfers it to their library of choice
That's possible but I would argue it's more convenient if it's in some way
already usable instead of just a pointer. Arro3 is that small that it's
negligible in size overall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
