shanipribadi commented on code in PR #6086:
URL: https://github.com/apache/opendal/pull/6086#discussion_r2105720410
##########
bindings/python/python/opendal/__init__.pyi:
##########
@@ -472,6 +475,20 @@ class AsyncFile:
Args:
bs (bytes): The content to write.
"""
+ async def write_from(self, other: AsyncFile) -> None:
Review Comment:
Possible confusion with naming it write_from is that on the rust lib there
is
[Writer::write_from](https://docs.rs/opendal/0.53.1/opendal/struct.Writer.html#method.write_from)
which accepts a buffer, rather than a reader.
I found this RFC
https://docs.rs/opendal/0.53.1/opendal/docs/rfcs/rfc_3017_remove_write_copy_from/index.html
where opendal used to have a Writer::copy_from(&mut self, size: u64, src:
oio::Reader) that got removed,
and on the RFC there is a mention of future possibilities
> Introduce utility functions such as Writer::copy_from(r: &dyn AsyncRead)
when possible.
I think API wise, the async_copyfileobj would be more familiar to python
users because of the similar shutil function. But as a user of opendal, I would
prefer that the API of the bindings mirror the rust API as much as possible
while it is still idiomatic to do so (so familiarity with the Rust API would
translate to the various bindings).
Perhaps we should revisit that RFC about copy_from one more time on the rust
side before introducing this on the python binding? @Xuanwo
Anyways, IMO the most valuable part of this PR is exposing the various
Reader/Writer options on the python binding, having the chunk_size and
concurrency option makes opendal performance comparable to boto3,
Not having the chunk_size/concurrency option is actually a blocker for me
personally to switch from boto3 to opendal due to performance reasons. (use
case is backing up many large files from disk to object storage).
Maybe splitting the PR would be a good way to go here (remove the write_from
on this PR, and have it as a separate PR?) @chitralverma
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
