Gaurav598 commented on issue #39176: URL: https://github.com/apache/beam/issues/39176#issuecomment-5494443034
The current Go filesystem implementation and the main design options for this issue. The existing filesystem.Interface.OpenRead only exposes io.ReadCloser, while several backends already have stronger capabilities underneath: local files support positional reads/seeking, memfs uses a bytes.Reader, and GCS/S3 can perform range reads. I don't think changing OpenRead directly to io.ReaderAt or io.ReadSeeker is the right approach, since that would change the existing interface contract and require every filesystem implementation/custom implementation to adopt the new semantics. The approach that appears to fit Beam's existing filesystem architecture better is to keep OpenRead unchanged and introduce an optional random-access capability, following the existing optional-interface pattern used by Copier, Renamer, etc. The remaining API-design question is whether that capability should expose io.ReaderAt-style positional reads with explicit lifecycle semantics, or a Beam-native range-read abstraction that maps directly to GCS/S3 range requests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
