That would be great to add, Matt. Of course it's important to make this backwards compatible, but other than that, the addition would be very welcome.
On Wed, May 19, 2021 at 9:41 AM Matt Rudary <matt.rud...@twosigma.com> wrote: > Hi, > > > > This is a quick sketch of a proposal – I wanted to get a sense of whether > there’s general support for this idea before fleshing it out further, > getting internal approvals, etc. > > > > I’m working with multiple storage systems that speak the S3 api. I would > like to support FileIO operations for these storage systems, but > S3FileSystem hardcodes the s3 scheme (the various systems use different URI > schemes) and it is in any case impossible to instantiate more than one in > the current design. > > > > I’d like to refactor the code in org.apache.beam.sdk.io.aws.s3 (and maybe > …aws.options) somewhat to enable this use-case. I haven’t worked out the > details yet, but it will take some thought to make this work in a non-hacky > way. > > > > Thanks > > Matt Rudary >
