Sorry for the very late reply. I would try to see if you can simply pass in an instance of filesystem into the source / sink you are using, rather than relying on getting the file system via the scheme.
The only point where you probably need to accept the file system by scheme is for checkpoints, but for sources / sinks it should be possible to pass the FS as an argument. If that is missing (a constructor that accepts a FS factory or so), then this makes sense to add. On Thu, Sep 6, 2018 at 3:07 AM Dave Brewster <davebrews...@me.com.invalid> wrote: > > Currently there doesn’t seem to be a way to do this. Am I correct in > that? I guess one could register multiple implementations each with their > own scheme but that seems somewhat hacky. > > It would be nice if the registration of the filesystem was done when the > DataSet (or DataStream) was defined and not have them preregistered in the > system. I’ve looked at the code and it doesn’t seem too terribly difficult. > > I was wondering if others have a need for this as well? > > Thanks, > > Dave Brewster > >
