Hi, I am working on BEAM-59 <https://issues.apache.org/jira/browse/BEAM-59> "IOChannelFactory redesign". The goals are:
1. Support file-based IOs (TextIO, AvorIO) with user-defined file system. 2. Support configuring any user-defined file system. And, I drafted the design proposal in two parts to address them in order: Part 1: IOChannelFactory Redesign <https://docs.google.com/document/d/11TdPyZ9_zmjokhNWM3Id-XJsVG3qel2lhdKTknmZ_7M/edit#> Summary: Old API: WritableByteChannel create(String spec, String mimeType); New API: WritableByteChannel create(URI uri, CreateOptions options); Noticeable proposed changes: 1. Includes the options parameter in most methods to specify behaviors. 2. Replace String with URI to include scheme for files/directories locations. 3. Require file systems to provide a SeekableByteChannel for read. 4. Additional methods, such as getMetadata(), rename() e.t.c Part 2: Configurable BeamFileSystem <https://docs.google.com/document/d/1-7vo9nLRsEEzDGnb562PuL4q9mUiq_ZVpCAiyyJw8p8/edit#heading=h.p3gc3colc2cs> Summary: Old API: IOChannelUtils.getFactory(glob).match(glob); New API: BeamFileSystems.getFileSystem(glob, config).match(glob); Looking for comments and feedback. Thanks -- Pei