[ https://issues.apache.org/jira/browse/BEAM-59?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898601#comment-15898601 ]
ASF GitHub Bot commented on BEAM-59: ------------------------------------ GitHub user peihe opened a pull request: https://github.com/apache/beam/pull/2175 [BEAM-59] Beam FileSystems: add match(), copy(), rename(), delete() utilities. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-<Jira issue #>] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/peihe/incubator-beam file-system-FileSystems Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2175.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2175 ---- commit 5508baac909acb935c6be4794c1cc4909e072517 Author: Pei He <peihe@peis-macbook-pro.local> Date: 2017-03-02T02:58:45Z [BEAM-59] Beam FileSystems: add match(), copy(), rename(), delete() utilities. ---- > IOChannelFactory rethinking/redesign > ------------------------------------ > > Key: BEAM-59 > URL: https://issues.apache.org/jira/browse/BEAM-59 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core, sdk-java-gcp > Reporter: Daniel Halperin > Assignee: Pei He > Fix For: First stable release > > > Right now, FileBasedSource and FileBasedSink communication is mediated by > IOChannelFactory. There are a number of issues: > * Global configuration -- e.g., all 'gs://' URIs use the same credentials. > This should be per-source/per-sink/etc. > * Supported APIs -- currently IOChannelFactory is in the "non-public API" > util package and subject to change. We need users to be able to add new > backends ('s3://', 'hdfs://', etc.) directly, without fear that they will be > broken. > * Per-backend features: e.g., creating buckets in GCS/s3, setting expiration > time, etc. > Updates: > Design docs posted on dev@ list: > Part 1: IOChannelFactory Redesign: > https://docs.google.com/document/d/11TdPyZ9_zmjokhNWM3Id-XJsVG3qel2lhdKTknmZ_7M/edit# > Part 2: Configurable BeamFileSystem: > https://docs.google.com/document/d/1-7vo9nLRsEEzDGnb562PuL4q9mUiq_ZVpCAiyyJw8p8/edit#heading=h.p3gc3colc2cs -- This message was sent by Atlassian JIRA (v6.3.15#6346)