[ https://issues.apache.org/jira/browse/FLINK-11838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479454#comment-17479454 ]
Galen Warren commented on FLINK-11838: -------------------------------------- [~MartijnVisser] I'd say let's handle documentation and release notes on the other one, https://issues.apache.org/jira/browse/FLINK-25577, since this one is now closed. Sorry if this is a dumb question, but what is involved in adding this to the release notes? > Create RecoverableWriter for GCS > -------------------------------- > > Key: FLINK-11838 > URL: https://issues.apache.org/jira/browse/FLINK-11838 > Project: Flink > Issue Type: New Feature > Components: Connectors / FileSystem > Affects Versions: 1.8.0 > Reporter: Fokko Driesprong > Assignee: Galen Warren > Priority: Major > Labels: pull-request-available, usability > Fix For: 1.15.0 > > Time Spent: 20m > Remaining Estimate: 0h > > GCS supports the resumable upload which we can use to create a Recoverable > writer similar to the S3 implementation: > https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload > After using the Hadoop compatible interface: > https://github.com/apache/flink/pull/7519 > We've noticed that the current implementation relies heavily on the renaming > of the files on the commit: > https://github.com/apache/flink/blob/master/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java#L233-L259 > This is suboptimal on an object store such as GCS. Therefore we would like to > implement a more GCS native RecoverableWriter -- This message was sent by Atlassian Jira (v8.20.1#820001)