[
https://issues.apache.org/jira/browse/FLINK-11378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767056#comment-16767056
]
Robert Metzger commented on FLINK-11378:
----------------------------------------
[~martijnvdgrift]: I gave you contributor permissions in our Jira (so that I
can assign you) and assigned you, because you've opened a PR for this.
> Allow HadoopRecoverableWriter to write to Hadoop compatible Filesystems
> -----------------------------------------------------------------------
>
> Key: FLINK-11378
> URL: https://issues.apache.org/jira/browse/FLINK-11378
> Project: Flink
> Issue Type: Improvement
> Components: FileSystem
> Reporter: Martijn van de Grift
> Assignee: Martijn van de Grift
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> At a client we're using Flink jobs to read data from Kafka and writing to
> GCS. In earlier versions, we've used `BucketingFileSink` for this, but we
> want to switch to the newer `StreamingFileSink`.
> Since we're running Flink on Google's DataProc, we're using the Hadoop
> compatible GCS
> [connector|https://github.com/GoogleCloudPlatform/bigdata-interop] made by
> Google. This currently doesn't work on Flink, because Flink checks for a HDFS
> scheme at 'HadoopRecoverableWriter'.
> We've successfully ran our jobs by creating a custom Flink Distro which has
> the hdfs scheme check removed.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
