[
https://issues.apache.org/jira/browse/FLINK-17364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094843#comment-17094843
]
Kostas Kloudas edited comment on FLINK-17364 at 4/28/20, 8:05 PM:
------------------------------------------------------------------
[~qqibrow] sorry for the delay. It would help if you could open a full PR with
the proposed solution or share with me a branch, so that I can test the code
and see it as a whole.
In the meantime, there are also some other places where you need to wire the
new presto-compatible {{S3AccessHelper}} in order to make it work. For example,
the {{S3FileSystemFactory.getS3AccessHelper()}}. In addition, there is the
{{PrestoS3RecoverableWriterTest}} class which, for now, simply tests that the
proper exception is thrown when using the recoverable writer with Presto. This
has to change maybe according to the {{HadoopS3RecoverableWriterITCase}}.
was (Author: kkl0u):
[~qqibrow] sorry for the delay, I will check it in the following days.
In the meantime, there are also some other places where you need to wire the
new presto-compatible {{S3AccessHelper}} in order to make it work. For example,
the {{S3FileSystemFactory.getS3AccessHelper()}}. In addition, there is the
{{PrestoS3RecoverableWriterTest}} class which, for now, simply tests that the
proper exception is thrown when using the recoverable writer with Presto. This
has to change maybe according to the {{HadoopS3RecoverableWriterITCase}}.
> Support StreamingFileSink in PrestoS3FileSystem
> -----------------------------------------------
>
> Key: FLINK-17364
> URL: https://issues.apache.org/jira/browse/FLINK-17364
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.10.0
> Reporter: Lu Niu
> Priority: Major
>
> For S3, currently the StreamingFileSink supports only the Hadoop-based
> FileSystem implementation, not the implementation based on Presto At the same
> time, presto is the recommended file system for checkpointing. implementing
> StreamingFileSink in PrestoS3FileSystem helps filling the gap, enables user
> to use PrestoS3FileSystem in all access to S3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
