[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-18 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-630081895 I am looking on it now. thanks This is an automated message from the Apache Git Service. To respond to the

[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-18 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-630035754 About the manual file clean up, we could move the template file to a temp folder. And use a special `RecovrableWriter` that could know the temp folder. WDYT?

[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-18 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-630028998 NP This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-17 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-629923445 HI @aljoscha , 1. First of all, I think you are right we should merge to master after all the participants agree with the pr. 2. I could split the commit to two commits:

[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-17 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-629779232 Hi, @aljoscha I could not restore bucket state through reading the snapshot of v1 in my local. I think one reason is that these snapshots contain absolute paths. such as

[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-16 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-629748020 HI, @aljoscha @kl0u I resolve all the comments. 1. Change the name of `PartFileFactory` to `BucketWriter`. Change the `PartFileWriter` to `InProgressFileWriter`. 2.

[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-16 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-629650094 Hi, @aljoscha thanks for your suggestion. 1. yes you are right I should delete the v1 code in the test. 1. The reason I used Operator to generate snapshots at the time is

[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-16 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-629637583 1. Yes. It is `BucketWriter` not `BulkWriter`. 2. I got what do you mean. I would follow your suggestion. But I would want to explain why I want to name it `PartFileFactory`

[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-16 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-629604823 Hi, @kl0u thanks for your review. 1. You are right. We could rename the `PartFileWriter` to `InProgressFileWriter`. 2. You are very careful. We should remove the

[GitHub] [flink] guoweiM commented on pull request #12132: [FLINK-17593][Connectors/FileSystem] Support arbitrary recovery mechanism for PartFileWriter

2020-05-14 Thread GitBox
guoweiM commented on pull request #12132: URL: https://github.com/apache/flink/pull/12132#issuecomment-628654263 Thanks for your reminding. I would update pr latter today. This is an automated message from the Apache Git