Hi Piotr, I’ve create an issue https://issues.apache.org/jira/browse/FLINK-9592
<https://issues.apache.org/jira/browse/FLINK-9592>
The third proposal looks great, may I try to contribute this issue ?
> On 14 Jun 2018, at 12:29, Piotr Nowojski <pi...@data-artisans.com> wrote:
>
> Hi,
>
> Couple of things:
>
> 1. Please create a Jira ticket with this proposal, so we can move discussion
> from user mailing list.
>
> I haven’t thought it through, so take my comments with a grain of salt,
> however:
>
> 2. If we were to go with such callback, I would prefer to have one
> BucketStateChangeCallback, with methods like `onInProgressToPending(…)`,
> `onPendingToFinal`, `onPendingToCancelled(…)`, etc, in oppose to having one
> interface passed three times/four times for different purposes.
>
> 3. Other thing that I had in mind is that BucketingSink could be rewritten to
> extend TwoPhaseCommitSinkFunction. In that case, with
>
> public class BucketingSink2 extends TwoPhaseCommitSinkFunction<???>
>
> user could add his own hooks by overriding following methods
>
> BucketingSink2#beginTransaction, BucketingSink2#preCommit,
> BucketingSink2#commit, BucketingSink2#abort. For example:
>
> public class MyBucketingSink extends BucketingSink2 {
> @Override
> protected void commit(??? txn) {
> super.commit(txn);
> // My hook on moving file from pending to commit state
> };
> }
>
> Alternatively, we could implement before mentioned callbacks support in
> TwoPhaseCommitSinkFunction and provide such feature to
> Kafka/Pravega/BucketingSink at once.
>
> Piotrek
Sincerely yours,
Rinat Sharipov
Software Engineer at 1DMP CORE Team
email: r.shari...@cleverdata.ru <mailto:a.totma...@cleverdata.ru>
mobile: +7 (925) 416-37-26
CleverDATA
make your data clever
