Hi, David, As far as a can see from streamer implementation, flush() call 1. won't return until all futures returned by addData() are (successfully) finished 2. will throw an exception if at least one future has been failed.
Could you please refine what kind of flaw do you suspect? As for callback added by listen(), it is called right _after_ the future gets to 'done' state, thus flush() completion does not guarantee callback's apply() completion. 2018-04-26 16:37 GMT+03:00 David Harvey <dhar...@jobcase.com>: > Thanks Yakov. I think this is more subtle. > > Our loading via IgniteDatastreamer is idempotent, but this depends on > being certain that a batch of work has successfully completed. It is > *not* sufficient for us to listen to the futures returned by addData, > then to call flush(), and then to record success if there have been no > exceptions. We must wait until get() is called on every future before > recording that the operation was successful. The fact that the future is > done is not sufficient, we need to know that it is done and there is no > exception. We can call flush and then do a future.get() on the incomplete > futures, but not as an assert. (It is valid to assert that fut.isDone(), > but that is not sufficient.) > > Based on by current understanding, I think this is a flaw in Ignite, even > if the fix might only be to clarify the comments for flush() to make this > behavior clear. > > > -- Best regards, Andrey Kuznetsov.
