yunfengzhou-hub commented on PR #112: URL: https://github.com/apache/flink-ml/pull/112#issuecomment-1159880670
Hi @lindong28, Thanks for reviewing this PR. Before this PR is implemented, Flink ML stages with iterations, like KMeans, have to spend at least 100 ms for each iteration they perform, even if the payload of the UDF in the iteration body is trivial. This is because Flink ML has to wait until buffer timeout before it can forward the iteration epoch watermark to the next round. With the flushing behavior introduced by this PR, the 100 ms overhead described above would be saved. Flink ML stages could progress to the next epoch as soon as it receives the iteration watermark, instead of being restricted by buffer timeout. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
