zhijiangW commented on pull request #2:
URL: https://github.com/apache/flink-benchmarks/pull/2#issuecomment-678116298
Thanks for the review and discussions, @pnowojski !
> but maybe it would be also better to use FLIP-27 sources here?
Considering the current situation of mostly used legacy source, it is also
meaningful to measure the performance in this case. But I agree we can also
supplement FLIP-27 source as an follow-up option.
> 1. how fast quickly can we trigger and complete N checkpoints, by setting
checkpointing interval to 0ms (or as small number as possible), make sure the
number of concurrent checkpoints is set to 1, and just measure how quick we can
complete those checkpoints (note, we should also in that case modify
@OperationsPerInvocation to N. This with an induced backpressure.
I think my current way is exactly for this purpose, just set `N =5` and the
default concurrent cp should be 1 if I remembered correctly. I am not quite
sure whether it is really necessary to set even smaller interval since 100ms is
already small enough in most practical cases. If we give much smaller values,
the backpressure might not be triggered before N checkpoints finished.
> 2. what is the throughput of the job with unaligned checkpoints compared
to aligned checkpoints. Set checkpointing interval to some reasonable value and
measure how fast can we process N records without any backpressure.
I agree to supplement this case as well, similar with we did in
`RemoteBalanceBenchmark`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
