pnowojski commented on pull request #2: URL: https://github.com/apache/flink-benchmarks/pull/2#issuecomment-678248194
> I think my current way is exactly for this purpose, just set N =5 and the default concurrent cp should be 1 if I remembered correctly. I am not quite sure whether it is really necessary to set even smaller interval since 100ms is already small enough in most practical cases. If we give much smaller values, the backpressure might not be triggered before N checkpoints finished. But the problem with the current setup is that if checkpointing time is small, you will be measuring just the time to complete last checkpoint, instead of 5. And the problem is that even if the current interval makes sense, the checkpointing time can change in the future, as we improve things, which would render this benchmark obsolete. That's why I suggested to just trigger checkpoint as quickly as possible, to avoid this problem. > If we give much smaller values, the backpressure might not be triggered before N checkpoints finished. Do you think this can happen? Sources will produce huge number of records immediately (millions of records per second). Even if checkpoint is triggered just a couple of `ms` after the source started, the task should be already fully back pressured. If this worries you, maybe there is a way of delaying the first checkpoint? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
