Hi Xiaogang and Stephan We're continuing to test and have now set up the cluster to disable incremental RocksDB checkpointing as well as increasing the checkpoint interval from 30s to 120s (not ideal really :-( )
We'll run it with a large number of jobs and report back if this setup shows improvement. Appreciate any another insights you might have around this problem. Thanks Prashant -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/S3-recovery-and-checkpoint-directories-exhibit-explosive-growth-tp14270p14392.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
