Re: Improved performance when using incremental checkpoints

2020-06-17 Thread Congxian Qiu
; Please provide a simple program which could reproduce this so that we can > help you more. > > Best > Yun Tang > -- > *From:* Aljoscha Krettek > *Sent:* Tuesday, June 16, 2020 19:53 > *To:* user@flink.apache.org > *Subject:* Re: Improved p

Re: Improved performance when using incremental checkpoints

2020-06-17 Thread Yun Tang
2020 19:53 To: user@flink.apache.org Subject: Re: Improved performance when using incremental checkpoints Hi, it might be that the operations that Flink performs on RocksDB during checkpointing will "poke" RocksDB somehow and make it clean up it's internal hierarchies of storage more. Other

Re: Improved performance when using incremental checkpoints

2020-06-16 Thread Aljoscha Krettek
20 15:44 *To:* user@flink.apache.org *Subject:* Improved performance when using incremental checkpoints Hello, We are using RocksDB as the backend state. At first we didn't enable the checkpoints mechanism. We observed the following behaviour and we are wondering why ? When using the rocksDB *

Re: Improved performance when using incremental checkpoints

2020-06-16 Thread nick toker
day, June 16, 2020 15:44 > *To:* user@flink.apache.org > *Subject:* Improved performance when using incremental checkpoints > > Hello, > > We are using RocksDB as the backend state. > At first we didn't enable the checkpoints mechanism. > > We observed the following beh

Re: Improved performance when using incremental checkpoints

2020-06-16 Thread Yun Tang
this problem. Best Yun Tang From: nick toker Sent: Tuesday, June 16, 2020 15:44 To: user@flink.apache.org Subject: Improved performance when using incremental checkpoints Hello, We are using RocksDB as the backend state. At first we didn't enable the checkpoints

Improved performance when using incremental checkpoints

2020-06-16 Thread nick toker
Hello, We are using RocksDB as the backend state. At first we didn't enable the checkpoints mechanism. We observed the following behaviour and we are wondering why ? When using the rocksDB *without* checkpoint the performance was very extremely bad. And when we enabled the checkpoint the