Re: RocksDB savepoint recovery performance improvements

2020-05-26 Thread Yun Tang
that is for quick fix at his scenario. Best Yun Tang From: Steven Wu Sent: Wednesday, May 27, 2020 0:36 To: Joey Pereira Cc: user@flink.apache.org ; Yun Tang ; Mike Mintz ; Shahid Chohan ; Aaron Levin Subject: Re: RocksDB savepoint recovery performanc

Re: RocksDB savepoint recovery performance improvements

2020-05-26 Thread Steven Wu
flink/contrib/streaming/state/snapshot/RocksFullSnapshotStrategy.java#L308 >>> >>> >>> Best >>> Yun Tang >>> -- >>> *From:* Joey Pereira >>> *Sent:* Tuesday, May 19, 2020 2:27 >>> *To:* user@flink.apach

Re: RocksDB savepoint recovery performance improvements

2020-05-26 Thread Joey Pereira
t; ---------- >> *From:* Joey Pereira >> *Sent:* Tuesday, May 19, 2020 2:27 >> *To:* user@flink.apache.org >> *Cc:* Mike Mintz ; Shahid Chohan ; >> Aaron Levin >> *Subject:* RocksDB savepoint recovery performance improvements >> >>

Re: RocksDB savepoint recovery performance improvements

2020-05-18 Thread Joey Pereira
hotStrategy.java#L308 > > > Best > Yun Tang > -- > *From:* Joey Pereira > *Sent:* Tuesday, May 19, 2020 2:27 > *To:* user@flink.apache.org > *Cc:* Mike Mintz ; Shahid Chohan ; > Aaron Levin > *Subject:* RocksDB savepoint recovery perfor

Re: RocksDB savepoint recovery performance improvements

2020-05-18 Thread Yun Tang
day, May 19, 2020 2:27 To: user@flink.apache.org Cc: Mike Mintz ; Shahid Chohan ; Aaron Levin Subject: RocksDB savepoint recovery performance improvements Hey, While running a Flink application with a large-state, savepoint recovery has been a painful part of operating the application becau

RocksDB savepoint recovery performance improvements

2020-05-18 Thread Joey Pereira
Hey, While running a Flink application with a large-state, savepoint recovery has been a painful part of operating the application because recovery time can be several hours. During some profiling that chohan (cc'd) had done, a red flag stood out — savepoint recovery consisted mostly of RocksDB Ge