Re: Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-14 Thread Piotr Nowojski
eral issues with memory > managed in Kubernetes, most of them actually went away when we upgraded > from Flink 1.9 to 1.13. > > > >Do we know why there's such a huge performance regression? Can we improve > this somehow with some flag tweaking? It would be great if we see a

Re:Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-11 Thread 张蛟
nk 1.9 >to 1.13. > >Do we know why there's such a huge performance regression? Can we improve this >somehow with some flag tweaking? It would be great if we see a more in depth >explanation of the gains vs losses of upgrading. > >On Wed, Aug 4, 2021 at 3:08 PM Stephan Ewen >mailto:se...@apache.org&g

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-08 Thread Jiayi Liao
e heavy users of RocksDB and have had several issues with memory > managed in Kubernetes, most of them actually went away when we upgraded > from Flink 1.9 to 1.13. > > Do we know why there's such a huge performance regression? Can we improve > this somehow with some flag twea

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-05 Thread Yuval Itzchakov
We are heavy users of RocksDB and have had several issues with memory managed in Kubernetes, most of them actually went away when we upgraded from Flink 1.9 to 1.13. Do we know why there's such a huge performance regression? Can we improve this somehow with some flag tweaking? It would be great if

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-05 Thread Yuval Itzchakov
hy there's such a huge performance regression? Can we improve > this somehow with some flag tweaking? It would be great if we see a more in > depth explanation of the gains vs losses of upgrading. > > On Wed, Aug 4, 2021 at 3:08 PM Stephan Ewen wrote: > > > > -

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-05 Thread Yun Tang
1 at 3:08 PM Stephan Ewen mailto:se...@apache.org>> wrote: From: Nico Kruber mailto:n...@apache.org>> Sent: Thursday, August 5, 2021 0:10 To: u...@flink.apache.org<mailto:u...@flink.apache.org> mailto:u...@flink.apache.org>>; dev mailto:dev@

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Piotr Nowojski
gt;> >> We are heavy users of RocksDB and have had several issues with memory >> managed in Kubernetes, most of them actually went away when we upgraded >> from Flink 1.9 to 1.13. >> >> Do we know why there's such a huge performance regression? Can we improve >>

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Yun Tang
sses of upgrading. On Wed, Aug 4, 2021 at 3:08 PM Stephan Ewen mailto:se...@apache.org>> wrote: From: Nico Kruber Sent: Thursday, August 5, 2021 0:10 To: u...@flink.apache.org ; dev Subject: Re: [ANNOUNCE] RocksDB Version Upgrade and Performance That'

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Nico Kruber
That's actually also what I'm seeing most of the time and what I'd expect to improve with the newer RocksDB version. Hence, I'd also favour the upgrade even if there is a slight catch with respect to performance - we should, however, continue to investigate this together with the RocksDB communi

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Stephan Ewen
For the details of what causes this regression, I would add @Yun Tang to this discussion. On Wed, Aug 4, 2021 at 2:36 PM Yuval Itzchakov wrote: > We are heavy users of RocksDB and have had several issues with memory > managed in Kubernetes, most of them actually went away when we upgraded > fro

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread David Anderson
I am hearing quite often from users who are struggling to manage memory usage, and these are all users using RocksDB. While I don't know for certain that RocksDB is the cause in every case, from my perspective, getting the better memory stability of version 6.20 in place is critical. Regards, Davi

[ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-04 Thread Stephan Ewen
Hi all! *!!! If you are a big user of the Embedded RocksDB State Backend and have performance sensitive workloads, please read this !!!* I want to quickly raise some awareness for a RocksDB version upgrade we plan to do, and some possible impact on application performance. *We plan to upgrade R