The iterator is inside a try-with-resources. And if the memory leak was inside our code, we will see it using visualvm or jmap, and that's not the case. This is not a memory leak in the heap. That's why my guess goes directly to rocksdb.
On Sat, Feb 4, 2017 at 5:31 PM, Damian Guy <damian....@gmail.com> wrote: > Hi Pierre, > > When you are iterating over the entries do you close the iterator once you > are finished? If you don't then that will cause a memory leak. > > Thanks, > Damian > > On Sat, 4 Feb 2017 at 16:18 Pierre Coquentin <pierre.coquen...@gmail.com> > wrote: > > > Hi, > > > > We ran a few tests with apache kafka 0.10.1.1. > > We use a Topology with only one processor and a KVStore configured as > > persistent backed by rocksdb 4.9.0. Each events received are stored using > > the method put(key, value) and in the punctuate method, we iterate over > all > > entries with all(), processing them and deleting them with delete(key). > > Then after a few days, the jvm crashed due to lack of memory. Visualvm or > > jmap doesn't show anything, so my guesses were there was a memory leak in > > rocksdb. We configured the KVStore to be in memory and as you can see in > > picture 2, the memory is stable. > > I didn't run the same test with a newer version of rocksdb yet. > > Any thoughts ? > > Regards, > > > > Pierre > > >
