On Wed, Dec 11, 2019 at 1:59 PM John Huss <johnth...@gmail.com> wrote:
> > > On Wed, Dec 11, 2019 at 10:59 AM Andrus Adamchik <and...@objectstyle.org> > wrote: > >> >> >> > On Dec 10, 2019, at 9:39 PM, John Huss <johnth...@gmail.com> wrote: >> > >> >> My projects are set up to use soft references using " >> >> cayenne.server.object_retain_strategy=soft". I wouldn't expect this to >> >> behave differently than weak under memory pressure, but my unit test >> with >> >> soft is not releasing this memory. I'll try running the test with >> "weak" >> >> and see if that changes it. And I'll look at your project (thanks for >> >> that). >> >> >> > >> > To clarify for future readers, we're talking about objects being >> retained >> > by this reference path: >> > QueryCache -> cached_object -> ObjectContext -> >> > other_objects_that_weren't_cached >> > >> > Sorry, I was wrong about the cause here. The persistent objects ARE >> > released by the context's objectStore (whether using weak or soft). But >> the >> > context is still retaining a lot of extra memory. I'm having a hard time >> > determining the specific cause. It might be the dataRowCache? >> >> If you have "Use Shared Cache" unchecked in the modeler, then you get a >> single dataRowCache per context, so that would definitely explain it. I am >> using the default - one dataRowCache per stack, shared by all contexts, so >> that's never a problem. >> > > I've narrowed it down - the "extra" memory being retained by the > ObjectContext is: entries in ObjectStore.objectMap - but not DataObjects > themselves (those get cleared), it's just the references to those objects: > the mapping from ObjectId -> WeakReference. These entries stay present > after the WeakReference is cleared. All those ObjectIds (though small) add > up to a significant amount of memory over time. It looks like these are > supposed to be cleared ReferenceMap.checkReferenceQueue(), which calls > ReferenceQueue.poll() to find the cleared WeakReferences and remove those > entries from the ObjectStore's objectMap. However, poll() doesn't ever seem > to return any results (a cleared WeakReference). If I add in a call to > ReferenceQueue.remove(5) manually (which will block for five milliseconds > while it finds cleared references), it does return them and clear that > memory. I need to read more about ReferenceQueue, but the current > implementation of checkReferenceQueue() does not appear to be working since > poll() never returns anything. > This is actually a timing problem, not a problem with poll(). checkReferenceQueue() would need to be called *after* the context is done being used *and* the gc has occurred. As it currently is checkReferenceQueue() is called while the context is being used, which can help clear *some* memory that can be freed while in use, but doesn't help after the context is unused (like when it is just sitting being referenced by an object in the QueryCache. > >> >> Andrus >> >>
