Hey Max, Took a heap dump using two different scenarios. In test 1, I set the - Xmx to 100M, and in test 2 I set the max to 512M, and am comparing the difference between the number of objects across the two memory scenarios. In my code, I'm running a for-loop that executes 5000 times, and most of the objects present in JHat show up no more than 500 times, and in equal amounts, in either memory scenario (leads me to believe these are not the culprits). However, there are a handful of datastore classes that show a 1:1 relationship between the amount of memory I allow in each test. Following are the outputs from JHat:
TEST #1 (100M Max) 101489 instances of class com.google.storage.onestore.v3.OnestoreEntity $PropertyValue 101488 instances of class com.google.storage.onestore.v3.OnestoreEntity $Property 52846 instances of class com.google.storage.onestore.v3.OnestoreEntity $Path$Element 43623 instances of class com.google.storage.onestore.v3.OnestoreEntity $Path 29782 instances of class com.google.storage.onestore.v3.OnestoreEntity $Reference 15944 instances of class com.google.apphosting.api.DatastorePb$Query 13839 instances of class com.google.storage.onestore.v3.OnestoreEntity $EntityProto 9237 instances of class com.google.appengine.api.datastore.dev.LocalCompositeIndexManager $IndexComponentsOnlyQuery 6705 instances of class com.google.appengine.api.datastore.dev.LocalDatastoreService $LiveQuery 4613 instances of class com.google.appengine.api.datastore.dev.LocalDatastoreService$Profile $EntityGroup TEST #2 (512M Max) 579558 instances of class com.google.storage.onestore.v3.OnestoreEntity $Property 579558 instances of class com.google.storage.onestore.v3.OnestoreEntity $PropertyValue 25863 instances of class com.google.storage.onestore.v3.OnestoreEntity $Path$Element 24805 instances of class com.google.storage.onestore.v3.OnestoreEntity $Path 17683 instances of class com.google.storage.onestore.v3.OnestoreEntity $Reference 10564 instances of class com.google.apphosting.api.DatastorePb$Query 8441 instances of class com.google.appengine.api.datastore.dev.LocalDatastoreService $LiveQuery 6065 instances of class com.google.storage.onestore.v3.OnestoreEntity $EntityProto 5002 instances of class com.google.appengine.api.datastore.dev.LocalDatastoreService$Profile $EntityGroup >From what I can tell, the OnestoreEntity$Property and OnestoreEntity $PropertyValue classes are growing the most (and somewhat linearly) with the number of Datastore operations I execute (capped only by the amount of memory I allow the JVM). Now, my first inclination is to explain my error away on the grounds that the local datastore is "in- memory", and so if I don't allow enough JVM memory for my datastore, this is expected behavior. However, I'm only persisting 5,000 User entities (and not even getting that far). My "User" is not terribly complicated (6 properties, 4 Lists, and a Preferences object). The lists and preferences are Lazy and empty -- I'm not creating anything for these in my test -- just a User....plus I'm only populating one of the field properties (username) with a very short string. For me to run out of 512MB of memory after creating only 4k-ish users means that each of my User entities is taking up around 125k each. Accounting for other datastore components taking up memory, we could say that my entities are only taking up 100k each, leaving the datastore the other 25k, or 100MB of memory. That still seems like my entities are taking up way too much memory (hence some sort of leak?), though I guess I don't know what's going on with the OnestoreEntity class to simulate the datastore, so I can't say for sure. Any thoughts here would be much appreciated. If I can provide any more data for you, please let me know. Thanks! David On Mar 12, 9:37 pm, "Max Ross (Google)" <maxr+appeng...@google.com> wrote: > Thanks for the report David, this certainly seems suspicious. There is at > least one memory leak I'm aware of but it's related to transactions so > that's probably not what you're bumping into. Have you tried taking a heap > dump to see what exactly is building up? > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
