Adam, I ran with -XX:+HeapDumpOnOutOfMemoryError and got a ~180 MB dump file. I can get it to you if you like, but Steve and my analysis only turned up the concerns I expressed above. About 25 MB was the "seralizedValue" byte array of BTreePersistentIndexedCache$DataBlock. About 30 MB (I think) was the file full paths stored as String keys in DefaultFileSnapshotter$FileCollectionSnapshotImpl. This does not fully explain the OOM when running with -Xmx256m. I bumped it to 300m and it still was not enough, but 350m was. So, we are running right now with -Xmx384m, but at some point that will probably start failing as well. I don't think this is a "drop everything" type of problem, but I would like to get it addressed before 1.0 ships.
On Wed, May 12, 2010 at 2:21 AM, Adam Murdoch <[email protected]> wrote: > > > On 11/05/10 7:36 AM, John Murph wrote: > >> We are getting OOMs occasionally. These seems to happen for many (if not >> all) our various builds on our project, but not consistently for any of >> them. Each of these builds is run with a -Xmx of 256m. An example stack >> trace is given below. It seems related to caching of task archives, and my >> .bin file is ~30 MB right now. I think (but might be wrong) that this file >> is cached in memory (with other Java overhead), meaning that a lot of memory >> might be used by this cache. >> > > Chunks of it are loaded into memory as they are needed and discarded. The > whole file is not loaded into memory. > > > Do you guys think this is a potential problem area, or should I be >> looking elsewhere? >> > > It might be a problem. Or it might be something else and you are seeing the > trigger rather than the cause. Can you run Gradle with > GRADLE_OPTS=-XX:+HeapDumpOnOutOfMemoryError and we can see what the heap > dump reveals? > > > > P.S. It looks like default serialization is used for much of this data, >> leading to a bloated .bin file. I zipped it and got ~4.7 MB for the same >> file, and looking into the .bin file shows lots of repeated information. >> First, the class names of the serialized objects get repeated a lot. >> Secondly, the pathing for the input/output files are the same so much of >> that is also redundant. This might not be related to the OOMs, but it >> seemed interesting. >> > > These are good points. > > Default serialization is easy, particularly when things are changing. At > some point we could look at customising the serialization, to make things > more efficient and/or to get some better control over versioning. Right now, > it's good enough, I think. Same with the absolute paths - we could encode > them relative to some base dir, say, and save some space, but it's good > enough for now. > > > -- > Adam Murdoch > Gradle Developer > http://www.gradle.org > > > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > -- John Murph Automated Logic Research Team
