Hi Andy, It's 32-bit -- no matter how much memory I feed the JVM, this always comes up. At first, I thought I wasn't giving it enough RAM - but it was only 80k triples and no amount of memory seemed enough. That's when I realized this particular program was "RDFizing" web content - in this case blog posts. So the literals could be quite large, e.g. several kilobytes.
I'll see if I can create a simple test case to reproduce the issue. On a related note, I just tried TDB on a 32bit and a 64bit 512 Linode - perhaps because of this issue, perhaps the app uses too much memory as it is. Either way, in both instances as that VM approaches complete memory usage on the server and it starts to swap - everything slows down to be completely unusable. At first I thought TDB 64-bit would help, as the memory-mapped files surely wouldn't wind up grinding the swap, but it only appeared to get to maximum memory usage faster than when on a 32bit system. It would be highly desirable if there was a "disable caching" for low-memory environments. I started poking around the TDB source tree last night, but nothing jumped out as where the caching would be - can you point me in the right direction here? Al On Mon, Jul 18, 2011 at 4:10 AM, Andy Seaborne < [email protected]> wrote: > > > On 18/07/11 07:17, Al Baker wrote: > >> Hi All, >> >> I've noticed after creating a number of triples, eventually all TDB calls >> fail around. Some of of the objects stored are large strings (e.g. >> several >> k). >> >> java.lang.OutOfMemoryError: Java heap space >> at java.nio.HeapByteBuffer.<init>**(HeapByteBuffer.java:39) >> at java.nio.ByteBuffer.allocate(**ByteBuffer.java:312) >> at >> com.hp.hpl.jena.tdb.base.**objectfile.ObjectFileStorage.** >> read(ObjectFileStorage.java:**249) >> at com.hp.hpl.jena.tdb.lib.**NodeLib.fetchDecode(NodeLib.**java:60) >> >> This fails on 0.8.10 or the 0.9 TxTDB. >> >> Al >> >> > Hi Al, > > How much RAM are you giving the process? 32-bit or 64-bit JVM? > > Many very large stings might cause the node caching to use an excessive > about of memory (this area has not changed between 0.8. and 0.9). > > Aside spaces sides, it might be better to store large strings in a separate > file space and point to them. (Maybe TDB should do that automatically.) > > Andy >
