[
https://issues.apache.org/jira/browse/JENA-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081295#comment-13081295
]
Simon Helsen commented on JENA-91:
----------------------------------
There is some additional observation. When I change the line ByteBuffer bb =
ByteBuffer.allocate(len) ; into ByteBuffer bb = ByteBuffer.allocateDirect(len)
; I notice something quite remarkable: I can see short, but massive memory
spikes. E.g. for the most part the memory occupied by my server takes about
500Mb, but it sometimes spikes to 1.8 or 2.3 in case even 3.8Gb, but only for a
very short moment and then goes back its "normal" consumption
This change does permit me to keep running my tests, but obviously this is not
a desirable change since allocateDirect relies on memory-mapped I/O. Strangely,
I did see the same problems when running TDB as a whole in memory mapped I/O.
I don't understand enough about how this bit of code is supposed to work to
debug this in other further meaningful way. Perhaps I need to understand first
why the allocations are so large. Also, do you run tests on windows systems
once in a while?
> extremely large buffer is being created in ObjectFileStorage
> ------------------------------------------------------------
>
> Key: JENA-91
> URL: https://issues.apache.org/jira/browse/JENA-91
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: Windows (and I presume any little endian system)
> Reporter: Simon Helsen
> Priority: Critical
>
> I tried to debug the OME and check why a bytebuffer is causing my native
> memory to explode in almost no time. It all seems to happen in this bit of
> code in com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage (lines 243
> onwards)
> // No - it's in the underlying file storage.
> lengthBuffer.clear() ;
> int x = file.read(lengthBuffer, loc) ;
> if ( x != 4 )
> throw new
> FileException("ObjectFile.read("+loc+")["+filesize+"]["+file.size()+"]:
> Failed to read the length : got "+x+" bytes") ;
> int len = lengthBuffer.getInt(0) ;
> ByteBuffer bb = ByteBuffer.allocate(len) ;
> My debugger shows that x==4. It also shows the lengthBuffer has the following
> content: [111, 110, 61, 95]. This amounts to the value of len=1869495647,
> which is rather a lot :-) Obviously, the next statement (ByteBuffer.allocate)
> causes the OME.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
