[
https://issues.apache.org/jira/browse/JENA-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087196#comment-13087196
]
Simon Helsen commented on JENA-91:
----------------------------------
ok, so I made a different version of changeproc to reflect more what we do and
I start seeing the following warnings:
WARN [pool-1-thread-14] (TestTransSystem.java:246) - WRITER: Txn[5]/W Change
seen: 0 + 184 != 0 : id=5: i=0
WARN [pool-1-thread-14] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-1-thread-6] (TestTransSystem.java:246) - WRITER: Txn[20]/W Change
seen: 0 + 74 != 0 : id=9: i=0
WARN [pool-1-thread-6] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-1-thread-7] (TestTransSystem.java:246) - WRITER: Txn[28]/W Change
seen: 0 + 302 != 0 : id=12: i=0
WARN [pool-1-thread-7] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-1-thread-13] (TestTransSystem.java:246) - WRITER: Txn[30]/W Change
seen: 0 + 477 != 0 : id=13: i=0
WARN [pool-1-thread-13] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-1-thread-15] (TestTransSystem.java:246) - WRITER: Txn[37]/W Change
seen: 0 + 380 != 0 : id=15: i=0
WARN [pool-1-thread-15] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-1-thread-12] (TestTransSystem.java:246) - WRITER: Txn[35]/W Change
seen: 0 + 231 != 0 : id=14: i=0
WARN [pool-1-thread-12] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
. WARN [pool-2-thread-6] (TestTransSystem.java:246) - WRITER: Txn[54]/W Change
seen: 0 + 107 != 0 : id=18: i=0
WARN [pool-2-thread-6] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-2-thread-8] (TestTransSystem.java:246) - WRITER: Txn[56]/W Change
seen: 0 + 260 != 0 : id=19: i=0
WARN [pool-2-thread-8] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-2-thread-10] (TestTransSystem.java:246) - WRITER: Txn[57]/W Change
seen: 0 + 342 != 0 : id=20: i=0
WARN [pool-2-thread-10] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-2-thread-12] (TestTransSystem.java:246) - WRITER: Txn[60]/W Change
seen: 0 + 48 != 0 : id=22: i=0
WARN [pool-2-thread-12] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-2-thread-14] (TestTransSystem.java:246) - WRITER: Txn[58]/W Change
seen: 0 + 92 != 0 : id=21: i=0
WARN [pool-2-thread-14] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-2-thread-7] (TestTransSystem.java:246) - WRITER: Txn[68]/W Change
seen: 0 + 232 != 0 : id=23: i=0
WARN [pool-2-thread-7] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-2-thread-9] (TestTransSystem.java:246) - WRITER: Txn[72]/W Change
seen: 0 + 429 != 0 : id=24: i=0
WARN [pool-2-thread-9] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-2-thread-13] (TestTransSystem.java:246) - WRITER: Txn[76]/W Change
seen: 0 + 202 != 0 : id=28: i=0
WARN [pool-2-thread-13] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-2-thread-15] (TestTransSystem.java:246) - WRITER: Txn[73]/W Change
seen: 0 + 231 != 0 : id=26: i=0
WARN [pool-2-thread-15] (TestTransSystem.java:247) - Active (R=0 W=1) :
Finished (R=0, WC=0, WA=0) Queue 0
WARN [pool-2-thread-11] (TestTransSystem.java:246) - WRITER: Txn[80]/W Change
seen: 0 + 375 != 0 : id=25: i=0
Not sure if they are any help. This is my version of changeproc:
static int changeProc2(DatasetGraphTxn dsg, int id, int i)
{
Dataset ds = dsg.toDataset();
int count = 0 ;
int maxN = 500 ;
int N = RandomLib.qrandom.nextInt(maxN) ;
for ( int j = 0 ; j < N; j++ )
{
String rName = "test" + id + "_" + i + "_" + j;
Model m = ds.getNamedModel(rName);
m.removeAll();
Resource r = m.createResource(rName);
Property p =
m.createProperty("http://purl.org/dc/terms/created";);
Literal o = m.createLiteral("Andy");
m.add(r, p, o);
count++;
}
log.debug("Change = "+dsg.getDefaultGraph().size()) ;
return count ;
}
> extremely large buffer is being created in ObjectFileStorage
> ------------------------------------------------------------
>
> Key: JENA-91
> URL: https://issues.apache.org/jira/browse/JENA-91
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: Windows (and I presume any little endian system)
> Reporter: Simon Helsen
> Assignee: Andy Seaborne
> Priority: Critical
> Attachments: TestTransSystem.patch
>
>
> I tried to debug the OME and check why a bytebuffer is causing my native
> memory to explode in almost no time. It all seems to happen in this bit of
> code in com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage (lines 243
> onwards)
> // No - it's in the underlying file storage.
> lengthBuffer.clear() ;
> int x = file.read(lengthBuffer, loc) ;
> if ( x != 4 )
> throw new
> FileException("ObjectFile.read("+loc+")["+filesize+"]["+file.size()+"]:
> Failed to read the length : got "+x+" bytes") ;
> int len = lengthBuffer.getInt(0) ;
> ByteBuffer bb = ByteBuffer.allocate(len) ;
> My debugger shows that x==4. It also shows the lengthBuffer has the following
> content: [111, 110, 61, 95]. This amounts to the value of len=1869495647,
> which is rather a lot :-) Obviously, the next statement (ByteBuffer.allocate)
> causes the OME.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
