[ https://issues.apache.org/jira/browse/JENA-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426696#comment-16426696 ]
ASF GitHub Bot commented on JENA-1516: -------------------------------------- Github user afs commented on a diff in the pull request: https://github.com/apache/jena/pull/391#discussion_r179411949 --- Diff: jena-tdb/src/main/java/org/apache/jena/tdb/lib/NodeLib.java --- @@ -53,26 +52,32 @@ // Characters in IRIs that are illegal and cause SSE problems, but we wish to keep. final private static char MarkerChar = '_' ; final private static char[] invalidIRIChars = { MarkerChar , ' ' } ; + final private static int SIZE = 1024; + // Marshalling space. + final private static ByteBuffer workspace = ByteBuffer.allocate(SIZE); --- End diff -- No problem about the review timing - any PR is as far as I'm concerned on permanent review. The more eyes on concurrency the better. It is safe. It gets called from `NodeTableNative.writeNodeToTable` which itself (see its comment) is inside synchronization. Several things need to happen together - write the node and update the node cache - so the sync is further out in `NodeTableNative`. The comment on `encodeStore` where the byte buffer is used applies "Callers must synchronize to ensure writing is not concurrent." For decode, the ByteBuffer comes from `ObjectFileStorage.read`. I can add a comment to the `workspace` declaration. (I'd _like_ scoped statics inside methods.) > Concurrency issues with ObjectFileStorage > ----------------------------------------- > > Key: JENA-1516 > URL: https://issues.apache.org/jira/browse/JENA-1516 > Project: Apache Jena > Issue Type: Bug > Components: TDB > Affects Versions: Jena 3.5.0, Jena 3.6.0 > Reporter: Andy Seaborne > Assignee: Andy Seaborne > Priority: Critical > Fix For: Jena 3.7.0 > > Attachments: FusekiConcurrencyTest.java, LOG-360, log4j-t.properties > > > Original [user@jena email > report|https://lists.apache.org/thread.html/876e6f69b7953bd0ac91ab85f5687ff6f2d9dfe202d8553602ba7c42@%3Cusers.jena.apache.org%3E]. > This is under high load. In the attached test case, at least 50 queries and > 50 updates issues by 25 threads are necessary to get any sort of likelihood > of a failure. > Failures of alloc-write are exasperated by the fact flag for checking is not > volatile. > The database is not permanently corrupted, however, updates can be lost. > Errors include the ones below - this is not exhaustive. > Errors, generated with a v3.6.0 server: > Some are consequences of earlier errors. > {noformat} > rm -rf DB ; fuseki-basic --loc DB --update /ds > {noformat} > {noformat} > 21:15:56 qtp826865256-16 ERROR BindingTDB :: get1(?s) > java.nio.BufferOverflowException > at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:214) > at sun.nio.ch.IOUtil.read(IOUtil.java:200) > at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:741) > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727) > at > org.apache.jena.tdb.base.file.BufferChannelFile.read(BufferChannelFile.java:112) > 21:15:56 qtp826865256-13 ERROR BindingTDB :: get1(?p) > java.lang.IllegalArgumentException > at java.nio.Buffer.position(Buffer.java:244) > at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:216) > at sun.nio.ch.IOUtil.read(IOUtil.java:200) > at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:741) > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727) > at > org.apache.jena.tdb.base.file.BufferChannelFile.read(BufferChannelFile.java:112) > 21:15:56 qtp826865256-16 ERROR BindingTDB :: get1(?s) > org.apache.jena.tdb.base.file.FileException: > ObjectFileStorage.read[nodes](486)[filesize=2280][file.size()=2280]: Failed > to read the length : got 0 bytes > at > org.apache.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:341) > at > org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57) > at > org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57) > at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78) > 21:15:56 qtp826865256-15 ERROR BindingTDB :: get1(?o) > org.apache.jena.tdb.base.file.FileException: > ObjectFileStorage.read[nodes](59)[filesize=2326][file.size()=2326]: Failed to > read the length : got 0 bytes > at > org.apache.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:341) > at > org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57) > at > org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57) > at > org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57) > at > org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57) > at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78) > 21:15:56 qtp826865256-13 ERROR BindingTDB :: get1(?s) > java.nio.BufferOverflowException > at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:214) > at sun.nio.ch.IOUtil.read(IOUtil.java:200) > at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:741) > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727) > at > org.apache.jena.tdb.base.file.BufferChannelFile.read(BufferChannelFile.java:112) > at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78) > 21:15:56 qtp826865256-13 ERROR BindingTDB :: get1(?s) > java.lang.IllegalArgumentException > at java.nio.Buffer.position(Buffer.java:244) > at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:216) > at sun.nio.ch.IOUtil.read(IOUtil.java:200) > at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:741) > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727) > at > org.apache.jena.tdb.base.file.BufferChannelFile.read(BufferChannelFile.java:112) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)