[ https://issues.apache.org/jira/browse/JENA-624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15041305#comment-15041305 ]
Andy Seaborne commented on JENA-624: ------------------------------------ We will also need to evaluate whether the code needs to {{.remove}} TheradLocals as it clears up a transaction. It is a trade off of removing vs reusing for the same thread. This why I am seeking to minimize their use. Elsewhere [TransactionalComponentLifecycle#L136|https://github.com/afs/mantis/blob/master/dboe-transaction/src/main/java/org/seaborne/dboe/transaction/txn/TransactionalComponentLifecycle.java#L136] ==> [TransactionalComponentLifecycle#L220|https://github.com/afs/mantis/blob/master/dboe-transaction/src/main/java/org/seaborne/dboe/transaction/txn/TransactionalComponentLifecycle.java#L220] The comment in the last one is experience that ThreadLocals cause objects to be held for much longer than expected and I was seeing total memory grow in transaction cycles even without threads being created/destroyed. A system that creates a thread, have it execute and then die, is worse again as the thread local and anything it can reference is still live. > Develop a new in-memory RDF Dataset implementation > -------------------------------------------------- > > Key: JENA-624 > URL: https://issues.apache.org/jira/browse/JENA-624 > Project: Apache Jena > Issue Type: Improvement > Reporter: Andy Seaborne > Assignee: A. Soroka > Labels: java, linked_data, rdf > > The current (Jan 2014) Jena in-memory dataset uses a general purpose > container that works for any storage technology for graphs together with > in-memory graphs. > This project would develop a new implementation design specifically for RDF > datasets (triples and quads) and efficient SPARQL execution, for example, > using multi-core parallel operations and/or multi-version concurrent > datastructures to maximise true parallel operation. > This is a system project suitable for someone interested in datatbase > implementation, datastructure design and implementation, operating systems or > distributed systems. > Note that TDB can operate in-memory using a simulated disk with > copy-in/copy-out semantics for disk-level operations. It is for faithful > testing TDB infrastructure and is not designed performance, general in-memory > use or use at scale. While lesson may be learnt from that system, TDB > in-memory is not the answer here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)