[
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989698#comment-16989698
]
Andy Seaborne edited comment on JENA-1785 at 12/6/19 12:27 PM:
---------------------------------------------------------------
Here is my understanding of the problem: this is mainly to make sure I
understand the details here!
----
The NodeTable Cache should reflect the node table. The NodeTable is append-only
and it does not matter if nodes are added to it by later transactions; what is
in the data is determined by the triples/quads, not by presence in the node
table.
Inside a write-transaction things are different because the W-txn may abort.
Nodes created should not be visible outside the W-txn until it commits
(JENA-1746). The fact nodes are cleared up from storage after an abort was the
problem in JENA-1746.
The underlying disk storage, a TransBinaryDataFile, reflects transactions. In a
W-txn, there is the visible part (all still running read-transactions) and the
additional nodes of the W-txn.
NodeTableCache has buffering caches that attempts to reflect this.
These are ThreadBufferingCache, which is a pair of normal LRU caches, one
"local" for the W-txn nodes and one "base" for the globally visible cache of
the node table.
The same structure is used for the not-present cache.
Problem:
A node is initially recorded as "not present". It is now in the
base-not-present cache.
A W-txn adds this node, putting it in the node/nodeId caches as well as writing
through to the TransBinaryDataFile. It is removed from the not-present-local
cache if in there. The code correctly checks to this point because the
node/nodeId caching is checked before not-present.
However, problems arise if it falls out of the local node/nodeId cache.
Now, a lookup will not find it in a local cache, but will get the wrong answer
when it finds it in the base caching because it is in the base-not-present
cache.
was (Author: andy.seaborne):
Here is my understanding of the problem: this is mainly to make sure I
understand the details here!
----
The NodeTable Cache should reflect the node table. The NodeTable is append-only
and it does not matter if nodes are added to it by later transactions; what is
in the data is determined by the triples/quads, not by presence in the node
table.
Inside a write-transaction things are different because the W-txn may abort.
Nodes created should not be visible outside the W-txn until it commits
(JENA-1746). The fact nodes are cleared up from storage after an abort was the
problem in JENA-1746.
The underlying disk storage, a TransBinaryDataFile, reflects transactions. In a
W-txn, there is the visible part (all still running read-transactions) and the
additional nodes of the W-txn.
NodeTableCache has buffering caches that attempts to reflect this.
These are ThreadBufferingCache, which is a pair of normal LRU caches, one
"local" for the W-txn nodes and one "base" for the globally visible cache of
the node table.
The same structure is used for the not-present cache.
Problem:
A node is initially recorded as "not present". It is now in the
base-not-present cache.
A W-txn adds this node, putting it in the node<->nodeId caches as well as
writing through to the TransBinaryDataFile. It is removed from the
not-present-local cache if in there. The code correctly checks to this point
because the node<->nodeId caching is checked before not-present.
However, problems arise if it falls out of the local node/nodeId cache.
Now, a lookup will not find it in a local cache, but will get the wrong answer
when it finds it in the base caching because it is in the base-not-present
cache.
> A newly created node can remain invisible after commit
> ------------------------------------------------------
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
> Affects Versions: Jena 3.13.0, Jena 3.13.1
> Reporter: Pavel Mikhailovskii
> Assignee: Andy Seaborne
> Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain
> invisible even after it's created and the transaction is committed. That
> might happen because there's no guarantee that *all* newly created nodes will
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed
> from the "base" version of the _nonPresent_ cache (even if they were, there
> would still be a chance of re-adding them by some read transaction).
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of
> limited use anyway. A more sophisticated fix would involve keeping track of
> all newly allocated nodes and their removal from the base version of
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
