[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-12-12 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994965#comment-16994965
 ] 

ASF subversion and git services commented on JENA-1785:
---

Commit 19a2480958fbf7e58647c112f60bf632826a50fc in jena's branch 
refs/heads/master from Andy Seaborne
[ https://gitbox.apache.org/repos/asf?p=jena.git;h=19a2480 ]

JENA-1785: Comment up and cleanup after fix


> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Fix For: Jena 3.14.0
>
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-12-10 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992440#comment-16992440
 ] 

ASF subversion and git services commented on JENA-1785:
---

Commit e877e6d305606ec9da655d69aba1dbfdbcacbcf7 in jena's branch 
refs/heads/master from strangepleasures
[ https://gitbox.apache.org/repos/asf?p=jena.git;h=e877e6d ]

JENA-1785: A newly created node can remain invisible after commit


> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-12-10 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992441#comment-16992441
 ] 

ASF subversion and git services commented on JENA-1785:
---

Commit cad8fa022672e875cf8c55eaaa523fb60740e46c in jena's branch 
refs/heads/master from Andy Seaborne
[ https://gitbox.apache.org/repos/asf?p=jena.git;h=cad8fa0 ]

Merge pull request #646 from strangepleasures/JENA-1785

JENA-1785: A newly created node can remain invisible after commit

> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-12-10 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992442#comment-16992442
 ] 

ASF subversion and git services commented on JENA-1785:
---

Commit cad8fa022672e875cf8c55eaaa523fb60740e46c in jena's branch 
refs/heads/master from Andy Seaborne
[ https://gitbox.apache.org/repos/asf?p=jena.git;h=cad8fa0 ]

Merge pull request #646 from strangepleasures/JENA-1785

JENA-1785: A newly created node can remain invisible after commit

> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-12-07 Thread Andy Seaborne (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990432#comment-16990432
 ] 

Andy Seaborne commented on JENA-1785:
-

Then I think [PR646|https://github.com/apache/jena/pull/646] is a good way to 
go. I have some suggestions for that to simplify and remove some locking but 
the general approach looks right.

I'll take the detailed discussion to the PR on GH.

> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-12-06 Thread Pavel Mikhailovskii (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989901#comment-16989901
 ] 

Pavel Mikhailovskii commented on JENA-1785:
---

This is indeed a correct explanation of the problem.
To put it short, the problem is that even though newly added nodes are removed 
from the local version of the not-present cache, they don't get removed from 
the global cache on commit. 

> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-12-06 Thread Andy Seaborne (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989698#comment-16989698
 ] 

Andy Seaborne commented on JENA-1785:
-

Here is my understanding of the problem: this is mainly to make sure I 
understand the details here!

The NodeTable Cache should reflect the node table. The NodeTable is append-only 
and it does not matter if nodes are added to it by later transactions; what is 
in the data is determined by the  triples/quads, not by presence in the node 
table.

Inside a write-transaction things are different because the W-txn may abort. 
Nodes created should not be visible outside the W-txn until it commits 
(JENA-1746). The fact nodes are cleared up from storage after an abort was the 
problem in JENA-1746.

The underlying disk storage, a TransBinaryDataFile, reflects transactions. In a 
W-txn, there is the visible part (all still running read-transactions) and the 
additional nodes of the W-txn.

NodeTableCache has buffering caches that attempts to reflect this.

These are ThreadBufferingCache, which is a pair of normal LRU caches, one 
"local" for the W-txn nodes and one "base" for the globally visible cache of 
the node table.

The same structure is used for the not-present cache.

Problem:

A node is initially recorded as "not present". It is now in the 
base-not-present cache.

A W-txn adds this node, putting it in the node<->nodeId caches as well as 
writing through to the TransBinaryDataFile. It is removed from the 
not-present-local cache if in there. The code correctly checks to this point 
because the node<->nodeId caching is checked before not-present.

However, problems arise if it falls out of the local node/nodeId cache.

Now, a lookup will not find it in a local cache, but will get the wrong answer
when it finds it in the base caching because it is in the base-not-present 
cache.




> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-11-30 Thread Pavel Mikhailovskii (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985446#comment-16985446
 ] 

Pavel Mikhailovskii commented on JENA-1785:
---

The problem with the miss cache making its correct implementation very 
difficult is that it requires not only application or rollback of changes made 
in write transactions but also proper handling of different versions of the 
cache even for read transactions - a long-running read transaction started a 
certain number of commits ago should see the database at that point in time 
(Transaction.dataVersion) and ignore all nodes added in subsequent commits.  


> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-11-30 Thread Andy Seaborne (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985399#comment-16985399
 ] 

Andy Seaborne commented on JENA-1785:
-

The updated one fails nearly every time for me - bumping up the 1000 to 2000 
makes it certain. Hashing is involved so it isn't so surprising.

> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-11-29 Thread Pavel Mikhailovskii (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985087#comment-16985087
 ] 

Pavel Mikhailovskii commented on JENA-1785:
---

I'm currently working on a better fix which will keep the miss cache.

> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-11-29 Thread Pavel Mikhailovskii (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985086#comment-16985086
 ] 

Pavel Mikhailovskii commented on JENA-1785:
---

That's strange. It's failing every time on my machine. It also looks quite 
deterministic. Maybe the underlying Guava caches add some non-determinism, e.g. 
asynchronous  eviction of entities?

> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-11-29 Thread Andy Seaborne (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985082#comment-16985082
 ] 

Andy Seaborne commented on JENA-1785:
-

TestVisibilityOfChanges (first version) is only failing for me intermittently 
(1 in 5 maybe). Is this what you see?

> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-11-29 Thread Pavel Mikhailovskii (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984881#comment-16984881
 ] 

Pavel Mikhailovskii commented on JENA-1785:
---

Actually, a newly created resource may become invisible even in same 
transaction (see the updated test).


> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (JENA-1785) A newly created node can remain invisible after commit

2019-11-28 Thread Andy Seaborne (Jira)


[ 
https://issues.apache.org/jira/browse/JENA-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984489#comment-16984489
 ] 

Andy Seaborne commented on JENA-1785:
-

Thank you for bring this up. It will will need some thinking about. In TDB1, 
the "miss" cache makes a difference because it stops going to the storage node 
table.

But the node cache is an area of difference between TDB1 and TDB2. 

>From a first pass over TDB2 NodeTableCache java comments, it looks like it may 
>be a failure to manage the miss cache correctly.

> A newly created node can remain invisible after commit
> --
>
> Key: JENA-1785
> URL: https://issues.apache.org/jira/browse/JENA-1785
> Project: Apache Jena
>  Issue Type: Bug
>  Components: TDB2
>Affects Versions: Jena 3.13.0, Jena 3.13.1
>Reporter: Pavel Mikhailovskii
>Assignee: Andy Seaborne
>Priority: Critical
> Attachments: TestVisibilityOfChanges.java
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> A node once marked as non-present (_NodeTableCache.nonPresent_) can remain 
> invisible even after it's created and the transaction is committed. That 
> might happen because there's no guarantee that *all* newly created nodes will 
> be eventually added to the "base" version _ThreadBufferingCache.baseCache_ of 
> the _node2id_Cache_ (as the _localCache_ has limited capacity) or removed 
> from the "base" version of the _nonPresent_ cache (even if they were, there 
> would still be a chance of re-adding them by some read transaction). 
> The simplest fix is to get rid of the _nonPresent_ cache which seems to be of 
> limited use anyway. A more sophisticated fix would involve keeping track of 
> all newly allocated nodes and their removal from the base version of 
> _nonPresent_ cache on transaction commit.
> To reproduce: see the attached test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)