[jira] [Commented] (OAK-3007) SegmentStore cache does not take "string" map into account
[ https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299717#comment-15299717 ] Alex Parvulescu commented on OAK-3007: -- Merged to 1.2 branch with r1745461. > SegmentStore cache does not take "string" map into account > -- > > Key: OAK-3007 > URL: https://issues.apache.org/jira/browse/OAK-3007 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segmentmk >Reporter: Thomas Mueller >Assignee: Michael Dürig > Labels: candidate_oak_1_0, doc-impacting, resilience, scalability > Fix For: 1.4, 1.3.3, 1.2.16 > > Attachments: OAK-3007-2.patch, OAK-3007-3.patch, OAK-3007.patch > > > The SegmentStore cache size calculation ignores the size of the field > Segment.string (a concurrent hash map). It looks like a regular segment in a > memory mapped file has the size 1024, no matter how many strings are loaded > in memory. This can lead to out of memory. There seems to be no way to limit > (configure) the amount of memory used by strings. In one example, 100'000 > segments are loaded in memory, and 5 GB are used for Strings in that map. > We need a way to configure the amount of memory used for that. This seems to > be basically a cache. OAK-2688 does this, but it would be better to have one > cache with a configurable size limit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3007) SegmentStore cache does not take string map into account
[
https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14620405#comment-14620405
]
Thomas Mueller commented on OAK-3007:
-
For future reference, a benchmark I ran was:
{noformat}
java -Dwarmup=5 -Druntime=2 -Dprofile=true -jar target/oak-run-1.4-SNAPSHOT.jar
benchmark ReadPropertyTest Oak-Tar | grep Segment.read
{noformat}
This does not show readString, but many readTemplate calls. Possibly the
following tests could be run as well:
ReadDeepTreeTest
ConcurrentReadDeepTreeTest
ConcurrentReadRandomNodeAndItsPropertiesTest
ConcurrentReadTest
ConcurrentTraversalTest
DescendantSearchTest
GetDeepNodeTest
GetNodeWithAdmin
ReadPropertyTest
SQL2DescendantSearchTest
SQL2SearchTest
SimpleSearchTest
SegmentStore cache does not take string map into account
--
Key: OAK-3007
URL: https://issues.apache.org/jira/browse/OAK-3007
Project: Jackrabbit Oak
Issue Type: Bug
Components: segmentmk
Reporter: Thomas Mueller
Fix For: 1.3.3
Attachments: OAK-3007-2.patch, OAK-3007-3.patch, OAK-3007.patch
The SegmentStore cache size calculation ignores the size of the field
Segment.string (a concurrent hash map). It looks like a regular segment in a
memory mapped file has the size 1024, no matter how many strings are loaded
in memory. This can lead to out of memory. There seems to be no way to limit
(configure) the amount of memory used by strings. In one example, 100'000
segments are loaded in memory, and 5 GB are used for Strings in that map.
We need a way to configure the amount of memory used for that. This seems to
be basically a cache. OAK-2688 does this, but it would be better to have one
cache with a configurable size limit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3007) SegmentStore cache does not take string map into account
[ https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618694#comment-14618694 ] Thomas Mueller commented on OAK-3007: - While writing tests, I found and fixed some issues and changed the API a bit to simplify testing. Still working on this. SegmentStore cache does not take string map into account -- Key: OAK-3007 URL: https://issues.apache.org/jira/browse/OAK-3007 Project: Jackrabbit Oak Issue Type: Bug Components: segmentmk Reporter: Thomas Mueller Fix For: 1.3.3 Attachments: OAK-3007-2.patch, OAK-3007.patch The SegmentStore cache size calculation ignores the size of the field Segment.string (a concurrent hash map). It looks like a regular segment in a memory mapped file has the size 1024, no matter how many strings are loaded in memory. This can lead to out of memory. There seems to be no way to limit (configure) the amount of memory used by strings. In one example, 100'000 segments are loaded in memory, and 5 GB are used for Strings in that map. We need a way to configure the amount of memory used for that. This seems to be basically a cache. OAK-2688 does this, but it would be better to have one cache with a configurable size limit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3007) SegmentStore cache does not take string map into account
[ https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616318#comment-14616318 ] Michael Dürig commented on OAK-3007: In some ad-hoc testing this patch together with the one from OAK-3055 proved beneficial. Pending the failing ITs and your review of OAK-3055 I think we should apply the patches to trunk. As soon as we have enough longevity coverage we should also merge them to the branches. SegmentStore cache does not take string map into account -- Key: OAK-3007 URL: https://issues.apache.org/jira/browse/OAK-3007 Project: Jackrabbit Oak Issue Type: Bug Components: segmentmk Reporter: Thomas Mueller Fix For: 1.3.3 Attachments: OAK-3007-2.patch, OAK-3007.patch The SegmentStore cache size calculation ignores the size of the field Segment.string (a concurrent hash map). It looks like a regular segment in a memory mapped file has the size 1024, no matter how many strings are loaded in memory. This can lead to out of memory. There seems to be no way to limit (configure) the amount of memory used by strings. In one example, 100'000 segments are loaded in memory, and 5 GB are used for Strings in that map. We need a way to configure the amount of memory used for that. This seems to be basically a cache. OAK-2688 does this, but it would be better to have one cache with a configurable size limit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3007) SegmentStore cache does not take string map into account
[ https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616304#comment-14616304 ] Thomas Mueller commented on OAK-3007: - Thomas Mueller, could you also add a unit test for StringCache? Sure. I believe that the root cause for the compaction case filling up the cache is is OAK-3075. but this patch should be applied nonetheless. As far as I understand, OAK-3075 is not the reason, but yes I understand that some other issue caused that this many strings are loaded. It would be nice to find the root cause. In order to find the root cause, the patch here (OAK-3007) could be applied and the compaction test could be re-run. That should no longer result in out-of-memory in SegmentStore cache, but in slow compaction, which in turn can be analyzed using a profiler. Or in yet another out-of-memory (but in a different place), which in turn can be analyzed... So the current patch of OAK-3007 should help to understand the _real_ problem. but this patch should be applied nonetheless. Yes, I think applying this patch eventually to trunk (after writing more unit tests) would be good. Backporting it might not be all that urgent I guess. SegmentStore cache does not take string map into account -- Key: OAK-3007 URL: https://issues.apache.org/jira/browse/OAK-3007 Project: Jackrabbit Oak Issue Type: Bug Components: segmentmk Reporter: Thomas Mueller Fix For: 1.3.3 Attachments: OAK-3007-2.patch, OAK-3007.patch The SegmentStore cache size calculation ignores the size of the field Segment.string (a concurrent hash map). It looks like a regular segment in a memory mapped file has the size 1024, no matter how many strings are loaded in memory. This can lead to out of memory. There seems to be no way to limit (configure) the amount of memory used by strings. In one example, 100'000 segments are loaded in memory, and 5 GB are used for Strings in that map. We need a way to configure the amount of memory used for that. This seems to be basically a cache. OAK-2688 does this, but it would be better to have one cache with a configurable size limit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3007) SegmentStore cache does not take string map into account
[
https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616306#comment-14616306
]
Michael Dürig commented on OAK-3007:
A problem I encountered with the patch is {{FileStore#FileStore(java.io.File,
org.apache.jackrabbit.oak.spi.state.NodeState, int)}} throwing an {{ISAE}}.
That constructor should create a {{FileStore}} without cache. So far it did so
by specifying a cache size of 0 to the {{SegmentTracker}}. With the LIRS cache
this stopped working.
Just run {{CompactionAndCleanupTest#propertyRetention}} to see this.
SegmentStore cache does not take string map into account
--
Key: OAK-3007
URL: https://issues.apache.org/jira/browse/OAK-3007
Project: Jackrabbit Oak
Issue Type: Bug
Components: segmentmk
Reporter: Thomas Mueller
Fix For: 1.3.3
Attachments: OAK-3007-2.patch, OAK-3007.patch
The SegmentStore cache size calculation ignores the size of the field
Segment.string (a concurrent hash map). It looks like a regular segment in a
memory mapped file has the size 1024, no matter how many strings are loaded
in memory. This can lead to out of memory. There seems to be no way to limit
(configure) the amount of memory used by strings. In one example, 100'000
segments are loaded in memory, and 5 GB are used for Strings in that map.
We need a way to configure the amount of memory used for that. This seems to
be basically a cache. OAK-2688 does this, but it would be better to have one
cache with a configurable size limit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3007) SegmentStore cache does not take string map into account
[ https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614846#comment-14614846 ] Alex Parvulescu commented on OAK-3007: -- I believe that the root cause for the compaction case filling up the cache is is OAK-3075. but this patch should be applied nonetheless. SegmentStore cache does not take string map into account -- Key: OAK-3007 URL: https://issues.apache.org/jira/browse/OAK-3007 Project: Jackrabbit Oak Issue Type: Bug Components: segmentmk Reporter: Thomas Mueller Fix For: 1.3.3 Attachments: OAK-3007-2.patch, OAK-3007.patch The SegmentStore cache size calculation ignores the size of the field Segment.string (a concurrent hash map). It looks like a regular segment in a memory mapped file has the size 1024, no matter how many strings are loaded in memory. This can lead to out of memory. There seems to be no way to limit (configure) the amount of memory used by strings. In one example, 100'000 segments are loaded in memory, and 5 GB are used for Strings in that map. We need a way to configure the amount of memory used for that. This seems to be basically a cache. OAK-2688 does this, but it would be better to have one cache with a configurable size limit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3007) SegmentStore cache does not take string map into account
[
https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609783#comment-14609783
]
Michael Dürig commented on OAK-3007:
I think this approach makes a lot of sense and we should give it a spin.
[~tmueller], could you also add a unit test for {{StringCache}}?
SegmentStore cache does not take string map into account
--
Key: OAK-3007
URL: https://issues.apache.org/jira/browse/OAK-3007
Project: Jackrabbit Oak
Issue Type: Bug
Components: segmentmk
Reporter: Thomas Mueller
Fix For: 1.3.2
Attachments: OAK-3007-2.patch, OAK-3007.patch
The SegmentStore cache size calculation ignores the size of the field
Segment.string (a concurrent hash map). It looks like a regular segment in a
memory mapped file has the size 1024, no matter how many strings are loaded
in memory. This can lead to out of memory. There seems to be no way to limit
(configure) the amount of memory used by strings. In one example, 100'000
segments are loaded in memory, and 5 GB are used for Strings in that map.
We need a way to configure the amount of memory used for that. This seems to
be basically a cache. OAK-2688 does this, but it would be better to have one
cache with a configurable size limit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
