[jira] [Commented] (OAK-8066) Nodes with many direct children can lead to OOME when saving
[
https://issues.apache.org/jira/browse/OAK-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911256#comment-16911256
]
Francesco Mari commented on OAK-8066:
-
Backported to 1.8 at 1865533.
> Nodes with many direct children can lead to OOME when saving
>
>
> Key: OAK-8066
> URL: https://issues.apache.org/jira/browse/OAK-8066
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: TarMK
> Fix For: 1.12.0, 1.10.5
>
>
> {{DefaultSegmentWriter}} keeps a map of [child
> nodes|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriter.java#L805]
> of a node being written. This can lead to high memory consumption in the
> case where many child nodes are added at the same time. The latter could
> happen in the case where a node needs to be rewritten because of an increase
> in the GC generation from a concurrently completed revision garbage
> collection.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OAK-8066) Nodes with many direct children can lead to OOME when saving
[
https://issues.apache.org/jira/browse/OAK-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911234#comment-16911234
]
Francesco Mari commented on OAK-8066:
-
Backported to 1.10 at 1865531.
> Nodes with many direct children can lead to OOME when saving
>
>
> Key: OAK-8066
> URL: https://issues.apache.org/jira/browse/OAK-8066
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: TarMK
> Fix For: 1.12.0
>
>
> {{DefaultSegmentWriter}} keeps a map of [child
> nodes|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriter.java#L805]
> of a node being written. This can lead to high memory consumption in the
> case where many child nodes are added at the same time. The latter could
> happen in the case where a node needs to be rewritten because of an increase
> in the GC generation from a concurrently completed revision garbage
> collection.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OAK-8066) Nodes with many direct children can lead to OOME when saving
[
https://issues.apache.org/jira/browse/OAK-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781739#comment-16781739
]
Michael Dürig commented on OAK-8066:
Fixed at http://svn.apache.org/viewvc?rev=1854596&view=rev
[~ahanikel], [~frm] could you double check my changes?
> Nodes with many direct children can lead to OOME when saving
>
>
> Key: OAK-8066
> URL: https://issues.apache.org/jira/browse/OAK-8066
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: TarMK
> Fix For: 1.12
>
>
> {{DefaultSegmentWriter}} keeps a map of [child
> nodes|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriter.java#L805]
> of a node being written. This can lead to high memory consumption in the
> case where many child nodes are added at the same time. The latter could
> happen in the case where a node needs to be rewritten because of an increase
> in the GC generation from a concurrently completed revision garbage
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-8066) Nodes with many direct children can lead to OOME when saving
[
https://issues.apache.org/jira/browse/OAK-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781633#comment-16781633
]
Michael Dürig commented on OAK-8066:
Test case at http://svn.apache.org/viewvc?rev=1854579&view=rev
> Nodes with many direct children can lead to OOME when saving
>
>
> Key: OAK-8066
> URL: https://issues.apache.org/jira/browse/OAK-8066
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: TarMK
> Fix For: 1.12
>
>
> {{DefaultSegmentWriter}} keeps a map of [child
> nodes|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriter.java#L805]
> of a node being written. This can lead to high memory consumption in the
> case where many child nodes are added at the same time. The latter could
> happen in the case where a node needs to be rewritten because of an increase
> in the GC generation from a concurrently completed revision garbage
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-8066) Nodes with many direct children can lead to OOME when saving
[
https://issues.apache.org/jira/browse/OAK-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773959#comment-16773959
]
Michael Dürig commented on OAK-8066:
Filed OAK-8069 for adding logging to better understand the situation.
> Nodes with many direct children can lead to OOME when saving
>
>
> Key: OAK-8066
> URL: https://issues.apache.org/jira/browse/OAK-8066
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: TarMK
>
> {{DefaultSegmentWriter}} keeps a map of [child
> nodes|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriter.java#L805]
> of a node being written. This can lead to high memory consumption in the
> case where many child nodes are added at the same time. The latter could
> happen in the case where a node needs to be rewritten because of an increase
> in the GC generation from a concurrently completed revision garbage
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-8066) Nodes with many direct children can lead to OOME when saving
[
https://issues.apache.org/jira/browse/OAK-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773822#comment-16773822
]
Michael Dürig commented on OAK-8066:
One way to fix this is to regularly flush the accumulated child nodes into a
map record. See [https://github.com/mduerig/jackrabbit-oak/tree/OAK-8066] for
WIP.
/cc [~frm]
> Nodes with many direct children can lead to OOME when saving
>
>
> Key: OAK-8066
> URL: https://issues.apache.org/jira/browse/OAK-8066
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: TarMK
>
> {{DefaultSegmentWriter}} keeps a map of [child
> nodes|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriter.java#L805]
> of a node being written. This can lead to high memory consumption in the
> case where many child nodes are added at the same time. The latter could
> happen in the case where a node needs to be rewritten because of an increase
> in the GC generation from a concurrently completed revision garbage
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
