[
https://issues.apache.org/jira/browse/OAK-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-1760:
---------------------------------
Attachment: OAK-1760.patch
Proposed patch after discussing with [~mduerig] and [~mreutegg]
Current logic relies on state being copied when actual diff is performed in
merge. This however leads to a one big branch commit which performs poorly in
DocumentNodeStore. Patch changes the Upgrade logic to pre emptively copy the
NodeState from JR2 to NodeStore before actual merge which leads to small
commits (~10000 changes) and should perform better
DocumentNodeStore still maintains some state proportional to the number of
nodes in form of path-> rev mapping. However this should be manageable to store
in memory even for decent size repositories
[~jukkaz]] [~mduerig] and [~mreutegg] can you review the patch
> RepositoryUpgrade leads to one large commit with DocumentNodeStore causing OOM
> ------------------------------------------------------------------------------
>
> Key: OAK-1760
> URL: https://issues.apache.org/jira/browse/OAK-1760
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk, upgrade
> Affects Versions: 0.20
> Reporter: Chetan Mehrotra
> Attachments: OAK-1760.patch
>
>
> RepositoryUpgrade logic currently creates when migrating content from JR2 to
> Oak currently creates a one large commit. This causes issue with
> DocumentNodeStore as all the commit state is maintained in memory in
> Commit#operations variable which leads to OOM when source repository content
> is large.
> On Analyzing the heap dump it can be seen that
> {{o.a.j.o.plugins.document.Commit#operations}} map contains large number of
> UpdateOp entries. These entries are created due to the diff being performed
> in {{o.a.j.o.plugins.document.DocumentNodeStoreBranch#persist}} which
> traverse down the whole tree (source being JR2 tree) and captures the diff as
> UpdateOp. As commit does not get applied till whole diff is complete the in
> memory state is proportional to repository size
--
This message was sent by Atlassian JIRA
(v6.2#6252)
