[ https://issues.apache.org/jira/browse/OAK-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stefan Egli updated OAK-3187: ----------------------------- Attachment: OAK-3187.1-0-branch.patch Attached [^OAK-3187.1-0-branch.patch] which is the cherry-picked merge of OAK-2829 (and thus also OAK-3002) from the 1.2 branch into 1.0 with the before-mentioned conflicts resolved. The tests all run through ({{mvn clean install}} on parent runs fine). Plus the [test mentioned by [~mreutegg] in OAK-2829|https://issues.apache.org/jira/browse/OAK-2829?focusedCommentId=14530655&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14530655] has also been verified: it runs fine with the journal, and fails without the journal active. Next steps: * [~mreutegg], could you pls verify if the approach makes sense (backporting the journal alone without other diffCache changes does help) and if the patch attached does the trick /cc [~chetanm], [~mmarth], [~Philipp Suter] > Backporting journal to 1.0-branch > --------------------------------- > > Key: OAK-3187 > URL: https://issues.apache.org/jira/browse/OAK-3187 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, mongomk > Affects Versions: 1.0.18 > Reporter: Stefan Egli > Assignee: Stefan Egli > Priority: Critical > Fix For: 1.0.19 > > Attachments: OAK-3187.1-0-branch.patch > > > OAK-2829 introduces the concept of a journal that keeps track of all paths > that have changed between particular revisions in a cluster. The idea is to > look into backporting OAK-2829 to the 1.0-branch - and that's tracked atm in > this ticket (if it's better to do it in any other way, eg track it in > OAK-2829 directly, we can always change it that way). > I've done a "{{git cherry-pick c28a5d504993d64501d59c94a59f6186e8a0ad6e}}" > (which [merges OAK-2829 changes from trunk to > 1.2-branch|https://github.com/apache/jackrabbit-oak/commit/c28a5d504993d64501d59c94a59f6186e8a0ad6e]) > and followed up on the numerous conflicts resulting from this. > Before I go into the details of those conflicts below, there are probably a > few higher-level things to discuss too - as backporting "only" OAK-2829 (the > journal) to 1.0 might only serve half of the story - the diffCache and > lastRev handling has experienced quite a few changes since 1.0.x and it might > be worth while considering if it is OK to not backport those and still see > enough improvement, or if that's not the case. Those that come to mind > include: > * OAK-2685: which introduces the {{TieredDiffCache}} > * OAK-2131: 'reduce usage of _lastRev' > * OAK-2850: 'Flag states from revision of an external change' > The complete history of changes is what I'd put into the commit message and > goes as follows (not that OAK-3002 is also merged with this, and that for > completeness sake OAK-3085 would *have to be* merged too, but the latter is > trivial and can be done after this jumbo one): > {code} > OAK-2829: Comparing node states for external changes is too slow > OAK-3002: Optimize docCache and docChildrenCache invalidation by > filtering using journal > > Merged revisions > 1678023,1678171,1684820,1685590,1685964,1685977,1685989,1686023,1686032,1688179 > from trunk > > git-svn-id: > https://svn.apache.org/repos/asf/jackrabbit/oak/branches/1.2@1688649 > 13f79535-47bb-0310-9956-ffa450edef68 > > Conflicts: > > oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/Commit.java > - [conflict with OAK-2685 not available in 1.0 branch] > - [therefore manually: applyToCache: newEntry signature > changed] > > oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java > - [conflicts due to OAK-2850 and OAK-2685 not available in 1.0 > branch] > > oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreService.java > - [conflicts: only imports affected] > > oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/LastRevRecoveryAgent.java > - [conflict with OAK-2074 not available in 1.0 branch] > - [conflict with OAK-2131 not available in 1.0 branch] > - [conflict with OAK-2324 not available in 1.0 branch] > > oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/LocalDiffCache.java > - [conflict with OAK-2685 / newEntry method signature changed] > > oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/TieredDiffCache.java > - [conflict with OAK-2685 / newEntry signature, plus > DiffCache.getStats not avail in 1.0 branch] > > oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/UnsavedModifications.java > - [conflict with OAK-2324 not available in 1.0 branch] > - [conflict with OAK-1768] > - [conflict with OAK-2888 only available in 1.0 branch] > > oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java > - [conflict with OAK-1641 not available in 1.0 branch] > - [conflict with OAK-2681 not available in 1.0 branch: > findUncachedWithRetry not avail in 1.0] > > oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/util/Utils.java > - [conflicts: only imports affected] > > oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/document/ClusterTest.java > - [conflict with OAK-2850 not available in 1.0 branch] > > oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java > - [conflicts: only imports affected] > > oak-run/src/main/java/org/apache/jackrabbit/oak/benchmark/ObservationTest.java > - [conflict with OAK-2717 not available in 1.0 branch] > - [ADDED JcrCreator class plus > OakRepositoryFixture.setUpCluster(int n, JcrCreator customizer) from trunk] > > Additional Compile Problems: > > oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/document/UnsavedModificationsTest.java > - [conflict with OAK-2888 only available in 1.0 branch, fixed, > Snapshot.IGNORE added] > > Additional Test Problems: > > oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/document/JournalTest.java > - [conflict with OAK-2131 not available in 1.0 branch, hence > test logic had to be adapted/fixed] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)