[jira] [Commented] (OAK-5253) Optimize AbstractBlob#equal to not do content equals when possible
[ https://issues.apache.org/jira/browse/OAK-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734550#comment-15734550 ] Amit Jain commented on OAK-5253: Thanks Matt! Was already wip. But for 1.4 needed some backports as well OAK-4789 & OAK-5009. > Optimize AbstractBlob#equal to not do content equals when possible > -- > > Key: OAK-5253 > URL: https://issues.apache.org/jira/browse/OAK-5253 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob >Reporter: Amit Jain >Assignee: Amit Jain > Fix For: 1.6, 1.5.16, 1.4.11 > > Attachments: OAK-5253.1.patch > > > AbstractBlob#equals tries to match content when length is equal and content > identities is not null and different. Matching content triggers an expensive > download of binaries for S3DataStore. > Since, right now the content identity is the content hash the check can be > short -circuited when the content identities is not null and not equal to > return false. > This can be revisited if we change the identity to something different. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-5253) Optimize AbstractBlob#equal to not do content equals when possible
[ https://issues.apache.org/jira/browse/OAK-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain resolved OAK-5253. Resolution: Fixed Fix Version/s: 1.4.11 * trunk - http://svn.apache.org/viewvc?rev=1773322=rev * 1.4 - http://svn.apache.org/viewvc?rev=1773326=rev > Optimize AbstractBlob#equal to not do content equals when possible > -- > > Key: OAK-5253 > URL: https://issues.apache.org/jira/browse/OAK-5253 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob >Reporter: Amit Jain >Assignee: Amit Jain > Fix For: 1.6, 1.5.16, 1.4.11 > > Attachments: OAK-5253.1.patch > > > AbstractBlob#equals tries to match content when length is equal and content > identities is not null and different. Matching content triggers an expensive > download of binaries for S3DataStore. > Since, right now the content identity is the content hash the check can be > short -circuited when the content identities is not null and not equal to > return false. > This can be revisited if we change the identity to something different. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-5009) ExternalToExternalMigrationTest failures on Windows
[
https://issues.apache.org/jira/browse/OAK-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-5009:
---
Fix Version/s: 1.4.11
> ExternalToExternalMigrationTest failures on Windows
> ---
>
> Key: OAK-5009
> URL: https://issues.apache.org/jira/browse/OAK-5009
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, segment-tar
>Reporter: Julian Reschke
>Assignee: Tomek Rękawek
> Labels: test-failure
> Fix For: 1.6, 1.5.13, 1.4.11
>
>
> {noformat}
> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 13.484 sec
> <<< FAILURE! - in
> org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest
> blobsCanBeReadAfterSwitchingBlobStore(org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest)
> Time elapsed: 0.463 sec <<< ERROR!
> java.io.IOException: Unable to delete file:
> C:\tmp\1477483643533-0\segmentstore\data1a.tar
> blobsExistsOnTheNewBlobStore(org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest)
> Time elapsed: 13.021 sec <<< ERROR!
> java.io.IOException: Unable to delete file:
> C:\tmp\1477483643996-0\segmentstore\data0a.tar
> Running
> org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest
> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 12.719 sec
> <<< FAILURE! - in
> org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest
> blobsCanBeReadAfterSwitchingBlobStore(org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest)
> Time elapsed: 0.157 sec <<< ERROR!
> java.io.IOException: Unable to delete file:
> C:\tmp\1477483657018-0\segmentstore\data1a.tar
> blobsExistsOnTheNewBlobStore(org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest)
> Time elapsed: 12.561 sec <<< ERROR!
> java.io.IOException: Unable to delete file:
> C:\tmp\1477483657193-0\segmentstore\data0a.tar
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-5009) ExternalToExternalMigrationTest failures on Windows
[
https://issues.apache.org/jira/browse/OAK-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734531#comment-15734531
]
Amit Jain commented on OAK-5009:
Merged into 1.4 branch http://svn.apache.org/viewvc?rev=1773324=rev
> ExternalToExternalMigrationTest failures on Windows
> ---
>
> Key: OAK-5009
> URL: https://issues.apache.org/jira/browse/OAK-5009
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, segment-tar
>Reporter: Julian Reschke
>Assignee: Tomek Rękawek
> Labels: test-failure
> Fix For: 1.6, 1.5.13, 1.4.11
>
>
> {noformat}
> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 13.484 sec
> <<< FAILURE! - in
> org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest
> blobsCanBeReadAfterSwitchingBlobStore(org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest)
> Time elapsed: 0.463 sec <<< ERROR!
> java.io.IOException: Unable to delete file:
> C:\tmp\1477483643533-0\segmentstore\data1a.tar
> blobsExistsOnTheNewBlobStore(org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest)
> Time elapsed: 13.021 sec <<< ERROR!
> java.io.IOException: Unable to delete file:
> C:\tmp\1477483643996-0\segmentstore\data0a.tar
> Running
> org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest
> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 12.719 sec
> <<< FAILURE! - in
> org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest
> blobsCanBeReadAfterSwitchingBlobStore(org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest)
> Time elapsed: 0.157 sec <<< ERROR!
> java.io.IOException: Unable to delete file:
> C:\tmp\1477483657018-0\segmentstore\data1a.tar
> blobsExistsOnTheNewBlobStore(org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest)
> Time elapsed: 12.561 sec <<< ERROR!
> java.io.IOException: Unable to delete file:
> C:\tmp\1477483657193-0\segmentstore\data0a.tar
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4789) SegmentBlob should return null contentIdentity for inlined blobs
[
https://issues.apache.org/jira/browse/OAK-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734532#comment-15734532
]
Amit Jain commented on OAK-4789:
Merged into 1.4 branch http://svn.apache.org/viewvc?rev=1773325=rev
> SegmentBlob should return null contentIdentity for inlined blobs
>
>
> Key: OAK-4789
> URL: https://issues.apache.org/jira/browse/OAK-4789
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.6, 1.5.13, 1.4.11
>
>
> When a external BlobStore is configured with SegmentNodeStore then blob can
> exist in 3 forms
> # Blob inlined in segment storage - If blob length is <= 16512 then blob
> would be inlined in segment store
> # Blob inlined in BlobStore - Most BlobStore implementation also support
> inlining of blob content as part of blobId if the size is less than certain
> threshold. For {{FileDataStore}} this is determined by {{minRecordLength}}.
> If this is less than #1 then such a case would not happen
> # Blob whose content are stored in BlobStore without inlining
> Currently {{SegmentBlob}} returns recordId for {{getContentIdentity}} call
> for inlined blobs. This would cause this value to change if same blob is
> stored in a different segmentstore. As discussed
> [here|https://issues.apache.org/jira/browse/OAK-4712?focusedCommentId=15476477=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15476477]
> this causes ambiguity in certain cases.
> Given that {{getContentIdentity}} can return null it would be better if
> SegmentBlob returns null for inlined blob.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4789) SegmentBlob should return null contentIdentity for inlined blobs
[
https://issues.apache.org/jira/browse/OAK-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-4789:
---
Fix Version/s: 1.4.11
> SegmentBlob should return null contentIdentity for inlined blobs
>
>
> Key: OAK-4789
> URL: https://issues.apache.org/jira/browse/OAK-4789
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.6, 1.5.13, 1.4.11
>
>
> When a external BlobStore is configured with SegmentNodeStore then blob can
> exist in 3 forms
> # Blob inlined in segment storage - If blob length is <= 16512 then blob
> would be inlined in segment store
> # Blob inlined in BlobStore - Most BlobStore implementation also support
> inlining of blob content as part of blobId if the size is less than certain
> threshold. For {{FileDataStore}} this is determined by {{minRecordLength}}.
> If this is less than #1 then such a case would not happen
> # Blob whose content are stored in BlobStore without inlining
> Currently {{SegmentBlob}} returns recordId for {{getContentIdentity}} call
> for inlined blobs. This would cause this value to change if same blob is
> stored in a different segmentstore. As discussed
> [here|https://issues.apache.org/jira/browse/OAK-4712?focusedCommentId=15476477=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15476477]
> this causes ambiguity in certain cases.
> Given that {{getContentIdentity}} can return null it would be better if
> SegmentBlob returns null for inlined blob.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-5253) Optimize AbstractBlob#equal to not do content equals when possible
[ https://issues.apache.org/jira/browse/OAK-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733366#comment-15733366 ] Matt Ryan commented on OAK-5253: I just checked the patch with 1.4 and it applies cleanly. My testing indicates it should work fine with 1.4. > Optimize AbstractBlob#equal to not do content equals when possible > -- > > Key: OAK-5253 > URL: https://issues.apache.org/jira/browse/OAK-5253 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob >Reporter: Amit Jain >Assignee: Amit Jain > Fix For: 1.6, 1.5.16 > > Attachments: OAK-5253.1.patch > > > AbstractBlob#equals tries to match content when length is equal and content > identities is not null and different. Matching content triggers an expensive > download of binaries for S3DataStore. > Since, right now the content identity is the content hash the check can be > short -circuited when the content identities is not null and not equal to > return false. > This can be revisited if we change the identity to something different. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-5237) Change default query limit configuration
[
https://issues.apache.org/jira/browse/OAK-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733325#comment-15733325
]
Alexander Klimetschek commented on OAK-5237:
This overlaps with OAK-4888 (failing unindexed queries, ad-hoc queries can add
option(traversal ok) to query to allow it). Having both "hurdles" might be a
bit confusing. Are there cases where this one would help better?
> Change default query limit configuration
>
>
> Key: OAK-5237
> URL: https://issues.apache.org/jira/browse/OAK-5237
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
> Fix For: 1.6
>
>
> By default, queries can read an unlimited number of nodes, and an unlimited
> number of entries in memory (for "order by" and "union").
> I would like to change the default settings to:
> {noformat}
> -Doak.queryLimitInMemory=50
> -Doak.queryLimitReads=10
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-5253) Optimize AbstractBlob#equal to not do content equals when possible
[ https://issues.apache.org/jira/browse/OAK-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Ryan updated OAK-5253: --- Attachment: OAK-5253.1.patch [~amjain] Please take a look at the attached patch (OAK-5253.1.patch) for 1.5.16 which I think addresses this issue. I'll try to apply to 1.4 and send a separate patch if one is required. > Optimize AbstractBlob#equal to not do content equals when possible > -- > > Key: OAK-5253 > URL: https://issues.apache.org/jira/browse/OAK-5253 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob >Reporter: Amit Jain >Assignee: Amit Jain > Fix For: 1.6, 1.5.16 > > Attachments: OAK-5253.1.patch > > > AbstractBlob#equals tries to match content when length is equal and content > identities is not null and different. Matching content triggers an expensive > download of binaries for S3DataStore. > Since, right now the content identity is the content hash the check can be > short -circuited when the content identities is not null and not equal to > return false. > This can be revisited if we change the identity to something different. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-5017) Standby test failures
[
https://issues.apache.org/jira/browse/OAK-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733105#comment-15733105
]
Gavin commented on OAK-5017:
I added an Axis to use the 'label' expression. The 'Slaves' Axis used to work
but stopped working a short time ago, I filed a Jenkins ticket with no response
so far.
> Standby test failures
> -
>
> Key: OAK-5017
> URL: https://issues.apache.org/jira/browse/OAK-5017
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: test-failure
> Fix For: 1.6, 1.5.16
>
>
> I've recently seen a couple of the standby tests fail. E.g. on Jenkins:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1245/
> {noformat}
> java.lang.AssertionError: expected:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }> but was:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }>
> at
> org.apache.jackrabbit.oak.segment.standby.StandbyTestIT.testSyncLoop(StandbyTestIT.java:122)
> {noformat}
> {noformat}
> java.lang.AssertionError: expected:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }> but was:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }>
> at
> org.apache.jackrabbit.oak.segment.standby.StandbyTestIT.testSyncLoop(StandbyTestIT.java:122)
> {noformat}
> {{org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT.testProxySkippedBytes}}:
> {noformat}
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-5238) IndexCopier causes concurrent update on NodeBuilder
[ https://issues.apache.org/jira/browse/OAK-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manfred Baedke updated OAK-5238: Assignee: Marcel Reutegger (was: Manfred Baedke) > IndexCopier causes concurrent update on NodeBuilder > --- > > Key: OAK-5238 > URL: https://issues.apache.org/jira/browse/OAK-5238 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Affects Versions: 1.2.3, 1.0.15, 1.4.0 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: candidate_oak_1_4 > Fix For: 1.6 > > Attachments: OAK-5238.patch > > > OAK-2247 introduced the copy-on-write feature for lucene index in Oak. This > feature may result in a NodeBuilder updated by multiple threads concurrently. > New index files are first stored on the local filesystem and then copied > asynchronously into the repository. At the same time the async index update > thread manipulates the node builders as well. > With MongoMK this results in unexpected conflicts and failed async index > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-5238) IndexCopier causes concurrent update on NodeBuilder
[ https://issues.apache.org/jira/browse/OAK-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manfred Baedke reassigned OAK-5238: --- Assignee: Manfred Baedke (was: Marcel Reutegger) > IndexCopier causes concurrent update on NodeBuilder > --- > > Key: OAK-5238 > URL: https://issues.apache.org/jira/browse/OAK-5238 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Affects Versions: 1.2.3, 1.0.15, 1.4.0 >Reporter: Marcel Reutegger >Assignee: Manfred Baedke > Labels: candidate_oak_1_4 > Fix For: 1.6 > > Attachments: OAK-5238.patch > > > OAK-2247 introduced the copy-on-write feature for lucene index in Oak. This > feature may result in a NodeBuilder updated by multiple threads concurrently. > New index files are first stored on the local filesystem and then copied > asynchronously into the repository. At the same time the async index update > thread manipulates the node builders as well. > With MongoMK this results in unexpected conflicts and failed async index > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-3976) journal should support large(r) entries
[
https://issues.apache.org/jira/browse/OAK-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732385#comment-15732385
]
Vikas Saurabh edited comment on OAK-3976 at 12/8/16 2:38 PM:
-
Good point. But, as you said, we'd probably be ok with minor accuracy losses. I
avoided {{+=paths.length}} (in one step higher level modified method) for that
reason though.
was (Author: catholicon):
Good point. But, as you said, we'd probably be ok with minor accuracy losses. I
avoided {{+=paths.length}} for that reason though.
> journal should support large(r) entries
> ---
>
> Key: OAK-3976
> URL: https://issues.apache.org/jira/browse/OAK-3976
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Affects Versions: 1.3.14
>Reporter: Stefan Egli
>Assignee: Vikas Saurabh
> Fix For: 1.6, 1.5.16
>
>
> Journal entries are created in the background write. Normally this happens
> every second. If for some reason there is a large delay between two
> background writes, the number of pending changes can also accumulate. Which
> can result in (arbitrary) large single journal entries (ie with large {{_c}}
> property).
> This can cause multiple problems down the road:
> * journal gc at this point loads 450 entries - and if some are large this can
> result in a very large memory consumption during gc (which can cause severe
> stability problems for the VM, if not OOM etc). This should be fixed with
> OAK-3001 (where we only get the id, thus do not care how big {{_c}} is)
> * before OAK-3001 is done (which is currently scheduled after 1.4) what we
> can do is reduce the delete batch size (OAK-3975)
> * background reads however also read the journal entries and even if
> OAK-3001/OAK-3975 are implemented the background read can still cause large
> memory consumption. So we need to improve this one way or another.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3976) journal should support large(r) entries
[
https://issues.apache.org/jira/browse/OAK-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732385#comment-15732385
]
Vikas Saurabh commented on OAK-3976:
Good point. But, as you said, we'd probably be ok with minor accuracy losses. I
avoided {{+=paths.length}} for that reason though.
> journal should support large(r) entries
> ---
>
> Key: OAK-3976
> URL: https://issues.apache.org/jira/browse/OAK-3976
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Affects Versions: 1.3.14
>Reporter: Stefan Egli
>Assignee: Vikas Saurabh
> Fix For: 1.6, 1.5.16
>
>
> Journal entries are created in the background write. Normally this happens
> every second. If for some reason there is a large delay between two
> background writes, the number of pending changes can also accumulate. Which
> can result in (arbitrary) large single journal entries (ie with large {{_c}}
> property).
> This can cause multiple problems down the road:
> * journal gc at this point loads 450 entries - and if some are large this can
> result in a very large memory consumption during gc (which can cause severe
> stability problems for the VM, if not OOM etc). This should be fixed with
> OAK-3001 (where we only get the id, thus do not care how big {{_c}} is)
> * before OAK-3001 is done (which is currently scheduled after 1.4) what we
> can do is reduce the delete batch size (OAK-3975)
> * background reads however also read the journal entries and even if
> OAK-3001/OAK-3975 are implemented the background read can still cause large
> memory consumption. So we need to improve this one way or another.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3976) journal should support large(r) entries
[
https://issues.apache.org/jira/browse/OAK-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732378#comment-15732378
]
Julian Sedding commented on OAK-3976:
-
[~catholicon] Note that "private volatile int numChangedPaths = 0" will not get
atomically updated, thus it is possible that the count is not exact (imagine
two threads incrementing at the same time from 0 to 1, results in 1 when you'd
expect 2). It looks like this is likely not a problem here, as the threshold is
arbitrary. To be correct you should use an {{AtomicInteger}} or {{AtomicLong}},
however.
> journal should support large(r) entries
> ---
>
> Key: OAK-3976
> URL: https://issues.apache.org/jira/browse/OAK-3976
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Affects Versions: 1.3.14
>Reporter: Stefan Egli
>Assignee: Vikas Saurabh
> Fix For: 1.6, 1.5.16
>
>
> Journal entries are created in the background write. Normally this happens
> every second. If for some reason there is a large delay between two
> background writes, the number of pending changes can also accumulate. Which
> can result in (arbitrary) large single journal entries (ie with large {{_c}}
> property).
> This can cause multiple problems down the road:
> * journal gc at this point loads 450 entries - and if some are large this can
> result in a very large memory consumption during gc (which can cause severe
> stability problems for the VM, if not OOM etc). This should be fixed with
> OAK-3001 (where we only get the id, thus do not care how big {{_c}} is)
> * before OAK-3001 is done (which is currently scheduled after 1.4) what we
> can do is reduce the delete batch size (OAK-3975)
> * background reads however also read the journal entries and even if
> OAK-3001/OAK-3975 are implemented the background read can still cause large
> memory consumption. So we need to improve this one way or another.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-3976) journal should support large(r) entries
[
https://issues.apache.org/jira/browse/OAK-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732050#comment-15732050
]
Vikas Saurabh edited comment on OAK-3976 at 12/8/16 1:55 PM:
-
[~mreutegg], here's roughly what I am thinking of doing:
{code}
diff --git
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
index 04ecfd7..c168ff0 100644
---
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
+++
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
@@ -185,6 +185,13 @@ public final class DocumentNodeStore
private boolean disableJournalDiff =
diff --git
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
index 04ecfd7..c168ff0 100644
---
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
+++
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
@@ -185,6 +185,13 @@ public final class DocumentNodeStore
private boolean disableJournalDiff =
Boolean.getBoolean(SYS_PROP_DISABLE_JOURNAL);
+public static final String SYS_PROP_JOURNAL_PUSH_THRESHOLD =
"oak.journalPushThreshold";
+/**
+ * Threshold for number of paths in journal entry to require a force push
during commit
+ * (instead of at background write)
+ */
+static int journalPushThreshold = 10; //non-final to allow for tests.
+
/**
* The document store (might be used by multiple node stores).
*/
@@ -790,6 +797,7 @@ public final class DocumentNodeStore
changes.addChangeSet(getChangeSet(info));
// update head revision
newHead[0] = before.update(c.getRevision());
+pushJournalEntry(c.getRevision(), false);
setRoot(newHead[0]);
commitQueue.headRevisionChanged();
dispatcher.contentChanged(getRoot(), info);
@@ -2142,19 +2150,32 @@ public final class DocumentNodeStore
return unsavedLastRevisions.persist(this, new
UnsavedModifications.Snapshot() {
@Override
public void acquiring(Revision mostRecent) {
-if (store.create(JOURNAL,
singletonList(changes.asUpdateOp(mostRecent {
-// success: start with a new document
-changes = newJournalEntry();
-} else {
-// fail: log and keep the changes
-LOG.error("Failed to write to journal, accumulating
changes for future write (~" + changes.getMemory()
-+ " bytes).");
-}
+pushJournalEntry(mostRecent, true);
}
}, backgroundOperationLock.writeLock());
}
//-< internal
>-
+private ReadWriteLock journalPushLock = new ReentrantReadWriteLock();
+void pushJournalEntry(Revision r, boolean force) {
+if (!force && changes.getNumChangedPaths() < journalPushThreshold) {
+return;
+}
+journalPushLock.writeLock().lock();
+try {
+if (store.create(JOURNAL, singletonList(changes.asUpdateOp(r {
+// success: start with a new document
+changes = newJournalEntry();
+} else {
+// fail: log and keep the changes
+LOG.error("Failed to write to journal, accumulating changes
for future write (~" + changes.getMemory()
++ " bytes).");
+}
+} finally {
+journalPushLock.writeLock().unlock();
+}
+
+}
/**
* Returns the binary size of a property value represented as a JSON or
diff --git
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java
index e3aab49..cadef3c 100644
---
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java
+++
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java
@@ -86,6 +86,8 @@ public final class JournalEntry extends Document {
private volatile TreeNode changes = null;
+private volatile int numChangedPaths = 0;
+
private boolean concurrent;
JournalEntry(DocumentStore store) {
@@ -318,6 +320,7 @@ public final class JournalEntry extends Document {
for (String name : PathUtils.elements(path)) {
node = node.getOrCreate(name);
}
+
[jira] [Updated] (OAK-5254) MultiplexingNodeStoreService does not pick up Observers registered through the whiteboard
[
https://issues.apache.org/jira/browse/OAK-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Munteanu updated OAK-5254:
-
Attachment: 0001-OAK-5254-MultiplexingNodeStoreService-does-not-pick-.patch
Patch attached. [~tomek.rekawek] - can you please review?
> MultiplexingNodeStoreService does not pick up Observers registered through
> the whiteboard
> -
>
> Key: OAK-5254
> URL: https://issues.apache.org/jira/browse/OAK-5254
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Reporter: Robert Munteanu
> Fix For: 1.6, 1.5.16
>
> Attachments:
> 0001-OAK-5254-MultiplexingNodeStoreService-does-not-pick-.patch
>
>
> The {{MultiplexingNodeStoreService}} fails to pick up observer instances from
> the whiteboard. I noticed this when using it in an OSGi context and it failed
> to picked up the Lucene indexes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-5238) IndexCopier causes concurrent update on NodeBuilder
[ https://issues.apache.org/jira/browse/OAK-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732180#comment-15732180 ] Stefan Eissing commented on OAK-5238: - And if one created/initialized and commited a subnode, took a node builder on that for the COWDirectory for async copying. On Directory close merged as final task that node and then moved all contained nodes upward? There can only ever by a single COWDirectory per index at a time, right? So, gc inside the node store would not be necessary. > IndexCopier causes concurrent update on NodeBuilder > --- > > Key: OAK-5238 > URL: https://issues.apache.org/jira/browse/OAK-5238 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Affects Versions: 1.2.3, 1.0.15, 1.4.0 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: candidate_oak_1_4 > Fix For: 1.6 > > Attachments: OAK-5238.patch > > > OAK-2247 introduced the copy-on-write feature for lucene index in Oak. This > feature may result in a NodeBuilder updated by multiple threads concurrently. > New index files are first stored on the local filesystem and then copied > asynchronously into the repository. At the same time the async index update > thread manipulates the node builders as well. > With MongoMK this results in unexpected conflicts and failed async index > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3606) Improvements for IndexStatsMBean usage
[ https://issues.apache.org/jira/browse/OAK-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manfred Baedke updated OAK-3606: Priority: Minor (was: Major) > Improvements for IndexStatsMBean usage > -- > > Key: OAK-3606 > URL: https://issues.apache.org/jira/browse/OAK-3606 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, lucene >Affects Versions: 1.3.9 >Reporter: Thierry Ygé >Assignee: Manfred Baedke >Priority: Minor > Fix For: 1.6 > > Attachments: adding_new_MBean.patch, > new_mbean_interface_and_implementation.patch > > > When running integration tests, it is common to have the need to wait for the > async indexes to have been executed. So that the test can successfully > validate operations that depend on the search result. > With the current IndexStatsMBean implementation it cannot return the start > time of the last successful indexing. It provide a "LastIndexedTime" which is > not sufficient to know if changes made recently are now indexed. > The idea is to set the start time as value of a new attribute (i.e > "StartLastSuccessIndexedTime") to the IndexStatsMBean. > Then create a new Mbean that calculate from all existing IndexStatsMBean (as > multiple are possible now) the oldest "StartLastSuccessIndexedTime". > That will allow integration tests to be able to wait until that oldest > "StartLastSuccessIndexedTime" is greater than the time it started to wait. > Attached is a sample patch containing the necessary changes (for a Oak core > 1.4.0-SNAPSHOT). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-5254) MultiplexingNodeStoreService does not pick up Observers registered through the whiteboard
Robert Munteanu created OAK-5254:
Summary: MultiplexingNodeStoreService does not pick up Observers
registered through the whiteboard
Key: OAK-5254
URL: https://issues.apache.org/jira/browse/OAK-5254
Project: Jackrabbit Oak
Issue Type: Bug
Components: core
Reporter: Robert Munteanu
Fix For: 1.6, 1.5.16
The {{MultiplexingNodeStoreService}} fails to pick up observer instances from
the whiteboard. I noticed this when using it in an OSGi context and it failed
to picked up the Lucene indexes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-3976) journal should support large(r) entries
[
https://issues.apache.org/jira/browse/OAK-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732050#comment-15732050
]
Vikas Saurabh edited comment on OAK-3976 at 12/8/16 12:56 PM:
--
[~mreutegg], here's roughly what I am thinking of doing:
{code}
diff --git
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
index 04ecfd7..c168ff0 100644
---
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
+++
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
@@ -185,6 +185,13 @@ public final class DocumentNodeStore
private boolean disableJournalDiff =
diff --git
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
index 04ecfd7..c168ff0 100644
---
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
+++
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
@@ -185,6 +185,13 @@ public final class DocumentNodeStore
private boolean disableJournalDiff =
Boolean.getBoolean(SYS_PROP_DISABLE_JOURNAL);
+public static final String SYS_PROP_JOURNAL_PUSH_THRESHOLD =
"oak.journalPushThreshold";
+/**
+ * Threshold for number of paths in journal entry to require a force push
during commit
+ * (instead of at background write)
+ */
+static int journalPushThreshold = 10; //non-final to allow for tests.
+
/**
* The document store (might be used by multiple node stores).
*/
@@ -790,6 +797,7 @@ public final class DocumentNodeStore
changes.addChangeSet(getChangeSet(info));
// update head revision
newHead[0] = before.update(c.getRevision());
+pushJournalEntry(c.getRevision(), false);
setRoot(newHead[0]);
commitQueue.headRevisionChanged();
dispatcher.contentChanged(getRoot(), info);
@@ -2142,19 +2150,32 @@ public final class DocumentNodeStore
return unsavedLastRevisions.persist(this, new
UnsavedModifications.Snapshot() {
@Override
public void acquiring(Revision mostRecent) {
-if (store.create(JOURNAL,
singletonList(changes.asUpdateOp(mostRecent {
-// success: start with a new document
-changes = newJournalEntry();
-} else {
-// fail: log and keep the changes
-LOG.error("Failed to write to journal, accumulating
changes for future write (~" + changes.getMemory()
-+ " bytes).");
-}
+pushJournalEntry(mostRecent, true);
}
}, backgroundOperationLock.writeLock());
}
//-< internal
>-
+private ReadWriteLock journalPushLock = new ReentrantReadWriteLock();
+void pushJournalEntry(Revision r, boolean force) {
+if (!force && changes.getNumChangedPaths() < journalPushThreshold) {
+return;
+}
+journalPushLock.writeLock().lock();
+try {
+if (store.create(JOURNAL, singletonList(changes.asUpdateOp(r {
+// success: start with a new document
+changes = newJournalEntry();
+} else {
+// fail: log and keep the changes
+LOG.error("Failed to write to journal, accumulating changes
for future write (~" + changes.getMemory()
++ " bytes).");
+}
+} finally {
+journalPushLock.writeLock().unlock();
+}
+
+}
/**
* Returns the binary size of a property value represented as a JSON or
diff --git
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java
index e3aab49..cadef3c 100644
---
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java
+++
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java
@@ -86,6 +86,8 @@ public final class JournalEntry extends Document {
private volatile TreeNode changes = null;
+private volatile int numChangedPaths = 0;
+
private boolean concurrent;
JournalEntry(DocumentStore store) {
@@ -318,6 +320,7 @@ public final class JournalEntry extends Document {
for (String name : PathUtils.elements(path)) {
node = node.getOrCreate(name);
}
+
[jira] [Commented] (OAK-4069) Use read concern majority when connected to a replica set
[
https://issues.apache.org/jira/browse/OAK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732092#comment-15732092
]
Marcel Reutegger commented on OAK-4069:
---
The command serverStatus requires a user to have the clusterMonitor role. Can
you add warning when the connection is authenticated and the user does not have
that role assigned? I understand in that case the MongoDocumentStore would stay
with the default read concern even if the majority read concern option is
enabled on the server.
It would also be good to run some performance tests to see whether this new
default has an impact in a replica set.
> Use read concern majority when connected to a replica set
> -
>
> Key: OAK-4069
> URL: https://issues.apache.org/jira/browse/OAK-4069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Labels: resilience
> Fix For: 1.6, 1.5.16
>
> Attachments: OAK-4069.patch
>
>
> Mongo 3.2 introduces new query option: {{readConcern}}. It allows to read
> only these changes that have been already committed to the majority of
> secondary instances.
> It prevents stale reads - a situation in which a change has been committed on
> the primary (and read from it), but due to the network partition a new
> primary is elected and the change is rolled back.
> We should use this new option (together with {{w:majority}} implemented in
> OAK-3554) when running Oak on MongoDB replica set.
> References:
> * [Jepsen: MongoDB stale
> reads|https://aphyr.com/posts/322-jepsen-mongodb-stale-reads]
> * [MongoDB documentation: Read Concern
> in|https://docs.mongodb.org/manual/reference/read-concern/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3976) journal should support large(r) entries
[
https://issues.apache.org/jira/browse/OAK-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732050#comment-15732050
]
Vikas Saurabh commented on OAK-3976:
[~mreutegg], here's roughly what I am thinking of doing:
{code}
diff --git
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
index 04ecfd7..c168ff0 100644
---
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
+++
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
@@ -185,6 +185,13 @@ public final class DocumentNodeStore
private boolean disableJournalDiff =
diff --git
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
index 04ecfd7..c168ff0 100644
---
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
+++
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
@@ -185,6 +185,13 @@ public final class DocumentNodeStore
private boolean disableJournalDiff =
Boolean.getBoolean(SYS_PROP_DISABLE_JOURNAL);
+public static final String SYS_PROP_JOURNAL_PUSH_THRESHOLD =
"oak.journalPushThreshold";
+/**
+ * Threshold for number of paths in journal entry to require a force push
during commit
+ * (instead of at background write)
+ */
+static int journalPushThreshold = 10; //non-final to allow for tests.
+
/**
* The document store (might be used by multiple node stores).
*/
@@ -790,6 +797,7 @@ public final class DocumentNodeStore
changes.addChangeSet(getChangeSet(info));
// update head revision
newHead[0] = before.update(c.getRevision());
+pushJournalEntry(c.getRevision(), false);
setRoot(newHead[0]);
commitQueue.headRevisionChanged();
dispatcher.contentChanged(getRoot(), info);
@@ -2142,19 +2150,32 @@ public final class DocumentNodeStore
return unsavedLastRevisions.persist(this, new
UnsavedModifications.Snapshot() {
@Override
public void acquiring(Revision mostRecent) {
-if (store.create(JOURNAL,
singletonList(changes.asUpdateOp(mostRecent {
-// success: start with a new document
-changes = newJournalEntry();
-} else {
-// fail: log and keep the changes
-LOG.error("Failed to write to journal, accumulating
changes for future write (~" + changes.getMemory()
-+ " bytes).");
-}
+pushJournalEntry(mostRecent, true);
}
}, backgroundOperationLock.writeLock());
}
//-< internal
>-
+private ReadWriteLock journalPushLock = new ReentrantReadWriteLock();
+void pushJournalEntry(Revision r, boolean force) {
+if (!force && changes.getNumChangedPaths() < journalPushThreshold) {
+return;
+}
+journalPushLock.writeLock().lock();
+try {
+if (store.create(JOURNAL, singletonList(changes.asUpdateOp(r {
+// success: start with a new document
+changes = newJournalEntry();
+} else {
+// fail: log and keep the changes
+LOG.error("Failed to write to journal, accumulating changes
for future write (~" + changes.getMemory()
++ " bytes).");
+}
+} finally {
+journalPushLock.writeLock().unlock();
+}
+
+}
/**
* Returns the binary size of a property value represented as a JSON or
{code}
A few things that I think were important to take care of:
* Commit (if required) should push journal entry before it sets new head
* minimize journal push critical section
This way things are currently, I'm not completely confident of:
# commit clears journal and pushes an entry with its own revision V/s bkWrite
checked unsavedLastRevs to find the latest rev
#* I'm not sure if commit's rev could be bigger that highest-last-rev. Also,
does it matter?
# BkWrite can now potentially create empty journal entries. Should we avoid
that?
> journal should support large(r) entries
> ---
>
> Key: OAK-3976
> URL: https://issues.apache.org/jira/browse/OAK-3976
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Affects Versions: 1.3.14
>
[jira] [Commented] (OAK-4069) Use read concern majority when connected to a replica set
[
https://issues.apache.org/jira/browse/OAK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731938#comment-15731938
]
Tomek Rękawek commented on OAK-4069:
Got it. I've uploaded new patch, it'll only enable the read concern = majority
if:
* no read concern level is passed in the URI (see
{{DocumentMK.Builder#setMongoDB()}}) and
* the server supports it (see {{MongoStatus#supportsReadConcern}}).
> Use read concern majority when connected to a replica set
> -
>
> Key: OAK-4069
> URL: https://issues.apache.org/jira/browse/OAK-4069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Labels: resilience
> Fix For: 1.6, 1.5.16
>
> Attachments: OAK-4069.patch
>
>
> Mongo 3.2 introduces new query option: {{readConcern}}. It allows to read
> only these changes that have been already committed to the majority of
> secondary instances.
> It prevents stale reads - a situation in which a change has been committed on
> the primary (and read from it), but due to the network partition a new
> primary is elected and the change is rolled back.
> We should use this new option (together with {{w:majority}} implemented in
> OAK-3554) when running Oak on MongoDB replica set.
> References:
> * [Jepsen: MongoDB stale
> reads|https://aphyr.com/posts/322-jepsen-mongodb-stale-reads]
> * [MongoDB documentation: Read Concern
> in|https://docs.mongodb.org/manual/reference/read-concern/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4069) Use read concern majority when connected to a replica set
[
https://issues.apache.org/jira/browse/OAK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4069:
---
Attachment: (was: OAK-4069.patch)
> Use read concern majority when connected to a replica set
> -
>
> Key: OAK-4069
> URL: https://issues.apache.org/jira/browse/OAK-4069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Labels: resilience
> Fix For: 1.6, 1.5.16
>
> Attachments: OAK-4069.patch
>
>
> Mongo 3.2 introduces new query option: {{readConcern}}. It allows to read
> only these changes that have been already committed to the majority of
> secondary instances.
> It prevents stale reads - a situation in which a change has been committed on
> the primary (and read from it), but due to the network partition a new
> primary is elected and the change is rolled back.
> We should use this new option (together with {{w:majority}} implemented in
> OAK-3554) when running Oak on MongoDB replica set.
> References:
> * [Jepsen: MongoDB stale
> reads|https://aphyr.com/posts/322-jepsen-mongodb-stale-reads]
> * [MongoDB documentation: Read Concern
> in|https://docs.mongodb.org/manual/reference/read-concern/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4069) Use read concern majority when connected to a replica set
[
https://issues.apache.org/jira/browse/OAK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4069:
---
Attachment: OAK-4069.patch
> Use read concern majority when connected to a replica set
> -
>
> Key: OAK-4069
> URL: https://issues.apache.org/jira/browse/OAK-4069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Labels: resilience
> Fix For: 1.6, 1.5.16
>
> Attachments: OAK-4069.patch
>
>
> Mongo 3.2 introduces new query option: {{readConcern}}. It allows to read
> only these changes that have been already committed to the majority of
> secondary instances.
> It prevents stale reads - a situation in which a change has been committed on
> the primary (and read from it), but due to the network partition a new
> primary is elected and the change is rolled back.
> We should use this new option (together with {{w:majority}} implemented in
> OAK-3554) when running Oak on MongoDB replica set.
> References:
> * [Jepsen: MongoDB stale
> reads|https://aphyr.com/posts/322-jepsen-mongodb-stale-reads]
> * [MongoDB documentation: Read Concern
> in|https://docs.mongodb.org/manual/reference/read-concern/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-5199) Test coverage for ExternalGroupRef
[
https://issues.apache.org/jira/browse/OAK-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella updated OAK-5199:
--
Fix Version/s: (was: 1.5.15)
1.5.16
> Test coverage for ExternalGroupRef
> --
>
> Key: OAK-5199
> URL: https://issues.apache.org/jira/browse/OAK-5199
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: auth-external
>Reporter: angela
>Assignee: Manfred Baedke
> Fix For: 1.6, 1.5.16
>
>
> [~baedke], in addition to the missing documentation, i just noticed that
> there is also no tests for the new {{ExternalGroupRef}} you added recently.
> IMHO its crucial that we don't introduce new features/improvements/fixes
> without tests... this has been one of the big issues in the past and I really
> want the old bad habits that of the {{oak-auth-external}} module to be
> continue. The lack of test and documentation coverage has just been so
> troublesome in the past.
> If that is not possible by mutual agreement among all devs working on this
> module, I would suggest that we introduce automatic control into the pom.xml
> that fails the build if the test-coverage drops with a given change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4954) SetPropertyTest benchmark fails on Segment Tar
[
https://issues.apache.org/jira/browse/OAK-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella updated OAK-4954:
--
Fix Version/s: (was: 1.5.15)
1.5.16
> SetPropertyTest benchmark fails on Segment Tar
> --
>
> Key: OAK-4954
> URL: https://issues.apache.org/jira/browse/OAK-4954
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
> Labels: test-failure
> Fix For: 1.6, 1.5.16
>
>
> The {{SetPropertyTest}} fails on Oak Segment Tar:
> {noformat}
> javax.jcr.InvalidItemStateException: This item
> [/testfb3e8f1a/ca1ef350-f650-4466-b9e3-7f77d83e6303] does not exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl$ItemWriteOperation.checkPreconditions(ItemImpl.java:96)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$35.checkPreconditions(NodeImpl.java:1366)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.internalSetProperty(NodeImpl.java:1363)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.setProperty(NodeImpl.java:506)
> at
> org.apache.jackrabbit.oak.benchmark.SetPropertyTest.runTest(SetPropertyTest.java:65)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.execute(AbstractTest.java:372)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.runTest(AbstractTest.java:221)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.run(AbstractTest.java:197)
> at
> org.apache.jackrabbit.oak.benchmark.BenchmarkRunner.main(BenchmarkRunner.java:456)
> at
> org.apache.jackrabbit.oak.run.BenchmarkCommand.execute(BenchmarkCommand.java:26)
> at org.apache.jackrabbit.oak.run.Mode.execute(Mode.java:63)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-5017) Standby test failures
[
https://issues.apache.org/jira/browse/OAK-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella updated OAK-5017:
--
Fix Version/s: (was: 1.5.15)
1.5.16
> Standby test failures
> -
>
> Key: OAK-5017
> URL: https://issues.apache.org/jira/browse/OAK-5017
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: test-failure
> Fix For: 1.6, 1.5.16
>
>
> I've recently seen a couple of the standby tests fail. E.g. on Jenkins:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1245/
> {noformat}
> java.lang.AssertionError: expected:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }> but was:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }>
> at
> org.apache.jackrabbit.oak.segment.standby.StandbyTestIT.testSyncLoop(StandbyTestIT.java:122)
> {noformat}
> {noformat}
> java.lang.AssertionError: expected:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }> but was:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }>
> at
> org.apache.jackrabbit.oak.segment.standby.StandbyTestIT.testSyncLoop(StandbyTestIT.java:122)
> {noformat}
> {{org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT.testProxySkippedBytes}}:
> {noformat}
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-5198) Javadoc and Documentation of ExternalGroupRef
[
https://issues.apache.org/jira/browse/OAK-5198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella updated OAK-5198:
--
Fix Version/s: (was: 1.5.15)
1.5.16
> Javadoc and Documentation of ExternalGroupRef
> -
>
> Key: OAK-5198
> URL: https://issues.apache.org/jira/browse/OAK-5198
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: auth-external, doc
>Reporter: angela
>Assignee: Manfred Baedke
>Priority: Critical
> Fix For: 1.6, 1.5.16
>
>
> as discussed on the phone, i would like to have the new API
> {{ExternalGroupRef}} to be documented in the javadoc (e.g. explaining the
> diff wrt ExternalIdentityRef and the expected usage) and in the oak
> documentation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3606) Improvements for IndexStatsMBean usage
[ https://issues.apache.org/jira/browse/OAK-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella updated OAK-3606: -- Fix Version/s: (was: 1.5.15) > Improvements for IndexStatsMBean usage > -- > > Key: OAK-3606 > URL: https://issues.apache.org/jira/browse/OAK-3606 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, lucene >Affects Versions: 1.3.9 >Reporter: Thierry Ygé >Assignee: Manfred Baedke > Fix For: 1.6 > > Attachments: adding_new_MBean.patch, > new_mbean_interface_and_implementation.patch > > > When running integration tests, it is common to have the need to wait for the > async indexes to have been executed. So that the test can successfully > validate operations that depend on the search result. > With the current IndexStatsMBean implementation it cannot return the start > time of the last successful indexing. It provide a "LastIndexedTime" which is > not sufficient to know if changes made recently are now indexed. > The idea is to set the start time as value of a new attribute (i.e > "StartLastSuccessIndexedTime") to the IndexStatsMBean. > Then create a new Mbean that calculate from all existing IndexStatsMBean (as > multiple are possible now) the oldest "StartLastSuccessIndexedTime". > That will allow integration tests to be able to wait until that oldest > "StartLastSuccessIndexedTime" is greater than the time it started to wait. > Attached is a sample patch containing the necessary changes (for a Oak core > 1.4.0-SNAPSHOT). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4954) SetPropertyTest benchmark fails on Segment Tar
[
https://issues.apache.org/jira/browse/OAK-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731778#comment-15731778
]
Michael Dürig commented on OAK-4954:
I just saw this again by running {{java -jar oak-run-1.6-SNAPSHOT.jar benchmark
SetPropertyTest Oak-Tar Oak-Segment-Tar}}. Don't think the additional Oak-Tar
fixture is related but rather it might only happen once in a while!?
> SetPropertyTest benchmark fails on Segment Tar
> --
>
> Key: OAK-4954
> URL: https://issues.apache.org/jira/browse/OAK-4954
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
> Labels: test-failure
> Fix For: 1.6, 1.5.15
>
>
> The {{SetPropertyTest}} fails on Oak Segment Tar:
> {noformat}
> javax.jcr.InvalidItemStateException: This item
> [/testfb3e8f1a/ca1ef350-f650-4466-b9e3-7f77d83e6303] does not exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl$ItemWriteOperation.checkPreconditions(ItemImpl.java:96)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$35.checkPreconditions(NodeImpl.java:1366)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.internalSetProperty(NodeImpl.java:1363)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.setProperty(NodeImpl.java:506)
> at
> org.apache.jackrabbit.oak.benchmark.SetPropertyTest.runTest(SetPropertyTest.java:65)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.execute(AbstractTest.java:372)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.runTest(AbstractTest.java:221)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.run(AbstractTest.java:197)
> at
> org.apache.jackrabbit.oak.benchmark.BenchmarkRunner.main(BenchmarkRunner.java:456)
> at
> org.apache.jackrabbit.oak.run.BenchmarkCommand.execute(BenchmarkCommand.java:26)
> at org.apache.jackrabbit.oak.run.Mode.execute(Mode.java:63)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-5253) Optimize AbstractBlob#equal to not do content equals when possible
Amit Jain created OAK-5253: -- Summary: Optimize AbstractBlob#equal to not do content equals when possible Key: OAK-5253 URL: https://issues.apache.org/jira/browse/OAK-5253 Project: Jackrabbit Oak Issue Type: Improvement Components: blob Reporter: Amit Jain Assignee: Amit Jain Fix For: 1.6, 1.5.16 AbstractBlob#equals tries to match content when length is equal and content identities is not null and different. Matching content triggers an expensive download of binaries for S3DataStore. Since, right now the content identity is the content hash the check can be short -circuited when the content identities is not null and not equal to return false. This can be revisited if we change the identity to something different. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4069) Use read concern majority when connected to a replica set
[
https://issues.apache.org/jira/browse/OAK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731776#comment-15731776
]
Marcel Reutegger commented on OAK-4069:
---
bq. Don't set the default readConcern for replica set?
That's my preference, yes. Simply requiring 3.2 is not enough. E.g. 3.2 with
MMAPv1 does not support majority read concern at all and for WiredTiger it is
[disabled by
default|https://docs.mongodb.com/manual/reference/program/mongod/#cmdoption--enableMajorityReadConcern].
> Use read concern majority when connected to a replica set
> -
>
> Key: OAK-4069
> URL: https://issues.apache.org/jira/browse/OAK-4069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Labels: resilience
> Fix For: 1.6, 1.5.16
>
> Attachments: OAK-4069.patch
>
>
> Mongo 3.2 introduces new query option: {{readConcern}}. It allows to read
> only these changes that have been already committed to the majority of
> secondary instances.
> It prevents stale reads - a situation in which a change has been committed on
> the primary (and read from it), but due to the network partition a new
> primary is elected and the change is rolled back.
> We should use this new option (together with {{w:majority}} implemented in
> OAK-3554) when running Oak on MongoDB replica set.
> References:
> * [Jepsen: MongoDB stale
> reads|https://aphyr.com/posts/322-jepsen-mongodb-stale-reads]
> * [MongoDB documentation: Read Concern
> in|https://docs.mongodb.org/manual/reference/read-concern/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-5252) Disable IPv6 tests on Jenkins nodes labelled "beam"
[ https://issues.apache.org/jira/browse/OAK-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francesco Mari resolved OAK-5252. - Resolution: Fixed Fix Version/s: 1.5.15 Fixed at r1773209. > Disable IPv6 tests on Jenkins nodes labelled "beam" > --- > > Key: OAK-5252 > URL: https://issues.apache.org/jira/browse/OAK-5252 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.6, 1.5.15 > > > Tests using the IPv6 stack consistently fail on the Apache Jenkins nodes > under the label "beam". The easiest solution to avoid this situation is to > disable those tests using a JUnit assumption when they are run in such a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-5252) Disable IPv6 tests on Jenkins nodes labelled "beam"
Francesco Mari created OAK-5252: --- Summary: Disable IPv6 tests on Jenkins nodes labelled "beam" Key: OAK-5252 URL: https://issues.apache.org/jira/browse/OAK-5252 Project: Jackrabbit Oak Issue Type: Technical task Components: segment-tar Reporter: Francesco Mari Assignee: Francesco Mari Fix For: 1.6 Tests using the IPv6 stack consistently fail on the Apache Jenkins nodes under the label "beam". The easiest solution to avoid this situation is to disable those tests using a JUnit assumption when they are run in such a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4069) Use read concern majority when connected to a replica set
[
https://issues.apache.org/jira/browse/OAK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731761#comment-15731761
]
Tomek Rękawek commented on OAK-4069:
[~mreutegg], ok, I'll update the driver to 3.4.
With regards to the read concern support - shouldn't we bump the required
MongoDB version to 3.2 in MongoDocumentStore#checkVersion? If not, what should
be correct behaviour for the old MongoDB? Don't set the default readConcern for
replica set?
> Use read concern majority when connected to a replica set
> -
>
> Key: OAK-4069
> URL: https://issues.apache.org/jira/browse/OAK-4069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Labels: resilience
> Fix For: 1.6, 1.5.16
>
> Attachments: OAK-4069.patch
>
>
> Mongo 3.2 introduces new query option: {{readConcern}}. It allows to read
> only these changes that have been already committed to the majority of
> secondary instances.
> It prevents stale reads - a situation in which a change has been committed on
> the primary (and read from it), but due to the network partition a new
> primary is elected and the change is rolled back.
> We should use this new option (together with {{w:majority}} implemented in
> OAK-3554) when running Oak on MongoDB replica set.
> References:
> * [Jepsen: MongoDB stale
> reads|https://aphyr.com/posts/322-jepsen-mongodb-stale-reads]
> * [MongoDB documentation: Read Concern
> in|https://docs.mongodb.org/manual/reference/read-concern/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4954) SetPropertyTest benchmark fails on Segment Tar
[
https://issues.apache.org/jira/browse/OAK-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731755#comment-15731755
]
Andrei Dulceanu commented on OAK-4954:
--
Right, I was looking at the {{oak-run}} documentation and mistakenly took
{{Oak-Tar}} from there. I switched to {{Oak-Segment-Tar}} and the results are
still good:
{code}
$ java -jar target/oak-run-1.6-SNAPSHOT.jar benchmark SetPropertyTest
Oak-Segment-Tar
Apache Jackrabbit Oak 1.6-SNAPSHOT
# SetPropertyTest C min 10% 50% 90% max
N
Oak-Segment-Tar1 180 195 207 220 258
289
{code}
> SetPropertyTest benchmark fails on Segment Tar
> --
>
> Key: OAK-4954
> URL: https://issues.apache.org/jira/browse/OAK-4954
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
> Labels: test-failure
> Fix For: 1.6, 1.5.15
>
>
> The {{SetPropertyTest}} fails on Oak Segment Tar:
> {noformat}
> javax.jcr.InvalidItemStateException: This item
> [/testfb3e8f1a/ca1ef350-f650-4466-b9e3-7f77d83e6303] does not exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl$ItemWriteOperation.checkPreconditions(ItemImpl.java:96)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$35.checkPreconditions(NodeImpl.java:1366)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.internalSetProperty(NodeImpl.java:1363)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.setProperty(NodeImpl.java:506)
> at
> org.apache.jackrabbit.oak.benchmark.SetPropertyTest.runTest(SetPropertyTest.java:65)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.execute(AbstractTest.java:372)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.runTest(AbstractTest.java:221)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.run(AbstractTest.java:197)
> at
> org.apache.jackrabbit.oak.benchmark.BenchmarkRunner.main(BenchmarkRunner.java:456)
> at
> org.apache.jackrabbit.oak.run.BenchmarkCommand.execute(BenchmarkCommand.java:26)
> at org.apache.jackrabbit.oak.run.Mode.execute(Mode.java:63)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-5251) Test failure: externalAddOffline(org.apache.jackrabbit.oak.plugins.blob.datastore.BlobIdTrackerTest)
[ https://issues.apache.org/jira/browse/OAK-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731740#comment-15731740 ] Amit Jain commented on OAK-5251: Ignored test for now with http://svn.apache.org/viewvc?rev=1773207=rev > Test failure: > externalAddOffline(org.apache.jackrabbit.oak.plugins.blob.datastore.BlobIdTrackerTest) > > > Key: OAK-5251 > URL: https://issues.apache.org/jira/browse/OAK-5251 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Minor > Fix For: 1.6 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4954) SetPropertyTest benchmark fails on Segment Tar
[
https://issues.apache.org/jira/browse/OAK-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731737#comment-15731737
]
Michael Dürig commented on OAK-4954:
You need to run it against the {{Oak-Segment-Tar}} fixture. {{Oak-Tar}} is the
old {{oak-segment}} module.
> SetPropertyTest benchmark fails on Segment Tar
> --
>
> Key: OAK-4954
> URL: https://issues.apache.org/jira/browse/OAK-4954
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
> Labels: test-failure
> Fix For: 1.6, 1.5.15
>
>
> The {{SetPropertyTest}} fails on Oak Segment Tar:
> {noformat}
> javax.jcr.InvalidItemStateException: This item
> [/testfb3e8f1a/ca1ef350-f650-4466-b9e3-7f77d83e6303] does not exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl$ItemWriteOperation.checkPreconditions(ItemImpl.java:96)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$35.checkPreconditions(NodeImpl.java:1366)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.internalSetProperty(NodeImpl.java:1363)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.setProperty(NodeImpl.java:506)
> at
> org.apache.jackrabbit.oak.benchmark.SetPropertyTest.runTest(SetPropertyTest.java:65)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.execute(AbstractTest.java:372)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.runTest(AbstractTest.java:221)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.run(AbstractTest.java:197)
> at
> org.apache.jackrabbit.oak.benchmark.BenchmarkRunner.main(BenchmarkRunner.java:456)
> at
> org.apache.jackrabbit.oak.run.BenchmarkCommand.execute(BenchmarkCommand.java:26)
> at org.apache.jackrabbit.oak.run.Mode.execute(Mode.java:63)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-5251) Test failure: externalAddOffline(org.apache.jackrabbit.oak.plugins.blob.datastore.BlobIdTrackerTest)
Amit Jain created OAK-5251: -- Summary: Test failure: externalAddOffline(org.apache.jackrabbit.oak.plugins.blob.datastore.BlobIdTrackerTest) Key: OAK-5251 URL: https://issues.apache.org/jira/browse/OAK-5251 Project: Jackrabbit Oak Issue Type: Bug Components: blob Reporter: Amit Jain Assignee: Amit Jain Priority: Minor Fix For: 1.6 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4954) SetPropertyTest benchmark fails on Segment Tar
[
https://issues.apache.org/jira/browse/OAK-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731727#comment-15731727
]
Andrei Dulceanu commented on OAK-4954:
--
[~mduerig], I've just run locally this testcase with the following command:
{code}
$ java -jar target/oak-run-1.6-SNAPSHOT.jar benchmark SetPropertyTest Oak-Tar
{code}
and here's the output obtained:
{code}
Apache Jackrabbit Oak 1.6-SNAPSHOT
# SetPropertyTest C min 10% 50% 90% max
N
Oak-Tar1 86 95 100 106 140
596
{code}
LGTM. Am I doing something wrong here?
> SetPropertyTest benchmark fails on Segment Tar
> --
>
> Key: OAK-4954
> URL: https://issues.apache.org/jira/browse/OAK-4954
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
> Labels: test-failure
> Fix For: 1.6, 1.5.15
>
>
> The {{SetPropertyTest}} fails on Oak Segment Tar:
> {noformat}
> javax.jcr.InvalidItemStateException: This item
> [/testfb3e8f1a/ca1ef350-f650-4466-b9e3-7f77d83e6303] does not exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl$ItemWriteOperation.checkPreconditions(ItemImpl.java:96)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$35.checkPreconditions(NodeImpl.java:1366)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.internalSetProperty(NodeImpl.java:1363)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.setProperty(NodeImpl.java:506)
> at
> org.apache.jackrabbit.oak.benchmark.SetPropertyTest.runTest(SetPropertyTest.java:65)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.execute(AbstractTest.java:372)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.runTest(AbstractTest.java:221)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.run(AbstractTest.java:197)
> at
> org.apache.jackrabbit.oak.benchmark.BenchmarkRunner.main(BenchmarkRunner.java:456)
> at
> org.apache.jackrabbit.oak.run.BenchmarkCommand.execute(BenchmarkCommand.java:26)
> at org.apache.jackrabbit.oak.run.Mode.execute(Mode.java:63)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4069) Use read concern majority when connected to a replica set
[
https://issues.apache.org/jira/browse/OAK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731717#comment-15731717
]
Marcel Reutegger commented on OAK-4069:
---
I think it would be better if we just upgrade to the 3.4 driver version and get
support for MongoDB 3.4 as well. What happens if you connect to a replica set
that does not support the read concern? IIRC that failed with an exception when
I last tried it. Otherwise looks good.
> Use read concern majority when connected to a replica set
> -
>
> Key: OAK-4069
> URL: https://issues.apache.org/jira/browse/OAK-4069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Labels: resilience
> Fix For: 1.6, 1.5.16
>
> Attachments: OAK-4069.patch
>
>
> Mongo 3.2 introduces new query option: {{readConcern}}. It allows to read
> only these changes that have been already committed to the majority of
> secondary instances.
> It prevents stale reads - a situation in which a change has been committed on
> the primary (and read from it), but due to the network partition a new
> primary is elected and the change is rolled back.
> We should use this new option (together with {{w:majority}} implemented in
> OAK-3554) when running Oak on MongoDB replica set.
> References:
> * [Jepsen: MongoDB stale
> reads|https://aphyr.com/posts/322-jepsen-mongodb-stale-reads]
> * [MongoDB documentation: Read Concern
> in|https://docs.mongodb.org/manual/reference/read-concern/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-5017) Standby test failures
[
https://issues.apache.org/jira/browse/OAK-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731707#comment-15731707
]
Francesco Mari commented on OAK-5017:
-
I tried to configure Jenkins to run the matrix tasks on the Ubuntu nodes only.
For posterity, this can be done by adding a "Slaves" axis in the matrix
configuration. The sad truth is that on Apache's Jenkins this doesn't work. I'm
resorting to the {{CIHelper}}.
> Standby test failures
> -
>
> Key: OAK-5017
> URL: https://issues.apache.org/jira/browse/OAK-5017
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: test-failure
> Fix For: 1.6, 1.5.15
>
>
> I've recently seen a couple of the standby tests fail. E.g. on Jenkins:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1245/
> {noformat}
> java.lang.AssertionError: expected:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }> but was:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }>
> at
> org.apache.jackrabbit.oak.segment.standby.StandbyTestIT.testSyncLoop(StandbyTestIT.java:122)
> {noformat}
> {noformat}
> java.lang.AssertionError: expected:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }> but was:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }>
> at
> org.apache.jackrabbit.oak.segment.standby.StandbyTestIT.testSyncLoop(StandbyTestIT.java:122)
> {noformat}
> {{org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT.testProxySkippedBytes}}:
> {noformat}
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-5250) Avoid temporary files on re-index
Marcel Reutegger created OAK-5250: - Summary: Avoid temporary files on re-index Key: OAK-5250 URL: https://issues.apache.org/jira/browse/OAK-5250 Project: Jackrabbit Oak Issue Type: Improvement Components: lucene Reporter: Marcel Reutegger Fix For: 1.8 A lucene index with copy-on-write writes back files to the repository asynchronously and the implementation may skip files that are created on the local filesystem and shortly after deleted again. For the re-index case this could be further improved. All intermediate lucene segment files that are later merged together can be skipped if the asynchronous write back is disabled and only the final files are copied to the repository when the directory is closed. This would avoid a lot of garbage in the blob store. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4069) Use read concern majority when connected to a replica set
[
https://issues.apache.org/jira/browse/OAK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731651#comment-15731651
]
Tomek Rękawek commented on OAK-4069:
Patch attached. I tried to stay as close as possible to the existing
writeConcern support / checks. I upgraded the Mongo driver to 3.3, since it has
better support for readConcern (DB class setter/getter). According to the
[docs|https://docs.mongodb.com/ecosystem/drivers/java/], the 3.3 driver should
be compatible with MongoDB 3.2.
[~mreutegg], could you take a look on the patch?
> Use read concern majority when connected to a replica set
> -
>
> Key: OAK-4069
> URL: https://issues.apache.org/jira/browse/OAK-4069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Labels: resilience
> Fix For: 1.6, 1.5.16
>
> Attachments: OAK-4069.patch
>
>
> Mongo 3.2 introduces new query option: {{readConcern}}. It allows to read
> only these changes that have been already committed to the majority of
> secondary instances.
> It prevents stale reads - a situation in which a change has been committed on
> the primary (and read from it), but due to the network partition a new
> primary is elected and the change is rolled back.
> We should use this new option (together with {{w:majority}} implemented in
> OAK-3554) when running Oak on MongoDB replica set.
> References:
> * [Jepsen: MongoDB stale
> reads|https://aphyr.com/posts/322-jepsen-mongodb-stale-reads]
> * [MongoDB documentation: Read Concern
> in|https://docs.mongodb.org/manual/reference/read-concern/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4069) Use read concern majority when connected to a replica set
[
https://issues.apache.org/jira/browse/OAK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4069:
---
Attachment: OAK-4069.patch
> Use read concern majority when connected to a replica set
> -
>
> Key: OAK-4069
> URL: https://issues.apache.org/jira/browse/OAK-4069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Labels: resilience
> Fix For: 1.6, 1.5.16
>
> Attachments: OAK-4069.patch
>
>
> Mongo 3.2 introduces new query option: {{readConcern}}. It allows to read
> only these changes that have been already committed to the majority of
> secondary instances.
> It prevents stale reads - a situation in which a change has been committed on
> the primary (and read from it), but due to the network partition a new
> primary is elected and the change is rolled back.
> We should use this new option (together with {{w:majority}} implemented in
> OAK-3554) when running Oak on MongoDB replica set.
> References:
> * [Jepsen: MongoDB stale
> reads|https://aphyr.com/posts/322-jepsen-mongodb-stale-reads]
> * [MongoDB documentation: Read Concern
> in|https://docs.mongodb.org/manual/reference/read-concern/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-5017) Standby test failures
[
https://issues.apache.org/jira/browse/OAK-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731590#comment-15731590
]
Francesco Mari commented on OAK-5017:
-
The latest group of problematic tests related to Cold Standby are the following.
{noformat}
FailoverIPRangeIT.testFailoverCorrectListIPv6UseIPv6:133->createTestWithConfig:164
expected:<{ root = { ... } }> but was:<{ root : { } }>
FailoverIPRangeIT.testFailoverCorrectListUseIPv6:126->createTestWithConfig:164
expected:<{ root = { ... } }> but was:<{ root : { } }>
FailoverIPRangeIT.testFailoverLocalClientUseIPv6:65->createTestWithConfig:164
expected:<{ root = { ... } }> but was:<{ root : { } }>
{noformat}
These tests always fail because the following error.
{noformat}
io.netty.channel.AbstractChannel$AnnotatedSocketException: Protocol family
unavailable: /0:0:0:0:0:0:0:1:50238
at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_102]
at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_102]
at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_102]
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
~[na:1.8.0_102]
at
io.netty.channel.socket.nio.NioSocketChannel.doConnect(NioSocketChannel.java:242)
~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
...
{noformat}
The test and the error hint that the issue might be related to the way IPv6 is
implemented in the JVM or configured on the host. The following table reports
the build, the node and the JVM where the tests failed.
{noformat}
1320
1319 beam2 (1.8) beam3 (1.8)
1318 beam3 (1.8) beam2 (1.8)
1317
1316 beam2 (1.8) beam3 (1.8)
1315
1314 beam1 (1.7)
1313
1312
1311
{noformat}
The following table reports the same information for every time the tests
passed.
{noformat}
1320 H18 (1.8) ubuntu-eu2 (1.8) H16 (1.8) H10 (1.8) H15 (1.7) H12 (1.7)
ubuntu-1 (1.7) proserpina-test (1.7)
1319 H18 (1.8) H16 (1.8) H15 (1.7) ubuntu-1 (1.7) H16 (1.7) proserpina-test
(1.7)
1318 H18 (1.8) H15 (1.8) ubuntu-1 (1.7) proserpina-test (1.7)
1317
1316 H18 (1.8) H16 (1.8) H15 (1.7) ubuntu-1 (1.7) proserpina-test (1.7)
1315
1314 ubuntu-6 (1.8) ubuntu-4 (1.8) ubuntu-2 (1.8) H10 (1.8) H12 (1.7) H17 (1.7)
H11 (1.7)
1313
1312
1311
{noformat}
The lack of overlapping between the node names hint that the problem might be
related to the configuration of the nodes grouped under the "beam" label.
> Standby test failures
> -
>
> Key: OAK-5017
> URL: https://issues.apache.org/jira/browse/OAK-5017
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: test-failure
> Fix For: 1.6, 1.5.15
>
>
> I've recently seen a couple of the standby tests fail. E.g. on Jenkins:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1245/
> {noformat}
> java.lang.AssertionError: expected:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }> but was:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }>
> at
> org.apache.jackrabbit.oak.segment.standby.StandbyTestIT.testSyncLoop(StandbyTestIT.java:122)
> {noformat}
> {noformat}
> java.lang.AssertionError: expected:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }> but was:
> org.apache.jackrabbit.oak.segment.SegmentNodeState<{ checkpoints = { ... },
> root = { ... } }>
> at
> org.apache.jackrabbit.oak.segment.standby.StandbyTestIT.testSyncLoop(StandbyTestIT.java:122)
> {noformat}
> {{org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT.testProxySkippedBytes}}:
> {noformat}
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-5238) IndexCopier causes concurrent update on NodeBuilder
[ https://issues.apache.org/jira/browse/OAK-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731586#comment-15731586 ] Chetan Mehrotra commented on OAK-5238: -- bq. Maybe the IndexCopier could create the blob asynchronously and then update the node builder while holding a lock? Yes but then also blob data needs to be held in memory. Or we have some sync points with main indexing thread where LuceneIndexEditor does a callback on OakDirectory to push the changes for e.g. in DefaultIndexWriter#updateDocument call > IndexCopier causes concurrent update on NodeBuilder > --- > > Key: OAK-5238 > URL: https://issues.apache.org/jira/browse/OAK-5238 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Affects Versions: 1.2.3, 1.0.15, 1.4.0 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: candidate_oak_1_4 > Fix For: 1.6 > > Attachments: OAK-5238.patch > > > OAK-2247 introduced the copy-on-write feature for lucene index in Oak. This > feature may result in a NodeBuilder updated by multiple threads concurrently. > New index files are first stored on the local filesystem and then copied > asynchronously into the repository. At the same time the async index update > thread manipulates the node builders as well. > With MongoMK this results in unexpected conflicts and failed async index > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-5249) DefaultAuthorizableActionProvider should use @Modified to react to config changes
Tobias Bocanegra created OAK-5249:
-
Summary: DefaultAuthorizableActionProvider should use @Modified to
react to config changes
Key: OAK-5249
URL: https://issues.apache.org/jira/browse/OAK-5249
Project: Jackrabbit Oak
Issue Type: Improvement
Components: security
Affects Versions: 1.5.14
Reporter: Tobias Bocanegra
The DefaultAuthorizableActionProvider [0] doesn't handle the @Modified
component description, causing entire OAK to restart when the config changes.
suggested change:
{code}
@Activate
private void activate(Map properties) {
this.modified(properties);
}
@Modified
private void modified(Map properties) {
config = ConfigurationParameters.of(properties);
enabledActions = config.getConfigValue(ENABLED_ACTIONS,
DEFAULT_ACTIONS);
}
{code}
[0]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/spi/security/user/action/DefaultAuthorizableActionProvider.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-5241) Test failure: TomcatIT.testTomcat()
[ https://issues.apache.org/jira/browse/OAK-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra reassigned OAK-5241: Assignee: Chetan Mehrotra > Test failure: TomcatIT.testTomcat() > --- > > Key: OAK-5241 > URL: https://issues.apache.org/jira/browse/OAK-5241 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration, webapp >Reporter: Hudson >Assignee: Chetan Mehrotra > Fix For: 1.6, 1.5.15 > > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.8 > (latest),nsfixtures=DOCUMENT_RDB,profile=unittesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.8 > (latest),nsfixtures=DOCUMENT_RDB,profile=unittesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.8%20(latest),nsfixtures=DOCUMENT_RDB,profile=unittesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.8%20(latest),nsfixtures=DOCUMENT_RDB,profile=unittesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-5241) Test failure: TomcatIT.testTomcat()
[ https://issues.apache.org/jira/browse/OAK-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-5241. -- Resolution: Fixed Fix Version/s: 1.5.15 Fixed as part of OAK-5248 > Test failure: TomcatIT.testTomcat() > --- > > Key: OAK-5241 > URL: https://issues.apache.org/jira/browse/OAK-5241 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration, webapp >Reporter: Hudson >Assignee: Chetan Mehrotra > Fix For: 1.6, 1.5.15 > > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.8 > (latest),nsfixtures=DOCUMENT_RDB,profile=unittesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.8 > (latest),nsfixtures=DOCUMENT_RDB,profile=unittesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.8%20(latest),nsfixtures=DOCUMENT_RDB,profile=unittesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.8%20(latest),nsfixtures=DOCUMENT_RDB,profile=unittesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-5202) Update Oak trunk to Jackrabbit 2.13.5
[ https://issues.apache.org/jira/browse/OAK-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-5202. - Resolution: Fixed > Update Oak trunk to Jackrabbit 2.13.5 > - > > Key: OAK-5202 > URL: https://issues.apache.org/jira/browse/OAK-5202 > Project: Jackrabbit Oak > Issue Type: Task > Components: parent >Affects Versions: 1.5.14 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Fix For: 1.6, 1.5.15 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-5238) IndexCopier causes concurrent update on NodeBuilder
[ https://issues.apache.org/jira/browse/OAK-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731537#comment-15731537 ] Marcel Reutegger commented on OAK-5238: --- bq. One possible approach can be to use a in memory builder in OakDirectory and merge that back to main builder upon close. This would ensure that concurrent access is avoided. Yes, this should work, but I would merge it back earlier and not on close. On re-index the directory may see a lot of changes and I think it's better to not buffer them all in memory, even if binaries go into the blob store. The node states still contain binary references of significant size. Maybe the IndexCopier could create the blob asynchronously and then update the node builder while holding a lock? I'll work on a patch. > IndexCopier causes concurrent update on NodeBuilder > --- > > Key: OAK-5238 > URL: https://issues.apache.org/jira/browse/OAK-5238 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Affects Versions: 1.2.3, 1.0.15, 1.4.0 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: candidate_oak_1_4 > Fix For: 1.6 > > Attachments: OAK-5238.patch > > > OAK-2247 introduced the copy-on-write feature for lucene index in Oak. This > feature may result in a NodeBuilder updated by multiple threads concurrently. > New index files are first stored on the local filesystem and then copied > asynchronously into the repository. At the same time the async index update > thread manipulates the node builders as well. > With MongoMK this results in unexpected conflicts and failed async index > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-5238) IndexCopier causes concurrent update on NodeBuilder
[ https://issues.apache.org/jira/browse/OAK-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger reassigned OAK-5238: - Assignee: Marcel Reutegger > IndexCopier causes concurrent update on NodeBuilder > --- > > Key: OAK-5238 > URL: https://issues.apache.org/jira/browse/OAK-5238 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Affects Versions: 1.2.3, 1.0.15, 1.4.0 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: candidate_oak_1_4 > Fix For: 1.6 > > Attachments: OAK-5238.patch > > > OAK-2247 introduced the copy-on-write feature for lucene index in Oak. This > feature may result in a NodeBuilder updated by multiple threads concurrently. > New index files are first stored on the local filesystem and then copied > asynchronously into the repository. At the same time the async index update > thread manipulates the node builders as well. > With MongoMK this results in unexpected conflicts and failed async index > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-5241) Test failure: TomcatIT.testTomcat()
[ https://issues.apache.org/jira/browse/OAK-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-5241: -- Component/s: webapp > Test failure: TomcatIT.testTomcat() > --- > > Key: OAK-5241 > URL: https://issues.apache.org/jira/browse/OAK-5241 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration, webapp >Reporter: Hudson > Fix For: 1.6 > > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.8 > (latest),nsfixtures=DOCUMENT_RDB,profile=unittesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.8 > (latest),nsfixtures=DOCUMENT_RDB,profile=unittesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.8%20(latest),nsfixtures=DOCUMENT_RDB,profile=unittesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.8%20(latest),nsfixtures=DOCUMENT_RDB,profile=unittesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-5245) Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 (latest),nsfixtures=DOCUMENT_NS,profile=integrationTesting #1320 failed
[ https://issues.apache.org/jira/browse/OAK-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-5245. --- Resolution: Duplicate > Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=DOCUMENT_NS,profile=integrationTesting #1320 failed > -- > > Key: OAK-5245 > URL: https://issues.apache.org/jira/browse/OAK-5245 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=DOCUMENT_NS,profile=integrationTesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=DOCUMENT_NS,profile=integrationTesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=DOCUMENT_NS,profile=integrationTesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=DOCUMENT_NS,profile=integrationTesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-5244) Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 (latest),nsfixtures=SEGMENT_MK,profile=integrationTesting #1320 failed
[ https://issues.apache.org/jira/browse/OAK-5244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-5244. --- Resolution: Duplicate > Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=SEGMENT_MK,profile=integrationTesting #1320 failed > - > > Key: OAK-5244 > URL: https://issues.apache.org/jira/browse/OAK-5244 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=SEGMENT_MK,profile=integrationTesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=SEGMENT_MK,profile=integrationTesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=SEGMENT_MK,profile=integrationTesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=SEGMENT_MK,profile=integrationTesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-5242) Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 (latest),nsfixtures=SEGMENT_MK,profile=unittesting #1320 failed
[ https://issues.apache.org/jira/browse/OAK-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-5242. --- Resolution: Duplicate > Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=SEGMENT_MK,profile=unittesting #1320 failed > -- > > Key: OAK-5242 > URL: https://issues.apache.org/jira/browse/OAK-5242 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=SEGMENT_MK,profile=unittesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=SEGMENT_MK,profile=unittesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=SEGMENT_MK,profile=unittesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=SEGMENT_MK,profile=unittesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-5243) Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 (latest),nsfixtures=SEGMENT_TAR,profile=integrationTesting #1320 failed
[ https://issues.apache.org/jira/browse/OAK-5243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-5243. --- Resolution: Duplicate > Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=SEGMENT_TAR,profile=integrationTesting #1320 failed > -- > > Key: OAK-5243 > URL: https://issues.apache.org/jira/browse/OAK-5243 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=SEGMENT_TAR,profile=integrationTesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=SEGMENT_TAR,profile=integrationTesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=SEGMENT_TAR,profile=integrationTesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=SEGMENT_TAR,profile=integrationTesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-5241) Test failure: TomcatIT.testTomcat()
[ https://issues.apache.org/jira/browse/OAK-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-5241: -- Fix Version/s: 1.6 Summary: Test failure: TomcatIT.testTomcat() (was: Build Apache Jackrabbit Oak matrix/jdk=JDK 1.8 (latest),nsfixtures=DOCUMENT_RDB,profile=unittesting #1320 failed) > Test failure: TomcatIT.testTomcat() > --- > > Key: OAK-5241 > URL: https://issues.apache.org/jira/browse/OAK-5241 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > Fix For: 1.6 > > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.8 > (latest),nsfixtures=DOCUMENT_RDB,profile=unittesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.8 > (latest),nsfixtures=DOCUMENT_RDB,profile=unittesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.8%20(latest),nsfixtures=DOCUMENT_RDB,profile=unittesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.8%20(latest),nsfixtures=DOCUMENT_RDB,profile=unittesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-5240) Test failure: IndexSanityCheckerTest.sizeMismatch()
[ https://issues.apache.org/jira/browse/OAK-5240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-5240: -- Fix Version/s: 1.6 Component/s: lucene Summary: Test failure: IndexSanityCheckerTest.sizeMismatch() (was: Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 (latest),nsfixtures=DOCUMENT_NS,profile=unittesting #1320 failed) > Test failure: IndexSanityCheckerTest.sizeMismatch() > --- > > Key: OAK-5240 > URL: https://issues.apache.org/jira/browse/OAK-5240 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration, lucene >Reporter: Hudson > Fix For: 1.6 > > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=DOCUMENT_NS,profile=unittesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=DOCUMENT_NS,profile=unittesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=DOCUMENT_NS,profile=unittesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=DOCUMENT_NS,profile=unittesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-5239) Test failure: ExternalPrivateStoreIT. testSyncUpdatedBinaryProperty()
[ https://issues.apache.org/jira/browse/OAK-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-5239: -- Fix Version/s: 1.6 Component/s: segment-tar Summary: Test failure: ExternalPrivateStoreIT. testSyncUpdatedBinaryProperty() (was: Build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 (latest),nsfixtures=DOCUMENT_RDB,profile=integrationTesting #1320 failed) > Test failure: ExternalPrivateStoreIT. testSyncUpdatedBinaryProperty() > - > > Key: OAK-5239 > URL: https://issues.apache.org/jira/browse/OAK-5239 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration, segment-tar >Reporter: Hudson > Fix For: 1.6 > > > Jenkins CI failure: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ > The build Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=DOCUMENT_RDB,profile=integrationTesting #1320 has failed. > First failed run: [Apache Jackrabbit Oak matrix/jdk=JDK 1.7 > (latest),nsfixtures=DOCUMENT_RDB,profile=integrationTesting > #1320|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=DOCUMENT_RDB,profile=integrationTesting/1320/] > [console > log|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=JDK%201.7%20(latest),nsfixtures=DOCUMENT_RDB,profile=integrationTesting/1320/console] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
