[jira] [Comment Edited] (OAK-3987) Indexer dry run mode
[
https://issues.apache.org/jira/browse/OAK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092576#comment-16092576
]
Chetan Mehrotra edited comment on OAK-3987 at 7/19/17 5:08 AM:
---
[~edivad] oak-run tool now supports out-of-band indexing where no changes are
done to NodeStore [1]. This meets the dry run requirements.
{noformat}
java -jar oak-run*.jar index --reindex \
--index-paths=/oak:index/indexName \
--checkpoint=head \
--fds-path=/path/to/datastore /path/to/segmentstore/
{noformat}
If this looks fine to you then I would like to resolve this issue as done
[1]
https://jackrabbit.apache.org/oak/docs/query/oak-run-indexing.html#out-of-band-indexing
was (Author: chetanm):
[~edivad] oak-run tool now supports out-of-band indexing where no changes are
done to NodeStore [1]. This meets the dry run requirements.
If this looks fine to you then I would like to resolve this issue as done
[1]
https://jackrabbit.apache.org/oak/docs/query/oak-run-indexing.html#out-of-band-indexing
> Indexer dry run mode
>
>
> Key: OAK-3987
> URL: https://issues.apache.org/jira/browse/OAK-3987
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing
>Reporter: Alex Deparvu
>Assignee: Davide Giannella
> Fix For: 1.8
>
>
> Based on a discussion on the dev list, it would be interesting to provide a
> {{dry run}} mode for the indexer which would give an indication of an average
> indexing time.
> Input could be:
> * path to index definition (mandatory)
> * path to the content (mandatory, default could be {{/}})
> * max number of nodes (optional, but I'd still cap this value so indexing
> doesn't take over the entire aem instance)
> Also, we could do this on a separate (dedicated) thread so it doesn't
> interfere with existing indexers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-3987) Indexer dry run mode
[
https://issues.apache.org/jira/browse/OAK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-3987:
-
Fix Version/s: 1.8
> Indexer dry run mode
>
>
> Key: OAK-3987
> URL: https://issues.apache.org/jira/browse/OAK-3987
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing
>Reporter: Alex Deparvu
>Assignee: Davide Giannella
> Fix For: 1.8
>
>
> Based on a discussion on the dev list, it would be interesting to provide a
> {{dry run}} mode for the indexer which would give an indication of an average
> indexing time.
> Input could be:
> * path to index definition (mandatory)
> * path to the content (mandatory, default could be {{/}})
> * max number of nodes (optional, but I'd still cap this value so indexing
> doesn't take over the entire aem instance)
> Also, we could do this on a separate (dedicated) thread so it doesn't
> interfere with existing indexers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-3987) Indexer dry run mode
[
https://issues.apache.org/jira/browse/OAK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092576#comment-16092576
]
Chetan Mehrotra commented on OAK-3987:
--
[~edivad] oak-run tool now supports out-of-band indexing where no changes are
done to NodeStore [1]. This meets the dry run requirements.
If this looks fine to you then I would like to resolve this issue as done
[1]
https://jackrabbit.apache.org/oak/docs/query/oak-run-indexing.html#out-of-band-indexing
> Indexer dry run mode
>
>
> Key: OAK-3987
> URL: https://issues.apache.org/jira/browse/OAK-3987
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing
>Reporter: Alex Deparvu
>Assignee: Davide Giannella
>
> Based on a discussion on the dev list, it would be interesting to provide a
> {{dry run}} mode for the indexer which would give an indication of an average
> indexing time.
> Input could be:
> * path to index definition (mandatory)
> * path to the content (mandatory, default could be {{/}})
> * max number of nodes (optional, but I'd still cap this value so indexing
> doesn't take over the entire aem instance)
> Also, we could do this on a separate (dedicated) thread so it doesn't
> interfere with existing indexers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6469) CompositePermissionProvider should implement AggregatedPermissionProvider
[
https://issues.apache.org/jira/browse/OAK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu updated OAK-6469:
--
Attachment: OAK-6469.patch
attaching patch: [^OAK-6469.patch]. this also overwrites the fix for OAK-6451
with a proper impl.
[~anchela] would appreciate a look!
> CompositePermissionProvider should implement AggregatedPermissionProvider
> -
>
> Key: OAK-6469
> URL: https://issues.apache.org/jira/browse/OAK-6469
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
> Attachments: OAK-6469.patch
>
>
> A more in-depth followup of OAK-6451. It turns out you can't mix CUG setup
> with the Multiplexing setup because the {{CompositePermissionProvider}}
> cannot contain another composite.
> Making {{CompositePermissionProvider}} implement
> {{AggregatedPermissionProvider}} opens the door for composites of composites.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6469) CompositePermissionProvider should implement AggregatedPermissionProvider
Alex Deparvu created OAK-6469:
-
Summary: CompositePermissionProvider should implement
AggregatedPermissionProvider
Key: OAK-6469
URL: https://issues.apache.org/jira/browse/OAK-6469
Project: Jackrabbit Oak
Issue Type: Improvement
Components: core, security
Reporter: Alex Deparvu
Assignee: Alex Deparvu
A more in-depth followup of OAK-6451. It turns out you can't mix CUG setup with
the Multiplexing setup because the {{CompositePermissionProvider}} cannot
contain another composite.
Making {{CompositePermissionProvider}} implement
{{AggregatedPermissionProvider}} opens the door for composites of composites.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6467) CompositeTreePermission can create an invalid TreePermsssion object
[
https://issues.apache.org/jira/browse/OAK-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu resolved OAK-6467.
---
Resolution: Won't Fix
Fix Version/s: (was: 1.7.4)
> CompositeTreePermission can create an invalid TreePermsssion object
> ---
>
> Key: OAK-6467
> URL: https://issues.apache.org/jira/browse/OAK-6467
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
>
> There's a case where the {{CompositePermissionProvider}} can create an
> invalid {{TreePermsssion}} instance via the {{CompositeTreePermission}}
> object. It can return a {{NO_RECOURSE}} if there's a single provider
> configured (like the CUG) that is not able to handle that specific check.
> {noformat}
> java.lang.UnsupportedOperationException: null
> at
> org.apache.jackrabbit.oak.spi.security.authorization.permission.TreePermission$3.canRead(TreePermission.java:212)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder.exists(SecureNodeBuilder.java:128)
> at
> org.apache.jackrabbit.oak.plugins.tree.impl.AbstractTree.exists(AbstractTree.java:225)
> at
> org.apache.jackrabbit.oak.core.MutableTree.exists(MutableTree.java:122)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getNode(SessionDelegate.java:427)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getRootNode(SessionDelegate.java:415)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getItem(SessionDelegate.java:440)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemInternal(SessionImpl.java:166)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.access$400(SessionImpl.java:81)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:228)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:225)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performNullable(SessionDelegate.java:243)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemOrNull(SessionImpl.java:225)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-5740) deliver overflow change even without new commit
[ https://issues.apache.org/jira/browse/OAK-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091828#comment-16091828 ] Vikas Saurabh commented on OAK-5740: [~egli], I could not read through the patch closely (but, I had seen the changes earlier anyway.. so, "patch lgtm" still holds :) ). I think we should fix this issue (... and if that sounds too risky to others, then close this issue noting the same)... I mean that I don't see any point in holding this issue open. > deliver overflow change even without new commit > --- > > Key: OAK-5740 > URL: https://issues.apache.org/jira/browse/OAK-5740 > Project: Jackrabbit Oak > Issue Type: Bug > Components: jcr >Affects Versions: 1.6.0 >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Minor > Fix For: 1.8 > > Attachments: OAK-5740.StuffAtop-v2-patch.patch, > OAK-5740.testcase.patch, OAK-5740.testcase.patch, OAK-5740.v2.patch, > OAK-5740.v3.patch > > > As [reported|http://markmail.org/message/2qxle24f6zu2vpms] by [~catholicon] > on oak-dev the observation queue only delivers the so-called _overflow > entry/change_ only when new commits are 'coming in'. We might want to > consider fixing this, even though arguably this is a very rare case (since > typically the observation queue is configured to be very large) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6467) CompositeTreePermission can create an invalid TreePermsssion object
[
https://issues.apache.org/jira/browse/OAK-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091767#comment-16091767
]
Alex Deparvu commented on OAK-6467:
---
yes, you are right. unfortunately I realized that this might actually be
related to OAK-6451 but wanted to double-check before closing this as won't fix
:)
> CompositeTreePermission can create an invalid TreePermsssion object
> ---
>
> Key: OAK-6467
> URL: https://issues.apache.org/jira/browse/OAK-6467
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
> Fix For: 1.7.4
>
>
> There's a case where the {{CompositePermissionProvider}} can create an
> invalid {{TreePermsssion}} instance via the {{CompositeTreePermission}}
> object. It can return a {{NO_RECOURSE}} if there's a single provider
> configured (like the CUG) that is not able to handle that specific check.
> {noformat}
> java.lang.UnsupportedOperationException: null
> at
> org.apache.jackrabbit.oak.spi.security.authorization.permission.TreePermission$3.canRead(TreePermission.java:212)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder.exists(SecureNodeBuilder.java:128)
> at
> org.apache.jackrabbit.oak.plugins.tree.impl.AbstractTree.exists(AbstractTree.java:225)
> at
> org.apache.jackrabbit.oak.core.MutableTree.exists(MutableTree.java:122)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getNode(SessionDelegate.java:427)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getRootNode(SessionDelegate.java:415)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getItem(SessionDelegate.java:440)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemInternal(SessionImpl.java:166)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.access$400(SessionImpl.java:81)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:228)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:225)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performNullable(SessionDelegate.java:243)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemOrNull(SessionImpl.java:225)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6467) CompositeTreePermission can create an invalid TreePermsssion object
[
https://issues.apache.org/jira/browse/OAK-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091734#comment-16091734
]
angela commented on OAK-6467:
-
[~stillalex], i am not sure this is really a bug. if your have a security setup
that ends up with a {{TreePermission.NO_RECOURSE}} as the only option left your
setup is wrong and this should be identified during testing of a new
authorization model.
> CompositeTreePermission can create an invalid TreePermsssion object
> ---
>
> Key: OAK-6467
> URL: https://issues.apache.org/jira/browse/OAK-6467
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
> Fix For: 1.7.4
>
>
> There's a case where the {{CompositePermissionProvider}} can create an
> invalid {{TreePermsssion}} instance via the {{CompositeTreePermission}}
> object. It can return a {{NO_RECOURSE}} if there's a single provider
> configured (like the CUG) that is not able to handle that specific check.
> {noformat}
> java.lang.UnsupportedOperationException: null
> at
> org.apache.jackrabbit.oak.spi.security.authorization.permission.TreePermission$3.canRead(TreePermission.java:212)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder.exists(SecureNodeBuilder.java:128)
> at
> org.apache.jackrabbit.oak.plugins.tree.impl.AbstractTree.exists(AbstractTree.java:225)
> at
> org.apache.jackrabbit.oak.core.MutableTree.exists(MutableTree.java:122)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getNode(SessionDelegate.java:427)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getRootNode(SessionDelegate.java:415)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getItem(SessionDelegate.java:440)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemInternal(SessionImpl.java:166)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.access$400(SessionImpl.java:81)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:228)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:225)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performNullable(SessionDelegate.java:243)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemOrNull(SessionImpl.java:225)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6468) Include the tail generation in the binary references index
Francesco Mari created OAK-6468: --- Summary: Include the tail generation in the binary references index Key: OAK-6468 URL: https://issues.apache.org/jira/browse/OAK-6468 Project: Jackrabbit Oak Issue Type: New Feature Components: segment-tar Reporter: Francesco Mari Assignee: Francesco Mari Fix For: 1.8 OAK-3349 introduces the tail generation, an additional generation number associated to each segment that is consulted during the cleanup phase to determine if a segment should be removed. The tail generation is currently only preset in the segments. For efficiency purposes, it should be added to the binary references index too, like the full generation number. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6467) CompositeTreePermission can create an invalid TreePermsssion object
[
https://issues.apache.org/jira/browse/OAK-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu updated OAK-6467:
--
Description:
There's a case where the {{CompositePermissionProvider}} can create an invalid
{{TreePermsssion}} instance via the {{CompositeTreePermission}} object. It can
return a {{NO_RECOURSE}} if there's a single provider configured (like the CUG)
that is not able to handle that specific check.
{noformat}
java.lang.UnsupportedOperationException: null
at
org.apache.jackrabbit.oak.spi.security.authorization.permission.TreePermission$3.canRead(TreePermission.java:212)
at
org.apache.jackrabbit.oak.core.SecureNodeBuilder.exists(SecureNodeBuilder.java:128)
at
org.apache.jackrabbit.oak.plugins.tree.impl.AbstractTree.exists(AbstractTree.java:225)
at
org.apache.jackrabbit.oak.core.MutableTree.exists(MutableTree.java:122)
at
org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getNode(SessionDelegate.java:427)
at
org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getRootNode(SessionDelegate.java:415)
at
org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getItem(SessionDelegate.java:440)
at
org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemInternal(SessionImpl.java:166)
at
org.apache.jackrabbit.oak.jcr.session.SessionImpl.access$400(SessionImpl.java:81)
at
org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:228)
at
org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:225)
at
org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performNullable(SessionDelegate.java:243)
at
org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemOrNull(SessionImpl.java:225)
{noformat}
was:There's a case where the {{CompositePermissionProvider}} can create an
invalid {{TreePermsssion}} instance via the {{CompositeTreePermission}} object.
It can return a {{NO_RECOURSE}} if there's a single provider configured (like
the CUG) that is not able to handle that specific check.
> CompositeTreePermission can create an invalid TreePermsssion object
> ---
>
> Key: OAK-6467
> URL: https://issues.apache.org/jira/browse/OAK-6467
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
> Fix For: 1.7.4
>
>
> There's a case where the {{CompositePermissionProvider}} can create an
> invalid {{TreePermsssion}} instance via the {{CompositeTreePermission}}
> object. It can return a {{NO_RECOURSE}} if there's a single provider
> configured (like the CUG) that is not able to handle that specific check.
> {noformat}
> java.lang.UnsupportedOperationException: null
> at
> org.apache.jackrabbit.oak.spi.security.authorization.permission.TreePermission$3.canRead(TreePermission.java:212)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder.exists(SecureNodeBuilder.java:128)
> at
> org.apache.jackrabbit.oak.plugins.tree.impl.AbstractTree.exists(AbstractTree.java:225)
> at
> org.apache.jackrabbit.oak.core.MutableTree.exists(MutableTree.java:122)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getNode(SessionDelegate.java:427)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getRootNode(SessionDelegate.java:415)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.getItem(SessionDelegate.java:440)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemInternal(SessionImpl.java:166)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.access$400(SessionImpl.java:81)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:228)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$3.performNullable(SessionImpl.java:225)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performNullable(SessionDelegate.java:243)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.getItemOrNull(SessionImpl.java:225)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6467) CompositeTreePermission can create an invalid TreePermsssion object
[
https://issues.apache.org/jira/browse/OAK-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu updated OAK-6467:
--
Fix Version/s: 1.7.4
> CompositeTreePermission can create an invalid TreePermsssion object
> ---
>
> Key: OAK-6467
> URL: https://issues.apache.org/jira/browse/OAK-6467
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
> Fix For: 1.7.4
>
>
> There's a case where the {{CompositePermissionProvider}} can create an
> invalid {{TreePermsssion}} instance via the {{CompositeTreePermission}}
> object. It can return a {{NO_RECOURSE}} if there's a single provider
> configured (like the CUG) that is not able to handle that specific check.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6467) CompositeTreePermission can create an invalid TreePermsssion object
Alex Deparvu created OAK-6467:
-
Summary: CompositeTreePermission can create an invalid
TreePermsssion object
Key: OAK-6467
URL: https://issues.apache.org/jira/browse/OAK-6467
Project: Jackrabbit Oak
Issue Type: Bug
Components: core, security
Reporter: Alex Deparvu
Assignee: Alex Deparvu
There's a case where the {{CompositePermissionProvider}} can create an invalid
{{TreePermsssion}} instance via the {{CompositeTreePermission}} object. It can
return a {{NO_RECOURSE}} if there's a single provider configured (like the CUG)
that is not able to handle that specific check.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-3710) Continuous revision GC
[
https://issues.apache.org/jira/browse/OAK-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091519#comment-16091519
]
Marcel Reutegger commented on OAK-3710:
---
Added an option to the DocumentNodeStoreService ({{versionGCContinuous}}) that
enable continuous Revision GC (default: false). Implemented in trunk:
http://svn.apache.org/r1802289
There are still some details to figure out. With the option enabled, the
{{VersionGarbageCollector}} logs info messages every five seconds. Those
messages are useful when GC runs e.g. every 24 hours, but are too verbose when
run frequently.
> Continuous revision GC
> --
>
> Key: OAK-3710
> URL: https://issues.apache.org/jira/browse/OAK-3710
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: documentmk
>Reporter: Marcel Reutegger
>
> Implement continuous revision GC cleaning up documents older than a given
> threshold (e.g. one day). This issue is related to OAK-3070 where each GC run
> starts where the last one finished.
> This will avoid peak load on the system as we see it right now, when GC is
> triggered once a day.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6463) Property index update fails in composite NodeStore setup
[ https://issues.apache.org/jira/browse/OAK-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6463. -- Resolution: Fixed Fix Version/s: 1.7.4 > Property index update fails in composite NodeStore setup > > > Key: OAK-6463 > URL: https://issues.apache.org/jira/browse/OAK-6463 > Project: Jackrabbit Oak > Issue Type: Bug > Components: composite, property-index >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8, 1.7.4 > > Attachments: OAK-6463-v1.patch > > > In a CompositeNodeStore setup involving 1 read only mount a commit involving > update of property index may fail even if the commit does not update content > under read only paths -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6463) Property index update fails in composite NodeStore setup
[ https://issues.apache.org/jira/browse/OAK-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091504#comment-16091504 ] Chetan Mehrotra commented on OAK-6463: -- bq. I'm wondering if the 'index' supplier should not be wrapped into a Suppliers.memoize call so you'd get the same NodeBuilder instance on each .get() call Makes sense. Applied a modified patch which uses memoize supplier with r1802288 > Property index update fails in composite NodeStore setup > > > Key: OAK-6463 > URL: https://issues.apache.org/jira/browse/OAK-6463 > Project: Jackrabbit Oak > Issue Type: Bug > Components: composite, property-index >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8, 1.7.4 > > Attachments: OAK-6463-v1.patch > > > In a CompositeNodeStore setup involving 1 read only mount a commit involving > update of property index may fail even if the commit does not update content > under read only paths -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6466) Suppress NOP info message from revision GC
[
https://issues.apache.org/jira/browse/OAK-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger resolved OAK-6466.
---
Resolution: Fixed
Fix Version/s: 1.7.4
Done in trunk: http://svn.apache.org/r1802287
> Suppress NOP info message from revision GC
> --
>
> Key: OAK-6466
> URL: https://issues.apache.org/jira/browse/OAK-6466
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, documentmk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Fix For: 1.8, 1.7.4
>
>
> The Revision GC sometimes logs messages like this:
> {noformat}
> Proceeding to reset [0] _deletedOnce flags
> {noformat}
> It should only log messages when it actually does some work.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6450) Stop relying on the service.pid property in SecurityProviderRegistration
[
https://issues.apache.org/jira/browse/OAK-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091487#comment-16091487
]
angela edited comment on OAK-6450 at 7/18/17 12:32 PM:
---
[~rombert], the patch does not include an update of the Oak documentation,
which actually describes the required service ids. The following pages need to
be updated:
- http://jackrabbit.apache.org/oak/docs/security/introduction.html
- http://jackrabbit.apache.org/oak/docs/security/authorization/composite.html
- http://jackrabbit.apache.org/oak/docs/security/authorization/cug.html
As far as the actual patch is concerned:
- why are the {{RegistrationConstants}} located in
_org.apache.jackrabbit.oak.security.authorization_ package? that looks wrong to
me as it is used also in non-authorization packages.
- shouldn't these {{RegistrationConstants}} be public in the SPI space in order
to make sure that custom implementations can use it? i would love to get rid of
the {{oak-core}} dependencies in non-core modules (see also OAK-6318)
- i am not sure about the PN prefix with the {{RegistrationConstants}}. what
does it stand for?
and one more thing: unless already present i think the patch should come with
explicit testing for the new way and the compat-mode (and a mixture of both) in
order to make sure we don't break existing customers that have custom lists of
required service (such as e.g. Adobe AEM).
was (Author: anchela):
[~rombert], the patch does not include an update of the Oak documentation,
which actually describes the required service ids. The following pages need to
be updated:
- http://jackrabbit.apache.org/oak/docs/security/introduction.html
- http://jackrabbit.apache.org/oak/docs/security/authorization/composite.html
- http://jackrabbit.apache.org/oak/docs/security/authorization/cug.html
As far as the actual patch is concerned:
- why are the {{RegistrationConstants}} located in
_org.apache.jackrabbit.oak.security.authorization_ package? that looks wrong to
me as it is used also in non-authorization packages.
- shouldn't these {{RegistrationConstants}} be public in the SPI space in order
to make sure that custom implementations can use it? i would love to get rid of
the {{oak-core}} dependencies in non-core modules (see also OAK-6318)
- i am not sure about the PN prefix with the {{RegistrationConstants}}. what
does it stand for?
> Stop relying on the service.pid property in SecurityProviderRegistration
>
>
> Key: OAK-6450
> URL: https://issues.apache.org/jira/browse/OAK-6450
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: security
>Reporter: Robert Munteanu
>Assignee: Robert Munteanu
> Fix For: 1.8, 1.7.4
>
> Attachments:
> 0001-OAK-6450-Stop-relying-on-the-service.pid-property-in.patch,
> 0002-OAK-6450-Stop-relying-on-the-service.pid-property-in.patch
>
>
> As discussed in OAK-6111, we should stop relying on the {{service.pid}} OSGi
> property for tracking required services as it was incorrectly set so far by
> the {{maven-scr-plugin}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6450) Stop relying on the service.pid property in SecurityProviderRegistration
[
https://issues.apache.org/jira/browse/OAK-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091487#comment-16091487
]
angela commented on OAK-6450:
-
[~rombert], the patch does not include an update of the Oak documentation,
which actually describes the required service ids. The following pages need to
be updated:
- http://jackrabbit.apache.org/oak/docs/security/introduction.html
- http://jackrabbit.apache.org/oak/docs/security/authorization/composite.html
- http://jackrabbit.apache.org/oak/docs/security/authorization/cug.html
As far as the actual patch is concerned:
- why are the {{RegistrationConstants}} located in
_org.apache.jackrabbit.oak.security.authorization_ package? that looks wrong to
me as it is used also in non-authorization packages.
- shouldn't these {{RegistrationConstants}} be public in the SPI space in order
to make sure that custom implementations can use it? i would love to get rid of
the {{oak-core}} dependencies in non-core modules (see also OAK-6318)
- i am not sure about the PN prefix with the {{RegistrationConstants}}. what
does it stand for?
> Stop relying on the service.pid property in SecurityProviderRegistration
>
>
> Key: OAK-6450
> URL: https://issues.apache.org/jira/browse/OAK-6450
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: security
>Reporter: Robert Munteanu
>Assignee: Robert Munteanu
> Fix For: 1.8, 1.7.4
>
> Attachments:
> 0001-OAK-6450-Stop-relying-on-the-service.pid-property-in.patch,
> 0002-OAK-6450-Stop-relying-on-the-service.pid-property-in.patch
>
>
> As discussed in OAK-6111, we should stop relying on the {{service.pid}} OSGi
> property for tracking required services as it was incorrectly set so far by
> the {{maven-scr-plugin}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6462) Incorrect memory calculation for bundled node states
[ https://issues.apache.org/jira/browse/OAK-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-6462. --- Resolution: Fixed Fix Version/s: 1.7.4 Fixed in trunk: http://svn.apache.org/r1802286 > Incorrect memory calculation for bundled node states > > > Key: OAK-6462 > URL: https://issues.apache.org/jira/browse/OAK-6462 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.6.0 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: candidate_oak_1_6 > Fix For: 1.8, 1.7.4 > > > OAK-1312 introduced bundling of nodes into a document and enabled the feature > for nt:file nodes. A DocumentNodeState of type nt:file now has an incorrect > memory calculation because the node state references a BundlingContext which > also references the binary property reference. Such a node state is now much > heavier than before and can cause OOME. > This is a clone of OAK-5226, which was supposed to fix the described issue, > however it turns out the fix is not sufficient and memory calculation is > still incorrect for bundled nodes/properties. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6463) Property index update fails in composite NodeStore setup
[
https://issues.apache.org/jira/browse/OAK-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091471#comment-16091471
]
Alex Deparvu commented on OAK-6463:
---
very interesting fix! I'm wondering if the 'index' supplier should not be
wrapped into a {{Suppliers.memoize}} call so you'd get the same {{NodeBuilder}}
instance on each {{.get()}} call. In the current form it will return a new
builder on each call and while it will still probably work, I'm a bit worried
about the perf impact.
> Property index update fails in composite NodeStore setup
>
>
> Key: OAK-6463
> URL: https://issues.apache.org/jira/browse/OAK-6463
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: composite, property-index
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6463-v1.patch
>
>
> In a CompositeNodeStore setup involving 1 read only mount a commit involving
> update of property index may fail even if the commit does not update content
> under read only paths
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6409) Oak-run indexing: improved (user friendly) output
[ https://issues.apache.org/jira/browse/OAK-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091414#comment-16091414 ] Paul Chibulcuteanu commented on OAK-6409: - [~chetanm], the logs/output look good IMO and everything requested is implemented. > Oak-run indexing: improved (user friendly) output > - > > Key: OAK-6409 > URL: https://issues.apache.org/jira/browse/OAK-6409 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: indexing >Reporter: Thomas Mueller >Assignee: Chetan Mehrotra > Fix For: 1.8 > > > The oak-run indexing (OAK-6081) output should be human readable, and if > possible minimal. Detailed output should be written to a log file, but stdout > should be easy for a user to understand. For example some header info when > starting, where to find the detailed output, then one line every 3 seconds > about the progress (in %, number of nodes read, ETA), and when done some info > on what to do next. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6463) Property index update fails in composite NodeStore setup
[ https://issues.apache.org/jira/browse/OAK-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-6463: - Attachment: OAK-6463-v1.patch [patch|^OAK-6463-v1.patch] for the same. Issue here was that for any property index update PropertyIndexEditor had to initialize NodeBuilder for all IndexStoreStrategy (which are 1 per mount). Now if the index did not had entries for private mount at setup time (when writes to it were allowed) then later this cases issue as on update the editor would try to eagerly initialize the NodeBuilder for :oak:mount-<>-index node even if no updates need to be done under that. This patch fixes this by using a lazy builder i.e. use Supplier where NodeBuilder is constructed on demand. With this a mount related index node is not created for normal index updates. [~stillalex] Please review! > Property index update fails in composite NodeStore setup > > > Key: OAK-6463 > URL: https://issues.apache.org/jira/browse/OAK-6463 > Project: Jackrabbit Oak > Issue Type: Bug > Components: composite, property-index >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > Attachments: OAK-6463-v1.patch > > > In a CompositeNodeStore setup involving 1 read only mount a commit involving > update of property index may fail even if the commit does not update content > under read only paths -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6466) Suppress NOP info message from revision GC
Marcel Reutegger created OAK-6466:
-
Summary: Suppress NOP info message from revision GC
Key: OAK-6466
URL: https://issues.apache.org/jira/browse/OAK-6466
Project: Jackrabbit Oak
Issue Type: Improvement
Components: core, documentmk
Reporter: Marcel Reutegger
Assignee: Marcel Reutegger
Priority: Minor
Fix For: 1.8
The Revision GC sometimes logs messages like this:
{noformat}
Proceeding to reset [0] _deletedOnce flags
{noformat}
It should only log messages when it actually does some work.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6407) Refactor oak.spi.query into a separate module/bundle
[ https://issues.apache.org/jira/browse/OAK-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091355#comment-16091355 ] angela commented on OAK-6407: - [~tmueller], regarding your questions: > Which ones? see the various issues I fixed recently within the query code base to fix the cyclic package dependencies. > Which ones? same here. > What does it solve / prevent exactly, and how? if the query implementation resides in oak-core and the query-spi gets refactored out, it will no longer be possible to introduce cyclic package dependencies from exported and non-exported classes as we used to have it in oak-core. while this caused a WARNING in the build these WARNINGs where just ignored... so i'd rather have it prevented upon build altogether. moving it out would also allow other modules (e.g. oak-lucene, oak-solr and oak-auth-external) to no longer depend on oak-core. > Why does org.apache.jackrabbit.oak.spi.query need to be moved to a new > project, but not for example org.apache.jackrabbit.oak.spi.security? I want to move out _org.apache.jackrabbit.oak.spi.security_ (OAK-6318) but that's blocked by OAK-6319 and consequently by this issue :-)... in fact refactoring spi.security out of oak-core is the main reason for looking into the query code base. > If we move those classes, why does oak-lucene still depends on oak-core? It > would be nice if oak-lucene just depends on this (and possibly other) spi > projects. yes sure... that's the ultimate goal (see above). i don't recall the very details but it might require some additional work in the wast plugins-query package space, which i didn't touch so far as i wanted to get the untangle the ties between the query implementation and query.spi in a first step and not reshuffle everything. > Refactor oak.spi.query into a separate module/bundle > -- > > Key: OAK-6407 > URL: https://issues.apache.org/jira/browse/OAK-6407 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, indexing, query >Reporter: angela >Assignee: angela > Labels: modularization > Fix For: 1.8 > > Attachments: OAK-6407.patch > > > now that OAK-6304 and OAK-6355 have been resolved, i would like to suggest > that we move the _o.a.j.oak.spi.query_ code base into a separate > module/bundle in order to prevent the introduction of bogus cycles and odd > package exports in the future. > [~tmueller], patch will follow asap. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6465) Path supporting fragments should be an unbounded property
[ https://issues.apache.org/jira/browse/OAK-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091348#comment-16091348 ] Tomek Rękawek commented on OAK-6465: Fixed for trunk in [r1802263|https://svn.apache.org/r1802263]. > Path supporting fragments should be an unbounded property > - > > Key: OAK-6465 > URL: https://issues.apache.org/jira/browse/OAK-6465 > Project: Jackrabbit Oak > Issue Type: Bug > Components: composite >Reporter: Tomek Rękawek > Fix For: 1.8, 1.7.4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6465) Path supporting fragments should be an unbounded property
[ https://issues.apache.org/jira/browse/OAK-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomek Rękawek updated OAK-6465: --- Fix Version/s: 1.7.4 1.8 Component/s: composite > Path supporting fragments should be an unbounded property > - > > Key: OAK-6465 > URL: https://issues.apache.org/jira/browse/OAK-6465 > Project: Jackrabbit Oak > Issue Type: Bug > Components: composite >Reporter: Tomek Rękawek > Fix For: 1.8, 1.7.4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6465) Path supporting fragments should be an unbounded property
Tomek Rękawek created OAK-6465: -- Summary: Path supporting fragments should be an unbounded property Key: OAK-6465 URL: https://issues.apache.org/jira/browse/OAK-6465 Project: Jackrabbit Oak Issue Type: Bug Reporter: Tomek Rękawek -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6462) Incorrect memory calculation for bundled node states
[ https://issues.apache.org/jira/browse/OAK-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-6462: -- Labels: candidate_oak_1_6 (was: ) > Incorrect memory calculation for bundled node states > > > Key: OAK-6462 > URL: https://issues.apache.org/jira/browse/OAK-6462 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.6.0 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: candidate_oak_1_6 > Fix For: 1.8 > > > OAK-1312 introduced bundling of nodes into a document and enabled the feature > for nt:file nodes. A DocumentNodeState of type nt:file now has an incorrect > memory calculation because the node state references a BundlingContext which > also references the binary property reference. Such a node state is now much > heavier than before and can cause OOME. > This is a clone of OAK-5226, which was supposed to fix the described issue, > however it turns out the fix is not sufficient and memory calculation is > still incorrect for bundled nodes/properties. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6462) Incorrect memory calculation for bundled node states
[ https://issues.apache.org/jira/browse/OAK-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091344#comment-16091344 ] Marcel Reutegger commented on OAK-6462: --- Added an ignored test to trunk: http://svn.apache.org/r1802262 > Incorrect memory calculation for bundled node states > > > Key: OAK-6462 > URL: https://issues.apache.org/jira/browse/OAK-6462 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.6.0 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.8 > > > OAK-1312 introduced bundling of nodes into a document and enabled the feature > for nt:file nodes. A DocumentNodeState of type nt:file now has an incorrect > memory calculation because the node state references a BundlingContext which > also references the binary property reference. Such a node state is now much > heavier than before and can cause OOME. > This is a clone of OAK-5226, which was supposed to fix the described issue, > however it turns out the fix is not sufficient and memory calculation is > still incorrect for bundled nodes/properties. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6463) Property index update fails in composite NodeStore setup
[ https://issues.apache.org/jira/browse/OAK-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091343#comment-16091343 ] Chetan Mehrotra commented on OAK-6463: -- Added ignored tests with 1802261,1802256 > Property index update fails in composite NodeStore setup > > > Key: OAK-6463 > URL: https://issues.apache.org/jira/browse/OAK-6463 > Project: Jackrabbit Oak > Issue Type: Bug > Components: composite, property-index >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > > In a CompositeNodeStore setup involving 1 read only mount a commit involving > update of property index may fail even if the commit does not update content > under read only paths -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6464) Public constructor for RandomStream
[ https://issues.apache.org/jira/browse/OAK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-6464. --- Resolution: Fixed Fix Version/s: 1.7.4 Done in trunk: http://svn.apache.org/r1802260 > Public constructor for RandomStream > --- > > Key: OAK-6464 > URL: https://issues.apache.org/jira/browse/OAK-6464 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Trivial > Fix For: 1.8, 1.7.4 > > > The public test utility class RandomStream has a package private constructor. > The constructor should be public to make it usable from tests in other > packages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6464) Public constructor for RandomStream
[ https://issues.apache.org/jira/browse/OAK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-6464: -- Component/s: documentmk > Public constructor for RandomStream > --- > > Key: OAK-6464 > URL: https://issues.apache.org/jira/browse/OAK-6464 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Trivial > Fix For: 1.8, 1.7.4 > > > The public test utility class RandomStream has a package private constructor. > The constructor should be public to make it usable from tests in other > packages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6464) Public constructor for RandomStream
Marcel Reutegger created OAK-6464: - Summary: Public constructor for RandomStream Key: OAK-6464 URL: https://issues.apache.org/jira/browse/OAK-6464 Project: Jackrabbit Oak Issue Type: Improvement Components: core Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Trivial Fix For: 1.8 The public test utility class RandomStream has a package private constructor. The constructor should be public to make it usable from tests in other packages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6461) Merge all security related validators into a single hook
[
https://issues.apache.org/jira/browse/OAK-6461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091321#comment-16091321
]
angela commented on OAK-6461:
-
[~stillalex], in general that looks good to me. the only thing that i would
like to mention: the composite-configurations have a ranking, which (in
particular for permission evaluation) is relevant for performance so the
order within the composite-configurations should be maintained, would that be
the case with your patch?
> Merge all security related validators into a single hook
>
>
> Key: OAK-6461
> URL: https://issues.apache.org/jira/browse/OAK-6461
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
> Attachments: OAK-6461.patch
>
>
> I'd like to see if it's feasible to merge all security related validators
> into a single hook, instead of a hook per _SecurityConfiguration_.
> Pros
> * all validators will be merged into a single hook, meaning processing will
> happen via a single diff over the content
> Cons
> * order of hooks will change, there will be commit hooks first, all
> aggregated validators next and post validation hooks last. I don't think
> there's any issue with validation itself as all data added by the hooks will
> be visible to the composite validator.
> This is how the chaining looks like in the current setup:
> {noformat}
> EditorHook :
> (TokenValidatorProvider),
> VersionablePathHook,
> EditorHook :
> (CompositeEditorProvider : ([
> PermissionStoreValidatorProvider,
> PermissionValidatorProvider,
> AccessControlValidatorProvider])),
> EditorHook :
> (PrivilegeValidatorProvider),
> EditorHook :
> (CompositeEditorProvider : ([
> UserValidatorProvider,
> CacheValidatorProvider])),
> PermissionHook,
> JcrAllCommitHook
> {noformat}
> If we merged them, this is the result:
> {noformat}
> VersionablePathHook,
> EditorHook :
> (CompositeEditorProvider : ([
> PermissionStoreValidatorProvider,
> PermissionValidatorProvider,
> AccessControlValidatorProvider,
> UserValidatorProvider,
> CacheValidatorProvider,
> PrivilegeValidatorProvider,
> TokenValidatorProvider])),
> PermissionHook,
> JcrAllCommitHook
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6463) Property index update fails in composite NodeStore setup
Chetan Mehrotra created OAK-6463: Summary: Property index update fails in composite NodeStore setup Key: OAK-6463 URL: https://issues.apache.org/jira/browse/OAK-6463 Project: Jackrabbit Oak Issue Type: Bug Components: composite, property-index Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Fix For: 1.8 In a CompositeNodeStore setup involving 1 read only mount a commit involving update of property index may fail even if the commit does not update content under read only paths -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (OAK-6462) Incorrect memory calculation for bundled node states
[ https://issues.apache.org/jira/browse/OAK-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger reassigned OAK-6462: - Assignee: Marcel Reutegger (was: Chetan Mehrotra) Affects Version/s: (was: 1.5.13) 1.6.0 Fix Version/s: (was: 1.5.15) (was: 1.6.0) 1.8 Description: OAK-1312 introduced bundling of nodes into a document and enabled the feature for nt:file nodes. A DocumentNodeState of type nt:file now has an incorrect memory calculation because the node state references a BundlingContext which also references the binary property reference. Such a node state is now much heavier than before and can cause OOME. This is a clone of OAK-5226, which was supposed to fix the described issue, however it turns out the fix is not sufficient and memory calculation is still incorrect for bundled nodes/properties. was:OAK-1312 introduced bundling of nodes into a document and enabled the feature for nt:file nodes. A DocumentNodeState of type nt:file now has an incorrect memory calculation because the node state references a BundlingContext which also references the binary property reference. Such a node state is now much heavier than before and can cause OOME. > Incorrect memory calculation for bundled node states > > > Key: OAK-6462 > URL: https://issues.apache.org/jira/browse/OAK-6462 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.6.0 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.8 > > > OAK-1312 introduced bundling of nodes into a document and enabled the feature > for nt:file nodes. A DocumentNodeState of type nt:file now has an incorrect > memory calculation because the node state references a BundlingContext which > also references the binary property reference. Such a node state is now much > heavier than before and can cause OOME. > This is a clone of OAK-5226, which was supposed to fix the described issue, > however it turns out the fix is not sufficient and memory calculation is > still incorrect for bundled nodes/properties. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6462) Incorrect memory calculation for bundled node states
Marcel Reutegger created OAK-6462: - Summary: Incorrect memory calculation for bundled node states Key: OAK-6462 URL: https://issues.apache.org/jira/browse/OAK-6462 Project: Jackrabbit Oak Issue Type: Bug Components: core, documentmk Affects Versions: 1.5.13 Reporter: Marcel Reutegger Assignee: Chetan Mehrotra Fix For: 1.5.15, 1.6.0 OAK-1312 introduced bundling of nodes into a document and enabled the feature for nt:file nodes. A DocumentNodeState of type nt:file now has an incorrect memory calculation because the node state references a BundlingContext which also references the binary property reference. Such a node state is now much heavier than before and can cause OOME. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6461) Merge all security related validators into a single hook
[
https://issues.apache.org/jira/browse/OAK-6461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu updated OAK-6461:
--
Attachment: OAK-6461.patch
proposed patch for review (IT tests are still running).
[~anchela] would love to hear your thoughts on this one!
> Merge all security related validators into a single hook
>
>
> Key: OAK-6461
> URL: https://issues.apache.org/jira/browse/OAK-6461
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
> Attachments: OAK-6461.patch
>
>
> I'd like to see if it's feasible to merge all security related validators
> into a single hook, instead of a hook per _SecurityConfiguration_.
> Pros
> * all validators will be merged into a single hook, meaning processing will
> happen via a single diff over the content
> Cons
> * order of hooks will change, there will be commit hooks first, all
> aggregated validators next and post validation hooks last. I don't think
> there's any issue with validation itself as all data added by the hooks will
> be visible to the composite validator.
> This is how the chaining looks like in the current setup:
> {noformat}
> EditorHook :
> (TokenValidatorProvider),
> VersionablePathHook,
> EditorHook :
> (CompositeEditorProvider : ([
> PermissionStoreValidatorProvider,
> PermissionValidatorProvider,
> AccessControlValidatorProvider])),
> EditorHook :
> (PrivilegeValidatorProvider),
> EditorHook :
> (CompositeEditorProvider : ([
> UserValidatorProvider,
> CacheValidatorProvider])),
> PermissionHook,
> JcrAllCommitHook
> {noformat}
> If we merged them, this is the result:
> {noformat}
> VersionablePathHook,
> EditorHook :
> (CompositeEditorProvider : ([
> PermissionStoreValidatorProvider,
> PermissionValidatorProvider,
> AccessControlValidatorProvider,
> UserValidatorProvider,
> CacheValidatorProvider,
> PrivilegeValidatorProvider,
> TokenValidatorProvider])),
> PermissionHook,
> JcrAllCommitHook
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6461) Merge all security related validators into a single hook
Alex Deparvu created OAK-6461:
-
Summary: Merge all security related validators into a single hook
Key: OAK-6461
URL: https://issues.apache.org/jira/browse/OAK-6461
Project: Jackrabbit Oak
Issue Type: Improvement
Components: core, security
Reporter: Alex Deparvu
Assignee: Alex Deparvu
I'd like to see if it's feasible to merge all security related validators into
a single hook, instead of a hook per _SecurityConfiguration_.
Pros
* all validators will be merged into a single hook, meaning processing will
happen via a single diff over the content
Cons
* order of hooks will change, there will be commit hooks first, all aggregated
validators next and post validation hooks last. I don't think there's any issue
with validation itself as all data added by the hooks will be visible to the
composite validator.
This is how the chaining looks like in the current setup:
{noformat}
EditorHook :
(TokenValidatorProvider),
VersionablePathHook,
EditorHook :
(CompositeEditorProvider : ([
PermissionStoreValidatorProvider,
PermissionValidatorProvider,
AccessControlValidatorProvider])),
EditorHook :
(PrivilegeValidatorProvider),
EditorHook :
(CompositeEditorProvider : ([
UserValidatorProvider,
CacheValidatorProvider])),
PermissionHook,
JcrAllCommitHook
{noformat}
If we merged them, this is the result:
{noformat}
VersionablePathHook,
EditorHook :
(CompositeEditorProvider : ([
PermissionStoreValidatorProvider,
PermissionValidatorProvider,
AccessControlValidatorProvider,
UserValidatorProvider,
CacheValidatorProvider,
PrivilegeValidatorProvider,
TokenValidatorProvider])),
PermissionHook,
JcrAllCommitHook
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6455) Don't call observer concurrently from the CompositeNodeStore
[
https://issues.apache.org/jira/browse/OAK-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091266#comment-16091266
]
Tomek Rękawek commented on OAK-6455:
So, it seems we'll need the lock after all. Re-introduced merge lock in
[r1802250|https://svn.apache.org/r1802250].
> Don't call observer concurrently from the CompositeNodeStore
>
>
> Key: OAK-6455
> URL: https://issues.apache.org/jira/browse/OAK-6455
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: composite
>Reporter: Tomek Rękawek
> Fix For: 1.8, 1.7.4
>
>
> Per Observer interface
> {noformat}
> However, each observer is
> * always guaranteed to see a linear sequence of changes. In other words the
> * method will not be called concurrently from multiple threads and successive
> * calls represent a linear sequence of repository states, i.e. the root
> * state passed to a call is guaranteed to represent a repository state
> {noformat}
> Currently concurrent merge would result in concurrent calls to Observer. So
> that would need to be serialized
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6455) Don't call observer concurrently from the CompositeNodeStore
[
https://issues.apache.org/jira/browse/OAK-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek resolved OAK-6455.
Resolution: Fixed
Fix Version/s: 1.7.4
> Don't call observer concurrently from the CompositeNodeStore
>
>
> Key: OAK-6455
> URL: https://issues.apache.org/jira/browse/OAK-6455
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: composite
>Reporter: Tomek Rękawek
> Fix For: 1.8, 1.7.4
>
>
> Per Observer interface
> {noformat}
> However, each observer is
> * always guaranteed to see a linear sequence of changes. In other words the
> * method will not be called concurrently from multiple threads and successive
> * calls represent a linear sequence of repository states, i.e. the root
> * state passed to a call is guaranteed to represent a repository state
> {noformat}
> Currently concurrent merge would result in concurrent calls to Observer. So
> that would need to be serialized
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (OAK-3349) Partial compaction
[ https://issues.apache.org/jira/browse/OAK-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francesco Mari reassigned OAK-3349: --- Assignee: Francesco Mari (was: Michael Dürig) > Partial compaction > -- > > Key: OAK-3349 > URL: https://issues.apache.org/jira/browse/OAK-3349 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: segment-tar >Reporter: Michael Dürig >Assignee: Francesco Mari > Labels: compaction, gc, scalability > Fix For: 1.8, 1.7.4 > > Attachments: compaction-time.png, cycle-count.png, post-gc-size.png > > > On big repositories compaction can take quite a while to run as it needs to > create a full deep copy of the current root node state. For such cases it > could be beneficial if we could partially compact the repository thus > splitting full compaction over multiple cycles. > Partial compaction would run compaction on a sub-tree just like we now run it > on the full tree. Afterwards it would create a new root node state by > referencing the previous root node state replacing said sub-tree with the > compacted one. > Todo: Asses feasibility and impact, implement prototype. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6459) VisibleValidator duplicated in chain for each hierarchy level
[
https://issues.apache.org/jira/browse/OAK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu resolved OAK-6459.
---
Resolution: Fixed
fixed with http://svn.apache.org/viewvc?rev=1802248=rev
> VisibleValidator duplicated in chain for each hierarchy level
> -
>
> Key: OAK-6459
> URL: https://issues.apache.org/jira/browse/OAK-6459
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Alexander Klimetschek
>Assignee: Alex Deparvu
>Priority: Minor
> Fix For: 1.7.4
>
>
> VisibleValidator gets chained multiple times, while each one working on the
> same NodeState will run the exact same check, NodeStateUtils.isHidden(name).
> One execution should be enough.
> Below stacktrace has 9 instances chained. Code for propertyAdded as in the
> stacktrace below in 1.4 is
> [here|https://github.com/apache/jackrabbit-oak/blob/1.4/oak-core/src/main/java/org/apache/jackrabbit/oak/spi/commit/VisibleValidator.java#L82-L83].
>
> Also found a similar stacktrace in OAK-6358 with 12 instances.
> This might be a small optimization to make, especially if there are many
> properties added, and if there is apparently another variable factor in how
> many instances there are.
> According to [~stillalex]:
> This is directly related to the hierarchy level (so if a change is 5 levels
> deep, you'll see 5 validators chained there) and the short fix is to fix
> calls [creating new
> VisibleValidators|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-store-spi/src/main/java/org/apache/jackrabbit/oak/spi/commit/VisibleValidator.java#L44]
> to unwrap if needed, like
> [here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-store-spi/src/main/java/org/apache/jackrabbit/oak/spi/commit/VisibleEditor.java#L38].
> {noformat}
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException:
> OakAccess: Access denied
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionValidator.checkPermissions(PermissionValidator.java:242)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionValidator.propertyAdded(PermissionValidator.java:112)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeEditor.propertyAdded(CompositeEditor.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.propertyAdded(EditorDiff.java:82)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareProperties(SegmentNodeState.java:593)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:491)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:531)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:584)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:584)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compareBranch(MapRecord.java:561)
> at
>
[jira] [Updated] (OAK-6459) VisibleValidator duplicated in chain for each hierarchy level
[
https://issues.apache.org/jira/browse/OAK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu updated OAK-6459:
--
Fix Version/s: 1.7.4
> VisibleValidator duplicated in chain for each hierarchy level
> -
>
> Key: OAK-6459
> URL: https://issues.apache.org/jira/browse/OAK-6459
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Alexander Klimetschek
>Assignee: Alex Deparvu
>Priority: Minor
> Fix For: 1.7.4
>
>
> VisibleValidator gets chained multiple times, while each one working on the
> same NodeState will run the exact same check, NodeStateUtils.isHidden(name).
> One execution should be enough.
> Below stacktrace has 9 instances chained. Code for propertyAdded as in the
> stacktrace below in 1.4 is
> [here|https://github.com/apache/jackrabbit-oak/blob/1.4/oak-core/src/main/java/org/apache/jackrabbit/oak/spi/commit/VisibleValidator.java#L82-L83].
>
> Also found a similar stacktrace in OAK-6358 with 12 instances.
> This might be a small optimization to make, especially if there are many
> properties added, and if there is apparently another variable factor in how
> many instances there are.
> According to [~stillalex]:
> This is directly related to the hierarchy level (so if a change is 5 levels
> deep, you'll see 5 validators chained there) and the short fix is to fix
> calls [creating new
> VisibleValidators|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-store-spi/src/main/java/org/apache/jackrabbit/oak/spi/commit/VisibleValidator.java#L44]
> to unwrap if needed, like
> [here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-store-spi/src/main/java/org/apache/jackrabbit/oak/spi/commit/VisibleEditor.java#L38].
> {noformat}
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException:
> OakAccess: Access denied
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionValidator.checkPermissions(PermissionValidator.java:242)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionValidator.propertyAdded(PermissionValidator.java:112)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeEditor.propertyAdded(CompositeEditor.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.propertyAdded(EditorDiff.java:82)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareProperties(SegmentNodeState.java:593)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:491)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:531)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:584)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:584)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compareBranch(MapRecord.java:561)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:466)
[jira] [Assigned] (OAK-6459) VisibleValidator duplicated in chain for each hierarchy level
[
https://issues.apache.org/jira/browse/OAK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu reassigned OAK-6459:
-
Assignee: Alex Deparvu
Priority: Minor (was: Major)
Component/s: core
> VisibleValidator duplicated in chain for each hierarchy level
> -
>
> Key: OAK-6459
> URL: https://issues.apache.org/jira/browse/OAK-6459
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Alexander Klimetschek
>Assignee: Alex Deparvu
>Priority: Minor
>
> VisibleValidator gets chained multiple times, while each one working on the
> same NodeState will run the exact same check, NodeStateUtils.isHidden(name).
> One execution should be enough.
> Below stacktrace has 9 instances chained. Code for propertyAdded as in the
> stacktrace below in 1.4 is
> [here|https://github.com/apache/jackrabbit-oak/blob/1.4/oak-core/src/main/java/org/apache/jackrabbit/oak/spi/commit/VisibleValidator.java#L82-L83].
>
> Also found a similar stacktrace in OAK-6358 with 12 instances.
> This might be a small optimization to make, especially if there are many
> properties added, and if there is apparently another variable factor in how
> many instances there are.
> According to [~stillalex]:
> This is directly related to the hierarchy level (so if a change is 5 levels
> deep, you'll see 5 validators chained there) and the short fix is to fix
> calls [creating new
> VisibleValidators|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-store-spi/src/main/java/org/apache/jackrabbit/oak/spi/commit/VisibleValidator.java#L44]
> to unwrap if needed, like
> [here|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-store-spi/src/main/java/org/apache/jackrabbit/oak/spi/commit/VisibleEditor.java#L38].
> {noformat}
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException:
> OakAccess: Access denied
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionValidator.checkPermissions(PermissionValidator.java:242)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionValidator.propertyAdded(PermissionValidator.java:112)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleValidator.propertyAdded(VisibleValidator.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeEditor.propertyAdded(CompositeEditor.java:83)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.propertyAdded(EditorDiff.java:82)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareProperties(SegmentNodeState.java:593)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:491)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:531)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:584)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:584)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compareBranch(MapRecord.java:561)
> at
>
[jira] [Comment Edited] (OAK-6409) Oak-run indexing: improved (user friendly) output
[
https://issues.apache.org/jira/browse/OAK-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091171#comment-16091171
]
Chetan Mehrotra edited comment on OAK-6409 at 7/18/17 6:22 AM:
---
Done following changes with 1802241,1802242
# oak-run index would now copy a logback-index.xml to work dir (default ./temp)
and use that for configuring
# Logback is configured to scan the config periodically and if any change is
done at runtime the changes would be picked up
# By default all info logs would be routed to temp/indexing.log
# In addition logs from following categories would be logged at info level to
stdout with just time and message
{noformat}
org.apache.jackrabbit.oak.index
org.apache.jackrabbit.oak.plugins.index.importer
org.apache.jackrabbit.oak.plugins.index.IndexUpdate
org.apache.jackrabbit.oak.plugins.index.progress
{noformat}
Some run
{noformat}
Apache Jackrabbit Oak
11:28:41 - Logging configured from
/mnt/hdd/data/oak/oak-run-indexing/temp/logback-indexing.xml
11:28:41 - Any change in logging config would be picked up
11:28:41 - Logs would be written to temp/indexing.log
/content/oak:index/enablementResourceName => valid
/oak:index/authorizables => valid
/oak:index/commerceLucene => valid
/oak:index/cqMobileAppLucene => valid
/oak:index/cqPageLucene => valid
/oak:index/cqProjectLucene => valid
/oak:index/cqTagLucene => valid
/oak:index/damAssetLucene => valid
/oak:index/lucene => valid
/oak:index/ntBaseLucene => valid
/oak:index/slingeventJob => valid
/oak:index/socialLucene => valid
/oak:index/versionStoreIndex => valid
/oak:index/workflowDataLucene => valid
Index consistency check report stored at
/mnt/hdd/data/oak/oak-run-indexing/indexing-result/index-consistency-check-report.txt
{noformat}
Sample run for indexing
{noformat}
Apache Jackrabbit Oak 1.8-SNAPSHOT
11:39:50 - Logging configured from
/mnt/hdd/data/oak/oak-run-indexing/import/temp/logback-indexing.xml
11:39:50 - Any change in logging config would be picked up
11:39:50 - Logs would be written to temp/indexing.log
11:39:51 - Created checkpoint [r15d54512839-0-2] for indexing
11:39:51 - Proceeding to reindex with read only access to NodeStore
11:39:51 - Proceeding to index [/oak:index/lucene] upto checkpoint
r15d54512839-0-2 {creator=IndexCommand, created=2017-07-18T11:39:51.160+05:30}
11:39:51 - Switched the async lane for indexes at [/oak:index/lucene] to
offline-reindex-async and marked them for reindex
11:39:51 - Reindexing will be performed for following indexes:
[/oak:index/lucene]
11:39:51 - Paths to be traversed includedPath : [/], excludedPaths :
[/etc/workflow/instances, /jcr:system, /etc/replication, /var]
11:39:51 - Estimated node count to be traversed for reindexing under / is
[131072]
11:40:02 - Reindexing Traversed #1
/content/mobileapps/we-retail/shell/jcr:content/pge-app/app-content/phonegap/res/screens/android/screen-hdpi-portrait.png/jcr:content
[999.90 nodes/s, 3599640.00 nodes/hr] (Elapsed 10.76 s, Expected 2.017 min,
Completed 7.63%)
11:40:02 - /oak:index/lucene => Indexed 1 nodes in 10.75 s ...
11:40:09 - Reindexing Traversed #2
/content/we-retail/us/en/experience/skiing-deep-powder-in-siberia/jcr:content/root/responsivegrid/content_fragment
[.06 nodes/s, 3999800.00 nodes/hr] (Elapsed 18.06 s, Expected 99.00 s,
Completed 15.26%)
11:40:09 - /oak:index/lucene => Indexed 2 nodes in 7.303 s ...
11:40:18 - Reindexing Traversed #3
/etc/clientlibs/social/thirdparty/ckeditor/plugins/docprops/dialogs/docprops.js
[1153.81 nodes/s, 4153707.69 nodes/hr] (Elapsed 26.80 s, Expected 87.00 s,
Completed 22.89%)
11:40:18 - /oak:index/lucene => Indexed 3 nodes in 8.742 s ...
11:40:26 - Reindexing Traversed #4
/etc/packages/day/cq560/social/calendar/cq-social-calendar-pkg-1.4.33.zip/jcr:content/vlt:definition/filter/f2
[1176.44 nodes/s, 4235188.24 nodes/hr] (Elapsed 34.57 s, Expected 77.00 s,
Completed 30.52%)
11:40:26 - /oak:index/lucene => Indexed 4 nodes in 7.771 s ...
11:40:32 - Reindexing Traversed #5
/libs/cq/contentsync/widgets/source/widgets/ConsolePanel.js/jcr:content
[1249.98 nodes/s, 4499910.00 nodes/hr] (Elapsed 40.73 s, Expected 64.00 s,
Completed 38.15%)
11:40:32 - /oak:index/lucene => Indexed 5 nodes in 6.158 s ...
11:40:41 - Reindexing Traversed #6
/libs/cq/personalization/clientlib/kernel/source/teaser/teaser-client.js/jcr:content
[1199.98 nodes/s, 4319928.00 nodes/hr] (Elapsed 50.30 s, Expected 59.00 s,
Completed 45.78%)
11:40:41 - /oak:index/lucene => Indexed 6 nodes in 9.569 s ...
11:40:48 - Reindexing Traversed #7
/libs/cq/workflow/components/model/step/tab_common/items/timeout/items/timeoutHandler
[1249.98 nodes/s, 4499935.71 nodes/hr] (Elapsed 56.54 s, Expected 48.00 s,
Completed 53.41%)
11:40:48 - /oak:index/lucene => Indexed 7 nodes in 6.240 s ...
11:40:55 - Reindexing Traversed #8
[jira] [Commented] (OAK-6409) Oak-run indexing: improved (user friendly) output
[
https://issues.apache.org/jira/browse/OAK-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091171#comment-16091171
]
Chetan Mehrotra commented on OAK-6409:
--
Done following changes with 1802241,1802242
# oak-run index would now copy a logback-index.xml to work dir (default ./temp)
and use that for configuring
# Logback is configured to scan the config periodically and if any change is
done at runtime the changes would be picked up
# By default all info logs would be routed to temp/indexing.log
# In addition logs from following categories would be logged at info level to
stdout with just time and message
{noformat}
org.apache.jackrabbit.oak.index
org.apache.jackrabbit.oak.plugins.index.importer
org.apache.jackrabbit.oak.plugins.index.IndexUpdate
org.apache.jackrabbit.oak.plugins.index.progress
{noformat}
Same run
{noformat}
Apache Jackrabbit Oak
11:28:41 - Logging configured from
/mnt/hdd/data/oak/oak-run-indexing/temp/logback-indexing.xml
11:28:41 - Any change in logging config would be picked up
11:28:41 - Logs would be written to temp/indexing.log
/content/oak:index/enablementResourceName => valid
/oak:index/authorizables => valid
/oak:index/commerceLucene => valid
/oak:index/cqMobileAppLucene => valid
/oak:index/cqPageLucene => valid
/oak:index/cqProjectLucene => valid
/oak:index/cqTagLucene => valid
/oak:index/damAssetLucene => valid
/oak:index/lucene => valid
/oak:index/ntBaseLucene => valid
/oak:index/slingeventJob => valid
/oak:index/socialLucene => valid
/oak:index/versionStoreIndex => valid
/oak:index/workflowDataLucene => valid
Index consistency check report stored at
/mnt/hdd/data/oak/oak-run-indexing/indexing-result/index-consistency-check-report.txt
{noformat}
Sample run for indexing
{noformat}
Apache Jackrabbit Oak 1.8-SNAPSHOT
11:39:50 - Logging configured from
/mnt/hdd/data/oak/oak-run-indexing/import/temp/logback-indexing.xml
11:39:50 - Any change in logging config would be picked up
11:39:50 - Logs would be written to temp/indexing.log
11:39:51 - Created checkpoint [r15d54512839-0-2] for indexing
11:39:51 - Proceeding to reindex with read only access to NodeStore
11:39:51 - Proceeding to index [/oak:index/lucene] upto checkpoint
r15d54512839-0-2 {creator=IndexCommand, created=2017-07-18T11:39:51.160+05:30}
11:39:51 - Switched the async lane for indexes at [/oak:index/lucene] to
offline-reindex-async and marked them for reindex
11:39:51 - Reindexing will be performed for following indexes:
[/oak:index/lucene]
11:39:51 - Paths to be traversed includedPath : [/], excludedPaths :
[/etc/workflow/instances, /jcr:system, /etc/replication, /var]
11:39:51 - Estimated node count to be traversed for reindexing under / is
[131072]
11:40:02 - Reindexing Traversed #1
/content/mobileapps/we-retail/shell/jcr:content/pge-app/app-content/phonegap/res/screens/android/screen-hdpi-portrait.png/jcr:content
[999.90 nodes/s, 3599640.00 nodes/hr] (Elapsed 10.76 s, Expected 2.017 min,
Completed 7.63%)
11:40:02 - /oak:index/lucene => Indexed 1 nodes in 10.75 s ...
11:40:09 - Reindexing Traversed #2
/content/we-retail/us/en/experience/skiing-deep-powder-in-siberia/jcr:content/root/responsivegrid/content_fragment
[.06 nodes/s, 3999800.00 nodes/hr] (Elapsed 18.06 s, Expected 99.00 s,
Completed 15.26%)
11:40:09 - /oak:index/lucene => Indexed 2 nodes in 7.303 s ...
11:40:18 - Reindexing Traversed #3
/etc/clientlibs/social/thirdparty/ckeditor/plugins/docprops/dialogs/docprops.js
[1153.81 nodes/s, 4153707.69 nodes/hr] (Elapsed 26.80 s, Expected 87.00 s,
Completed 22.89%)
11:40:18 - /oak:index/lucene => Indexed 3 nodes in 8.742 s ...
11:40:26 - Reindexing Traversed #4
/etc/packages/day/cq560/social/calendar/cq-social-calendar-pkg-1.4.33.zip/jcr:content/vlt:definition/filter/f2
[1176.44 nodes/s, 4235188.24 nodes/hr] (Elapsed 34.57 s, Expected 77.00 s,
Completed 30.52%)
11:40:26 - /oak:index/lucene => Indexed 4 nodes in 7.771 s ...
11:40:32 - Reindexing Traversed #5
/libs/cq/contentsync/widgets/source/widgets/ConsolePanel.js/jcr:content
[1249.98 nodes/s, 4499910.00 nodes/hr] (Elapsed 40.73 s, Expected 64.00 s,
Completed 38.15%)
11:40:32 - /oak:index/lucene => Indexed 5 nodes in 6.158 s ...
11:40:41 - Reindexing Traversed #6
/libs/cq/personalization/clientlib/kernel/source/teaser/teaser-client.js/jcr:content
[1199.98 nodes/s, 4319928.00 nodes/hr] (Elapsed 50.30 s, Expected 59.00 s,
Completed 45.78%)
11:40:41 - /oak:index/lucene => Indexed 6 nodes in 9.569 s ...
11:40:48 - Reindexing Traversed #7
/libs/cq/workflow/components/model/step/tab_common/items/timeout/items/timeoutHandler
[1249.98 nodes/s, 4499935.71 nodes/hr] (Elapsed 56.54 s, Expected 48.00 s,
Completed 53.41%)
11:40:48 - /oak:index/lucene => Indexed 7 nodes in 6.240 s ...
11:40:55 - Reindexing Traversed #8
/libs/dam/gui/content/assets/moveassetwizard/jcr:content/head [1249.98 nodes/s,
[jira] [Resolved] (OAK-6341) oak-run redirects reindexing info to STDERR
[ https://issues.apache.org/jira/browse/OAK-6341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6341. -- Resolution: Fixed Fix Version/s: 1.7.4 Done as part of OAK-6409. Now logs are routed to stdout > oak-run redirects reindexing info to STDERR > --- > > Key: OAK-6341 > URL: https://issues.apache.org/jira/browse/OAK-6341 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene, run >Affects Versions: 1.8, 1.7.1 >Reporter: Paul Chibulcuteanu >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8, 1.7.4 > > > Reindexing run via oak-run tool redirects all the important reindexing > information to STDERR. > This is due to > https://github.com/apache/jackrabbit-oak/blob/trunk/oak-run/src/main/resources/logback.xml#L55-L57 > In this particular case, the information should be redirected to STDOUT since > it is important information related to reindexing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (OAK-6409) Oak-run indexing: improved (user friendly) output
[ https://issues.apache.org/jira/browse/OAK-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra reassigned OAK-6409: Assignee: Chetan Mehrotra > Oak-run indexing: improved (user friendly) output > - > > Key: OAK-6409 > URL: https://issues.apache.org/jira/browse/OAK-6409 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: indexing >Reporter: Thomas Mueller >Assignee: Chetan Mehrotra > Fix For: 1.8 > > > The oak-run indexing (OAK-6081) output should be human readable, and if > possible minimal. Detailed output should be written to a log file, but stdout > should be easy for a user to understand. For example some header info when > starting, where to find the detailed output, then one line every 3 seconds > about the progress (in %, number of nodes read, ETA), and when done some info > on what to do next. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
