[jira] [Updated] (OAK-2776) Upgrade should allow to skip copying versions
[ https://issues.apache.org/jira/browse/OAK-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Sedding updated OAK-2776: Attachment: OAK-2776.patch Proposed patch. Note that the attached patch depends on other upgrade improvements in OAK-2586, OAK-2619 and OAK-2626. See also \[0\] for the commits in sequence. \[0\] https://github.com/apache/jackrabbit-oak/compare/trunk...code-distillery:trunk > Upgrade should allow to skip copying versions > - > > Key: OAK-2776 > URL: https://issues.apache.org/jira/browse/OAK-2776 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: upgrade >Affects Versions: 1.2 >Reporter: Julian Sedding > Attachments: OAK-2776.patch > > > In some cases it is not necessary to copy version histories during an > upgrade. Skipping to copy versions can result in a lot less content that > needs copying and thus a significant speedup. > Additionally, OAK-2586 introduces the possibility to include and exclude > paths for an upgrade. Version histories should thus only be copied if their > respective versionable node is present in the copied part of the content. > Also reducing content being copied redundantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2777) Minimize the cost calculation for queries using reference restrictions.
[ https://issues.apache.org/jira/browse/OAK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Przemo Pakulski updated OAK-2777: - Labels: performance (was: ) > Minimize the cost calculation for queries using reference restrictions. > --- > > Key: OAK-2777 > URL: https://issues.apache.org/jira/browse/OAK-2777 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Affects Versions: 1.1.2, 1.2 >Reporter: Przemo Pakulski > Labels: performance > Attachments: oak-2777.patch > > > According to the javadocs (QueryIndex) minimum cost for index is 1. Currently > ReferenceIndex returns this minimum value, when it can be used for the query. > But even then cost for remaining indexes is still calculated. We could skip > cost calculation of remaining indexes if we achieved the minimum cost already. > It will speed up all queries which can leverage the reference Index. > Example query: > SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = > '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2777) Minimize the cost calculation for queries using reference restrictions.
[ https://issues.apache.org/jira/browse/OAK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Przemo Pakulski updated OAK-2777: - Component/s: query > Minimize the cost calculation for queries using reference restrictions. > --- > > Key: OAK-2777 > URL: https://issues.apache.org/jira/browse/OAK-2777 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Affects Versions: 1.1.2, 1.2 >Reporter: Przemo Pakulski > Attachments: oak-2777.patch > > > According to the javadocs (QueryIndex) minimum cost for index is 1. Currently > ReferenceIndex returns this minimum value, when it can be used for the query. > But even then cost for remaining indexes is still calculated. We could skip > cost calculation of remaining indexes if we achieved the minimum cost already. > It will speed up all queries which can leverage the reference Index. > Example query: > SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = > '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2777) Minimize the cost calculation for queries using reference restrictions.
[ https://issues.apache.org/jira/browse/OAK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496459#comment-14496459 ] Przemo Pakulski commented on OAK-2777: -- We could do similar thing for unique property index, which I believe is (should be) as fast as reference index. > Minimize the cost calculation for queries using reference restrictions. > --- > > Key: OAK-2777 > URL: https://issues.apache.org/jira/browse/OAK-2777 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Affects Versions: 1.1.2, 1.2 >Reporter: Przemo Pakulski > Attachments: oak-2777.patch > > > According to the javadocs (QueryIndex) minimum cost for index is 1. Currently > ReferenceIndex returns this minimum value, when it can be used for the query. > But even then cost for remaining indexes is still calculated. We could skip > cost calculation of remaining indexes if we achieved the minimum cost already. > It will speed up all queries which can leverage the reference Index. > Example query: > SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = > '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2777) Minimize the cost calculation for queries using reference restrictions.
[ https://issues.apache.org/jira/browse/OAK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Przemo Pakulski updated OAK-2777: - Attachment: (was: oak-2777.patch) > Minimize the cost calculation for queries using reference restrictions. > --- > > Key: OAK-2777 > URL: https://issues.apache.org/jira/browse/OAK-2777 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Affects Versions: 1.1.2, 1.2 >Reporter: Przemo Pakulski > Attachments: oak-2777.patch > > > According to the javadocs (QueryIndex) minimum cost for index is 1. Currently > ReferenceIndex returns this minimum value, when it can be used for the query. > But even then cost for remaining indexes is still calculated. We could skip > cost calculation of remaining indexes if we achieved the minimum cost already. > It will speed up all queries which can leverage the reference Index. > Example query: > SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = > '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2777) Minimize the cost calculation for queries using reference restrictions.
[ https://issues.apache.org/jira/browse/OAK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Przemo Pakulski updated OAK-2777: - Attachment: oak-2777.patch > Minimize the cost calculation for queries using reference restrictions. > --- > > Key: OAK-2777 > URL: https://issues.apache.org/jira/browse/OAK-2777 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Affects Versions: 1.1.2, 1.2 >Reporter: Przemo Pakulski > Attachments: oak-2777.patch, oak-2777.patch > > > According to the javadocs (QueryIndex) minimum cost for index is 1. Currently > ReferenceIndex returns this minimum value, when it can be used for the query. > But even then cost for remaining indexes is still calculated. We could skip > cost calculation of remaining indexes if we achieved the minimum cost already. > It will speed up all queries which can leverage the reference Index. > Example query: > SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = > '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2777) Minimize the cost calculation for queries using reference restrictions.
[ https://issues.apache.org/jira/browse/OAK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Przemo Pakulski updated OAK-2777: - Attachment: oak-2777.patch patch details * set low service.ranking for ReferenceIndexProvider to process it first * skips cost calculation if minimum cost achieved already > Minimize the cost calculation for queries using reference restrictions. > --- > > Key: OAK-2777 > URL: https://issues.apache.org/jira/browse/OAK-2777 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Affects Versions: 1.1.2, 1.2 >Reporter: Przemo Pakulski > Attachments: oak-2777.patch > > > According to the javadocs (QueryIndex) minimum cost for index is 1. Currently > ReferenceIndex returns this minimum value, when it can be used for the query. > But even then cost for remaining indexes is still calculated. We could skip > cost calculation of remaining indexes if we achieved the minimum cost already. > It will speed up all queries which can leverage the reference Index. > Example query: > SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = > '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2755) Consolidated JMX view of all EventListener related statistics
[
https://issues.apache.org/jira/browse/OAK-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496442#comment-14496442
]
Michael Dürig commented on OAK-2755:
Looks good! Thanks for taking care of this.
> Consolidated JMX view of all EventListener related statistics
> -
>
> Key: OAK-2755
> URL: https://issues.apache.org/jira/browse/OAK-2755
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: monitoring, observation
> Fix For: 1.0.13, 1.3.0
>
> Attachments: OAK-2755-2.patch, OAK-2755-3.patch, OAK-2755.patch,
> consolidated-listener-stats-2.png, consolidated-listener-stats.png
>
>
> Oak Observation support exposes a {{EventListenerMBean}} [1] which provide
> quite a bit of details around registered observation listeners. However in a
> typical application there would be multiple listeners registered. To simplify
> monitoring it would be helpful to have a _consolidated_ view of all listeners
> related statistics.
> Further the stats can also include some more details which are Oak specific
> * Subtree paths to which the listener listens to - By default JCR Api allows
> single path however Oak allows a listener to register to multiple paths
> * If listener is enabled to listen to cluster local and cluster external
> changes
> * Size of queue in BackgroundObserver
> * Distribution of change types present in the queue - Local, External etc
> [1]
> https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-api/src/main/java/org/apache/jackrabbit/api/jmx/EventListenerMBean.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2777) Minimize the cost calculation for queries using reference restrictions.
[ https://issues.apache.org/jira/browse/OAK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Przemo Pakulski updated OAK-2777: - Description: According to the javadocs (QueryIndex) minimum cost for index is 1. Currently ReferenceIndex returns this minimum value, when it can be used for the query. But even then cost for remaining indexes is still calculated. We could skip cost calculation of remaining indexes if we achieved the minimum cost already. It will speed up all queries which can leverage the reference Index. Example query: SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' was: According to the javadocs (QueryIndex) minimum cost for index is 1. Currently ReferenceIndex returns this minimum value, when it can be used for the query. But even than remaining indexes are still calculated. We could skip cost calculation of remaining indexes if we achieved the minimum cost already. It will speed up all queries which can leverage the reference Index. Example query: SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' > Minimize the cost calculation for queries using reference restrictions. > --- > > Key: OAK-2777 > URL: https://issues.apache.org/jira/browse/OAK-2777 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Affects Versions: 1.1.2, 1.2 >Reporter: Przemo Pakulski > > According to the javadocs (QueryIndex) minimum cost for index is 1. Currently > ReferenceIndex returns this minimum value, when it can be used for the query. > But even then cost for remaining indexes is still calculated. We could skip > cost calculation of remaining indexes if we achieved the minimum cost already. > It will speed up all queries which can leverage the reference Index. > Example query: > SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = > '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2777) Minimize the cost calculation for queries using reference restrictions.
[ https://issues.apache.org/jira/browse/OAK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496432#comment-14496432 ] Przemo Pakulski commented on OAK-2777: -- I have working patch which makes above query running about 3 times faster on my env. What do you think, is it way to go ? > Minimize the cost calculation for queries using reference restrictions. > --- > > Key: OAK-2777 > URL: https://issues.apache.org/jira/browse/OAK-2777 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Affects Versions: 1.1.2, 1.2 >Reporter: Przemo Pakulski > > According to the javadocs (QueryIndex) minimum cost for index is 1. Currently > ReferenceIndex returns this minimum value, when it can be used for the query. > But even than remaining indexes are still calculated. We could skip cost > calculation of remaining indexes if we achieved the minimum cost already. > It will speed up all queries which can leverage the reference Index. > Example query: > SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = > '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2777) Minimize the cost calculation for queries using reference restrictions.
Przemo Pakulski created OAK-2777: Summary: Minimize the cost calculation for queries using reference restrictions. Key: OAK-2777 URL: https://issues.apache.org/jira/browse/OAK-2777 Project: Jackrabbit Oak Issue Type: Improvement Components: core Affects Versions: 1.2, 1.1.2 Reporter: Przemo Pakulski According to the javadocs (QueryIndex) minimum cost for index is 1. Currently ReferenceIndex returns this minimum value, when it can be used for the query. But even than remaining indexes are still calculated. We could skip cost calculation of remaining indexes if we achieved the minimum cost already. It will speed up all queries which can leverage the reference Index. Example query: SELECT * FROM [nt:base] WHERE PROPERTY([rep:members], 'WeakReference') = '345bef9b-ffa1-3e09-85df-1e03cfa0fb37' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-2755) Consolidated JMX view of all EventListener related statistics
[
https://issues.apache.org/jira/browse/OAK-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495748#comment-14495748
]
Chetan Mehrotra edited comment on OAK-2755 at 4/15/15 3:47 PM:
---
[updated patch|^OAK-2755-3.patch] based on above approach
[~mduerig] Can you have a look?
was (Author: chetanm):
[updated patch|^OAK-2755-2.patch] based on above approach
[~mduerig] Can you have a look?
> Consolidated JMX view of all EventListener related statistics
> -
>
> Key: OAK-2755
> URL: https://issues.apache.org/jira/browse/OAK-2755
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: monitoring, observation
> Fix For: 1.0.13, 1.3.0
>
> Attachments: OAK-2755-2.patch, OAK-2755-3.patch, OAK-2755.patch,
> consolidated-listener-stats-2.png, consolidated-listener-stats.png
>
>
> Oak Observation support exposes a {{EventListenerMBean}} [1] which provide
> quite a bit of details around registered observation listeners. However in a
> typical application there would be multiple listeners registered. To simplify
> monitoring it would be helpful to have a _consolidated_ view of all listeners
> related statistics.
> Further the stats can also include some more details which are Oak specific
> * Subtree paths to which the listener listens to - By default JCR Api allows
> single path however Oak allows a listener to register to multiple paths
> * If listener is enabled to listen to cluster local and cluster external
> changes
> * Size of queue in BackgroundObserver
> * Distribution of change types present in the queue - Local, External etc
> [1]
> https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-api/src/main/java/org/apache/jackrabbit/api/jmx/EventListenerMBean.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2755) Consolidated JMX view of all EventListener related statistics
[
https://issues.apache.org/jira/browse/OAK-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-2755:
-
Attachment: OAK-2755-3.patch
> Consolidated JMX view of all EventListener related statistics
> -
>
> Key: OAK-2755
> URL: https://issues.apache.org/jira/browse/OAK-2755
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: monitoring, observation
> Fix For: 1.0.13, 1.3.0
>
> Attachments: OAK-2755-2.patch, OAK-2755-3.patch, OAK-2755.patch,
> consolidated-listener-stats-2.png, consolidated-listener-stats.png
>
>
> Oak Observation support exposes a {{EventListenerMBean}} [1] which provide
> quite a bit of details around registered observation listeners. However in a
> typical application there would be multiple listeners registered. To simplify
> monitoring it would be helpful to have a _consolidated_ view of all listeners
> related statistics.
> Further the stats can also include some more details which are Oak specific
> * Subtree paths to which the listener listens to - By default JCR Api allows
> single path however Oak allows a listener to register to multiple paths
> * If listener is enabled to listen to cluster local and cluster external
> changes
> * Size of queue in BackgroundObserver
> * Distribution of change types present in the queue - Local, External etc
> [1]
> https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-api/src/main/java/org/apache/jackrabbit/api/jmx/EventListenerMBean.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2755) Consolidated JMX view of all EventListener related statistics
[
https://issues.apache.org/jira/browse/OAK-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496412#comment-14496412
]
Chetan Mehrotra commented on OAK-2755:
--
My mistake. Updated the link to correct patch
> Consolidated JMX view of all EventListener related statistics
> -
>
> Key: OAK-2755
> URL: https://issues.apache.org/jira/browse/OAK-2755
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: monitoring, observation
> Fix For: 1.0.13, 1.3.0
>
> Attachments: OAK-2755-2.patch, OAK-2755-3.patch, OAK-2755.patch,
> consolidated-listener-stats-2.png, consolidated-listener-stats.png
>
>
> Oak Observation support exposes a {{EventListenerMBean}} [1] which provide
> quite a bit of details around registered observation listeners. However in a
> typical application there would be multiple listeners registered. To simplify
> monitoring it would be helpful to have a _consolidated_ view of all listeners
> related statistics.
> Further the stats can also include some more details which are Oak specific
> * Subtree paths to which the listener listens to - By default JCR Api allows
> single path however Oak allows a listener to register to multiple paths
> * If listener is enabled to listen to cluster local and cluster external
> changes
> * Size of queue in BackgroundObserver
> * Distribution of change types present in the queue - Local, External etc
> [1]
> https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-api/src/main/java/org/apache/jackrabbit/api/jmx/EventListenerMBean.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-2662) SegmentOverflowException in HeavyWriteIT on Jenkins
[
https://issues.apache.org/jira/browse/OAK-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-2662.
Resolution: Fixed
Fixed in 1.0 branch at http://svn.apache.org/r1673817
> SegmentOverflowException in HeavyWriteIT on Jenkins
> ---
>
> Key: OAK-2662
> URL: https://issues.apache.org/jira/browse/OAK-2662
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: CI, jenkins
> Fix For: 1.0.13, 1.3.0, 1.2.1
>
> Attachments: OAK-2662.patch
>
>
> {noformat}
> heavyWrite(org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT) Time
> elapsed: 96.384 sec <<< ERROR!
> org.apache.jackrabbit.oak.plugins.segment.SegmentOverflowException: Segment
> cannot have more than 255 references 47a9dc3c-c6f9-4b5f-a61a-6711da8b68c2
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.getSegmentRef(SegmentWriter.java:353)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeRecordId(SegmentWriter.java:382)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapLeaf(SegmentWriter.java:426)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:484)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:511)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMap(SegmentWriter.java:720)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1108)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1091)
> at
> org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1082)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1110)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:97)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:83)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.updated(MemoryNodeBuilder.java:214)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:79)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:501)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:507)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createProperties(HeavyWriteIT.java:137)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:129)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:130)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.heavyWrite(HeavyWriteIT.java:110)
> {noformat}
> See
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/35/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/
> cc [~alex.parvulescu]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2755) Consolidated JMX view of all EventListener related statistics
[
https://issues.apache.org/jira/browse/OAK-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-2755:
-
Attachment: (was: OAK-2755-2.patch)
> Consolidated JMX view of all EventListener related statistics
> -
>
> Key: OAK-2755
> URL: https://issues.apache.org/jira/browse/OAK-2755
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: monitoring, observation
> Fix For: 1.0.13, 1.3.0
>
> Attachments: OAK-2755-2.patch, OAK-2755.patch,
> consolidated-listener-stats-2.png, consolidated-listener-stats.png
>
>
> Oak Observation support exposes a {{EventListenerMBean}} [1] which provide
> quite a bit of details around registered observation listeners. However in a
> typical application there would be multiple listeners registered. To simplify
> monitoring it would be helpful to have a _consolidated_ view of all listeners
> related statistics.
> Further the stats can also include some more details which are Oak specific
> * Subtree paths to which the listener listens to - By default JCR Api allows
> single path however Oak allows a listener to register to multiple paths
> * If listener is enabled to listen to cluster local and cluster external
> changes
> * Size of queue in BackgroundObserver
> * Distribution of change types present in the queue - Local, External etc
> [1]
> https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-api/src/main/java/org/apache/jackrabbit/api/jmx/EventListenerMBean.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2755) Consolidated JMX view of all EventListener related statistics
[
https://issues.apache.org/jira/browse/OAK-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496374#comment-14496374
]
Michael Dürig commented on OAK-2755:
Did you attach the right patch? Both are the same for me.
> Consolidated JMX view of all EventListener related statistics
> -
>
> Key: OAK-2755
> URL: https://issues.apache.org/jira/browse/OAK-2755
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: monitoring, observation
> Fix For: 1.0.13, 1.3.0
>
> Attachments: OAK-2755-2.patch, OAK-2755-2.patch, OAK-2755.patch,
> consolidated-listener-stats-2.png, consolidated-listener-stats.png
>
>
> Oak Observation support exposes a {{EventListenerMBean}} [1] which provide
> quite a bit of details around registered observation listeners. However in a
> typical application there would be multiple listeners registered. To simplify
> monitoring it would be helpful to have a _consolidated_ view of all listeners
> related statistics.
> Further the stats can also include some more details which are Oak specific
> * Subtree paths to which the listener listens to - By default JCR Api allows
> single path however Oak allows a listener to register to multiple paths
> * If listener is enabled to listen to cluster local and cluster external
> changes
> * Size of queue in BackgroundObserver
> * Distribution of change types present in the queue - Local, External etc
> [1]
> https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-api/src/main/java/org/apache/jackrabbit/api/jmx/EventListenerMBean.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2755) Consolidated JMX view of all EventListener related statistics
[
https://issues.apache.org/jira/browse/OAK-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496375#comment-14496375
]
Michael Dürig commented on OAK-2755:
Did you attach the right patch? Both are the same for me.
> Consolidated JMX view of all EventListener related statistics
> -
>
> Key: OAK-2755
> URL: https://issues.apache.org/jira/browse/OAK-2755
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: monitoring, observation
> Fix For: 1.0.13, 1.3.0
>
> Attachments: OAK-2755-2.patch, OAK-2755-2.patch, OAK-2755.patch,
> consolidated-listener-stats-2.png, consolidated-listener-stats.png
>
>
> Oak Observation support exposes a {{EventListenerMBean}} [1] which provide
> quite a bit of details around registered observation listeners. However in a
> typical application there would be multiple listeners registered. To simplify
> monitoring it would be helpful to have a _consolidated_ view of all listeners
> related statistics.
> Further the stats can also include some more details which are Oak specific
> * Subtree paths to which the listener listens to - By default JCR Api allows
> single path however Oak allows a listener to register to multiple paths
> * If listener is enabled to listen to cluster local and cluster external
> changes
> * Size of queue in BackgroundObserver
> * Distribution of change types present in the queue - Local, External etc
> [1]
> https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-api/src/main/java/org/apache/jackrabbit/api/jmx/EventListenerMBean.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496338#comment-14496338 ] Clinton H Goudie-Nice commented on OAK-2752: Thanks everyone for the extremely rapid fix. We are testing it soon and will let you know. > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.3.0, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-tm3.patch, > OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2768) Consider fair mode for backgroundOperationLock
[ https://issues.apache.org/jira/browse/OAK-2768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496323#comment-14496323 ] Marcel Reutegger commented on OAK-2768: --- Ran ObservationTest with 16 writers and a save interval of 10 nodes with fair and non-fair mode of the backgroundOperationLock. The attached 'lock time.png' shows the time it took to acquire the lock for each of the test runs. At least for this test, there is not much difference between the two modes. > Consider fair mode for backgroundOperationLock > -- > > Key: OAK-2768 > URL: https://issues.apache.org/jira/browse/OAK-2768 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, mongomk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.3.0 > > Attachments: lock time.png > > > The backgroundOperationLock in DocumentNodeStore uses the default non-fair > acquisition order. According to JavaDoc of ReentrantReadWriteLock it is > possible that a background operation task gets delayed for a long time when > the system is under load. We should probably consider using the fair mode for > the backgroundOperationLock to make sure background operation tasks do not > get delayed excessively. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2768) Consider fair mode for backgroundOperationLock
[ https://issues.apache.org/jira/browse/OAK-2768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2768: -- Attachment: lock time.png > Consider fair mode for backgroundOperationLock > -- > > Key: OAK-2768 > URL: https://issues.apache.org/jira/browse/OAK-2768 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, mongomk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.3.0 > > Attachments: lock time.png > > > The backgroundOperationLock in DocumentNodeStore uses the default non-fair > acquisition order. According to JavaDoc of ReentrantReadWriteLock it is > possible that a background operation task gets delayed for a long time when > the system is under load. We should probably consider using the fair mode for > the backgroundOperationLock to make sure background operation tasks do not > get delayed excessively. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2776) Upgrade should allow to skip copying versions
Julian Sedding created OAK-2776: --- Summary: Upgrade should allow to skip copying versions Key: OAK-2776 URL: https://issues.apache.org/jira/browse/OAK-2776 Project: Jackrabbit Oak Issue Type: Improvement Components: upgrade Affects Versions: 1.2 Reporter: Julian Sedding In some cases it is not necessary to copy version histories during an upgrade. Skipping to copy versions can result in a lot less content that needs copying and thus a significant speedup. Additionally, OAK-2586 introduces the possibility to include and exclude paths for an upgrade. Version histories should thus only be copied if their respective versionable node is present in the copied part of the content. Also reducing content being copied redundantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2662) SegmentOverflowException in HeavyWriteIT on Jenkins
[
https://issues.apache.org/jira/browse/OAK-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-2662:
---
Fix Version/s: 1.0.13
> SegmentOverflowException in HeavyWriteIT on Jenkins
> ---
>
> Key: OAK-2662
> URL: https://issues.apache.org/jira/browse/OAK-2662
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: CI, jenkins
> Fix For: 1.0.13, 1.3.0, 1.2.1
>
> Attachments: OAK-2662.patch
>
>
> {noformat}
> heavyWrite(org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT) Time
> elapsed: 96.384 sec <<< ERROR!
> org.apache.jackrabbit.oak.plugins.segment.SegmentOverflowException: Segment
> cannot have more than 255 references 47a9dc3c-c6f9-4b5f-a61a-6711da8b68c2
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.getSegmentRef(SegmentWriter.java:353)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeRecordId(SegmentWriter.java:382)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapLeaf(SegmentWriter.java:426)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:484)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:511)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMap(SegmentWriter.java:720)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1108)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1091)
> at
> org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1082)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1110)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:97)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:83)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.updated(MemoryNodeBuilder.java:214)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:79)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:501)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:507)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createProperties(HeavyWriteIT.java:137)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:129)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:130)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.heavyWrite(HeavyWriteIT.java:110)
> {noformat}
> See
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/35/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/
> cc [~alex.parvulescu]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (OAK-2662) SegmentOverflowException in HeavyWriteIT on Jenkins
[
https://issues.apache.org/jira/browse/OAK-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reopened OAK-2662:
Reopening for merging into 1.0
> SegmentOverflowException in HeavyWriteIT on Jenkins
> ---
>
> Key: OAK-2662
> URL: https://issues.apache.org/jira/browse/OAK-2662
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: CI, jenkins
> Fix For: 1.0.13, 1.3.0, 1.2.1
>
> Attachments: OAK-2662.patch
>
>
> {noformat}
> heavyWrite(org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT) Time
> elapsed: 96.384 sec <<< ERROR!
> org.apache.jackrabbit.oak.plugins.segment.SegmentOverflowException: Segment
> cannot have more than 255 references 47a9dc3c-c6f9-4b5f-a61a-6711da8b68c2
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.getSegmentRef(SegmentWriter.java:353)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeRecordId(SegmentWriter.java:382)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapLeaf(SegmentWriter.java:426)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:484)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:511)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMap(SegmentWriter.java:720)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1108)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1091)
> at
> org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1082)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1110)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:97)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:83)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.updated(MemoryNodeBuilder.java:214)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:79)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:501)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:507)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createProperties(HeavyWriteIT.java:137)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:129)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:130)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.heavyWrite(HeavyWriteIT.java:110)
> {noformat}
> See
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/35/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/
> cc [~alex.parvulescu]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2713) High memory usage of CompactionMap
[
https://issues.apache.org/jira/browse/OAK-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496271#comment-14496271
]
Michael Dürig commented on OAK-2713:
Initial fix in trunk at http://svn.apache.org/r1673791
> High memory usage of CompactionMap
> --
>
> Key: OAK-2713
> URL: https://issues.apache.org/jira/browse/OAK-2713
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: compaction, gc
> Fix For: 1.3.0, 1.2.2
>
>
> In environments with a lot of volatile content the {{CompactionMap}} can end
> up eating a lot of memory. From
> {{CompactionStrategyMBean#getCompactionMapStats}}:
> {noformat}
> [Estimated Weight: 317,5 MB, Records: 39500094, Segments: 36698],
> [Estimated Weight: 316,4 MB, Records: 39374593, Segments: 36660],
> [Estimated Weight: 315,4 MB, Records: 39253205, Segments: 36620],
> [Estimated Weight: 315,1 MB, Records: 39221882, Segments: 36614],
> [Estimated Weight: 314,9 MB, Records: 39195490, Segments: 36604],
> [Estimated Weight: 315,0 MB, Records: 39182753, Segments: 36602],
> [Estimated Weight: 360 B, Records: 0, Segments: 0],
> {noformat}
> This causes compaction to be skipped:
> {noformat}
> 2015-03-30:30.03.2015 02:00:00.038 *INFO* [] [TarMK compaction thread
> [/foo/bar/crx-quickstart/repository/segmentstore], active since Mon Mar 30
> 02:00:00 CEST 2015, previous max duration 3854982ms]
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore Not enough available
> memory 5,5 GB, needed 6,3 GB, last merge delta 1,3 GB, so skipping compaction
> for now
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2713) High memory usage of CompactionMap
[
https://issues.apache.org/jira/browse/OAK-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-2713:
---
Fix Version/s: 1.3.0
> High memory usage of CompactionMap
> --
>
> Key: OAK-2713
> URL: https://issues.apache.org/jira/browse/OAK-2713
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: compaction, gc
> Fix For: 1.3.0, 1.2.2
>
>
> In environments with a lot of volatile content the {{CompactionMap}} can end
> up eating a lot of memory. From
> {{CompactionStrategyMBean#getCompactionMapStats}}:
> {noformat}
> [Estimated Weight: 317,5 MB, Records: 39500094, Segments: 36698],
> [Estimated Weight: 316,4 MB, Records: 39374593, Segments: 36660],
> [Estimated Weight: 315,4 MB, Records: 39253205, Segments: 36620],
> [Estimated Weight: 315,1 MB, Records: 39221882, Segments: 36614],
> [Estimated Weight: 314,9 MB, Records: 39195490, Segments: 36604],
> [Estimated Weight: 315,0 MB, Records: 39182753, Segments: 36602],
> [Estimated Weight: 360 B, Records: 0, Segments: 0],
> {noformat}
> This causes compaction to be skipped:
> {noformat}
> 2015-03-30:30.03.2015 02:00:00.038 *INFO* [] [TarMK compaction thread
> [/foo/bar/crx-quickstart/repository/segmentstore], active since Mon Mar 30
> 02:00:00 CEST 2015, previous max duration 3854982ms]
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore Not enough available
> memory 5,5 GB, needed 6,3 GB, last merge delta 1,3 GB, so skipping compaction
> for now
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller resolved OAK-2752. - Resolution: Fixed > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.3.0, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-tm3.patch, > OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496262#comment-14496262 ] Thomas Mueller commented on OAK-2752: - http://svn.apache.org/r1673786 (1.2 branch) http://svn.apache.org/r1673789 (1.0 branch) > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.3.0, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-tm3.patch, > OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2771) Longevity tests in oak-run
[ https://issues.apache.org/jira/browse/OAK-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496259#comment-14496259 ] Davide Giannella commented on OAK-2771: --- Sometime ago we coded the ScalabilityTestSuite in oak-run. Maybe it can be extended/reused/generalised. Please have a look if it could be. > Longevity tests in oak-run > -- > > Key: OAK-2771 > URL: https://issues.apache.org/jira/browse/OAK-2771 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run >Reporter: Alex Parvulescu >Priority: Minor > > How about adding a new category of tests in oak-run related to longevity > testing. Around the SegmentMk bits I've seen emerge a new type of tests that > setup an infinite sequence of operations waiting for a specific problem to > surface and if the error doesn't happen, we have a never-ending test. > Moreover running an IT test for a few hours is not reasonable on any of the > current build platforms we have now. > My proposal would be to add some of these tests to oak-run, provide a common > way to instrument them and possibly expose some simple (http) UI to stop them > once we think the results are satisfactory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2751) Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch
[
https://issues.apache.org/jira/browse/OAK-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496258#comment-14496258
]
Vikas Saurabh commented on OAK-2751:
Attached a [patch|^OAK-2751-set-prop-on-delete-should-conflict.patch] which
calls out conflict in {{NodeDocument.isConflicting}} while setting a property's
value for an already deleted node.
> Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch
> --
>
> Key: OAK-2751
> URL: https://issues.apache.org/jira/browse/OAK-2751
> Project: Jackrabbit Oak
> Issue Type: Bug
>Reporter: Chetan Mehrotra
>Assignee: Marcel Reutegger
>Priority: Blocker
> Fix For: 1.0.13
>
> Attachments: OAK-2751-set-prop-on-delete-should-conflict.patch
>
>
> With merge of OAK-2673 some failures were seen in oak-it/mk which are not
> seen on trunk. This require further change in patch to shutdown the feature
> completely.
> We still need to investigate why this happened and ensure that test pass with
> this feature enabled
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2751) Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch
[ https://issues.apache.org/jira/browse/OAK-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Saurabh updated OAK-2751: --- Attachment: OAK-2751-set-prop-on-delete-should-conflict.patch > Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch > -- > > Key: OAK-2751 > URL: https://issues.apache.org/jira/browse/OAK-2751 > Project: Jackrabbit Oak > Issue Type: Bug >Reporter: Chetan Mehrotra >Assignee: Marcel Reutegger >Priority: Blocker > Fix For: 1.0.13 > > Attachments: OAK-2751-set-prop-on-delete-should-conflict.patch > > > With merge of OAK-2673 some failures were seen in oak-it/mk which are not > seen on trunk. This require further change in patch to shutdown the feature > completely. > We still need to investigate why this happened and ensure that test pass with > this feature enabled -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1985) TokenLoginModule can't handle case insensitive userids
[ https://issues.apache.org/jira/browse/OAK-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496242#comment-14496242 ] angela commented on OAK-1985: - merged changes (plus additionally 1599160 which was required by the test-cases) into 1.0 branch at rev. 1673782 > TokenLoginModule can't handle case insensitive userids > -- > > Key: OAK-1985 > URL: https://issues.apache.org/jira/browse/OAK-1985 > Project: Jackrabbit Oak > Issue Type: Bug >Reporter: Tobias Bocanegra >Assignee: angela > Fix For: 1.1.0, 1.0.13 > > > Login against TokenLoginModule with an userid different in case throws: > javax.security.auth.login.LoginException: Invalid token credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1985) TokenLoginModule can't handle case insensitive userids
[ https://issues.apache.org/jira/browse/OAK-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angela updated OAK-1985: Fix Version/s: 1.0.13 > TokenLoginModule can't handle case insensitive userids > -- > > Key: OAK-1985 > URL: https://issues.apache.org/jira/browse/OAK-1985 > Project: Jackrabbit Oak > Issue Type: Bug >Reporter: Tobias Bocanegra >Assignee: angela > Fix For: 1.1.0, 1.0.13 > > > Login against TokenLoginModule with an userid different in case throws: > javax.security.auth.login.LoginException: Invalid token credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2775) Oak builder changes its state during repository creation
[
https://issues.apache.org/jira/browse/OAK-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496237#comment-14496237
]
Francesco Mari commented on OAK-2775:
-
I will wait until OAK-2760 is fixed. Having both a {{Repository}} and a
{{ContentRepository}} from the same configuration is a very important use case
for me.
> Oak builder changes its state during repository creation
>
>
> Key: OAK-2775
> URL: https://issues.apache.org/jira/browse/OAK-2775
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.2
>Reporter: Francesco Mari
> Attachments: OAK-2775-01.patch
>
>
> The {{Oak#createContentRepository()}} method changes the state of the builder
> at every invocation. In particular, it always adds a new {{CommitHook}}.
> The observable behavior is that all the {{IndexEditor}} instances are
> executed twice when the {{Oak}} and {{Jcr}} builders are used together - i.e.
> when both an instance of {{Repository}} and {{ContentRepository}} are needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2713) High memory usage of CompactionMap
[
https://issues.apache.org/jira/browse/OAK-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-2713:
---
Fix Version/s: (was: 1.2.1)
1.2.2
> High memory usage of CompactionMap
> --
>
> Key: OAK-2713
> URL: https://issues.apache.org/jira/browse/OAK-2713
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: compaction, gc
> Fix For: 1.2.2
>
>
> In environments with a lot of volatile content the {{CompactionMap}} can end
> up eating a lot of memory. From
> {{CompactionStrategyMBean#getCompactionMapStats}}:
> {noformat}
> [Estimated Weight: 317,5 MB, Records: 39500094, Segments: 36698],
> [Estimated Weight: 316,4 MB, Records: 39374593, Segments: 36660],
> [Estimated Weight: 315,4 MB, Records: 39253205, Segments: 36620],
> [Estimated Weight: 315,1 MB, Records: 39221882, Segments: 36614],
> [Estimated Weight: 314,9 MB, Records: 39195490, Segments: 36604],
> [Estimated Weight: 315,0 MB, Records: 39182753, Segments: 36602],
> [Estimated Weight: 360 B, Records: 0, Segments: 0],
> {noformat}
> This causes compaction to be skipped:
> {noformat}
> 2015-03-30:30.03.2015 02:00:00.038 *INFO* [] [TarMK compaction thread
> [/foo/bar/crx-quickstart/repository/segmentstore], active since Mon Mar 30
> 02:00:00 CEST 2015, previous max duration 3854982ms]
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore Not enough available
> memory 5,5 GB, needed 6,3 GB, last merge delta 1,3 GB, so skipping compaction
> for now
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2775) Oak builder changes its state during repository creation
[
https://issues.apache.org/jira/browse/OAK-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496210#comment-14496210
]
Michael Dürig commented on OAK-2775:
Right, there is a TODO about this in the code. See OAK-2736 for why this is
currently pending.
> Oak builder changes its state during repository creation
>
>
> Key: OAK-2775
> URL: https://issues.apache.org/jira/browse/OAK-2775
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.2
>Reporter: Francesco Mari
> Attachments: OAK-2775-01.patch
>
>
> The {{Oak#createContentRepository()}} method changes the state of the builder
> at every invocation. In particular, it always adds a new {{CommitHook}}.
> The observable behavior is that all the {{IndexEditor}} instances are
> executed twice when the {{Oak}} and {{Jcr}} builders are used together - i.e.
> when both an instance of {{Repository}} and {{ContentRepository}} are needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-2751) Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch
[ https://issues.apache.org/jira/browse/OAK-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger reassigned OAK-2751: - Assignee: Marcel Reutegger (was: Chetan Mehrotra) > Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch > -- > > Key: OAK-2751 > URL: https://issues.apache.org/jira/browse/OAK-2751 > Project: Jackrabbit Oak > Issue Type: Bug >Reporter: Chetan Mehrotra >Assignee: Marcel Reutegger >Priority: Blocker > Fix For: 1.0.13 > > > With merge of OAK-2673 some failures were seen in oak-it/mk which are not > seen on trunk. This require further change in patch to shutdown the feature > completely. > We still need to investigate why this happened and ensure that test pass with > this feature enabled -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2751) Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch
[ https://issues.apache.org/jira/browse/OAK-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2751: -- Priority: Blocker (was: Major) > Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch > -- > > Key: OAK-2751 > URL: https://issues.apache.org/jira/browse/OAK-2751 > Project: Jackrabbit Oak > Issue Type: Bug >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Blocker > Fix For: 1.0.13 > > > With merge of OAK-2673 some failures were seen in oak-it/mk which are not > seen on trunk. This require further change in patch to shutdown the feature > completely. > We still need to investigate why this happened and ensure that test pass with > this feature enabled -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496198#comment-14496198 ] Thomas Mueller commented on OAK-2752: - http://svn.apache.org/r1673761 (trunk) > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.3.0, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-tm3.patch, > OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2738) Possible StackOverflowException with many "or" conditions
[ https://issues.apache.org/jira/browse/OAK-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller resolved OAK-2738. - Resolution: Fixed > Possible StackOverflowException with many "or" conditions > - > > Key: OAK-2738 > URL: https://issues.apache.org/jira/browse/OAK-2738 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.0.13, 1.3.0, 1.2.1 > > > For XPath queries with many "or" conditions of the form "@x = 1 or @x = 2 or > @x = 3", the converter could throw a StackOverflowException (during the > optimization phase). > Such conditions are converted to "x in (1, 2, 3)", however this conversion is > recursive and relatively slow. We need to make sure at least 10'000 > conditions can be processed efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2775) Oak builder changes its state during repository creation
[
https://issues.apache.org/jira/browse/OAK-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496192#comment-14496192
]
Francesco Mari commented on OAK-2775:
-
If {{Oak}} and {{Jcr}} are not supposed to be used more than once, it would be
nice if the builder would "deactivate" itself after the first usage and throw
{{IllegalStateException}} when factory methods are called more than once. I
also couldn't find any reference in the Javadoc about the single-usage
assumption.
> Oak builder changes its state during repository creation
>
>
> Key: OAK-2775
> URL: https://issues.apache.org/jira/browse/OAK-2775
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.2
>Reporter: Francesco Mari
> Attachments: OAK-2775-01.patch
>
>
> The {{Oak#createContentRepository()}} method changes the state of the builder
> at every invocation. In particular, it always adds a new {{CommitHook}}.
> The observable behavior is that all the {{IndexEditor}} instances are
> executed twice when the {{Oak}} and {{Jcr}} builders are used together - i.e.
> when both an instance of {{Repository}} and {{ContentRepository}} are needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2775) Oak builder changes its state during repository creation
[
https://issues.apache.org/jira/browse/OAK-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496187#comment-14496187
]
Michael Dürig commented on OAK-2775:
This is somewhat a duplicate of OAK-2736. The Oak builder is not mean to be
used more than once. If needed it would be an option to add a {{clone}} method
to it so you could obtain a fresh instance from a stale one.
> Oak builder changes its state during repository creation
>
>
> Key: OAK-2775
> URL: https://issues.apache.org/jira/browse/OAK-2775
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.2
>Reporter: Francesco Mari
> Attachments: OAK-2775-01.patch
>
>
> The {{Oak#createContentRepository()}} method changes the state of the builder
> at every invocation. In particular, it always adds a new {{CommitHook}}.
> The observable behavior is that all the {{IndexEditor}} instances are
> executed twice when the {{Oak}} and {{Jcr}} builders are used together - i.e.
> when both an instance of {{Repository}} and {{ContentRepository}} are needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2751) Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch
[
https://issues.apache.org/jira/browse/OAK-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496186#comment-14496186
]
Vikas Saurabh commented on OAK-2751:
In 1.0 branch with
{{addRemoveSetMoveCopy(org.apache.jackrabbit.mk.test.RandomizedTest)}} fails if
{{enableConcurrentAddDelete}} is set (see OAK-2673). The step that fails is
attempting setting a property on a deleted node. This is an expected failure --
but it doesn't with current fix of OAK-2673.
-The reason the test passes on trunk is that trunk is using
{{NodeStoreKernel.applyJsop}} which itself throws the exception while trying to
set a property on an unavailable node.- This piece of code doesn't exist
anymore on trunk ... it was roughly 1 week back (I believe it's due to removing
MicroKernel implementation effort).
While fixing jsop parsing on 1.0 can fix the error, I belive, the fix should
really be done in {{NodeDocument.isConflicting}} i.e. setting a property on a
known deleted node should be called out as conflict.
[~mreutegg], thoughts?
> Test failures with EnableConcurrentAddRemove feature enabled on 1.0 branch
> --
>
> Key: OAK-2751
> URL: https://issues.apache.org/jira/browse/OAK-2751
> Project: Jackrabbit Oak
> Issue Type: Bug
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.0.13
>
>
> With merge of OAK-2673 some failures were seen in oak-it/mk which are not
> seen on trunk. This require further change in patch to shutdown the feature
> completely.
> We still need to investigate why this happened and ensure that test pass with
> this feature enabled
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2734) Compaction does not finish on repository with continuous writes
[ https://issues.apache.org/jira/browse/OAK-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Marth updated OAK-2734: --- Fix Version/s: (was: 1.2.1) 1.3.0 > Compaction does not finish on repository with continuous writes > > > Key: OAK-2734 > URL: https://issues.apache.org/jira/browse/OAK-2734 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segmentmk >Reporter: Michael Dürig >Assignee: Michael Dürig > Labels: compaction, gc > Fix For: 1.3.0 > > > A repository with continuous writes can keep the compactor from completing > causing the repository size to grow indefinitely. > This effect is caused by the compactor trying to catch up with changes that > occurred while compacting. I.e. compacting them on top of the already > compacted head. When there is a steady stream of incoming changes it can > happen that the compactor never actually catches up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2744) Change default cache distribution ratio if persistent cache is enabled
[ https://issues.apache.org/jira/browse/OAK-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-2744: - Fix Version/s: (was: 1.2.1) (was: 1.0.13) > Change default cache distribution ratio if persistent cache is enabled > -- > > Key: OAK-2744 > URL: https://issues.apache.org/jira/browse/OAK-2744 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: mongomk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.3.0 > > > By default the cache memory in DocumentNodeStore is distributed in following > ratio > * nodeCache - 25% > * childrenCache - 10% > * docChildrenCache - 3% > * diffCache - 5% > * documentCache - Is given the rest i.e. 57% > However off late we have found that with persistent cache enabled we can > lower the cache allocated to Document cache. That would reduce the time spent > in invalidating cache entries in periodic reads. So far we are using > following ration in few setup and that is turning out well > * nodeCachePercentage=35 > * childrenCachePercentage=20 > * diffCachePercentage=30 > * docChildrenCachePercentage=10 > * documentCache - Is given the rest i.e. 5% > We should use the above distribution by default if the persistent cache is > found to be enabled -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2775) Oak builder changes its state during repository creation
[
https://issues.apache.org/jira/browse/OAK-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-2775:
Attachment: OAK-2775-01.patch
This patch avoids to change the state of the builder when a
{{ContentRepository}} is created.
> Oak builder changes its state during repository creation
>
>
> Key: OAK-2775
> URL: https://issues.apache.org/jira/browse/OAK-2775
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.2
>Reporter: Francesco Mari
> Attachments: OAK-2775-01.patch
>
>
> The {{Oak#createContentRepository()}} method changes the state of the builder
> at every invocation. In particular, it always adds a new {{CommitHook}}.
> The observable behavior is that all the {{IndexEditor}} instances are
> executed twice when the {{Oak}} and {{Jcr}} builders are used together - i.e.
> when both an instance of {{Repository}} and {{ContentRepository}} are needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-2775) Oak builder changes its state during repository creation
Francesco Mari created OAK-2775:
---
Summary: Oak builder changes its state during repository creation
Key: OAK-2775
URL: https://issues.apache.org/jira/browse/OAK-2775
Project: Jackrabbit Oak
Issue Type: Bug
Components: oak-core
Affects Versions: 1.2
Reporter: Francesco Mari
The {{Oak#createContentRepository()}} method changes the state of the builder
at every invocation. In particular, it always adds a new {{CommitHook}}.
The observable behavior is that all the {{IndexEditor}} instances are executed
twice when the {{Oak}} and {{Jcr}} builders are used together - i.e. when both
an instance of {{Repository}} and {{ContentRepository}} are needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2774) Invalid characters not escaped while fulltext query parsing
[
https://issues.apache.org/jira/browse/OAK-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496152#comment-14496152
]
Rishabh Maurya commented on OAK-2774:
-
Added following test case at LuceneIndexTest.java and it fails -
{code}
@Test
public void testLuceneQueryParserEscape() throws Exception {
NodeBuilder nb =
newLuceneIndexDefinition(builder.child(INDEX_DEFINITIONS_NAME), "lucene",
of(TYPENAME_STRING));
TestUtil.useV2(nb);
NodeState before = builder.getNodeState();
builder.child("a").setProperty("foo", "/bar");
NodeState after = builder.getNodeState();
NodeState indexed = HOOK.processCommit(before, after, CommitInfo.EMPTY);
IndexTracker tracker = new IndexTracker();
tracker.update(indexed);
AdvancedQueryIndex queryIndex = new LucenePropertyIndex(tracker);
FilterImpl filter = createFilter(NT_BASE);
filter.setFullTextConstraint(new FullTextTerm(null, "/bar", false,
true, null));
List plans = queryIndex.getPlans(filter, null, indexed);
Cursor cursor = queryIndex.query(plans.get(0), indexed);
assertTrue(cursor.hasNext());
assertEquals("/a", cursor.next().getPath());
}
{code}
> Invalid characters not escaped while fulltext query parsing
> ---
>
> Key: OAK-2774
> URL: https://issues.apache.org/jira/browse/OAK-2774
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-lucene
>Affects Versions: 1.2
>Reporter: Rishabh Maurya
>
> Invalid characters such as forward slash {{/}} should be escaped, as fulltext
> search on them results in below error -
> {code}
> java.lang.RuntimeException: INVALID_SYNTAX_CANNOT_PARSE: Syntax Error, cannot
> parse /bar: Lexical error at line 1, column 5. Encountered: after :
> "/bar"
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.tokenToQuery(LucenePropertyIndex.java:1042)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$2.visitTerm(LucenePropertyIndex.java:983)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$2.visit(LucenePropertyIndex.java:978)
> at
> org.apache.jackrabbit.oak.query.fulltext.FullTextTerm.accept(FullTextTerm.java:215)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.getFullTextQuery(LucenePropertyIndex.java:933)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.getLuceneRequest(LucenePropertyIndex.java:519)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.access$200(LucenePropertyIndex.java:160)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.loadDocs(LucenePropertyIndex.java:321)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.computeNext(LucenePropertyIndex.java:274)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.computeNext(LucenePropertyIndex.java:265)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$LucenePathCursor$1.hasNext(LucenePropertyIndex.java:1138)
> ...
> {code}
> http://lucene.apache.org/core/4_0_0/queryparser/org/apache/lucene/queryparser/classic/QueryParserBase.html#escape(java.lang.String)
> should be used to escape such invalid characters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-2774) Invalid characters not escaped while fulltext query parsing
Rishabh Maurya created OAK-2774:
---
Summary: Invalid characters not escaped while fulltext query
parsing
Key: OAK-2774
URL: https://issues.apache.org/jira/browse/OAK-2774
Project: Jackrabbit Oak
Issue Type: Bug
Components: oak-lucene
Affects Versions: 1.2
Reporter: Rishabh Maurya
Invalid characters such as forward slash {{/}} should be escaped, as fulltext
search on them results in below error -
{code}
java.lang.RuntimeException: INVALID_SYNTAX_CANNOT_PARSE: Syntax Error, cannot
parse /bar: Lexical error at line 1, column 5. Encountered: after :
"/bar"
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.tokenToQuery(LucenePropertyIndex.java:1042)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$2.visitTerm(LucenePropertyIndex.java:983)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$2.visit(LucenePropertyIndex.java:978)
at
org.apache.jackrabbit.oak.query.fulltext.FullTextTerm.accept(FullTextTerm.java:215)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.getFullTextQuery(LucenePropertyIndex.java:933)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.getLuceneRequest(LucenePropertyIndex.java:519)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.access$200(LucenePropertyIndex.java:160)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.loadDocs(LucenePropertyIndex.java:321)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.computeNext(LucenePropertyIndex.java:274)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.computeNext(LucenePropertyIndex.java:265)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$LucenePathCursor$1.hasNext(LucenePropertyIndex.java:1138)
...
{code}
http://lucene.apache.org/core/4_0_0/queryparser/org/apache/lucene/queryparser/classic/QueryParserBase.html#escape(java.lang.String)
should be used to escape such invalid characters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496143#comment-14496143 ] Thomas Mueller edited comment on OAK-2752 at 4/15/15 1:05 PM: -- OAK-2752-tm3.patch with more documentation I'm running the tests now (all of them, so it will take a while) and will then commit the change to trunk and branches was (Author: tmueller): OAK-2752-tm3.patch with more documentation I'm running the tests now and will then commit the change to trunk > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.3.0, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-tm3.patch, > OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-2752: Fix Version/s: 1.3.0 > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.3.0, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-tm3.patch, > OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-2752: Attachment: OAK-2752-tm3.patch OAK-2752-tm3.patch with more documentation I'm running the tests now and will then commit the change to trunk > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-tm3.patch, > OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496140#comment-14496140 ] Thomas Mueller commented on OAK-2752: - I have added more comments to the code. Removing the line in clearSegmentIdTables is needed because refresh() only re-builds the map if there is a hash collision _and_ there is an empty reference. If the reference is removed as well, then the map is not re-build, in which case you can end up with two segment id objects for the same segment. > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-v2.patch, OAK-2752-v3.patch, > OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2065) JMX stats for operations being performed in DocumentStore
[ https://issues.apache.org/jira/browse/OAK-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-2065: - Priority: Major (was: Minor) > JMX stats for operations being performed in DocumentStore > - > > Key: OAK-2065 > URL: https://issues.apache.org/jira/browse/OAK-2065 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: mongomk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.3.0 > > > Currently DocumentStore performs various background operations like > # Cache consistency check > # Pushing the lastRev updates > # Synchrnizing the root node version > We should capture some stats like time taken in various task and expose them > over JMX to determine if those background operations are performing well or > not. For example its important that all tasks performed in background task > should be completed under 1 sec (default polling interval). If the time taken > increases then it would be cause of concern > See http://markmail.org/thread/57fax4nyabbubbef -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2676) Expose stats around BackgroundObserver queue
[
https://issues.apache.org/jira/browse/OAK-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-2676.
--
Resolution: Duplicate
Fix Version/s: (was: 1.3.0)
> Expose stats around BackgroundObserver queue
>
>
> Key: OAK-2676
> URL: https://issues.apache.org/jira/browse/OAK-2676
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
>
> To aid in debugging growth of observation queue it would be helpful to expose
> following details from {{BackgroundObserver}}
> # Size of queue
> # Number of
> ## External change
> ## Internal change
> ## _Collapsed_ change
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2543) Service user session creation isn't fast enough
[ https://issues.apache.org/jira/browse/OAK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Marth updated OAK-2543: --- Fix Version/s: 1.3.1 > Service user session creation isn't fast enough > --- > > Key: OAK-2543 > URL: https://issues.apache.org/jira/browse/OAK-2543 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Reporter: Laurie byrum > Labels: performance > Fix For: 1.3.1 > > > We have some (very commonly hit) bits of code that need to read configs (for > example), and thus need higher privileges. At one point, we were advised to > make short-lived service sessions to handle this. We did this and found our > performance was absolutely abysmal. We're on our 3rd bottleneck that we > are working through. They have all pointed to session creation. Maybe each > creation isn't too bad, but in aggregate, it's much slower than, for > example, the actual reads or anything else. > I was able to make the code usually avoid session creation in the first 2 > cases, but earlier this week we hit the third example where the answer seems > to > be one of 1) make creating sessions ignorably fast even when they are > created a lot 2) cache whatever read is requiring the escalation and clean > up in event listeners (those listeners will invariably have to listen to > non-local events, but the events should be uncommon so far) 3) long-lived > sessions used for reads across threads. > Per Michael Duerig, #1 is the goal. Can we see if the current situation can > be improved? Because it isn't ignorably fast today. Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2749) Provide a "different lane" for slow indexers in async indexing
[ https://issues.apache.org/jira/browse/OAK-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496133#comment-14496133 ] Chetan Mehrotra commented on OAK-2749: -- Looks better. However make enabling this configurable as otherwise it would unnecessary perform a diff even if no index is assigned to it. So have it configurable and leave it to class initializing Oak to configure this mode > Provide a "different lane" for slow indexers in async indexing > -- > > Key: OAK-2749 > URL: https://issues.apache.org/jira/browse/OAK-2749 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Reporter: Davide Giannella >Assignee: Davide Giannella > Fix For: 1.3.0 > > Attachments: OAK-2749-rc1.diff, OAK-2749-rc2.diff > > > In case of big repositories, asynchronous index like Lucene Property, > could lag behind as slow indexes, for example Full Text, are taken > care in the same thread pool. > Provide a separate thread pool in which such indexes could be > registered. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2749) Provide a "different lane" for slow indexers in async indexing
[
https://issues.apache.org/jira/browse/OAK-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella updated OAK-2749:
--
Attachment: OAK-2749-rc2.diff
[~chetanm] in [^OAK-2749-rc2.diff] I reverted the forcing of new
executor. If you could give a look to see if everything is in place it
would be awesome.
{quote}
Or we can simply expose a JMX opertaion to perform this sort of migration
{quote}
I'll have a look at the JMX thing. Will follow up here once I have a
more detailed understanding of what we should do.
> Provide a "different lane" for slow indexers in async indexing
> --
>
> Key: OAK-2749
> URL: https://issues.apache.org/jira/browse/OAK-2749
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Fix For: 1.3.0
>
> Attachments: OAK-2749-rc1.diff, OAK-2749-rc2.diff
>
>
> In case of big repositories, asynchronous index like Lucene Property,
> could lag behind as slow indexes, for example Full Text, are taken
> care in the same thread pool.
> Provide a separate thread pool in which such indexes could be
> registered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2773) Speed up creation of JCR sessions
[ https://issues.apache.org/jira/browse/OAK-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496110#comment-14496110 ] Michael Dürig commented on OAK-2773: Probably a duplicate of OAK-2543? > Speed up creation of JCR sessions > - > > Key: OAK-2773 > URL: https://issues.apache.org/jira/browse/OAK-2773 > Project: Jackrabbit Oak > Issue Type: Wish > Components: core >Affects Versions: 1.0.12, 1.2 >Reporter: Przemo Pakulski > Labels: performance > > JCR sessions used to be lightweight objects and fast to create in > Jackrabbit2/CRX. > I noticed that in Oak creation of session is significantly more expensive. On > my environment creating Oak sessions is ~10 times slower than in crx2. > The most of the time is spent in PrincipalProviderImpl class which executes > many queries. > Possible solutions are: > * introduce MembershipCache as it was in Jackrabbit2 > * speed up the queries, actually the queries itself are pretty fast, > calculation of execution plans is heavy and adds much overhead > * both -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[
https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496104#comment-14496104
]
Michael Dürig commented on OAK-2752:
Patch looks good to me except for the removed line in {{clearSegmentIdTables}}.
Could you elaborate why this is necessary?
Thanks for the tests and the comments. Very helpful.
> SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
> -
>
> Key: OAK-2752
> URL: https://issues.apache.org/jira/browse/OAK-2752
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.2
>Reporter: Will McGauley
>Assignee: Thomas Mueller
>Priority: Critical
> Fix For: 1.0.13, 1.2.1
>
> Attachments: OAK-2752-md.patch, OAK-2752-md2.patch,
> OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-v2.patch, OAK-2752-v3.patch,
> OAK-2752.patch
>
>
> The while loop getSegmentId(msb, lsb) loops forever in the situation where
> the segment id is not found.
> Looping occurs from 'first' and loops to the end of the SegmentId references,
> and then loops back to first. If the segmentid is not found in any of the
> referenced items then looping continues past 'first' again and loops for
> eternity through all the references.
> See attached patch for possible fix to break out of the loop after getting
> back to 'first' again.
> note: I have tested this patch on my system and it seems to work, but I do
> not know if the patch provides the best fix, I am a bit new to this code.
> The most important part of the patch would be the break condition from the
> loop so the loop does not continue after testing each reference
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2773) Speed up creation of JCR sessions
[ https://issues.apache.org/jira/browse/OAK-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Przemo Pakulski updated OAK-2773: - Description: JCR sessions used to be lightweight objects and fast to create in Jackrabbit2/CRX. I noticed that in Oak creation of session is significantly more expensive. On my environment creating Oak sessions is ~10 times slower than in crx2. The most of the time is spent in PrincipalProviderImpl class which executes many queries. Possible solutions are: * introduce MembershipCache as it was in Jackrabbit2 * speed up the queries, actually the queries itself are pretty fast, calculation of execution plans is heavy and adds much overhead * both was: JCR sessions used to be lightweight objects and chip to create in Jackrabbit2/CRX. I noticed that in Oak creation of session is significantly more expensive. On my environment creating Oak sessions is ~10 times slower than in crx2. The most of the time is spent in PrincipalProviderImpl class which executes many queries. Possible solutions are: * introduce MembershipCache as it was in Jackrabbit2 * speed up the queries, actually the queries itself are pretty fast, calculation of execution plans is heavy and adds much overhead * both > Speed up creation of JCR sessions > - > > Key: OAK-2773 > URL: https://issues.apache.org/jira/browse/OAK-2773 > Project: Jackrabbit Oak > Issue Type: Wish > Components: core >Affects Versions: 1.0.12, 1.2 >Reporter: Przemo Pakulski > Labels: performance > > JCR sessions used to be lightweight objects and fast to create in > Jackrabbit2/CRX. > I noticed that in Oak creation of session is significantly more expensive. On > my environment creating Oak sessions is ~10 times slower than in crx2. > The most of the time is spent in PrincipalProviderImpl class which executes > many queries. > Possible solutions are: > * introduce MembershipCache as it was in Jackrabbit2 > * speed up the queries, actually the queries itself are pretty fast, > calculation of execution plans is heavy and adds much overhead > * both -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2773) Speedup creation of JCR session
Przemo Pakulski created OAK-2773: Summary: Speedup creation of JCR session Key: OAK-2773 URL: https://issues.apache.org/jira/browse/OAK-2773 Project: Jackrabbit Oak Issue Type: Wish Components: core Affects Versions: 1.2, 1.0.12 Reporter: Przemo Pakulski JCR sessions used to be lightweight objects and chip to create in Jackrabbit2/CRX. I noticed that in Oak creation of session is significantly more expensive. On my environment creating Oak sessions is ~10 times slower than in crx2. The most of the time is spent in PrincipalProviderImpl class which executes many queries. Possible solutions are: * introduce MembershipCache as it was in Jackrabbit2 * speed up the queries, actually the queries itself are pretty fast, calculation of execution plans is heavy and adds much overhead * both -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2773) Speed up creation of JCR sessions
[ https://issues.apache.org/jira/browse/OAK-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Przemo Pakulski updated OAK-2773: - Summary: Speed up creation of JCR sessions (was: Speedup creation of JCR session) > Speed up creation of JCR sessions > - > > Key: OAK-2773 > URL: https://issues.apache.org/jira/browse/OAK-2773 > Project: Jackrabbit Oak > Issue Type: Wish > Components: core >Affects Versions: 1.0.12, 1.2 >Reporter: Przemo Pakulski > Labels: performance > > JCR sessions used to be lightweight objects and chip to create in > Jackrabbit2/CRX. > I noticed that in Oak creation of session is significantly more expensive. On > my environment creating Oak sessions is ~10 times slower than in crx2. > The most of the time is spent in PrincipalProviderImpl class which executes > many queries. > Possible solutions are: > * introduce MembershipCache as it was in Jackrabbit2 > * speed up the queries, actually the queries itself are pretty fast, > calculation of execution plans is heavy and adds much overhead > * both -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2731) NPE when calling Event.getInfo()
[ https://issues.apache.org/jira/browse/OAK-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2731: --- Fix Version/s: 1.3.0 > NPE when calling Event.getInfo() > > > Key: OAK-2731 > URL: https://issues.apache.org/jira/browse/OAK-2731 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core >Affects Versions: 1.1.6 >Reporter: Dominique Pfister > Fix For: 1.3.0 > > Attachments: OAK-2731.txt > > > On a very busy site, we're observing an NPE in the code that should gather > information about a JCR event for our custom event handler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2731) NPE when calling Event.getInfo()
[
https://issues.apache.org/jira/browse/OAK-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496052#comment-14496052
]
Timothee Maret commented on OAK-2731:
-
Other observations from debugging it
{code}
org.apache.jackrabbit.oak.jcr.observation.EventFactory#nodeReordered
primaryType = null
mixins = empty iterable
parent = /content/dam/home/users/0/0y3nN2ysAP8bVU4IUFdZ/files
name = untitled folder 3
identifier = ee22e405-1598-442f-a0d3-422d7af8e530/untitled folder 3
destName = untitled folder 2
org.apache.jackrabbit.oak.jcr.observation.QueueingHandler#nodeReordered
destName = untitled folder 3
name = untitled folder 4
reordered = {N/A} (empty node state)
org.apache.jackrabbit.oak.plugins.observation.FilteredHandler#nodeReordered
handler = QueueingHandler
filter = org.apache.jackrabbit.oak.plugins.observation.filter.Filters$3
{code}
The QueueingHandler receives an empty node state. This may have been filtered
out in the filter.
> NPE when calling Event.getInfo()
>
>
> Key: OAK-2731
> URL: https://issues.apache.org/jira/browse/OAK-2731
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.1.6
>Reporter: Dominique Pfister
> Attachments: OAK-2731.txt
>
>
> On a very busy site, we're observing an NPE in the code that should gather
> information about a JCR event for our custom event handler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2749) Provide a "different lane" for slow indexers in async indexing
[
https://issues.apache.org/jira/browse/OAK-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496034#comment-14496034
]
Chetan Mehrotra commented on OAK-2749:
--
{quote}
I forced a new executor so that it would be sure the "slow" part won't
affect the "fast" one. Didn't get how to add a new scheduled job
yet. Do you mind to provide it against my fork (pull request?)? I've
just merged the latest trunk in it.
{quote}
Did not still get it. Executor is backed by a pool. So you simply call
WhiteboardUtils.scheduleWithFixedDelay multiple times with different runnables
and they would not conflict with each other
{quote}
True. Didn't think about it. I think it's more a task that could be
taken care by some groovy script in the oak shell. Otherwise we'll
need some commit hook and in case of changes on the async aspect
will take care of the checkpoints.
{quote}
Or we can simply expose a JMX opertaion to perform this sort of migration
> Provide a "different lane" for slow indexers in async indexing
> --
>
> Key: OAK-2749
> URL: https://issues.apache.org/jira/browse/OAK-2749
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Fix For: 1.3.0
>
> Attachments: OAK-2749-rc1.diff
>
>
> In case of big repositories, asynchronous index like Lucene Property,
> could lag behind as slow indexes, for example Full Text, are taken
> care in the same thread pool.
> Provide a separate thread pool in which such indexes could be
> registered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2749) Provide a "different lane" for slow indexers in async indexing
[
https://issues.apache.org/jira/browse/OAK-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496015#comment-14496015
]
Davide Giannella commented on OAK-2749:
---
[~chetanm]
{quote}
Did not understood about forcing new executor requirement. I think we just need
to add a new scheduled job (possibly provide a config to get such a job enabled)
{quote}
I forced a new executor so that it would be sure the "slow" part won't
affect the "fast" one. Didn't get how to add a new scheduled job
yet. Do you mind to provide it against my fork (pull request?)? I've
just merged the latest trunk in it.
{quote}
Further we would also need to take care on how to do the switch of an existing
index. Alex Parvulescu mentioned that if we start a new async index we would
need to take care of checkpoint state also. So we switch an existing index to
say 'async-slow' then we would also need to bootstrap the checkpoint state
otherwise it would trigger a full reindex.
{quote}
True. Didn't think about it. I think it's more a task that could be
taken care by some groovy script in the oak shell. Otherwise we'll
need some commit hook and in case of changes on the {{async}} aspect
will take care of the checkpoints.
{quote}
Also I am not sure if we would should allow such task to run in parallel or
ensure at a time one indexer is running (so as to not put load)
{quote}
As far as I understood, one indexer running at time is the current
implementation. That's why we could experience some delays on updating
the async indexes as in big repositories with frequent updates and big
documents the full-text could take some time to index.
Running the full-text on a separate thread in parallel will allow the
propery indexes (Lucene etc) to complete faster. The side effect is of
course more load on the system but I don't have any meaning on how to
quantify it now.
> Provide a "different lane" for slow indexers in async indexing
> --
>
> Key: OAK-2749
> URL: https://issues.apache.org/jira/browse/OAK-2749
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Fix For: 1.3.0
>
> Attachments: OAK-2749-rc1.diff
>
>
> In case of big repositories, asynchronous index like Lucene Property,
> could lag behind as slow indexes, for example Full Text, are taken
> care in the same thread pool.
> Provide a separate thread pool in which such indexes could be
> registered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-2752: Attachment: OAK-2752-tm2.patch OAK-2752-tm2.patch: new patch, with more tests, less changes, and an additional bug fixed. > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-tm2.patch, OAK-2752-v2.patch, OAK-2752-v3.patch, > OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2771) Longevity tests in oak-run
[ https://issues.apache.org/jira/browse/OAK-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496001#comment-14496001 ] Michael Dürig commented on OAK-2771: Sounds like a good idea. I think we have already a few candidates for such tests. I'd be happy to pilot with my work on OAK-2713 once something is ready. > Longevity tests in oak-run > -- > > Key: OAK-2771 > URL: https://issues.apache.org/jira/browse/OAK-2771 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run >Reporter: Alex Parvulescu >Priority: Minor > > How about adding a new category of tests in oak-run related to longevity > testing. Around the SegmentMk bits I've seen emerge a new type of tests that > setup an infinite sequence of operations waiting for a specific problem to > surface and if the error doesn't happen, we have a never-ending test. > Moreover running an IT test for a few hours is not reasonable on any of the > current build platforms we have now. > My proposal would be to add some of these tests to oak-run, provide a common > way to instrument them and possibly expose some simple (http) UI to stop them > once we think the results are satisfactory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2772) Make Event Subjects Available in All Cases
[
https://issues.apache.org/jira/browse/OAK-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496000#comment-14496000
]
Michael Dürig commented on OAK-2772:
Commits are merge when the observation commit queue becomes full as means of
graceful degradation. If increasing the queue is not an option we need to
investigate in how to apply back pressure to the system. Some initial support
is already there. See {{CommitRateLimiter}}.
OTOH, the userId - which pertains to the commit boundary - is never available
for cluster external changes. For such changes we cannot retain the commit
boundary without changing Oak's underlying consistency model. Oak is
sequentially consistent, which is slightly weaker then linearisable and which
allows us to achieve horizontal scalability.
> Make Event Subjects Available in All Cases
> --
>
> Key: OAK-2772
> URL: https://issues.apache.org/jira/browse/OAK-2772
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.2
>Reporter: Dominique Jäggi
>
> as per SLING-4624 subject support is needed for repository events in order to
> overcome the use of loginAdministrative calls which is a major security
> concern.
> currently oak cannot guarantee availability of subject (userid) for
> propagated events as commits may be merged under certain conditions and the
> information is thus lost.
> we should explore whether and how this issue can be resolved.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495996#comment-14495996 ] Michael Dürig commented on OAK-2752: The duplicate instances indeed created problems. See OAK-2662. But this should be fixed now. > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2713) High memory usage of CompactionMap
[
https://issues.apache.org/jira/browse/OAK-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495993#comment-14495993
]
Michael Dürig commented on OAK-2713:
Right, \[0] is the actual fix. The rest is auxiliary as you noted.
I'd rather keeps this together here as all of the commits relate to the ongoing
work on this issue. We can move {{SegmentCompactionIT}} as soon as we have
OAK-2771 ready.
> High memory usage of CompactionMap
> --
>
> Key: OAK-2713
> URL: https://issues.apache.org/jira/browse/OAK-2713
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: compaction, gc
> Fix For: 1.2.1
>
>
> In environments with a lot of volatile content the {{CompactionMap}} can end
> up eating a lot of memory. From
> {{CompactionStrategyMBean#getCompactionMapStats}}:
> {noformat}
> [Estimated Weight: 317,5 MB, Records: 39500094, Segments: 36698],
> [Estimated Weight: 316,4 MB, Records: 39374593, Segments: 36660],
> [Estimated Weight: 315,4 MB, Records: 39253205, Segments: 36620],
> [Estimated Weight: 315,1 MB, Records: 39221882, Segments: 36614],
> [Estimated Weight: 314,9 MB, Records: 39195490, Segments: 36604],
> [Estimated Weight: 315,0 MB, Records: 39182753, Segments: 36602],
> [Estimated Weight: 360 B, Records: 0, Segments: 0],
> {noformat}
> This causes compaction to be skipped:
> {noformat}
> 2015-03-30:30.03.2015 02:00:00.038 *INFO* [] [TarMK compaction thread
> [/foo/bar/crx-quickstart/repository/segmentstore], active since Mon Mar 30
> 02:00:00 CEST 2015, previous max duration 3854982ms]
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore Not enough available
> memory 5,5 GB, needed 6,3 GB, last merge delta 1,3 GB, so skipping compaction
> for now
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495991#comment-14495991 ] Thomas Mueller commented on OAK-2752: - While writing test, I think I found a second issue, with the (relatively new) method clearSegmentIdTables. After calling this method, two different instances for the same segment id can be created. The problem is that clearSegmentIdTables _removes_ the reference, and not just _clears_ the reference. The refresh() method is then called, however it does not recreate the internal map, because there were only hash conflicts but no empty references. > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2732) NPE in lucene search
[
https://issues.apache.org/jira/browse/OAK-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-2732.
--
Resolution: Fixed
Applied the patch
* trunk - http://svn.apache.org/r1673695
* 1.0 - http://svn.apache.org/r1673714
* 1.2 - http://svn.apache.org/r1673698
> NPE in lucene search
>
>
> Key: OAK-2732
> URL: https://issues.apache.org/jira/browse/OAK-2732
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene, query
>Reporter: Shashank Gupta
>Assignee: Chetan Mehrotra
> Fix For: 1.0.13, 1.3.0, 1.2.1
>
> Attachments: OAK-2732.patch, error.log, query-npe.log
>
>
> REST invocation [1] result in NPE .
> [1]http://localhost:4502/bin/security/authorizables.json?_charset_=utf-8&filter=dam-user1-1&ml=0&limit=25
> {code}
>
> java.lang.NullPointerException
>at
> org.apache.lucene.util.automaton.CompiledAutomaton.getTermsEnum(CompiledAutomaton.java:243)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.extractMatchingTokens(LuceneIndex.java:901)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.tokenToQuery(LuceneIndex.java:870)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visitTerm(LuceneIndex.java:828)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visit(LuceneIndex.java:820)
>at
> org.apache.jackrabbit.oak.query.fulltext.FullTextTerm.accept(FullTextTerm.java:215)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visit(LuceneIndex.java:780)
>at
> org.apache.jackrabbit.oak.query.fulltext.FullTextContains.accept(FullTextContains.java:63)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.getFullTextQuery(LuceneIndex.java:776)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.getLuceneRequest(LuceneIndex.java:509)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.access$100(LuceneIndex.java:155)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.loadDocs(LuceneIndex.java:344)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:292)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:283)
>at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor$1.hasNext(LuceneIndex.java:1056)
>at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
>at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>at
> org.apache.jackrabbit.oak.spi.query.Cursors$PathCursor.hasNext(Cursors.java:198)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor.hasNext(LuceneIndex.java:1077)
>at
> org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.fetchNext(AggregationCursor.java:88)
>at
> org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.hasNext(AggregationCursor.java:75)
>at
> org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:401)
>at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:664)
>at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:689)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:203)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:237)
>at
> com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:137)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:151)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:203)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:237)
>at
> com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
>at
> org.apache.jackrabbi
[jira] [Commented] (OAK-2713) High memory usage of CompactionMap
[
https://issues.apache.org/jira/browse/OAK-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495990#comment-14495990
]
Alex Parvulescu commented on OAK-2713:
--
if I get this correctly, only a single commit has the actual fix [0], the rest
of them can be classified as either better stats (the depth and the jmx
AnnotatedStandardMBean thing) or the infinite test.
does it make sense to break this up a bit? or possibly make the _
SegmentCompactionIT_ a subtask of OAK-2771 and commit everything else together.
either way +1, patch looks good to me.
[0]
https://github.com/mduerig/jackrabbit-oak/commit/284b9fc633de0ae8eab2a140c2501853b3366142
> High memory usage of CompactionMap
> --
>
> Key: OAK-2713
> URL: https://issues.apache.org/jira/browse/OAK-2713
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: compaction, gc
> Fix For: 1.2.1
>
>
> In environments with a lot of volatile content the {{CompactionMap}} can end
> up eating a lot of memory. From
> {{CompactionStrategyMBean#getCompactionMapStats}}:
> {noformat}
> [Estimated Weight: 317,5 MB, Records: 39500094, Segments: 36698],
> [Estimated Weight: 316,4 MB, Records: 39374593, Segments: 36660],
> [Estimated Weight: 315,4 MB, Records: 39253205, Segments: 36620],
> [Estimated Weight: 315,1 MB, Records: 39221882, Segments: 36614],
> [Estimated Weight: 314,9 MB, Records: 39195490, Segments: 36604],
> [Estimated Weight: 315,0 MB, Records: 39182753, Segments: 36602],
> [Estimated Weight: 360 B, Records: 0, Segments: 0],
> {noformat}
> This causes compaction to be skipped:
> {noformat}
> 2015-03-30:30.03.2015 02:00:00.038 *INFO* [] [TarMK compaction thread
> [/foo/bar/crx-quickstart/repository/segmentstore], active since Mon Mar 30
> 02:00:00 CEST 2015, previous max duration 3854982ms]
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore Not enough available
> memory 5,5 GB, needed 6,3 GB, last merge delta 1,3 GB, so skipping compaction
> for now
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2731) NPE when calling Event.getInfo()
[
https://issues.apache.org/jira/browse/OAK-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495982#comment-14495982
]
Timothee Maret commented on OAK-2731:
-
Looking at the stack trace it seems the NPE is throw when trying to create an
info map without mixin types and without primary type (null).
{code}
return ImmutableMap.of(JCR_PRIMARYTYPE, mapper.getJcrName(primaryType)); //
L193
{code}
> NPE when calling Event.getInfo()
>
>
> Key: OAK-2731
> URL: https://issues.apache.org/jira/browse/OAK-2731
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.1.6
>Reporter: Dominique Pfister
> Attachments: OAK-2731.txt
>
>
> On a very busy site, we're observing an NPE in the code that should gather
> information about a JCR event for our custom event handler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495964#comment-14495964 ] Michael Dürig commented on OAK-2752: FTR, I just hit this issue during a compaction cycle in my work on OAK-2713. Compaction would never finish because of this. > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2772) Make Event Subjects Available in All Cases
[ https://issues.apache.org/jira/browse/OAK-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495931#comment-14495931 ] Dominique Jäggi commented on OAK-2772: -- added SLING-4624 > Make Event Subjects Available in All Cases > -- > > Key: OAK-2772 > URL: https://issues.apache.org/jira/browse/OAK-2772 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Affects Versions: 1.2 >Reporter: Dominique Jäggi > > as per SLING-4624 subject support is needed for repository events in order to > overcome the use of loginAdministrative calls which is a major security > concern. > currently oak cannot guarantee availability of subject (userid) for > propagated events as commits may be merged under certain conditions and the > information is thus lost. > we should explore whether and how this issue can be resolved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2772) Make Event Subjects Available in All Cases
Dominique Jäggi created OAK-2772: Summary: Make Event Subjects Available in All Cases Key: OAK-2772 URL: https://issues.apache.org/jira/browse/OAK-2772 Project: Jackrabbit Oak Issue Type: Improvement Components: core Affects Versions: 1.2 Reporter: Dominique Jäggi as per SLING-4624 subject support is needed for repository events in order to overcome the use of loginAdministrative calls which is a major security concern. currently oak cannot guarantee availability of subject (userid) for propagated events as commits may be merged under certain conditions and the information is thus lost. we should explore whether and how this issue can be resolved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2771) Longevity tests in oak-run
[ https://issues.apache.org/jira/browse/OAK-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495924#comment-14495924 ] Alex Parvulescu commented on OAK-2771: -- I would like to propose using OAK-2713 as a pilot for setting up the base, and of course this should be kept as simple as possible. [~mduerig] thoughts? > Longevity tests in oak-run > -- > > Key: OAK-2771 > URL: https://issues.apache.org/jira/browse/OAK-2771 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run >Reporter: Alex Parvulescu >Priority: Minor > > How about adding a new category of tests in oak-run related to longevity > testing. Around the SegmentMk bits I've seen emerge a new type of tests that > setup an infinite sequence of operations waiting for a specific problem to > surface and if the error doesn't happen, we have a never-ending test. > Moreover running an IT test for a few hours is not reasonable on any of the > current build platforms we have now. > My proposal would be to add some of these tests to oak-run, provide a common > way to instrument them and possibly expose some simple (http) UI to stop them > once we think the results are satisfactory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2771) Longevity tests in oak-run
Alex Parvulescu created OAK-2771: Summary: Longevity tests in oak-run Key: OAK-2771 URL: https://issues.apache.org/jira/browse/OAK-2771 Project: Jackrabbit Oak Issue Type: Improvement Components: run Reporter: Alex Parvulescu Priority: Minor How about adding a new category of tests in oak-run related to longevity testing. Around the SegmentMk bits I've seen emerge a new type of tests that setup an infinite sequence of operations waiting for a specific problem to surface and if the error doesn't happen, we have a never-ending test. Moreover running an IT test for a few hours is not reasonable on any of the current build platforms we have now. My proposal would be to add some of these tests to oak-run, provide a common way to instrument them and possibly expose some simple (http) UI to stop them once we think the results are satisfactory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2770) Configurable mode for backgroundOperationLock
[ https://issues.apache.org/jira/browse/OAK-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2770: -- Fix Version/s: 1.2.1 Merged into 1.2 branch: http://svn.apache.org/r1673684 > Configurable mode for backgroundOperationLock > - > > Key: OAK-2770 > URL: https://issues.apache.org/jira/browse/OAK-2770 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: core, mongomk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.3.0, 1.2.1 > > > In a first step I would like to make the mode configurable. This allows to > run tests with both fair and non-fair mode and compare results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2766) Log time to acquire backgroundOperationLock in background operation tasks
[ https://issues.apache.org/jira/browse/OAK-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2766: -- Fix Version/s: 1.2.1 Merged into 1.2: http://svn.apache.org/r1673683 > Log time to acquire backgroundOperationLock in background operation tasks > - > > Key: OAK-2766 > URL: https://issues.apache.org/jira/browse/OAK-2766 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, mongomk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.3.0, 1.2.1 > > > The ReadStats has a dispatch time, which also includes the time it takes to > acquire the backgroundOperationLock. These should be separated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2762) Configurable maxLockTryTimeMS
[ https://issues.apache.org/jira/browse/OAK-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2762: -- Fix Version/s: 1.2.1 Merged to 1.2: http://svn.apache.org/r1673681 > Configurable maxLockTryTimeMS > - > > Key: OAK-2762 > URL: https://issues.apache.org/jira/browse/OAK-2762 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, mongomk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.3.0, 1.2.1 > > > OAK-2127 introduced a maxLockTryTimeMS to allow a writer to proceed with a > merge even if the merge lock is currently acquired by another thread. The > value is currently hardcoded. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495867#comment-14495867 ] Alex Parvulescu commented on OAK-2752: -- bq. I think the fast tests (all of them?) should not be IT tests, so we find problems earlier. I fully agree, I wasn't sure initially how long the test will take to find the mentioned issues (see other heavy IT tests around the segment bits). now the tests look fast enough to be run as unit tests, so feel free to rename the test class (remove the 'IT'). > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495859#comment-14495859 ] Michael Dürig commented on OAK-2752: Could you add some tests then? > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2656) Test failures in LDAP authentication: Failed to bind an LDAP service
[
https://issues.apache.org/jira/browse/OAK-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-2656:
---
Description:
The following tests all fail with the same error message "Failed to bind an
LDAP service (1024) to the service registry.".
{noformat}
testAuthenticateFail(org.apache.jackrabbit.oak.security.authentication.ldap.LdapProviderTest):
Failed to bind an LDAP service (1024) to the service registry.
testGetGroups2(org.apache.jackrabbit.oak.security.authentication.ldap.LdapProviderTest):
Failed to bind an LDAP service (1024) to the service registry.
org.apache.jackrabbit.oak.security.authentication.ldap.LdapDefaultLoginModuleTest:
Failed to bind an LDAP service (1024) to the service registry.
testGetUserByUserId(org.apache.jackrabbit.oak.security.authentication.ldap.LdapProviderTest):
Failed to bind an LDAP service (1024) to the service registry.
{noformat}
The stacktrace is always similar:
{noformat}
java.net.BindException: Address already in use]
at
org.apache.directory.server.ldap.LdapServer.startNetwork(LdapServer.java:528)
at
org.apache.directory.server.ldap.LdapServer.start(LdapServer.java:394)
at
org.apache.directory.server.unit.AbstractServerTest.setUp(AbstractServerTest.java:273)
at
org.apache.jackrabbit.oak.security.authentication.ldap.InternalLdapServer.setUp(InternalLdapServer.java:37)
at
org.apache.jackrabbit.oak.security.authentication.ldap.LdapLoginTestBase.beforeClass(LdapLoginTestBase.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:444)
at sun.nio.ch.Net.bind(Net.java:436)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at
org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:198)
at
org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:51)
at
org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:547)
at
org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$400(AbstractPollingIoAcceptor.java:68)
at
org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:422)
at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{noformat}
Failure seen at builds: 31, 40, 52, 62, 63, 64, 72, 95, 103
See
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/31/jdk=latest1.7,label=Ubuntu,nsfixtures=DOCUMENT_NS,profile=unittesting/
https://
[jira] [Updated] (OAK-2714) Test failures on Jenkins
[ https://issues.apache.org/jira/browse/OAK-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2714: --- Description: This issue is for tracking test failures seen at our Jenkins instance that might yet be transient. Once a failure happens too often we should remove it here and create a dedicated issue for it. || Test || Builds || Fixture || JVM || | org.apache.jackrabbit.oak.plugins.index.solr.configuration.DefaultAnalyzersConfigurationTest | 61, 63, 92, 94, 103 | DOCUMENT_RDB | 1.8 | | org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopierTest.reuseLocalDir | 81 | DOCUMENT_RDB | 1.7 | | org.apache.jackrabbit.oak.jcr.OrderableNodesTest.orderableFolder | 81, 87, 92, 95, 96 | DOCUMENT_NS, DOCUMENT_RDB (2) | 1.6, 1.7 | | org.apache.jackrabbit.oak.jcr.OrderedIndexIT.oak2035 | 76 | SEGMENT_MK | 1.6 | | org.apache.jackrabbit.oak.jcr.OrderableNodesTest.setPrimaryType | 69, 83, 97 | DOCUMENT_RDB | 1.6 | | org.apache.jackrabbit.oak.plugins.segment.standby.StandbyTestIT.testSyncLoop | 64 | ?| ? | | org.apache.jackrabbit.oak.jcr.observation.ObservationRefreshTest.observation | 48, 55 | ?| ? | | org.apache.jackrabbit.oak.run.osgi.DocumentNodeStoreConfigTest.testRDBDocumentStore_CustomBlobStore | 52 | ? | ? | | org.apache.jackrabbit.oak.run.osgi.JsonConfigRepFactoryTest.testRepositoryTar | 41 | ?| ? | | org.apache.jackrabbit.oak.jcr.AutoCreatedItemsTest.autoCreatedItems | 41, 88 | DOCUMENT_RDB | 1.7 | | org.apache.jackrabbit.test.api.observation.PropertyAddedTest.testMultiPropertyAdded | 29 | ?| ? | | org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.heavyWrite | 35 | SEGMENT_MK | ? | | org.apache.jackrabbit.oak.jcr.query.QueryPlanTest.propertyIndexVersusNodeTypeIndex | 90 | DOCUMENT_RDB | 1.6 | | org.apache.jackrabbit.oak.jcr.observation.ObservationTest.removeSubtreeFilter | 94 | DOCUMENT_RDB | 1.6 | was: This issue is for tracking test failures seen at our Jenkins instance that might yet be transient. Once a failure happens too often we should remove it here and create a dedicated issue for it. || Test || Builds || Fixture || JVM || | org.apache.jackrabbit.oak.plugins.index.solr.configuration.DefaultAnalyzersConfigurationTest | 61, 63, 92, 94 | ?| ? | | org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopierTest.reuseLocalDir | 81 | DOCUMENT_RDB | 1.7 | | org.apache.jackrabbit.oak.jcr.OrderableNodesTest.orderableFolder | 81, 87, 92, 95, 96 | DOCUMENT_NS, DOCUMENT_RDB (2) | 1.6, 1.7 | | org.apache.jackrabbit.oak.jcr.OrderedIndexIT.oak2035 | 76 | SEGMENT_MK | 1.6 | | org.apache.jackrabbit.oak.jcr.OrderableNodesTest.setPrimaryType | 69, 83, 97 | DOCUMENT_RDB | 1.6 | | org.apache.jackrabbit.oak.plugins.segment.standby.StandbyTestIT.testSyncLoop | 64 | ?| ? | | org.apache.jackrabbit.oak.jcr.observation.ObservationRefreshTest.observation | 48, 55 | ?| ? | | org.apache.jackrabbit.oak.run.osgi.DocumentNodeStoreConfigTest.testRDBDocumentStore_CustomBlobStore | 52 | ? | ? | | org.apache.jackrabbit.oak.run.osgi.JsonConfigRepFactoryTest.testRepositoryTar | 41 | ?| ? | | org.apache.jackrabbit.oak.jcr.AutoCreatedItemsTest.autoCreatedItems | 41, 88 | DOCUMENT_RDB | 1.7 | | org.apache.jackrabbit.test.api.observation.PropertyAddedTest.testMultiPropertyAdded | 29 | ?| ? | | org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.heavyWrite | 35 | SEGMENT_MK | ? | | org.apache.jackrabbit.oak.jcr.query.QueryPlanTest.propertyIndexVersusNodeTypeIndex | 90 | DOCUMENT_RDB | 1.6 | | org.apache.jackrabbit.oak.jcr.observation.ObservationTest.removeSubtreeFilter | 94 | DOCUMENT_RDB | 1.6 | > Test failures on Jenkins > > > Key: OAK-2714 > URL: https://issues.apache.org/jira/browse/OAK-2714 > Project: Jackrabbit Oak > Issue Type: Bug > Environment: Jenkins, Ubuntu: > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/ >Reporter: Michael Dürig > Labels: CI, Jenkins > Fix Fo
[jira] [Commented] (OAK-2752) SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb)
[ https://issues.apache.org/jira/browse/OAK-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495857#comment-14495857 ] Thomas Mueller commented on OAK-2752: - In any case, I think we need more tests, because this is quite a central class. Any bug here is very problematic, and it's easy to make mistakes here (bugs that can affect performance a lot, and bugs that can break things in very bad ways, for example having two segment id objects for the same segment). I like the testing approach taken by Alex, however I think the fast tests (all of them?) should not be IT tests, so we find problems earlier. > we can save bigger refactoring endeavours for later on. I agree my patch is big. I'm trying to write a smaller patch. > SegmentIdTable can sometimes hang when calling getSegmentId(msb, lsb) > - > > Key: OAK-2752 > URL: https://issues.apache.org/jira/browse/OAK-2752 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-core >Affects Versions: 1.2 >Reporter: Will McGauley >Assignee: Thomas Mueller >Priority: Critical > Fix For: 1.0.13, 1.2.1 > > Attachments: OAK-2752-md.patch, OAK-2752-md2.patch, > OAK-2752-tm1.patch, OAK-2752-v2.patch, OAK-2752-v3.patch, OAK-2752.patch > > > The while loop getSegmentId(msb, lsb) loops forever in the situation where > the segment id is not found. > Looping occurs from 'first' and loops to the end of the SegmentId references, > and then loops back to first. If the segmentid is not found in any of the > referenced items then looping continues past 'first' again and loops for > eternity through all the references. > See attached patch for possible fix to break out of the loop after getting > back to 'first' again. > note: I have tested this patch on my system and it seems to work, but I do > not know if the patch provides the best fix, I am a bit new to this code. > The most important part of the patch would be the break condition from the > loop so the loop does not continue after testing each reference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2662) SegmentOverflowException in HeavyWriteIT on Jenkins
[
https://issues.apache.org/jira/browse/OAK-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-2662.
Resolution: Fixed
> SegmentOverflowException in HeavyWriteIT on Jenkins
> ---
>
> Key: OAK-2662
> URL: https://issues.apache.org/jira/browse/OAK-2662
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: CI, jenkins
> Fix For: 1.3.0, 1.2.1
>
> Attachments: OAK-2662.patch
>
>
> {noformat}
> heavyWrite(org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT) Time
> elapsed: 96.384 sec <<< ERROR!
> org.apache.jackrabbit.oak.plugins.segment.SegmentOverflowException: Segment
> cannot have more than 255 references 47a9dc3c-c6f9-4b5f-a61a-6711da8b68c2
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.getSegmentRef(SegmentWriter.java:353)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeRecordId(SegmentWriter.java:382)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapLeaf(SegmentWriter.java:426)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:484)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:511)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMap(SegmentWriter.java:720)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1108)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1091)
> at
> org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1082)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1110)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:97)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:83)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.updated(MemoryNodeBuilder.java:214)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:79)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:501)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:507)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createProperties(HeavyWriteIT.java:137)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:129)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:130)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.heavyWrite(HeavyWriteIT.java:110)
> {noformat}
> See
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/35/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/
> cc [~alex.parvulescu]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2723) FileStore does not scale because of precomputed graph on TarReader
[
https://issues.apache.org/jira/browse/OAK-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495852#comment-14495852
]
Michael Dürig commented on OAK-2723:
Fixed in 1.2 at http://svn.apache.org/r1673673
> FileStore does not scale because of precomputed graph on TarReader
> --
>
> Key: OAK-2723
> URL: https://issues.apache.org/jira/browse/OAK-2723
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.1.8
>Reporter: Andrei Dulvac
>Assignee: Michael Dürig
>Priority: Critical
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 0001-TarReader-fix-for-precomputed-graph.patch
>
>
> The {{FileStore}} keeps a reference to all {{TarReader}} object, one per each
> file. In my test, for an ~350 Gb repository, that was ~1100 tar files, with a
> {{TarReader}} for each.
> The problem is {{TarReader}} keeps a reference to a precomputed _graph_
> {{ByteBuffer}}, which is not really used that much. That means that through
> the {{readers}} field, there's a reference to these _graphs_, which means
> they can't be GC'ed.
> The construction of {{FileStore}} is from oak-run:
> bq. FileStore store = new FileStore(directory, 256,
> TAR_STORAGE_MEMORY_MAPPED);
> The effect is you need more that 6GB of Ram just to instantiate the
> {{FileStore}} object.
> The attached patch fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-2723) FileStore does not scale because of precomputed graph on TarReader
[
https://issues.apache.org/jira/browse/OAK-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-2723.
Resolution: Fixed
> FileStore does not scale because of precomputed graph on TarReader
> --
>
> Key: OAK-2723
> URL: https://issues.apache.org/jira/browse/OAK-2723
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.1.8
>Reporter: Andrei Dulvac
>Assignee: Michael Dürig
>Priority: Critical
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 0001-TarReader-fix-for-precomputed-graph.patch
>
>
> The {{FileStore}} keeps a reference to all {{TarReader}} object, one per each
> file. In my test, for an ~350 Gb repository, that was ~1100 tar files, with a
> {{TarReader}} for each.
> The problem is {{TarReader}} keeps a reference to a precomputed _graph_
> {{ByteBuffer}}, which is not really used that much. That means that through
> the {{readers}} field, there's a reference to these _graphs_, which means
> they can't be GC'ed.
> The construction of {{FileStore}} is from oak-run:
> bq. FileStore store = new FileStore(directory, 256,
> TAR_STORAGE_MEMORY_MAPPED);
> The effect is you need more that 6GB of Ram just to instantiate the
> {{FileStore}} object.
> The attached patch fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2662) SegmentOverflowException in HeavyWriteIT on Jenkins
[
https://issues.apache.org/jira/browse/OAK-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495851#comment-14495851
]
Michael Dürig commented on OAK-2662:
Fixed in 1.2 at http://svn.apache.org/r1673671
> SegmentOverflowException in HeavyWriteIT on Jenkins
> ---
>
> Key: OAK-2662
> URL: https://issues.apache.org/jira/browse/OAK-2662
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: CI, jenkins
> Fix For: 1.3.0, 1.2.1
>
> Attachments: OAK-2662.patch
>
>
> {noformat}
> heavyWrite(org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT) Time
> elapsed: 96.384 sec <<< ERROR!
> org.apache.jackrabbit.oak.plugins.segment.SegmentOverflowException: Segment
> cannot have more than 255 references 47a9dc3c-c6f9-4b5f-a61a-6711da8b68c2
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.getSegmentRef(SegmentWriter.java:353)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeRecordId(SegmentWriter.java:382)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapLeaf(SegmentWriter.java:426)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:484)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:511)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMap(SegmentWriter.java:720)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1108)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1091)
> at
> org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1082)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1110)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:97)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:83)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.updated(MemoryNodeBuilder.java:214)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:79)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:501)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:507)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createProperties(HeavyWriteIT.java:137)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:129)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.createNodes(HeavyWriteIT.java:130)
> at
> org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.heavyWrite(HeavyWriteIT.java:110)
> {noformat}
> See
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/35/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/
> cc [~alex.parvulescu]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-2770) Configurable mode for backgroundOperationLock
[
https://issues.apache.org/jira/browse/OAK-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger resolved OAK-2770.
---
Resolution: Fixed
Added a system property to control mode.
Use {{-Doak.fairBackgroundOperationLock=true}} to enable fair mode.
Done in trunk: http://svn.apache.org/r1673669
> Configurable mode for backgroundOperationLock
> -
>
> Key: OAK-2770
> URL: https://issues.apache.org/jira/browse/OAK-2770
> Project: Jackrabbit Oak
> Issue Type: Sub-task
> Components: core, mongomk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
> Fix For: 1.3.0
>
>
> In a first step I would like to make the mode configurable. This allows to
> run tests with both fair and non-fair mode and compare results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2732) NPE in lucene search
[
https://issues.apache.org/jira/browse/OAK-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495825#comment-14495825
]
Alex Parvulescu commented on OAK-2732:
--
+1 looks good
> NPE in lucene search
>
>
> Key: OAK-2732
> URL: https://issues.apache.org/jira/browse/OAK-2732
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene, query
>Reporter: Shashank Gupta
>Assignee: Chetan Mehrotra
> Fix For: 1.0.13, 1.3.0, 1.2.1
>
> Attachments: OAK-2732.patch, error.log, query-npe.log
>
>
> REST invocation [1] result in NPE .
> [1]http://localhost:4502/bin/security/authorizables.json?_charset_=utf-8&filter=dam-user1-1&ml=0&limit=25
> {code}
>
> java.lang.NullPointerException
>at
> org.apache.lucene.util.automaton.CompiledAutomaton.getTermsEnum(CompiledAutomaton.java:243)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.extractMatchingTokens(LuceneIndex.java:901)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.tokenToQuery(LuceneIndex.java:870)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visitTerm(LuceneIndex.java:828)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visit(LuceneIndex.java:820)
>at
> org.apache.jackrabbit.oak.query.fulltext.FullTextTerm.accept(FullTextTerm.java:215)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visit(LuceneIndex.java:780)
>at
> org.apache.jackrabbit.oak.query.fulltext.FullTextContains.accept(FullTextContains.java:63)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.getFullTextQuery(LuceneIndex.java:776)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.getLuceneRequest(LuceneIndex.java:509)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.access$100(LuceneIndex.java:155)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.loadDocs(LuceneIndex.java:344)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:292)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:283)
>at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor$1.hasNext(LuceneIndex.java:1056)
>at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
>at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>at
> org.apache.jackrabbit.oak.spi.query.Cursors$PathCursor.hasNext(Cursors.java:198)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor.hasNext(LuceneIndex.java:1077)
>at
> org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.fetchNext(AggregationCursor.java:88)
>at
> org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.hasNext(AggregationCursor.java:75)
>at
> org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:401)
>at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:664)
>at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:689)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:203)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:237)
>at
> com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:137)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:151)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:203)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:237)
>at
> com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:137)
>
[jira] [Created] (OAK-2770) Configurable mode for backgroundOperationLock
Marcel Reutegger created OAK-2770: - Summary: Configurable mode for backgroundOperationLock Key: OAK-2770 URL: https://issues.apache.org/jira/browse/OAK-2770 Project: Jackrabbit Oak Issue Type: Sub-task Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Fix For: 1.3.0 In a first step I would like to make the mode configurable. This allows to run tests with both fair and non-fair mode and compare results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2732) NPE in lucene search
[
https://issues.apache.org/jira/browse/OAK-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-2732:
-
Attachment: OAK-2732.patch
[attached patch|^OAK-2732.patch] with a testcase and fix
As [~teofili] mentioned if there is no document index with given field name
then Term would be null. As a fix in case of null just return an empty array
On branch currently we use the same logic in LuceneIndex in LucenePropertyIndex
also. So when we merge we would need to fix it there also
[~tmueller][[~alexparvulescu] Can you review the patch
> NPE in lucene search
>
>
> Key: OAK-2732
> URL: https://issues.apache.org/jira/browse/OAK-2732
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene, query
>Reporter: Shashank Gupta
> Fix For: 1.0.13, 1.3.0, 1.2.1
>
> Attachments: OAK-2732.patch, error.log, query-npe.log
>
>
> REST invocation [1] result in NPE .
> [1]http://localhost:4502/bin/security/authorizables.json?_charset_=utf-8&filter=dam-user1-1&ml=0&limit=25
> {code}
>
> java.lang.NullPointerException
>at
> org.apache.lucene.util.automaton.CompiledAutomaton.getTermsEnum(CompiledAutomaton.java:243)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.extractMatchingTokens(LuceneIndex.java:901)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.tokenToQuery(LuceneIndex.java:870)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visitTerm(LuceneIndex.java:828)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visit(LuceneIndex.java:820)
>at
> org.apache.jackrabbit.oak.query.fulltext.FullTextTerm.accept(FullTextTerm.java:215)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visit(LuceneIndex.java:780)
>at
> org.apache.jackrabbit.oak.query.fulltext.FullTextContains.accept(FullTextContains.java:63)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.getFullTextQuery(LuceneIndex.java:776)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.getLuceneRequest(LuceneIndex.java:509)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.access$100(LuceneIndex.java:155)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.loadDocs(LuceneIndex.java:344)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:292)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:283)
>at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor$1.hasNext(LuceneIndex.java:1056)
>at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
>at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>at
> org.apache.jackrabbit.oak.spi.query.Cursors$PathCursor.hasNext(Cursors.java:198)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor.hasNext(LuceneIndex.java:1077)
>at
> org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.fetchNext(AggregationCursor.java:88)
>at
> org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.hasNext(AggregationCursor.java:75)
>at
> org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:401)
>at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:664)
>at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:689)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:203)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:237)
>at
> com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:137)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:151)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:203)
>
[jira] [Assigned] (OAK-2732) NPE in lucene search
[
https://issues.apache.org/jira/browse/OAK-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra reassigned OAK-2732:
Assignee: Chetan Mehrotra
> NPE in lucene search
>
>
> Key: OAK-2732
> URL: https://issues.apache.org/jira/browse/OAK-2732
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene, query
>Reporter: Shashank Gupta
>Assignee: Chetan Mehrotra
> Fix For: 1.0.13, 1.3.0, 1.2.1
>
> Attachments: OAK-2732.patch, error.log, query-npe.log
>
>
> REST invocation [1] result in NPE .
> [1]http://localhost:4502/bin/security/authorizables.json?_charset_=utf-8&filter=dam-user1-1&ml=0&limit=25
> {code}
>
> java.lang.NullPointerException
>at
> org.apache.lucene.util.automaton.CompiledAutomaton.getTermsEnum(CompiledAutomaton.java:243)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.extractMatchingTokens(LuceneIndex.java:901)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.tokenToQuery(LuceneIndex.java:870)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visitTerm(LuceneIndex.java:828)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visit(LuceneIndex.java:820)
>at
> org.apache.jackrabbit.oak.query.fulltext.FullTextTerm.accept(FullTextTerm.java:215)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$3.visit(LuceneIndex.java:780)
>at
> org.apache.jackrabbit.oak.query.fulltext.FullTextContains.accept(FullTextContains.java:63)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.getFullTextQuery(LuceneIndex.java:776)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.getLuceneRequest(LuceneIndex.java:509)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.access$100(LuceneIndex.java:155)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.loadDocs(LuceneIndex.java:344)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:292)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:283)
>at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor$1.hasNext(LuceneIndex.java:1056)
>at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
>at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>at
> org.apache.jackrabbit.oak.spi.query.Cursors$PathCursor.hasNext(Cursors.java:198)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor.hasNext(LuceneIndex.java:1077)
>at
> org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.fetchNext(AggregationCursor.java:88)
>at
> org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.hasNext(AggregationCursor.java:75)
>at
> org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:401)
>at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:664)
>at
> org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:689)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:203)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:237)
>at
> com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:137)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:151)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:203)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:237)
>at
> com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
>at
> org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:137)
>at
> org.apache.jackrabbit.
[jira] [Commented] (OAK-2723) FileStore does not scale because of precomputed graph on TarReader
[
https://issues.apache.org/jira/browse/OAK-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495795#comment-14495795
]
Michael Dürig commented on OAK-2723:
Fixed in trunk: http://svn.apache.org/r1673664
> FileStore does not scale because of precomputed graph on TarReader
> --
>
> Key: OAK-2723
> URL: https://issues.apache.org/jira/browse/OAK-2723
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.1.8
>Reporter: Andrei Dulvac
>Assignee: Michael Dürig
>Priority: Critical
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 0001-TarReader-fix-for-precomputed-graph.patch
>
>
> The {{FileStore}} keeps a reference to all {{TarReader}} object, one per each
> file. In my test, for an ~350 Gb repository, that was ~1100 tar files, with a
> {{TarReader}} for each.
> The problem is {{TarReader}} keeps a reference to a precomputed _graph_
> {{ByteBuffer}}, which is not really used that much. That means that through
> the {{readers}} field, there's a reference to these _graphs_, which means
> they can't be GC'ed.
> The construction of {{FileStore}} is from oak-run:
> bq. FileStore store = new FileStore(directory, 256,
> TAR_STORAGE_MEMORY_MAPPED);
> The effect is you need more that 6GB of Ram just to instantiate the
> {{FileStore}} object.
> The attached patch fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2723) FileStore does not scale because of precomputed graph on TarReader
[
https://issues.apache.org/jira/browse/OAK-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-2723:
---
Fix Version/s: 1.3.0
> FileStore does not scale because of precomputed graph on TarReader
> --
>
> Key: OAK-2723
> URL: https://issues.apache.org/jira/browse/OAK-2723
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-core
>Affects Versions: 1.1.8
>Reporter: Andrei Dulvac
>Assignee: Michael Dürig
>Priority: Critical
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 0001-TarReader-fix-for-precomputed-graph.patch
>
>
> The {{FileStore}} keeps a reference to all {{TarReader}} object, one per each
> file. In my test, for an ~350 Gb repository, that was ~1100 tar files, with a
> {{TarReader}} for each.
> The problem is {{TarReader}} keeps a reference to a precomputed _graph_
> {{ByteBuffer}}, which is not really used that much. That means that through
> the {{readers}} field, there's a reference to these _graphs_, which means
> they can't be GC'ed.
> The construction of {{FileStore}} is from oak-run:
> bq. FileStore store = new FileStore(directory, 256,
> TAR_STORAGE_MEMORY_MAPPED);
> The effect is you need more that 6GB of Ram just to instantiate the
> {{FileStore}} object.
> The attached patch fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
