[jira] [Created] (OAK-4920) Performance: DefaultSyncHandler.listIdentities() search too broad, triggers traversal warning
Alexander Klimetschek created OAK-4920:
--
Summary: Performance: DefaultSyncHandler.listIdentities() search
too broad, triggers traversal warning
Key: OAK-4920
URL: https://issues.apache.org/jira/browse/OAK-4920
Project: Jackrabbit Oak
Issue Type: Improvement
Components: auth-external
Affects Versions: 1.5.11, 1.4.8
Reporter: Alexander Klimetschek
DefaultSyncHandler.listIdentities() collects users by [searching for all nodes
under
/home|https://github.com/apache/jackrabbit-oak/blob/b3e62e3467bf6433b5a419c2f371331f33e57820/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/DefaultSyncHandler.java#L143]
– the xpath query executed is
{{/jcr:root/home//element(\*)\[@jcr:primaryType]}}. With a few hundred users
this easily gives an oak index traversal warning:
{noformat}
org.apache.jackrabbit.oak.spi.query.Cursors$TraversingCursor Traversed 1000
nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [nt:base]
as a where [jcr:primaryType] is not null and isdescendantnode(a, '/home') /*
xpath: /jcr:root/home//element(*)[@jcr:primaryType] */, path=/home//*,
property=[jcr:primaryType=[is not null]]); consider creating an index or
changing the query
{noformat}
However, [later it
reduces|https://github.com/apache/jackrabbit-oak/blob/b3e62e3467bf6433b5a419c2f371331f33e57820/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/DefaultSyncHandler.java#L151]
the result to authorizables which have a {{rep:externalId}}.
Since OAK-4301 there is an oak index for {{rep:externalId}}, so the query can
be optimized by searching for anything with {{rep:externalId}} instead:
{code:java}
userManager.findAuthorizables("rep:externalId", null);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4916) Add support for excluding commits to BackgroundObserver
[
https://issues.apache.org/jira/browse/OAK-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-4916:
-
Attachment: OAK-4916.patch
Attaching [^OAK-4916.patch] which implements the {{isExcluded}} subclass hook
as well as the {{NOOP_CHANGED}} CommitInfo in BackgroundObserver as described,
including test cases.
> Add support for excluding commits to BackgroundObserver
> ---
>
> Key: OAK-4916
> URL: https://issues.apache.org/jira/browse/OAK-4916
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: core
>Affects Versions: 1.5.11
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Fix For: 1.6
>
> Attachments: OAK-4916.patch
>
>
> As part of pre-filtering commits it would be useful to have support in the
> BackgroundObserver (in general) that would allow to exclude certain commits
> from being added to the (BackgroundObserver's) queue, thus keeping the queue
> smaller. The actual filtering is up to subclasses.
> The suggested implementation is as follows:
> * a new method {{isExcluded}} is introduced which represents a subclass hook
> for filtering
> * excluded commits are not added to the queue
> * when multiple commits are excluded subsequently, this is collapsed
> * the first non-excluded commit (ContentChange) added to the queue is marked
> with the last non-excluded root state as the 'previous root'
> * downstream Observers are notified of the exclusion of a commit via a
> special CommitInfo {{NOOP_CHANGE}}: this instructs it to exclude this change
> while at the same time 'fast-forwarding' the root state to the new one.
> ** this extra token is one way of solving the problem that
> {{Observer.contentChanged}} represents a diff between two states but does not
> transport the 'from' state explicitly - that is implicitly taken from the
> previous call to {{contentChanged}}. Thus using such a gap token
> ({{NOOP_CHANGE}}) seems to be the only way to instruct Observers to skip a
> change.
> To repeat: whoever extends BackgroundObserver with filtering must be aware of
> the new {{NOOP_CHANGE}} token. Anyone not doing filtering will not get any
> {{NOOP_CHANGE}} tokens though.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
[
https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu resolved OAK-4917.
--
Resolution: Fixed
thanks [~marett]!
applied the patch at http://svn.apache.org/viewvc?rev=1764144=rev
> DefaultStandbyReferencesReader does not re-attempt to read segment upon
> failure
>
>
> Key: OAK-4917
> URL: https://issues.apache.org/jira/browse/OAK-4917
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.14
>Reporter: Timothee Maret
>Assignee: Timothee Maret
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4917.patch
>
>
> This following logs occurred while running integration testing.
> {code}
> 07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder
> Sending head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.0020 to client
> qastandby1
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get
> references' message
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler
> Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client
> qastandby1
> 07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader
> Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
> 9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285)
> at
> org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
>
[jira] [Updated] (OAK-4913) Interrupt online revision cleanup on segment-tar
[
https://issues.apache.org/jira/browse/OAK-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4913:
---
Labels: management (was: )
> Interrupt online revision cleanup on segment-tar
>
>
> Key: OAK-4913
> URL: https://issues.apache.org/jira/browse/OAK-4913
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Alex Parvulescu
>Assignee: Michael Dürig
> Labels: management
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4913.patch
>
>
> Sub task of OAK-4835 for the {{segment-tar}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4914) Interrupt online revision cleanup on segmentmk
[
https://issues.apache.org/jira/browse/OAK-4914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4914:
---
Labels: management (was: )
> Interrupt online revision cleanup on segmentmk
> --
>
> Key: OAK-4914
> URL: https://issues.apache.org/jira/browse/OAK-4914
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Alex Parvulescu
> Labels: management
>
> Sub task of OAK-4835 for the {{segmentmk}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4915) Interrupt online revision cleanup on documentmk
[
https://issues.apache.org/jira/browse/OAK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4915:
---
Labels: management (was: )
> Interrupt online revision cleanup on documentmk
> ---
>
> Key: OAK-4915
> URL: https://issues.apache.org/jira/browse/OAK-4915
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: documentmk
>Reporter: Alex Parvulescu
> Labels: management
>
> Sub task of OAK-4835 for the {{document}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4835) Provide generic option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562671#comment-15562671 ] Michael Dürig commented on OAK-4835: Filed OAK-4919 for providing better feedback of ongoing gc. > Provide generic option to interrupt online revision cleanup > --- > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc, management, monitoring > Fix For: 1.6, 1.5.13 > > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4835) Provide generic option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4835: --- Labels: compaction gc management monitoring (was: compaction gc) > Provide generic option to interrupt online revision cleanup > --- > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc, management, monitoring > Fix For: 1.6, 1.5.13 > > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4919) Better feedback from method invocations on RevisionGCMBean
[
https://issues.apache.org/jira/browse/OAK-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562667#comment-15562667
]
Michael Dürig commented on OAK-4919:
[~alex.parvulescu] FYI
> Better feedback from method invocations on RevisionGCMBean
> --
>
> Key: OAK-4919
> URL: https://issues.apache.org/jira/browse/OAK-4919
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Michael Dürig
> Labels: monitoring
> Fix For: 1.5.13
>
>
> The methods to invoke and cancel revision gc return void. This is by design
> as those calls are asynchronous. The idea is that
> {{RevisionGC.getRevisionGCStatus()}} would return the current status of an
> ongoing gc operation. However, currently that method only returns the status
> of the asynchronous task that was fired off. It should instead be able to
> convey back the real status of the underlying operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4919) Better feedback from method invocations on RevisionGCMBean
Michael Dürig created OAK-4919:
--
Summary: Better feedback from method invocations on RevisionGCMBean
Key: OAK-4919
URL: https://issues.apache.org/jira/browse/OAK-4919
Project: Jackrabbit Oak
Issue Type: Improvement
Components: core
Reporter: Michael Dürig
Fix For: 1.5.13
The methods to invoke and cancel revision gc return void. This is by design as
those calls are asynchronous. The idea is that
{{RevisionGC.getRevisionGCStatus()}} would return the current status of an
ongoing gc operation. However, currently that method only returns the status of
the asynchronous task that was fired off. It should instead be able to convey
back the real status of the underlying operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
[
https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothee Maret updated OAK-4917:
Flags: Patch
> DefaultStandbyReferencesReader does not re-attempt to read segment upon
> failure
>
>
> Key: OAK-4917
> URL: https://issues.apache.org/jira/browse/OAK-4917
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.14
>Reporter: Timothee Maret
>Assignee: Timothee Maret
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4917.patch
>
>
> This following logs occurred while running integration testing.
> {code}
> 07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder
> Sending head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.0020 to client
> qastandby1
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get
> references' message
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler
> Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client
> qastandby1
> 07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader
> Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
> 9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285)
> at
> org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
>
[jira] [Commented] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
[
https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562611#comment-15562611
]
Timothee Maret commented on OAK-4917:
-
[~alexparvulescu], [~frm], [~mduerig] could you review the patch ?
> DefaultStandbyReferencesReader does not re-attempt to read segment upon
> failure
>
>
> Key: OAK-4917
> URL: https://issues.apache.org/jira/browse/OAK-4917
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.14
>Reporter: Timothee Maret
>Assignee: Timothee Maret
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4917.patch
>
>
> This following logs occurred while running integration testing.
> {code}
> 07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder
> Sending head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.0020 to client
> qastandby1
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get
> references' message
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler
> Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client
> qastandby1
> 07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader
> Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
> 9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285)
> at
> org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
>
[jira] [Updated] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
[
https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothee Maret updated OAK-4917:
Attachment: OAK-4917.patch
Attaching a svn compatible patch otherwise available at
https://github.com/tmaret/jackrabbit-oak/commit/bbe213e5fad7e32aca09aa6a225db6246ea80f10.patch
> DefaultStandbyReferencesReader does not re-attempt to read segment upon
> failure
>
>
> Key: OAK-4917
> URL: https://issues.apache.org/jira/browse/OAK-4917
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.14
>Reporter: Timothee Maret
>Assignee: Timothee Maret
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4917.patch
>
>
> This following logs occurred while running integration testing.
> {code}
> 07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder
> Sending head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.0020 to client
> qastandby1
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get
> references' message
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler
> Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client
> qastandby1
> 07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader
> Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
> 9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285)
> at
> org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
>
[jira] [Created] (OAK-4918) Upgrade Oak dependency to 1.5.12
Michael Dürig created OAK-4918: -- Summary: Upgrade Oak dependency to 1.5.12 Key: OAK-4918 URL: https://issues.apache.org/jira/browse/OAK-4918 Project: Jackrabbit Oak Issue Type: Task Components: segment-tar Reporter: Michael Dürig -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4913) Interrupt online revision cleanup on segment-tar
[
https://issues.apache.org/jira/browse/OAK-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4913:
---
Attachment: OAK-4913.patch
[^OAK-4913.patch]: patch for once we have upgrade our Oak dependency to 1.5.12
> Interrupt online revision cleanup on segment-tar
>
>
> Key: OAK-4913
> URL: https://issues.apache.org/jira/browse/OAK-4913
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Alex Parvulescu
>Assignee: Michael Dürig
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4913.patch
>
>
> Sub task of OAK-4835 for the {{segment-tar}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
[
https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothee Maret reassigned OAK-4917:
---
Assignee: Timothee Maret
> DefaultStandbyReferencesReader does not re-attempt to read segment upon
> failure
>
>
> Key: OAK-4917
> URL: https://issues.apache.org/jira/browse/OAK-4917
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.14
>Reporter: Timothee Maret
>Assignee: Timothee Maret
> Fix For: Segment Tar 0.0.16
>
>
> This following logs occurred while running integration testing.
> {code}
> 07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder
> Sending head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.0020 to client
> qastandby1
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get
> references' message
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler
> Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client
> qastandby1
> 07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader
> Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
> 9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285)
> at
> org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
>
[jira] [Commented] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
[
https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562576#comment-15562576
]
Timothee Maret commented on OAK-4917:
-
Discussing this with [~alexparvulescu], it seems this is expected and in fact
the cold stdby code deal with it by re-trying to load in a loop with a delay.
This looping behaviour is already implemented in the
{{DefaultStandbySegmentReader}} [0]. However the
{{DefaultStandbyReferencesReader}} does not contains the loop, causing the
issue.
[0]
https://github.com/apache/jackrabbit-oak/blob/066edc9eb5ef500339de0ebf7859e7ee9dd3ab4b/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/standby/server/DefaultStandbySegmentReader.java
> DefaultStandbyReferencesReader does not re-attempt to read segment upon
> failure
>
>
> Key: OAK-4917
> URL: https://issues.apache.org/jira/browse/OAK-4917
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.14
>Reporter: Timothee Maret
> Fix For: Segment Tar 0.0.16
>
>
> This following logs occurred while running integration testing.
> {code}
> 07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder
> Sending head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.0020 to client
> qastandby1
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get
> references' message
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler
> Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client
> qastandby1
> 07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader
> Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
> 9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285)
> at
> org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
>
[jira] [Commented] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
[
https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562564#comment-15562564
]
Timothee Maret commented on OAK-4917:
-
Looking a bit further, both readers ({{DefaultStandbyHeadReader}} and
{{DefaultStandbyReferencesReader}}) do go through the
{{org.apache.jackrabbit.oak.segment.file.FileStore}} in order to fetch data.
However, both methods seem to use different caches.
The root cause of this issue seems to be those two cache not being in sync.
{{org.apache.jackrabbit.oak.segment.file.FileStore#getHead}} does use the
CachingSegmentReader segment Reader.
{{org.apache.jackrabbit.oak.segment.file.FileStore#readSegment}} does use the
SegmentCache segment cache in the FileStore.
> DefaultStandbyReferencesReader does not re-attempt to read segment upon
> failure
>
>
> Key: OAK-4917
> URL: https://issues.apache.org/jira/browse/OAK-4917
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.14
>Reporter: Timothee Maret
> Fix For: Segment Tar 0.0.16
>
>
> This following logs occurred while running integration testing.
> {code}
> 07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder
> Sending head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.0020 to client
> qastandby1
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get
> references' message
> 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler
> Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client
> qastandby1
> 07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2]
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader
> Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
> 9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285)
> at
> org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66)
> at
> org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41)
> at
> org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
> at
>
[jira] [Updated] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
[
https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothee Maret updated OAK-4917:
Description:
This following logs occurred while running integration testing.
{code}
07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2]
org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder Sending
head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.0020 to client qastandby1
07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get
references' message
07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler
Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client
qastandby1
07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2]
org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader
Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc
org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found
at
org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364)
at
org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304)
at
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013)
at
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974)
at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285)
at
org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
at
org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304)
at
org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66)
at
org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49)
at
org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41)
at
org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
org.apache.jackrabbit.oak.segment.standby.server.RequestObserverHandler.channelRead(RequestObserverHandler.java:53)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
org.apache.jackrabbit.oak.segment.standby.server.StateHandler.channelRead(StateHandler.java:62)
at
[jira] [Updated] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
[
https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothee Maret updated OAK-4917:
Description:
This following logs occurred while running integration testing.
{code}07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2]
org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder Sending
head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.0020 to client qastandby1
07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get
references' message
07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2]
org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler
Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client
qastandby1
07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2]
org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader
Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc
org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found
at
org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364)
at
org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304)
at
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013)
at
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974)
at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285)
at
org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
at
org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304)
at
org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66)
at
org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49)
at
org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41)
at
org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
org.apache.jackrabbit.oak.segment.standby.server.RequestObserverHandler.channelRead(RequestObserverHandler.java:53)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
org.apache.jackrabbit.oak.segment.standby.server.StateHandler.channelRead(StateHandler.java:62)
at
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562550#comment-15562550
]
Michael Dürig commented on OAK-4617:
Moved the functionality that was so far provided by the {{GCMonitorMBean}} to
the {{SegmentRevisionGCMBean}} and removed the former at
http://svn.apache.org/viewvc?rev=1764117=rev
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4917) DefaultStandbyReferencesReader does not re-attempt to read segment upon failure
Timothee Maret created OAK-4917: --- Summary: DefaultStandbyReferencesReader does not re-attempt to read segment upon failure Key: OAK-4917 URL: https://issues.apache.org/jira/browse/OAK-4917 Project: Jackrabbit Oak Issue Type: Bug Components: segment-tar Affects Versions: Segment Tar 0.0.14 Reporter: Timothee Maret Fix For: Segment Tar 0.0.16 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4835) Provide generic option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562546#comment-15562546 ] Michael Dürig commented on OAK-4835: New API for cancelling revision gc at http://svn.apache.org/viewvc?rev=1764115=rev > Provide generic option to interrupt online revision cleanup > --- > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > Fix For: 1.6, 1.5.13 > > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4911) Can not read node Property when move code into a Thread (Java)
[
https://issues.apache.org/jira/browse/OAK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amin zamani updated OAK-4911:
-
Summary: Can not read node Property when move code into a Thread (Java)
(was: Can not read node Property in a Thread (Java))
> Can not read node Property when move code into a Thread (Java)
> --
>
> Key: OAK-4911
> URL: https://issues.apache.org/jira/browse/OAK-4911
> Project: Jackrabbit Oak
> Issue Type: Bug
>Affects Versions: 1.3.10
> Environment: MAC OS X
>Reporter: amin zamani
> Attachments: java-code.txt
>
>
> Hallo,
> we develop an application and everything is working as it should. We upload
> documents and for each document a new preview file (node) should be
> generated. It works well when we execute the code synchron. But because of
> the fact that the generation of a preview file (like PDF format) can take
> very long time we decided to move the preview generation code into a usual
> java thread so that after the user is upload a document the website is
> responding very quick and does not wait till the preview file is generated
> but the preview file is generated parallel.
> No the problem: As soon as I move the same code regarding to the preview file
> generation into a usual thread (no modifications to the logic, only a thread
> is bordered to it) suddenly the node path does not exist, this is the error:
> -
> Caused by: javax.jcr.PathNotFoundException: jcr:lastModifiedBy not found on
> /jcr:system/jcr:versionStorage/7e/f8/5b/7ef85b25-4598-4476-82df-446eb3a08a90/1.0/jcr:frozenNode/jcr:content
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:631)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:625)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:207)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getProperty(NodeImpl.java:625)
> at
> com.westernacher.noah.util.content_repository.JcrUtil.getStringProperty(JcrUtil.java:35)
> ---
> In my point of view this is a bug. This is the code that works:
> Example:
> /// 1) User Upload document through browser in our web app..
> // 2) ... document uploaded and saved in OAK CR
> // 3) Now generate the preview:
> updateDocumentPreview(documentId);
>
> Now the same in a thread that does not work:
> => I have only moved the method "updateDocumentPreview(documentId);" in a
> Thread (Java 8):
> //Same code as before except the last line replaced by following lines:
> Runnable run = ()-> {updateDocumentPreview(documentId);}
> new Thread(run).start(); //Start the thread to generate a document preview
> --
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4908) Best-effort prefiltering in ChangeProcessor based on ChangeSet
[
https://issues.apache.org/jira/browse/OAK-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-4908:
-
Description:
This is a subtask as a result of
[discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962]
around design in OAK-4796:
Based on the ChangeSet provided with OAK-4907 in the CommitContext, the
ChangeProcessor should do a best-effort prefiltering of commits before they get
added to the (BackgroundObserver's) queue.
This consists of the following parts:
* -the support for optionally excluding commits from being added to the queue
in the BackgroundObserver- EDIT: factored that out into OAK-4916
* -the BackgroundObserver signaling downstream Observers that a change should
be excluded via a {{NOOP_CHANGE}} CommitInfo- EDIT: factored that out into
OAK-4916
* the ChangeProcessor using OAK-4907's ChangeSet of the CommitContext for
best-effort prefiltering - and handling the {{NOOP_CHANGED}} CommitInfo
introduced in OAK-4916
was:
This is a subtask as a result of
[discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962]
around design in OAK-4796:
Based on the ChangeSet provided with OAK-4907 in the CommitContext, the
ChangeProcessor should do a best-effort prefiltering of commits before they get
added to the (BackgroundObserver's) queue.
This consists of the following parts:
* -the support for optionally excluding commits from being added to the queue
in the BackgroundObserver- EDIT: factored that out into OAK-4916
* the ChangeProcessor using OAK-4907's ChangeSet of the CommitContext for
best-effort prefiltering
* the BackgroundObserver signaling downstream Observers that a change should be
excluded via a {{NOOP_CHANGE}} CommitInfo
> Best-effort prefiltering in ChangeProcessor based on ChangeSet
> --
>
> Key: OAK-4908
> URL: https://issues.apache.org/jira/browse/OAK-4908
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: core, jcr
>Affects Versions: 1.5.11
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Fix For: 1.6
>
>
> This is a subtask as a result of
> [discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962]
> around design in OAK-4796:
> Based on the ChangeSet provided with OAK-4907 in the CommitContext, the
> ChangeProcessor should do a best-effort prefiltering of commits before they
> get added to the (BackgroundObserver's) queue.
> This consists of the following parts:
> * -the support for optionally excluding commits from being added to the queue
> in the BackgroundObserver- EDIT: factored that out into OAK-4916
> * -the BackgroundObserver signaling downstream Observers that a change should
> be excluded via a {{NOOP_CHANGE}} CommitInfo- EDIT: factored that out into
> OAK-4916
> * the ChangeProcessor using OAK-4907's ChangeSet of the CommitContext for
> best-effort prefiltering - and handling the {{NOOP_CHANGED}} CommitInfo
> introduced in OAK-4916
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4913) Interrupt online revision cleanup on segment-tar
[
https://issues.apache.org/jira/browse/OAK-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-4913:
--
Assignee: Michael Dürig
> Interrupt online revision cleanup on segment-tar
>
>
> Key: OAK-4913
> URL: https://issues.apache.org/jira/browse/OAK-4913
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Alex Parvulescu
>Assignee: Michael Dürig
> Fix For: Segment Tar 0.0.16
>
>
> Sub task of OAK-4835 for the {{segment-tar}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4835) Provide generic option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4835: --- Fix Version/s: (was: Segment Tar 0.0.16) 1.6 1.5.13 > Provide generic option to interrupt online revision cleanup > --- > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > Fix For: 1.6, 1.5.13 > > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4835) Provide generic option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4835: --- Component/s: (was: segment-tar) (was: documentmk) > Provide generic option to interrupt online revision cleanup > --- > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > Fix For: Segment Tar 0.0.16 > > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4796) filter events before adding to ChangeProcessor's queue
[
https://issues.apache.org/jira/browse/OAK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561548#comment-15561548
]
Stefan Egli edited comment on OAK-4796 at 10/10/16 1:52 PM:
split up this task into 3 sub-tasks, things will be thus followed up there:
* OAK-4907 : collect ChangeSet in a validator, store in CommitContext
* OAK-4916 : add filter capability to BackgroundObserver - thus introduce
{{NOOP_CHANGE}}
* OAK-4908 : add 'exclusion' support to BackgroundObserver and ChangeProcessor
to use that for best-effort-prefiltering
was (Author: egli):
split up this task into 3 sub-tasks, things will be thus followed up there:
* OAK-4907 : collect ChangeSet in a validator, store in CommitContext
* OAK-4916 : add filter capability to BackgroundObserver - thus introduce
{{NOOP_CHANGED}}
* OAK-4908 : add 'exclusion' support to BackgroundObserver and ChangeProcessor
to use that for best-effort-prefiltering
> filter events before adding to ChangeProcessor's queue
> --
>
> Key: OAK-4796
> URL: https://issues.apache.org/jira/browse/OAK-4796
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Affects Versions: 1.5.9
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4796.changeSet.patch, OAK-4796.patch
>
>
> Currently the
> [ChangeProcessor.contentChanged|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L335]
> is in charge of doing the event diffing and filtering and does so in a
> pooled Thread, ie asynchronously, at a later stage independent from the
> commit. This has the advantage that the commit is fast, but has the following
> potentially negative effects:
> # events (in the form of ContentChange Objects) occupy a slot of the queue
> even if the listener is not interested in it - any commit lands on any
> listener's queue. This reduces the capacity of the queue for 'actual' events
> to be delivered. It therefore increases the risk that the queue fills - and
> when full has various consequences such as loosing the CommitInfo etc.
> # each event==ContentChange later on must be evaluated, and for that a diff
> must be calculated. Depending on runtime behavior that diff might be
> expensive if no longer in the cache (documentMk specifically).
> As an improvement, this diffing+filtering could be done at an earlier stage
> already, nearer to the commit, and in case the filter would ignore the event,
> it would not have to be put into the queue at all, thus avoiding occupying a
> slot and later potentially slower diffing.
> The suggestion is to implement this via the following algorithm:
> * During the commit, in a {{Validator}} the listener's filters are evaluated
> - in an as-efficient-as-possible manner (Reason for doing it in a Validator
> is that this doesn't add overhead as oak already goes through all changes for
> other Validators). As a result a _list of potentially affected observers_ is
> added to the {{CommitInfo}} (false positives are fine).
> ** Note that the above adds cost to the commit and must therefore be
> carefully done and measured
> ** One potential measure could be to only do filtering when listener's queues
> are larger than a certain threshold (eg 10)
> * The ChangeProcessor in {{contentChanged}} (in the one created in
> [createObserver|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L224])
> then checks the new commitInfo's _potentially affected observers_ list and
> if it's not in the list, adds a {{NOOP}} token at the end of the queue. If
> there's already a NOOP there, the two are collapsed (this way when a filter
> is not affected it would have a NOOP at the end of the queue). If later on a
> no-NOOP item is added, the NOOP's {{root}} is used as the {{previousRoot}}
> for the newly added {{ContentChange}} obj.
> ** To achieve that, the ContentChange obj is extended to not only have the
> "to" {{root}} pointer, but also the "from" {{previousRoot}} pointer which
> currently is implicitly maintained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4796) filter events before adding to ChangeProcessor's queue
[
https://issues.apache.org/jira/browse/OAK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561548#comment-15561548
]
Stefan Egli edited comment on OAK-4796 at 10/10/16 1:51 PM:
split up this task into 3 sub-tasks, things will be thus followed up there:
* OAK-4907 : collect ChangeSet in a validator, store in CommitContext
* OAK-4916 : add filter capability to BackgroundObserver - thus introduce
{{NOOP_CHANGED}}
* OAK-4908 : add 'exclusion' support to BackgroundObserver and ChangeProcessor
to use that for best-effort-prefiltering
was (Author: egli):
split up this task into 2 sub-tasks, things will be thus followed up there:
* OAK-4907 : collect ChangeSet in a validator, store in CommitContext
* OAK-4916 : add filter capability to BackgroundObserver - thus introduce
{{NOOP_CHANGED}}
* OAK-4908 : add 'exclusion' support to BackgroundObserver and ChangeProcessor
to use that for best-effort-prefiltering
> filter events before adding to ChangeProcessor's queue
> --
>
> Key: OAK-4796
> URL: https://issues.apache.org/jira/browse/OAK-4796
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Affects Versions: 1.5.9
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4796.changeSet.patch, OAK-4796.patch
>
>
> Currently the
> [ChangeProcessor.contentChanged|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L335]
> is in charge of doing the event diffing and filtering and does so in a
> pooled Thread, ie asynchronously, at a later stage independent from the
> commit. This has the advantage that the commit is fast, but has the following
> potentially negative effects:
> # events (in the form of ContentChange Objects) occupy a slot of the queue
> even if the listener is not interested in it - any commit lands on any
> listener's queue. This reduces the capacity of the queue for 'actual' events
> to be delivered. It therefore increases the risk that the queue fills - and
> when full has various consequences such as loosing the CommitInfo etc.
> # each event==ContentChange later on must be evaluated, and for that a diff
> must be calculated. Depending on runtime behavior that diff might be
> expensive if no longer in the cache (documentMk specifically).
> As an improvement, this diffing+filtering could be done at an earlier stage
> already, nearer to the commit, and in case the filter would ignore the event,
> it would not have to be put into the queue at all, thus avoiding occupying a
> slot and later potentially slower diffing.
> The suggestion is to implement this via the following algorithm:
> * During the commit, in a {{Validator}} the listener's filters are evaluated
> - in an as-efficient-as-possible manner (Reason for doing it in a Validator
> is that this doesn't add overhead as oak already goes through all changes for
> other Validators). As a result a _list of potentially affected observers_ is
> added to the {{CommitInfo}} (false positives are fine).
> ** Note that the above adds cost to the commit and must therefore be
> carefully done and measured
> ** One potential measure could be to only do filtering when listener's queues
> are larger than a certain threshold (eg 10)
> * The ChangeProcessor in {{contentChanged}} (in the one created in
> [createObserver|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L224])
> then checks the new commitInfo's _potentially affected observers_ list and
> if it's not in the list, adds a {{NOOP}} token at the end of the queue. If
> there's already a NOOP there, the two are collapsed (this way when a filter
> is not affected it would have a NOOP at the end of the queue). If later on a
> no-NOOP item is added, the NOOP's {{root}} is used as the {{previousRoot}}
> for the newly added {{ContentChange}} obj.
> ** To achieve that, the ContentChange obj is extended to not only have the
> "to" {{root}} pointer, but also the "from" {{previousRoot}} pointer which
> currently is implicitly maintained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4796) filter events before adding to ChangeProcessor's queue
[
https://issues.apache.org/jira/browse/OAK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561548#comment-15561548
]
Stefan Egli edited comment on OAK-4796 at 10/10/16 1:51 PM:
split up this task into 2 sub-tasks, things will be thus followed up there:
* OAK-4907 : collect ChangeSet in a validator, store in CommitContext
* OAK-4916 : add filter capability to BackgroundObserver - thus introduce
{{NOOP_CHANGED}}
* OAK-4908 : add 'exclusion' support to BackgroundObserver and ChangeProcessor
to use that for best-effort-prefiltering
was (Author: egli):
split up this task into 2 sub-tasks, things will be thus followed up there:
* OAK-4907 : collect ChangeSet in a validator, store in CommitContext
* OAK-4908 : add 'exclusion' support to BackgroundObserver and ChangeProcessor
to use that for best-effort-prefiltering
> filter events before adding to ChangeProcessor's queue
> --
>
> Key: OAK-4796
> URL: https://issues.apache.org/jira/browse/OAK-4796
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Affects Versions: 1.5.9
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4796.changeSet.patch, OAK-4796.patch
>
>
> Currently the
> [ChangeProcessor.contentChanged|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L335]
> is in charge of doing the event diffing and filtering and does so in a
> pooled Thread, ie asynchronously, at a later stage independent from the
> commit. This has the advantage that the commit is fast, but has the following
> potentially negative effects:
> # events (in the form of ContentChange Objects) occupy a slot of the queue
> even if the listener is not interested in it - any commit lands on any
> listener's queue. This reduces the capacity of the queue for 'actual' events
> to be delivered. It therefore increases the risk that the queue fills - and
> when full has various consequences such as loosing the CommitInfo etc.
> # each event==ContentChange later on must be evaluated, and for that a diff
> must be calculated. Depending on runtime behavior that diff might be
> expensive if no longer in the cache (documentMk specifically).
> As an improvement, this diffing+filtering could be done at an earlier stage
> already, nearer to the commit, and in case the filter would ignore the event,
> it would not have to be put into the queue at all, thus avoiding occupying a
> slot and later potentially slower diffing.
> The suggestion is to implement this via the following algorithm:
> * During the commit, in a {{Validator}} the listener's filters are evaluated
> - in an as-efficient-as-possible manner (Reason for doing it in a Validator
> is that this doesn't add overhead as oak already goes through all changes for
> other Validators). As a result a _list of potentially affected observers_ is
> added to the {{CommitInfo}} (false positives are fine).
> ** Note that the above adds cost to the commit and must therefore be
> carefully done and measured
> ** One potential measure could be to only do filtering when listener's queues
> are larger than a certain threshold (eg 10)
> * The ChangeProcessor in {{contentChanged}} (in the one created in
> [createObserver|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L224])
> then checks the new commitInfo's _potentially affected observers_ list and
> if it's not in the list, adds a {{NOOP}} token at the end of the queue. If
> there's already a NOOP there, the two are collapsed (this way when a filter
> is not affected it would have a NOOP at the end of the queue). If later on a
> no-NOOP item is added, the NOOP's {{root}} is used as the {{previousRoot}}
> for the newly added {{ContentChange}} obj.
> ** To achieve that, the ContentChange obj is extended to not only have the
> "to" {{root}} pointer, but also the "from" {{previousRoot}} pointer which
> currently is implicitly maintained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4908) Best-effort prefiltering in ChangeProcessor based on ChangeSet
[
https://issues.apache.org/jira/browse/OAK-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-4908:
-
Description:
This is a subtask as a result of
[discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962]
around design in OAK-4796:
Based on the ChangeSet provided with OAK-4907 in the CommitContext, the
ChangeProcessor should do a best-effort prefiltering of commits before they get
added to the (BackgroundObserver's) queue.
This consists of the following parts:
* -the support for optionally excluding commits from being added to the queue
in the BackgroundObserver- EDIT: factored that out into OAK-4916
* the ChangeProcessor using OAK-4907's ChangeSet of the CommitContext for
best-effort prefiltering
* the BackgroundObserver signaling downstream Observers that a change should be
excluded via a {{NOOP_CHANGE}} CommitInfo
was:
This is a subtask as a result of
[discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962]
around design in OAK-4796:
Based on the ChangeSet provided with OAK-4907 in the CommitContext, the
ChangeProcessor should do a best-effort prefiltering of commits before they get
added to the (BackgroundObserver's) queue.
This consists of the following parts:
* the support for optionally excluding commits from being added to the queue in
the BackgroundObserver
* the ChangeProcessor using OAK-4907's ChangeSet of the CommitContext for
best-effort prefiltering
* the BackgroundObserver signaling downstream Observers that a change should be
excluded via a {{NOOP_CHANGE}} CommitInfo
> Best-effort prefiltering in ChangeProcessor based on ChangeSet
> --
>
> Key: OAK-4908
> URL: https://issues.apache.org/jira/browse/OAK-4908
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: core, jcr
>Affects Versions: 1.5.11
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Fix For: 1.6
>
>
> This is a subtask as a result of
> [discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962]
> around design in OAK-4796:
> Based on the ChangeSet provided with OAK-4907 in the CommitContext, the
> ChangeProcessor should do a best-effort prefiltering of commits before they
> get added to the (BackgroundObserver's) queue.
> This consists of the following parts:
> * -the support for optionally excluding commits from being added to the queue
> in the BackgroundObserver- EDIT: factored that out into OAK-4916
> * the ChangeProcessor using OAK-4907's ChangeSet of the CommitContext for
> best-effort prefiltering
> * the BackgroundObserver signaling downstream Observers that a change should
> be excluded via a {{NOOP_CHANGE}} CommitInfo
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4916) Add support for excluding commits to BackgroundObserver
Stefan Egli created OAK-4916:
Summary: Add support for excluding commits to BackgroundObserver
Key: OAK-4916
URL: https://issues.apache.org/jira/browse/OAK-4916
Project: Jackrabbit Oak
Issue Type: Technical task
Components: core
Affects Versions: 1.5.11
Reporter: Stefan Egli
Assignee: Stefan Egli
Fix For: 1.6
As part of pre-filtering commits it would be useful to have support in the
BackgroundObserver (in general) that would allow to exclude certain commits
from being added to the (BackgroundObserver's) queue, thus keeping the queue
smaller. The actual filtering is up to subclasses.
The suggested implementation is as follows:
* a new method {{isExcluded}} is introduced which represents a subclass hook
for filtering
* excluded commits are not added to the queue
* when multiple commits are excluded subsequently, this is collapsed
* the first non-excluded commit (ContentChange) added to the queue is marked
with the last non-excluded root state as the 'previous root'
* downstream Observers are notified of the exclusion of a commit via a special
CommitInfo {{NOOP_CHANGE}}: this instructs it to exclude this change while at
the same time 'fast-forwarding' the root state to the new one.
** this extra token is one way of solving the problem that
{{Observer.contentChanged}} represents a diff between two states but does not
transport the 'from' state explicitly - that is implicitly taken from the
previous call to {{contentChanged}}. Thus using such a gap token
({{NOOP_CHANGE}}) seems to be the only way to instruct Observers to skip a
change.
To repeat: whoever extends BackgroundObserver with filtering must be aware of
the new {{NOOP_CHANGE}} token. Anyone not doing filtering will not get any
{{NOOP_CHANGE}} tokens though.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4915) Interrupt online revision cleanup on documentmk
[
https://issues.apache.org/jira/browse/OAK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4915:
-
Component/s: (was: segmentmk)
documentmk
> Interrupt online revision cleanup on documentmk
> ---
>
> Key: OAK-4915
> URL: https://issues.apache.org/jira/browse/OAK-4915
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: documentmk
>Reporter: Alex Parvulescu
>
> Sub task of OAK-4835 for the {{segmentmk}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4915) Interrupt online revision cleanup on documentmk
[
https://issues.apache.org/jira/browse/OAK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4915:
-
Description: Sub task of OAK-4835 for the {{document}} specific changes
(was: Sub task of OAK-4835 for the {{segmentmk}} specific changes)
> Interrupt online revision cleanup on documentmk
> ---
>
> Key: OAK-4915
> URL: https://issues.apache.org/jira/browse/OAK-4915
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: documentmk
>Reporter: Alex Parvulescu
>
> Sub task of OAK-4835 for the {{document}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4915) Interrupt online revision cleanup on documentmk
[
https://issues.apache.org/jira/browse/OAK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4915:
-
Fix Version/s: (was: Segment Tar 0.0.16)
> Interrupt online revision cleanup on documentmk
> ---
>
> Key: OAK-4915
> URL: https://issues.apache.org/jira/browse/OAK-4915
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: documentmk
>Reporter: Alex Parvulescu
>
> Sub task of OAK-4835 for the {{segmentmk}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4915) Interrupt online revision cleanup on documentmk
Alex Parvulescu created OAK-4915:
Summary: Interrupt online revision cleanup on documentmk
Key: OAK-4915
URL: https://issues.apache.org/jira/browse/OAK-4915
Project: Jackrabbit Oak
Issue Type: Technical task
Components: segmentmk
Reporter: Alex Parvulescu
Sub task of OAK-4835 for the {{segmentmk}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4907) Collect changes (paths, nts, props..) of a commit in a validator
[ https://issues.apache.org/jira/browse/OAK-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-4907: - Attachment: OAK-4907.patch Attaching [^OAK-4907.patch] which contains mainly: * ChangeSet : the data object holding actual items changed (paths, names, types, properties) * ChangeCollectorProvider: a type ValidationProvider that can be hooked into Oak to have above ChangeSets be generated and stored in CommitContexts > Collect changes (paths, nts, props..) of a commit in a validator > > > Key: OAK-4907 > URL: https://issues.apache.org/jira/browse/OAK-4907 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: core >Affects Versions: 1.5.11 >Reporter: Stefan Egli >Assignee: Stefan Egli > Fix For: 1.6 > > Attachments: OAK-4907.patch > > > It would be useful to collect a set of changes of a commit (eg in a > validator) that could later be used in an Observer for eg prefiltering. > Such a change collector should collect paths, nodetypes, properties, > node-names (and perhaps more at a later stage) of all changes and store the > result in the CommitInfo's CommitContext. > Note that this is a result of > [discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962] > around design in OAK-4796 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4914) Interrupt online revision cleanup on segmentmk
[
https://issues.apache.org/jira/browse/OAK-4914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4914:
-
Fix Version/s: (was: Segment Tar 0.0.16)
Description: Sub task of OAK-4835 for the {{segmentmk}} specific changes
(was: Sub task of OAK-4835 for the {{segment-tar}} specific changes)
> Interrupt online revision cleanup on segmentmk
> --
>
> Key: OAK-4914
> URL: https://issues.apache.org/jira/browse/OAK-4914
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Alex Parvulescu
>
> Sub task of OAK-4835 for the {{segmentmk}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4914) Interrupt online revision cleanup on segmentmk
[
https://issues.apache.org/jira/browse/OAK-4914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4914:
-
Component/s: (was: segment-tar)
segmentmk
> Interrupt online revision cleanup on segmentmk
> --
>
> Key: OAK-4914
> URL: https://issues.apache.org/jira/browse/OAK-4914
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Alex Parvulescu
>
> Sub task of OAK-4835 for the {{segmentmk}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4913) Interrupt online revision cleanup on segment-tar
Alex Parvulescu created OAK-4913:
Summary: Interrupt online revision cleanup on segment-tar
Key: OAK-4913
URL: https://issues.apache.org/jira/browse/OAK-4913
Project: Jackrabbit Oak
Issue Type: Technical task
Components: segment-tar
Reporter: Alex Parvulescu
Sub task of OAK-4835 for the {{segment-tar}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4914) Interrupt online revision cleanup on segmentmk
Alex Parvulescu created OAK-4914:
Summary: Interrupt online revision cleanup on segmentmk
Key: OAK-4914
URL: https://issues.apache.org/jira/browse/OAK-4914
Project: Jackrabbit Oak
Issue Type: Technical task
Components: segment-tar
Reporter: Alex Parvulescu
Fix For: Segment Tar 0.0.16
Sub task of OAK-4835 for the {{segment-tar}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4912) MongoDB: ReadPreferenceIT.testMongoReadPreferencesForLocalChanges() ocasionally fails
Julian Reschke created OAK-4912:
---
Summary: MongoDB:
ReadPreferenceIT.testMongoReadPreferencesForLocalChanges() ocasionally fails
Key: OAK-4912
URL: https://issues.apache.org/jira/browse/OAK-4912
Project: Jackrabbit Oak
Issue Type: Bug
Reporter: Julian Reschke
Priority: Minor
{noformat}
testMongoReadPreferencesForLocalChanges(org.apache.jackrabbit.oak.plugins.document.mongo.ReadPreferenceIT)
Time elapsed: 0.024 sec <<< FAILURE!
java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.jackrabbit.oak.plugins.document.mongo.ReadPreferenceIT.testMongoReadPreferencesForLocalChanges(ReadPreferenceIT.java:195)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4835) Provide generic option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4835: --- Component/s: (was: segmentmk) segment-tar > Provide generic option to interrupt online revision cleanup > --- > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk, segment-tar >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > Fix For: Segment Tar 0.0.16 > > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4910) Update segment tar to 0.0.14
[ https://issues.apache.org/jira/browse/OAK-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-4910. Resolution: Fixed Fixed at http://svn.apache.org/viewvc?rev=1764069=rev > Update segment tar to 0.0.14 > > > Key: OAK-4910 > URL: https://issues.apache.org/jira/browse/OAK-4910 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, segment-tar >Reporter: Davide Giannella >Assignee: Michael Dürig >Priority: Blocker > Fix For: 1.6, 1.5.12 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4911) Can not read node Property in a Thread (Java)
[
https://issues.apache.org/jira/browse/OAK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amin zamani updated OAK-4911:
-
Attachment: java-code.txt
I also have attached the important parts of the code as you can see. The error
is thrown in the "toDocument" method were access to the property
"lastModifiedBy" is not working for example.
Any ideas? Thank you very much for your help!
Best regards,
Amin Zamani
> Can not read node Property in a Thread (Java)
> -
>
> Key: OAK-4911
> URL: https://issues.apache.org/jira/browse/OAK-4911
> Project: Jackrabbit Oak
> Issue Type: Bug
>Affects Versions: 1.3.10
> Environment: MAC OS X
>Reporter: amin zamani
> Attachments: java-code.txt
>
>
> Hallo,
> we develop an application and everything is working as it should. We upload
> documents and for each document a new preview file (node) should be
> generated. It works well when we execute the code synchron. But because of
> the fact that the generation of a preview file (like PDF format) can take
> very long time we decided to move the preview generation code into a usual
> java thread so that after the user is upload a document the website is
> responding very quick and does not wait till the preview file is generated
> but the preview file is generated parallel.
> No the problem: As soon as I move the same code regarding to the preview file
> generation into a usual thread (no modifications to the logic, only a thread
> is bordered to it) suddenly the node path does not exist, this is the error:
> -
> Caused by: javax.jcr.PathNotFoundException: jcr:lastModifiedBy not found on
> /jcr:system/jcr:versionStorage/7e/f8/5b/7ef85b25-4598-4476-82df-446eb3a08a90/1.0/jcr:frozenNode/jcr:content
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:631)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:625)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:207)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getProperty(NodeImpl.java:625)
> at
> com.westernacher.noah.util.content_repository.JcrUtil.getStringProperty(JcrUtil.java:35)
> ---
> In my point of view this is a bug. This is the code that works:
> Example:
> /// 1) User Upload document through browser in our web app..
> // 2) ... document uploaded and saved in OAK CR
> // 3) Now generate the preview:
> updateDocumentPreview(documentId);
>
> Now the same in a thread that does not work:
> => I have only moved the method "updateDocumentPreview(documentId);" in a
> Thread (Java 8):
> //Same code as before except the last line replaced by following lines:
> Runnable run = ()-> {updateDocumentPreview(documentId);}
> new Thread(run).start(); //Start the thread to generate a document preview
> --
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4911) Can not read node Property in a Thread (Java)
[
https://issues.apache.org/jira/browse/OAK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562053#comment-15562053
]
amin zamani commented on OAK-4911:
--
Here some information of implementation of the method for updating the node -
but as mentioned before, it is doing nothing different but is only called in a
thread - it only writes an input stream of a file into the preview node in oak,
nothing more and then returns deep inside of it some properties of the node :
void updateDocumentPreview(String originalDocumentId){
//Get inputstream of the preview (preview file is in local directory of OS)
//Now save the preview file content into oak:
//Call of content update method:
updateDocumentContent(previewNodeIdToUpdate, InputStream);
}
And this is the implementation of the method with the input stream of the
preview file:
public Document updateDocumentContent(String documentId, InputStream
fileContents) {
try {
Binary binary =
jcrSession.getValueFactory().createBinary(fileContents);
try {
Node file = jcrSession.getNodeByIdentifier(documentId);
Version updatedVersion = versionFile(file, binary);
return toDocument(updatedVersion);
} finally {
binary.dispose();
}
} catch (Exception e) {
throw new RuntimeItemNotFoundException(e);
}
}
=> The exception is thrown in the method toDocument(updatedVersion)
First, This is the implementation of the "versionFile(file, binary)" method:
private Version versionFile(Node file, Binary fileContent) {
try {
VersionManager versionManager =
file.getSession().getWorkspace().getVersionManager();
versionManager.checkout(file.getPath());
file.getNode(Property.JCR_CONTENT).setProperty(Property.JCR_LAST_MODIFIED,
Calendar.getInstance());
file.getNode(Property.JCR_CONTENT).setProperty(Property.JCR_LAST_MODIFIED_BY,
SecurityUtil.getCurrentUserLogin());
file.getNode(Property.JCR_CONTENT).setProperty(Property.JCR_DATA,
fileContent);
jcrSession.save();
Version version = versionManager.checkin(file.getPath());
versionManager.getVersionHistory(file.getPath()).addVersionLabel(version.getName(),
NOAH_LAST_VERSION_LABEL, DO_MOVE_LABEL);
return version;
} catch (RepositoryException e) {
throw new RuntimeRepositoryException(e);
}
}
> Can not read node Property in a Thread (Java)
> -
>
> Key: OAK-4911
> URL: https://issues.apache.org/jira/browse/OAK-4911
> Project: Jackrabbit Oak
> Issue Type: Bug
>Affects Versions: 1.3.10
> Environment: MAC OS X
>Reporter: amin zamani
>
> Hallo,
> we develop an application and everything is working as it should. We upload
> documents and for each document a new preview file (node) should be
> generated. It works well when we execute the code synchron. But because of
> the fact that the generation of a preview file (like PDF format) can take
> very long time we decided to move the preview generation code into a usual
> java thread so that after the user is upload a document the website is
> responding very quick and does not wait till the preview file is generated
> but the preview file is generated parallel.
> No the problem: As soon as I move the same code regarding to the preview file
> generation into a usual thread (no modifications to the logic, only a thread
> is bordered to it) suddenly the node path does not exist, this is the error:
> -
> Caused by: javax.jcr.PathNotFoundException: jcr:lastModifiedBy not found on
> /jcr:system/jcr:versionStorage/7e/f8/5b/7ef85b25-4598-4476-82df-446eb3a08a90/1.0/jcr:frozenNode/jcr:content
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:631)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:625)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:207)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getProperty(NodeImpl.java:625)
> at
> com.westernacher.noah.util.content_repository.JcrUtil.getStringProperty(JcrUtil.java:35)
> ---
> In my point of view this is a bug. This is the code that works:
> Example:
> /// 1) User Upload document through browser in our web app..
> // 2) ... document uploaded and saved in OAK CR
> // 3) Now generate the preview:
> updateDocumentPreview(documentId);
>
> Now the same in a thread that does not work:
> => I have only moved the method
[jira] [Commented] (OAK-4889) Update Oak trunk to Jackrabbit 2.13.4
[ https://issues.apache.org/jira/browse/OAK-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562044#comment-15562044 ] Julian Reschke commented on OAK-4889: - trunk: [r1764061|http://svn.apache.org/r1764061] > Update Oak trunk to Jackrabbit 2.13.4 > - > > Key: OAK-4889 > URL: https://issues.apache.org/jira/browse/OAK-4889 > Project: Jackrabbit Oak > Issue Type: Task > Components: parent >Affects Versions: 1.5.11 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Fix For: 1.5.12 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4889) Update Oak trunk to Jackrabbit 2.13.4
[ https://issues.apache.org/jira/browse/OAK-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-4889. - Resolution: Fixed > Update Oak trunk to Jackrabbit 2.13.4 > - > > Key: OAK-4889 > URL: https://issues.apache.org/jira/browse/OAK-4889 > Project: Jackrabbit Oak > Issue Type: Task > Components: parent >Affects Versions: 1.5.11 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Fix For: 1.5.12 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4889) Update Oak trunk to Jackrabbit 2.13.4
[ https://issues.apache.org/jira/browse/OAK-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4889: Fix Version/s: 1.5.12 > Update Oak trunk to Jackrabbit 2.13.4 > - > > Key: OAK-4889 > URL: https://issues.apache.org/jira/browse/OAK-4889 > Project: Jackrabbit Oak > Issue Type: Task > Components: parent >Affects Versions: 1.5.11 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Fix For: 1.5.12 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4835) Provide generic option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4835: --- Fix Version/s: Segment Tar 0.0.16 > Provide generic option to interrupt online revision cleanup > --- > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk, segmentmk >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > Fix For: Segment Tar 0.0.16 > > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4911) Can not read node Property in a Thread (Java)
amin zamani created OAK-4911:
Summary: Can not read node Property in a Thread (Java)
Key: OAK-4911
URL: https://issues.apache.org/jira/browse/OAK-4911
Project: Jackrabbit Oak
Issue Type: Bug
Affects Versions: 1.3.10
Environment: MAC OS X
Reporter: amin zamani
Hallo,
we develop an application and everything is working as it should. We upload
documents and for each document a new preview file (node) should be generated.
It works well when we execute the code synchron. But because of the fact that
the generation of a preview file (like PDF format) can take very long time we
decided to move the preview generation code into a usual java thread so that
after the user is upload a document the website is responding very quick and
does not wait till the preview file is generated but the preview file is
generated parallel.
No the problem: As soon as I move the same code regarding to the preview file
generation into a usual thread (no modifications to the logic, only a thread is
bordered to it) suddenly the node path does not exist, this is the error:
-
Caused by: javax.jcr.PathNotFoundException: jcr:lastModifiedBy not found on
/jcr:system/jcr:versionStorage/7e/f8/5b/7ef85b25-4598-4476-82df-446eb3a08a90/1.0/jcr:frozenNode/jcr:content
at
org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:631)
at
org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:625)
at
org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:207)
at
org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
at
org.apache.jackrabbit.oak.jcr.session.NodeImpl.getProperty(NodeImpl.java:625)
at
com.westernacher.noah.util.content_repository.JcrUtil.getStringProperty(JcrUtil.java:35)
---
In my point of view this is a bug. This is the code that works:
Example:
/// 1) User Upload document through browser in our web app..
// 2) ... document uploaded and saved in OAK CR
// 3) Now generate the preview:
updateDocumentPreview(documentId);
Now the same in a thread that does not work:
=> I have only moved the method "updateDocumentPreview(documentId);" in a
Thread (Java 8):
//Same code as before except the last line replaced by following lines:
Runnable run = ()-> {updateDocumentPreview(documentId);}
new Thread(run).start(); //Start the thread to generate a document preview
--
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561888#comment-15561888
]
Michael Dürig commented on OAK-4617:
See http://markmail.org/message/pjb6psc46p2thd76 for my proposal for an initial
step.
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4835) Provide generic option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561887#comment-15561887 ] Michael Dürig commented on OAK-4835: See http://markmail.org/message/pjb6psc46p2thd76 for my proposal > Provide generic option to interrupt online revision cleanup > --- > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk, segmentmk >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4909) NRTIndex can get closed while in use
[
https://issues.apache.org/jira/browse/OAK-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-4909.
--
Resolution: Fixed
Fix Version/s: 1.5.12
> NRTIndex can get closed while in use
>
>
> Key: OAK-4909
> URL: https://issues.apache.org/jira/browse/OAK-4909
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.6, 1.5.12
>
>
> With hybrid indexing enabled it can happen that NRTIndex gets closed while
> its in use
> {noformat}
> 2016-10-10 14:22:41,094 WARN NA [Workflow Starter Thread]
> o.a.j.o.p.i.l.h.DocumentQueue - Error occurred while indexing index
> [/oak:index/hybridIndex] java.lang.IllegalStateException: null
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:134)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.createReader(NRTIndex.java:168)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getPrimaryReader(NRTIndex.java:87)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getReaders(NRTIndex.java:118)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.getNRTReaders(IndexNode.java:200)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.refreshReaders(IndexNode.java:172)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.access$000(IndexNode.java:50)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode$1.run(IndexNode.java:84)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.RefreshOnWritePolicy.refreshOnWriteIfRequired(RefreshOnWritePolicy.java:41)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.refreshReadersOnWriteIfRequired(IndexNode.java:168)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.processDocs(DocumentQueue.java:210)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4909) NRTIndex can get closed while in use
[
https://issues.apache.org/jira/browse/OAK-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561856#comment-15561856
]
Chetan Mehrotra commented on OAK-4909:
--
In a running system there can be at max 2 {{IndexNode}} instances opened for
same index say /oak:index/fooIndex. The older node ( IndexNode1) is being used
by some query while newer node (IndexNode2) is being constructed via
{{IndexTracker}} as index content has changed. These 2 node can refer to 3
{{NRTIndex}} instances
* IndexNode1 - NRTIndex0, NRTIndex1 - Here NRTIndex0 is the previous NRTIndex
from older IndexNode0
* IndexNode2 - NRTIndex1, NRTIndex2 - Here NRTIndex1 is the previous NRTIndex
which would not receive any new updates while NRTIndex2 is created just now
Currently logic only accounted for one IndexNode being active hence kept max 2
NRTIndex instance opened. However given above it should keep 3 NRTIndex
instances.
Fixed this in r1764047
> NRTIndex can get closed while in use
>
>
> Key: OAK-4909
> URL: https://issues.apache.org/jira/browse/OAK-4909
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.6
>
>
> With hybrid indexing enabled it can happen that NRTIndex gets closed while
> its in use
> {noformat}
> 2016-10-10 14:22:41,094 WARN NA [Workflow Starter Thread]
> o.a.j.o.p.i.l.h.DocumentQueue - Error occurred while indexing index
> [/oak:index/hybridIndex] java.lang.IllegalStateException: null
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:134)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.createReader(NRTIndex.java:168)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getPrimaryReader(NRTIndex.java:87)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getReaders(NRTIndex.java:118)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.getNRTReaders(IndexNode.java:200)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.refreshReaders(IndexNode.java:172)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.access$000(IndexNode.java:50)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode$1.run(IndexNode.java:84)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.RefreshOnWritePolicy.refreshOnWriteIfRequired(RefreshOnWritePolicy.java:41)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.refreshReadersOnWriteIfRequired(IndexNode.java:168)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.processDocs(DocumentQueue.java:210)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4754) Remove ExternalPrivateStoreIT and ExternalSharedStoreIT
[
https://issues.apache.org/jira/browse/OAK-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4754:
-
Fix Version/s: (was: Segment Tar 0.0.20)
> Remove ExternalPrivateStoreIT and ExternalSharedStoreIT
> ---
>
> Key: OAK-4754
> URL: https://issues.apache.org/jira/browse/OAK-4754
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.10
>Reporter: Timothee Maret
>Assignee: Timothee Maret
>
> The classes
> {{org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT}} and
> {{org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT}} do not
> contain any tests and are not referenced.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4040) Some classes from o.a.j.o.plugins.segment.compaction should be exported
[
https://issues.apache.org/jira/browse/OAK-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4040:
-
Fix Version/s: (was: Segment Tar 0.0.22)
> Some classes from o.a.j.o.plugins.segment.compaction should be exported
> ---
>
> Key: OAK-4040
> URL: https://issues.apache.org/jira/browse/OAK-4040
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Labels: technical_debt
>
> Classes
> {{org.apache.jackrabbit.oak.plugins.segment.compaction.CompactionStrategy}}
> and
> {{org.apache.jackrabbit.oak.plugins.segment.compaction.CompactionStrategyMBean}}
> should be exported. The former is used in the public API of multiple classes
> from {{org.apache.jackrabbit.oak.plugins.segment.file}} and
> {{org.apache.jackrabbit.oak.plugins.segment}}, while the latter is used as
> interface type for a service registered in the whiteboard.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3695) Expose ratio between waste and real data
[ https://issues.apache.org/jira/browse/OAK-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-3695: - Fix Version/s: (was: Segment Tar 0.0.16) > Expose ratio between waste and real data > > > Key: OAK-3695 > URL: https://issues.apache.org/jira/browse/OAK-3695 > Project: Jackrabbit Oak > Issue Type: Story > Components: segment-tar >Reporter: Valentin Olteanu > Labels: gc > > As a user, I want to know the ratio (or precise absolute values) between > waste and real data on TarMK, so that I can decide if Revision GC needs to be > run. The measurement has to be done on a running repository and without > impacting the performance. > This would also help measure the efficiency of Revision GC and see the effect > of improvements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4752) Coldstandby runs tests with NoopStats
[
https://issues.apache.org/jira/browse/OAK-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4752:
-
Fix Version/s: (was: Segment Tar 0.0.20)
> Coldstandby runs tests with NoopStats
> -
>
> Key: OAK-4752
> URL: https://issues.apache.org/jira/browse/OAK-4752
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.10
>Reporter: Timothee Maret
>Assignee: Timothee Maret
> Attachments: OAK-4752.patch, OAK-4752.patch, OAK-4752.patch,
> OAK-4752.patch
>
>
> As described in OAK-4736, the
> {{org.apache.jackrabbit.oak.segment.standby.TestBase}} runs with a
> {{org.apache.jackrabbit.oak.stats.NoopStats}} which makes makes the
> assertions related to statistics to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4105) Implement FileStore.size through FileStore.approximateSize
[
https://issues.apache.org/jira/browse/OAK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4105:
-
Fix Version/s: (was: Segment Tar 0.0.6)
> Implement FileStore.size through FileStore.approximateSize
> --
>
> Key: OAK-4105
> URL: https://issues.apache.org/jira/browse/OAK-4105
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: resilience
> Attachments: OAK-4105-01.patch, OAK-4105-02.patch
>
>
> {{FileStore.size()}} is prone to lock contention and should not be called too
> often. As OAK-2879 already introduced an approach for tracking the current
> size of the file store without having to lock, we might as well promote his
> to be "the official" implementation.
> [~frm] WDYT?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4910) Update segment tar to 0.0.14
Davide Giannella created OAK-4910: - Summary: Update segment tar to 0.0.14 Key: OAK-4910 URL: https://issues.apache.org/jira/browse/OAK-4910 Project: Jackrabbit Oak Issue Type: Improvement Components: core, segment-tar Reporter: Davide Giannella Assignee: Michael Dürig Priority: Blocker Fix For: 1.6, 1.5.12 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4909) NRTIndex can get closed while in use
[
https://issues.apache.org/jira/browse/OAK-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-4909:
-
Summary: NRTIndex can get closed while in use (was: NRTIndex can get
closed when in use)
> NRTIndex can get closed while in use
>
>
> Key: OAK-4909
> URL: https://issues.apache.org/jira/browse/OAK-4909
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.6
>
>
> With hybrid indexing enabled it can happen that NRTIndex gets closed while
> its in use
> {noformat}
> 2016-10-10 14:22:41,094 WARN NA [Workflow Starter Thread]
> o.a.j.o.p.i.l.h.DocumentQueue - Error occurred while indexing index
> [/oak:index/hybridIndex] java.lang.IllegalStateException: null
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:134)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.createReader(NRTIndex.java:168)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getPrimaryReader(NRTIndex.java:87)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getReaders(NRTIndex.java:118)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.getNRTReaders(IndexNode.java:200)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.refreshReaders(IndexNode.java:172)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.access$000(IndexNode.java:50)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode$1.run(IndexNode.java:84)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.RefreshOnWritePolicy.refreshOnWriteIfRequired(RefreshOnWritePolicy.java:41)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.refreshReadersOnWriteIfRequired(IndexNode.java:168)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.processDocs(DocumentQueue.java:210)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4901) Improve SizeDeltaGcEstimation logging
[
https://issues.apache.org/jira/browse/OAK-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu resolved OAK-4901.
--
Resolution: Fixed
Assignee: Alex Parvulescu
Fix Version/s: Segment Tar 0.0.16
thanks [~volteanu] for the contribution!
fixed with http://svn.apache.org/viewvc?rev=1764039=rev
> Improve SizeDeltaGcEstimation logging
> -
>
> Key: OAK-4901
> URL: https://issues.apache.org/jira/browse/OAK-4901
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Affects Versions: Segment Tar 0.0.14
>Reporter: Valentin Olteanu
>Assignee: Alex Parvulescu
>Priority: Minor
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4901.patch
>
>
> The current message logged after estimation is not very clear for someone
> that does not know how it works:
> {code}
> 06.10.2016 18:53:07.819 *INFO* [TarMK revision gc
> [/Users/volteanu/workspace/test/author/crx-quickstart/repository/segmentstore]]
> org.apache.jackrabbit.oak.segment.file.FileStore TarMK GC #1: estimation
> completed in 7.362 ms (7 ms). Size delta is 0% or 880.3 MB/880.9 MB
> (880258560/880892416 bytes), so skipping compaction for now
> {code}
> I think it should also list the delta in absolute value and the threshold it
> is compared against.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4909) NRTIndex can get closed when in use
Chetan Mehrotra created OAK-4909:
Summary: NRTIndex can get closed when in use
Key: OAK-4909
URL: https://issues.apache.org/jira/browse/OAK-4909
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Fix For: 1.6
With hybrid indexing enabled it can happen that NRTIndex gets closed while its
in use
{noformat}
2016-10-10 14:22:41,094 WARN NA [Workflow Starter Thread]
o.a.j.o.p.i.l.h.DocumentQueue - Error occurred while indexing index
[/oak:index/hybridIndex] java.lang.IllegalStateException: null
at
com.google.common.base.Preconditions.checkState(Preconditions.java:134)
at
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.createReader(NRTIndex.java:168)
at
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getPrimaryReader(NRTIndex.java:87)
at
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getReaders(NRTIndex.java:118)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.getNRTReaders(IndexNode.java:200)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.refreshReaders(IndexNode.java:172)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.access$000(IndexNode.java:50)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode$1.run(IndexNode.java:84)
at
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.RefreshOnWritePolicy.refreshOnWriteIfRequired(RefreshOnWritePolicy.java:41)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.refreshReadersOnWriteIfRequired(IndexNode.java:168)
at
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.processDocs(DocumentQueue.java:210)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4796) filter events before adding to ChangeProcessor's queue
[
https://issues.apache.org/jira/browse/OAK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561548#comment-15561548
]
Stefan Egli commented on OAK-4796:
--
split up this task into 2 sub-tasks, things will be thus followed up there:
* OAK-4907 : collect ChangeSet in a validator, store in CommitContext
* OAK-4908 : add 'exclusion' support to BackgroundObserver and ChangeProcessor
to use that for best-effort-prefiltering
> filter events before adding to ChangeProcessor's queue
> --
>
> Key: OAK-4796
> URL: https://issues.apache.org/jira/browse/OAK-4796
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Affects Versions: 1.5.9
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4796.changeSet.patch, OAK-4796.patch
>
>
> Currently the
> [ChangeProcessor.contentChanged|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L335]
> is in charge of doing the event diffing and filtering and does so in a
> pooled Thread, ie asynchronously, at a later stage independent from the
> commit. This has the advantage that the commit is fast, but has the following
> potentially negative effects:
> # events (in the form of ContentChange Objects) occupy a slot of the queue
> even if the listener is not interested in it - any commit lands on any
> listener's queue. This reduces the capacity of the queue for 'actual' events
> to be delivered. It therefore increases the risk that the queue fills - and
> when full has various consequences such as loosing the CommitInfo etc.
> # each event==ContentChange later on must be evaluated, and for that a diff
> must be calculated. Depending on runtime behavior that diff might be
> expensive if no longer in the cache (documentMk specifically).
> As an improvement, this diffing+filtering could be done at an earlier stage
> already, nearer to the commit, and in case the filter would ignore the event,
> it would not have to be put into the queue at all, thus avoiding occupying a
> slot and later potentially slower diffing.
> The suggestion is to implement this via the following algorithm:
> * During the commit, in a {{Validator}} the listener's filters are evaluated
> - in an as-efficient-as-possible manner (Reason for doing it in a Validator
> is that this doesn't add overhead as oak already goes through all changes for
> other Validators). As a result a _list of potentially affected observers_ is
> added to the {{CommitInfo}} (false positives are fine).
> ** Note that the above adds cost to the commit and must therefore be
> carefully done and measured
> ** One potential measure could be to only do filtering when listener's queues
> are larger than a certain threshold (eg 10)
> * The ChangeProcessor in {{contentChanged}} (in the one created in
> [createObserver|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L224])
> then checks the new commitInfo's _potentially affected observers_ list and
> if it's not in the list, adds a {{NOOP}} token at the end of the queue. If
> there's already a NOOP there, the two are collapsed (this way when a filter
> is not affected it would have a NOOP at the end of the queue). If later on a
> no-NOOP item is added, the NOOP's {{root}} is used as the {{previousRoot}}
> for the newly added {{ContentChange}} obj.
> ** To achieve that, the ContentChange obj is extended to not only have the
> "to" {{root}} pointer, but also the "from" {{previousRoot}} pointer which
> currently is implicitly maintained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4908) Best-effort prefiltering in ChangeProcessor based on ChangeSet
Stefan Egli created OAK-4908:
Summary: Best-effort prefiltering in ChangeProcessor based on
ChangeSet
Key: OAK-4908
URL: https://issues.apache.org/jira/browse/OAK-4908
Project: Jackrabbit Oak
Issue Type: Technical task
Components: core, jcr
Affects Versions: 1.5.11
Reporter: Stefan Egli
Assignee: Stefan Egli
Fix For: 1.6
This is a subtask as a result of
[discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962]
around design in OAK-4796:
Based on the ChangeSet provided with OAK-4907 in the CommitContext, the
ChangeProcessor should do a best-effort prefiltering of commits before they get
added to the (BackgroundObserver's) queue.
This consists of the following parts:
* the support for optionally excluding commits from being added to the queue in
the BackgroundObserver
* the ChangeProcessor using OAK-4907's ChangeSet of the CommitContext for
best-effort prefiltering
* the BackgroundObserver signaling downstream Observers that a change should be
excluded via a {{NOOP_CHANGE}} CommitInfo
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4907) Collect changes (paths, nts, props..) of a commit in a validator
[ https://issues.apache.org/jira/browse/OAK-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-4907: - Description: It would be useful to collect a set of changes of a commit (eg in a validator) that could later be used in an Observer for eg prefiltering. Such a change collector should collect paths, nodetypes, properties, node-names (and perhaps more at a later stage) of all changes and store the result in the CommitInfo's CommitContext. Note that this is a result of [discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962] around design in OAK-4796 was: It would be useful to collect a set of changes of a commit (eg in a validator) that could later be used in an Observer for eg prefiltering. Such a change collector should collect paths, nodetypes, properties, node-names (and perhaps more at a later stage) of all changes and store the result in the CommitInfo's CommitContext. > Collect changes (paths, nts, props..) of a commit in a validator > > > Key: OAK-4907 > URL: https://issues.apache.org/jira/browse/OAK-4907 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: core >Affects Versions: 1.5.11 >Reporter: Stefan Egli >Assignee: Stefan Egli > Fix For: 1.6 > > > It would be useful to collect a set of changes of a commit (eg in a > validator) that could later be used in an Observer for eg prefiltering. > Such a change collector should collect paths, nodetypes, properties, > node-names (and perhaps more at a later stage) of all changes and store the > result in the CommitInfo's CommitContext. > Note that this is a result of > [discussions|https://issues.apache.org/jira/browse/OAK-4796?focusedCommentId=15550962=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15550962] > around design in OAK-4796 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4907) Collect changes (paths, nts, props..) of a commit in a validator
Stefan Egli created OAK-4907: Summary: Collect changes (paths, nts, props..) of a commit in a validator Key: OAK-4907 URL: https://issues.apache.org/jira/browse/OAK-4907 Project: Jackrabbit Oak Issue Type: Technical task Components: core Affects Versions: 1.5.11 Reporter: Stefan Egli Assignee: Stefan Egli Fix For: 1.6 It would be useful to collect a set of changes of a commit (eg in a validator) that could later be used in an Observer for eg prefiltering. Such a change collector should collect paths, nodetypes, properties, node-names (and perhaps more at a later stage) of all changes and store the result in the CommitInfo's CommitContext. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4906) Support relative property based query by transforming the path
Chetan Mehrotra created OAK-4906:
Summary: Support relative property based query by transforming the
path
Key: OAK-4906
URL: https://issues.apache.org/jira/browse/OAK-4906
Project: Jackrabbit Oak
Issue Type: Improvement
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.6
PropertyIndex supports query like below
{noformat}
/jcr:root//*[jcr:content/@keywords]
{noformat}
For such query _keywords_ property is indexed and the as part of Cursor
returned for the query the path is transformed to return the parent path if the
parent node is _jcr:content_
Currently LucenePropertyIndex supports indexing relative properties wrt base
node like {{jcr:content/metadata/status}}. Here it would index the relative
property by refererring to its relative name.
To simplify switch to LucenePropertyIndex (as part of hybrid index support)
LucenePropertyIndex should also support for such queries where property index
definition only specifies indexing _keywords_ but the query is using a relative
property. So LucenePropertyIndex should
# First check if it has a property definition for _jcr:content/keywords_
# If not check if the indexingRule is for nt:base
# If yes check if it has property definition for _keywords_
# If yes then use the index to evaluate the query and return transformed result
LucenePropertyIndex already do such transformations for fulltext constraints.
With this we extend that support for property restrictions also
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
