[jira] [Resolved] (OAK-6354) Move DocumentNodeStoreMBean implementation in separate file

2017-06-19 Thread Marcel Reutegger (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcel Reutegger resolved OAK-6354.
---
   Resolution: Fixed
Fix Version/s: 1.7.2

Done in trunk: http://svn.apache.org/r1798836

> Move DocumentNodeStoreMBean implementation in separate file
> ---
>
> Key: OAK-6354
> URL: https://issues.apache.org/jira/browse/OAK-6354
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: documentmk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Fix For: 1.8, 1.7.2
>
>
> For readability it would be better to move the class to its own file. The 
> DocumentNodeStore class already has too many lines of code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6358) Moving a page failing to update internal references

2017-06-19 Thread Marcel Reutegger (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcel Reutegger resolved OAK-6358.
---
Resolution: Invalid

This issue looks related to the Adobe product AEM and contains a lot of product 
specific information unrelated to Oak.

If you think there is indeed a problem with Apache Oak, then please file an 
issue with detailed information how to reproduce this problem with just the 
repository implementation. Better yet, provide a test that shows the problem 
you think there is.

> Moving a page failing to update internal references
> ---
>
> Key: OAK-6358
> URL: https://issues.apache.org/jira/browse/OAK-6358
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: authorization-cug, jcr, security
>Affects Versions: 1.4.1
>Reporter: abdul hameed pathan
>Priority: Critical
> Attachments: ACL_Snapshot.PNG
>
>
> When moving a page in AEM, reference adjustment service user, updates the 
> property that contains reference to page/asset. It fails with access denied 
> if it is a multi-valued properties.
> reference-adjustment-service user has jcr:read that node and its parent and 
> rep:alterProperties on that node...
> [^ACL_Snapshot.PNG]
> code that modifies the references is at
> https://git.corp.adobe.com/CQ/wcm/blob/master/bundles/cq-wcm-commons/src/main/java/com/day/cq/wcm/commons/ReferenceSearch.java#L466
> it works for simple properties i.e. OOTB sample. But fails for multi-valued 
> properties, as in case of customer. If we give higher level permission i.e. 
> jcr:all it is working well for multi-valued properties as well
> Below is the Error Message:
> {noformat}
> 16.06.2017 12:44:13.228 *ERROR* [127.0.0.1 [1497597252043] POST 
> /bin/wcmcommand HTTP/1.1] com.day.cq.wcm.commons.ReferenceSearch Error while 
> adjusting references.
> org.apache.sling.api.resource.PersistenceException: Unable to commit changes 
> to session.
>   at 
> org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProvider.commit(JcrResourceProvider.java:482)
>   at 
> org.apache.sling.resourceresolver.impl.providers.stateful.AuthenticatedResourceProvider.commit(AuthenticatedResourceProvider.java:215)
>   at 
> org.apache.sling.resourceresolver.impl.helper.ResourceResolverControl.commit(ResourceResolverControl.java:411)
>   at 
> org.apache.sling.resourceresolver.impl.ResourceResolverImpl.commit(ResourceResolverImpl.java:1224)
>   at 
> com.day.cq.wcm.commons.ReferenceSearch.adjustReferences(ReferenceSearch.java:379)
>   at 
> com.day.cq.wcm.core.impl.PageManagerImpl.move(PageManagerImpl.java:521)
>   at 
> com.day.cq.wcm.core.impl.commands.AbstractCopyMoveCommand.doCommand(AbstractCopyMoveCommand.java:182)
>   at 
> com.day.cq.wcm.core.impl.commands.AbstractCopyMoveCommand.performCommand(AbstractCopyMoveCommand.java:86)
>   at 
> com.day.cq.wcm.core.impl.commands.WCMCommandService$CommandHolder.performCommand(WCMCommandService.java:178)
>   at 
> com.day.cq.wcm.core.impl.commands.WCMCommandServlet.performCommand(WCMCommandServlet.java:111)
>   at 
> com.day.cq.commons.servlets.AbstractCommandServlet.doPost(AbstractCommandServlet.java:49)
>   at 
> org.apache.sling.api.servlets.SlingAllMethodsServlet.mayService(SlingAllMethodsServlet.java:149)
>   at 
> org.apache.sling.api.servlets.SlingSafeMethodsServlet.service(SlingSafeMethodsServlet.java:345)
>   at 
> org.apache.sling.api.servlets.SlingSafeMethodsServlet.service(SlingSafeMethodsServlet.java:376)
>   at 
> org.apache.sling.engine.impl.request.RequestData.service(RequestData.java:546)
>   at 
> org.apache.sling.engine.impl.filter.SlingComponentFilterChain.render(SlingComponentFilterChain.java:44)
>   at 
> org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:77)
>   at 
> com.day.cq.personalization.impl.TargetComponentFilter.doFilter(TargetComponentFilter.java:96)
>   at 
> org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
>   at 
> com.day.cq.wcm.core.impl.WCMDebugFilter.doFilter(WCMDebugFilter.java:151)
>   at 
> org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
>   at 
> com.day.cq.wcm.core.impl.WCMComponentFilter.filterRootInclude(WCMComponentFilter.java:362)
>   at 
> com.day.cq.wcm.core.impl.WCMComponentFilter.doFilter(WCMComponentFilter.java:177)
>   at 
> org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
>   at 
> org.apache.sling.engine.impl.SlingRequestProcessorImpl.processComponent(SlingRequestProcessorImpl.java:282)
>   at 
> org.apache.sling.engine.impl.filter.RequestSlingFilterChain.render(Reque

[jira] [Resolved] (OAK-6357) Build Jackrabbit Oak #424 failed

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig resolved OAK-6357.

Resolution: Duplicate

> Build Jackrabbit Oak #424 failed
> 
>
> Key: OAK-6357
> URL: https://issues.apache.org/jira/browse/OAK-6357
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: continuous integration
>Reporter: Hudson
>
> Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/
> The build Jackrabbit Oak #424 has failed.
> First failed run: [Jackrabbit Oak 
> #424|https://builds.apache.org/job/Jackrabbit%20Oak/424/] [console 
> log|https://builds.apache.org/job/Jackrabbit%20Oak/424/console]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Davide Giannella (JIRA)
Davide Giannella created OAK-6365:
-

 Summary: oak-store-spi fails on javadoc
 Key: OAK-6365
 URL: https://issues.apache.org/jira/browse/OAK-6365
 Project: Jackrabbit Oak
  Issue Type: Bug
  Components: store-spi
Affects Versions: 1.7.1
Reporter: Davide Giannella
Priority: Blocker
 Fix For: 1.7.2


oak-store-spi is blocking the release by failing on the javadoc. More details 
in the attached build logs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-6365:
--
Attachment: build-2017-06-19-085854.log.gz

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-6365:
--
Description: oak-store-spi is blocking the release by failing on the 
javadoc. More details in the attached build logs: 
[^build-2017-06-19-085854.log.gz].  (was: oak-store-spi is blocking the release 
by failing on the javadoc. More details in the attached build logs.)

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Robert Munteanu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053589#comment-16053589
 ] 

Robert Munteanu commented on OAK-6365:
--

Specific error seems to be

{noformat}[INFO] [ERROR] 
/work/sources/apache/releases/oak-svn-trunk/oak-store-spi/src/main/java/org/apache/jackrabbit/oak/spi/commit/CompositeConflictHandler.java:56:
 error: reference not found
[INFO] [ERROR]  * backing handler. Use {@link 
#addHandler(PartialConflictHandler)} to add additional{noformat}

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Chetan Mehrotra (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053601#comment-16053601
 ] 

Chetan Mehrotra commented on OAK-6365:
--

May be we should disable the failing of build via solution suggested at [1]. Or 
we should run javadoc as part of build to fail the build itself

[1] http://blog.joda.org/2014/02/turning-off-doclint-in-jdk-8-javadoc.html

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (OAK-6366) Detect unsupported combination of read and write concern

2017-06-19 Thread Marcel Reutegger (JIRA)
Marcel Reutegger created OAK-6366:
-

 Summary: Detect unsupported combination of read and write concern
 Key: OAK-6366
 URL: https://issues.apache.org/jira/browse/OAK-6366
 Project: Jackrabbit Oak
  Issue Type: Improvement
  Components: mongomk
Reporter: Marcel Reutegger
Priority: Minor
 Fix For: 1.8


With OAK-4069 Oak will try to use a majority read concern when connected to a 
replica set. The implementation should be more careful when setting a majority 
read concern automatically and take the write concern into account. It is 
currently possible to end up with a system that has a user configured write 
concern of 1 and a majority read concern. This results in a non-functional 
system and should be prevented.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6366) Detect unsupported combination of read and write concern

2017-06-19 Thread Marcel Reutegger (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053617#comment-16053617
 ] 

Marcel Reutegger commented on OAK-6366:
---

Either the system should refuse to start with this configuration or fall back 
to a local read concern. I slightly prefer the first because the fallback isn't 
a recommended configuration.

> Detect unsupported combination of read and write concern
> 
>
> Key: OAK-6366
> URL: https://issues.apache.org/jira/browse/OAK-6366
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: mongomk
>Reporter: Marcel Reutegger
>Priority: Minor
> Fix For: 1.8
>
>
> With OAK-4069 Oak will try to use a majority read concern when connected to a 
> replica set. The implementation should be more careful when setting a 
> majority read concern automatically and take the write concern into account. 
> It is currently possible to end up with a system that has a user configured 
> write concern of 1 and a majority read concern. This results in a 
> non-functional system and should be prevented.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Alex Deparvu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053619#comment-16053619
 ] 

Alex Deparvu commented on OAK-6365:
---

that's part of my commit. I'll take a look.

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Thomas Mueller (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053620#comment-16053620
 ] 

Thomas Mueller commented on OAK-6365:
-

FYI {{-Xdoclint:none}} only works with Java 8. I wonder if we (sometimes) 
compile with Java 7.

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-3712) Clean up uncommitted changes

2017-06-19 Thread Marcel Reutegger (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcel Reutegger resolved OAK-3712.
---
   Resolution: Fixed
Fix Version/s: 1.7.2

> Clean up uncommitted changes
> 
>
> Key: OAK-3712
> URL: https://issues.apache.org/jira/browse/OAK-3712
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: core, documentmk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
> Fix For: 1.8, 1.7.2
>
> Attachments: 
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreSweepTest.txt, 
> TEST-org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreSweepTest.xml,
>  unit-tests.log
>
>
> Clean up old and uncommitted changes in the main document. This issue is 
> related to OAK-2392, which is specifically about changes on binary properties 
> and effect on blob GC.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Chetan Mehrotra (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053631#comment-16053631
 ] 

Chetan Mehrotra commented on OAK-6365:
--

bq. FYI -Xdoclint:none only works with Java 8. I wonder if we (sometimes) 
compile with Java 7.

Not for trunk since we switched to 1.8 [1]. So should be safe to enable for 
trunk

[1] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-parent/pom.xml#L66

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6361) Oak run tika command should connect to NodeStore in read only mode

2017-06-19 Thread Chetan Mehrotra (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Mehrotra resolved OAK-6361.
--
   Resolution: Fixed
Fix Version/s: 1.7.2

Done with 1799155

> Oak run tika command should connect to NodeStore in read only mode
> --
>
> Key: OAK-6361
> URL: https://issues.apache.org/jira/browse/OAK-6361
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
>  Labels: candidate_oak_1_6
> Fix For: 1.8, 1.7.2
>
>
> The tika command in oak-run used for text pre extraction should connect to 
> NodeStore in read only mode



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Alex Deparvu (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Deparvu resolved OAK-6365.
---
Resolution: Fixed
  Assignee: Alex Deparvu

fixed with http://svn.apache.org/viewvc?rev=1799159&view=rev
FYI, running the javadoc build with this error does not fail the build for me 
locally.

sorry for the noise!

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Assignee: Alex Deparvu
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-6361) Oak run tika command should connect to NodeStore in read only mode

2017-06-19 Thread Chetan Mehrotra (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Mehrotra updated OAK-6361:
-
Labels: candidate_oak_1_6  (was: )

> Oak run tika command should connect to NodeStore in read only mode
> --
>
> Key: OAK-6361
> URL: https://issues.apache.org/jira/browse/OAK-6361
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
>  Labels: candidate_oak_1_6
> Fix For: 1.8, 1.7.2
>
>
> The tika command in oak-run used for text pre extraction should connect to 
> NodeStore in read only mode



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6363) BlobStoreFixtureProvider should support '.cfg' files also

2017-06-19 Thread Chetan Mehrotra (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Mehrotra resolved OAK-6363.
--
   Resolution: Fixed
Fix Version/s: 1.7.2

Done with 1799158. Now both '.config' and '.cfg' files are supported

> BlobStoreFixtureProvider should support '.cfg' files also
> -
>
> Key: OAK-6363
> URL: https://issues.apache.org/jira/browse/OAK-6363
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8, 1.7.2
>
>
> BlobStoreFixtureProvider currently support only '.config' files which 
> contains typed properties [1]. This should be enhanced to support '.cfg' 
> files also
> [1] 
> https://sling.apache.org/documentation/bundles/configuration-installer-factory.html#configuration-files-config



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6364) BlobStoreFixtureProvider should configure a default 'secret' value if none specified

2017-06-19 Thread Chetan Mehrotra (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Mehrotra resolved OAK-6364.
--
   Resolution: Fixed
Fix Version/s: 1.7.2

Done with 1799156. Now for any caching datastore a default value for "secret" 
property is set if none is specified

> BlobStoreFixtureProvider should configure a default 'secret' value if none 
> specified
> 
>
> Key: OAK-6364
> URL: https://issues.apache.org/jira/browse/OAK-6364
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8, 1.7.2
>
>
> When BlobStoreFixtureProvider constructs a S3DataStore then currently it does 
> not configure the "secret" property. This causes issue with Tika command as 
> it tries to convert the blobId to references which result in NPE as secret is 
> null
> {noformat}
> 11:56:35.915 [main] ERROR o.a.j.core.data.AbstractDataStore - Failed to hash 
> identifier using MAC (Message Authentication Code) algorithm.
> java.lang.NullPointerException: null
> at 
> org.apache.jackrabbit.core.data.CachingDataStore.getOrCreateReferenceKey(CachingDataStore.java:685)
>  ~[oak-run-1.8-SNAPSHOT.jar:1.8-SNAPSHOT]
> at 
> org.apache.jackrabbit.core.data.AbstractDataStore.getReferenceKey(AbstractDataStore.java:141)
>  ~[oak-run-1.8-SNAPSHOT.jar:1.8-SNAPSHOT]
> at 
> org.apache.jackrabbit.core.data.AbstractDataStore.getReferenceFromIdentifier(AbstractDataStore.java:100)
>  ~[oak-run-1.8-SNAPSHOT.jar:1.8-SNAPSHOT]
> at 
> org.apache.jackrabbit.core.data.AbstractDataRecord.getReference(AbstractDataRecord.java:60)
>  [oak-run-1.8-SNAPSHOT.jar:1.8-SNAPSHOT]
> at 
> org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore.getReference(DataStoreBlobStore.java:306)
>  [oak-run-1.8-SNAPSHOT.jar:1.8-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> {noformat}
> To avoid such a behaviour the BlobStoreFixtureProvider should set "secret" to 
> some random value if its not set already



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6362) Use NodeStoreFixtureProvider in tika command of oak-run

2017-06-19 Thread Chetan Mehrotra (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Mehrotra resolved OAK-6362.
--
   Resolution: Fixed
Fix Version/s: 1.7.2

Done with 1799157

> Use NodeStoreFixtureProvider in tika command of oak-run
> ---
>
> Key: OAK-6362
> URL: https://issues.apache.org/jira/browse/OAK-6362
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.2
>
>
> Oak run tika command currently has custom logic to construct NodeStore. This 
> should be refactored to switch to new NodeStoreFixtureProvider support as 
> implemented in OAK-6210



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Thomas Mueller (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053648#comment-16053648
 ] 

Thomas Mueller commented on OAK-6365:
-

[~stillalex] it fails for me, with Java 8 ({{mvn javadoc:javadoc}}). Maybe you 
use Java 7, or another way to create the javadocs?

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Assignee: Alex Deparvu
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (OAK-6276) expose way to detect "eventual consistency delay"

2017-06-19 Thread Stefan Egli (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Egli reassigned OAK-6276:


Assignee: Stefan Egli

> expose way to detect "eventual consistency delay"
> -
>
> Key: OAK-6276
> URL: https://issues.apache.org/jira/browse/OAK-6276
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: api
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>
> I have a requirement to support an external messaging channel (eg Kafka) 
> between Oak-based instances of the same cluster. As part of handling those 
> messages the target instance in some cases might have to access data from the 
> repository. 
> Now with DocumentNodeStore's eventual consistency that data might not 
> 'travel' from the source to the target instance as fast as is the case for 
> the external message.
> Therefore the need arises to be able to delay such messages (on the target 
> instance) until the repository sees (at least) the data the source instance 
> wrote when sending off the message.
> This ticket is to equip Oak with any feasible way for higher level code to 
> generally speaking detect such an "eventual consistency delay".
> One simple idea that comes to mind is to expose the current _head revision 
> vector_ (or that from a particular session, but that might not be required, 
> ie be too complicated). The source instance could get the local head revision 
> vector, piggyback that on the message, then that could be compared on the 
> target instance with its head state. If that turns out to be older, then it 
> could do a wait and retry. (Nicer would of course be if there would even be a 
> call-back - but in theory that could also be implemented ontop of an 
> Observer).
> One means to expose the head revision vector would be via a repository 
> descriptor (which on access returns the current value, similar to [how 
> discovery-lite does 
> it|https://github.com/apache/jackrabbit-oak/blob/2634dbde9aedc2549f0512285e9abee5858b256f/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentDiscoveryLiteService.java#L246]).
>  And the format could be normalized as eg {{longs}} (eg {{\[1496071927014, 
> 1496071926243]}} (instead of {{\["r15c54d532ec-0-1", "r15c54d532ec-0-2"]}} to 
> avoid leaking the revision format explicitly).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Alex Deparvu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053659#comment-16053659
 ] 

Alex Deparvu commented on OAK-6365:
---

Could you post the error? It can't be the same one as before.

bq. Maybe you use Java 7, or another way to create the javadocs?
{noformat}
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
{noformat}

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Assignee: Alex Deparvu
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-4637) Property index: include/exclude key pattern list

2017-06-19 Thread Thomas Mueller (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053711#comment-16053711
 ] 

Thomas Mueller commented on OAK-4637:
-

http://svn.apache.org/r1799169 (docs)

> Property index: include/exclude key pattern list
> 
>
> Key: OAK-4637
> URL: https://issues.apache.org/jira/browse/OAK-4637
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: property-index
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Critical
> Fix For: 1.8
>
>
> In some cases, property indexes contain many nodes, and updating them can be 
> slow. Right now we have filters for node and mixin types, path (include and 
> exclude). 
> An include and exclude list of values (patterns) would be useful. For example 
> the property "status", if we only ever run queries with the condition "status 
> = 'ACTIVE'", then nodes with status INACTIVE and DONE don't need to be 
> indexed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5902) Cold standby should allow syncing of blobs bigger than 2.2 GB

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-5902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-5902:
---
Fix Version/s: 1.7.4

> Cold standby should allow syncing of blobs bigger than 2.2 GB
> -
>
> Key: OAK-5902
> URL: https://issues.apache.org/jira/browse/OAK-5902
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segment-tar
>Affects Versions: 1.6.1
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Fix For: 1.8, 1.7.4
>
>
> Currently there is a limitation for the maximum binary size (in bytes) to be 
> synced between primary and standby instances. This matches 
> {{Integer.MAX_VALUE}} (2,147,483,647) bytes and no binaries bigger than this 
> limit can be synced between the instances.
> Per comment at [1], the current protocol needs to be changed to allow sending 
> of binaries in chunks, to surpass this limitation.
> [1] 
> https://github.com/apache/jackrabbit-oak/blob/1.6/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/standby/client/StandbyClient.java#L125



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-4637) Property index: include/exclude key pattern list

2017-06-19 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller updated OAK-4637:

Fix Version/s: 1.7.2

> Property index: include/exclude key pattern list
> 
>
> Key: OAK-4637
> URL: https://issues.apache.org/jira/browse/OAK-4637
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: property-index
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Critical
> Fix For: 1.8, 1.7.2
>
>
> In some cases, property indexes contain many nodes, and updating them can be 
> slow. Right now we have filters for node and mixin types, path (include and 
> exclude). 
> An include and exclude list of values (patterns) would be useful. For example 
> the property "status", if we only ever run queries with the condition "status 
> = 'ACTIVE'", then nodes with status INACTIVE and DONE don't need to be 
> indexed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-4637) Property index: include/exclude key pattern list

2017-06-19 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller resolved OAK-4637.
-
Resolution: Fixed

> Property index: include/exclude key pattern list
> 
>
> Key: OAK-4637
> URL: https://issues.apache.org/jira/browse/OAK-4637
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: property-index
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Critical
> Fix For: 1.8, 1.7.2
>
>
> In some cases, property indexes contain many nodes, and updating them can be 
> slow. Right now we have filters for node and mixin types, path (include and 
> exclude). 
> An include and exclude list of values (patterns) would be useful. For example 
> the property "status", if we only ever run queries with the condition "status 
> = 'ACTIVE'", then nodes with status INACTIVE and DONE don't need to be 
> indexed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5885) segment-tar should have a tarmkrecovery command

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-5885:
---
Labels: tooling  (was: )

> segment-tar should have a tarmkrecovery command
> ---
>
> Key: OAK-5885
> URL: https://issues.apache.org/jira/browse/OAK-5885
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: run, segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
>  Labels: tooling
> Fix For: 1.8, 1.7.3
>
>
> {{oak-segment}} had a {{tarmkrecovery}} command responsible with listing 
> candidates for head journal entries. We should re-enable this also for 
> {{oak-segment-tar}}.
> /cc [~mduerig] [~frm]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6365) oak-store-spi fails on javadoc

2017-06-19 Thread Amit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053725#comment-16053725
 ] 

Amit Jain commented on OAK-6365:


I tried {{mvn javadoc:javadoc}} and it works fine for me

> oak-store-spi fails on javadoc
> --
>
> Key: OAK-6365
> URL: https://issues.apache.org/jira/browse/OAK-6365
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: store-spi
>Affects Versions: 1.7.1
>Reporter: Davide Giannella
>Assignee: Alex Deparvu
>Priority: Blocker
> Fix For: 1.7.2
>
> Attachments: build-2017-06-19-085854.log.gz
>
>
> oak-store-spi is blocking the release by failing on the javadoc. More details 
> in the attached build logs: [^build-2017-06-19-085854.log.gz].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5278) Improved compaction estimator

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-5278:
---
Priority: Minor  (was: Major)

> Improved compaction estimator
> -
>
> Key: OAK-5278
> URL: https://issues.apache.org/jira/browse/OAK-5278
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: segment-tar
>Reporter: Michael Dürig
>Priority: Minor
>  Labels: gc, operations
> Fix For: 1.8
>
>
> OAK-4293 introduced a new approach for estimating whether we actually want to 
> run or skip a gc cycle. That approach is purely based on the absolute growth 
> of the repository's on disk footprint. 
> I think this can be further refined as with the {{GCJournal}} we can 
> effectively extrapolate the amount of garbage at a given point in time given 
> the history of previous gc cycles. E.g. let {{S_n}} be the size of the 
> repository and {{G_n}} the percentage of garbage right before the {{n}}-th gc 
> cycle. We can then linearly extrapolate the garbage {{G_n+1}} for the 
> {{n+1}}-the gc cycle along the repository sizes:
> {code}
> G_n+1 = G_n * (S_k+1 - S_k)/(S_k - S_k-1)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-6359) Change behavior for very complex queries

2017-06-19 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller updated OAK-6359:

Priority: Critical  (was: Major)

> Change behavior for very complex queries
> 
>
> Key: OAK-6359
> URL: https://issues.apache.org/jira/browse/OAK-6359
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Thomas Mueller
>Priority: Critical
> Fix For: 1.8
>
>
> Very complex queries can cause take a long time to parse, and can possibly 
> cause OOME. It would be good if processing queries automatically stops after 
> some number of loops, after some time, or some amount of memory was used. And 
> then throw an exception "query too complex".
> Example query:
> {noformat}
> //element(*,rep:Authorizable)[ ( (  
>  jcr:like(fn:lower-case(fn:name()), 'audio%')
>  or jcr:contains(@rep:principalName,'audio*')
>  or jcr:contains(@cq:first-name,'audio*')
>  or jcr:contains(@cq:last-name,'audio*')
>  or jcr:contains(profile/@givenName,'audio*')
>  or jcr:contains(profile/@familyName,'audio*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'details:%')
>  or jcr:contains(@rep:principalName,'details:*')
>  or jcr:contains(@cq:first-name,'details:*')
>  or jcr:contains(@cq:last-name,'details:*')
>  or jcr:contains(profile/@givenName,'details:*')
>  or jcr:contains(profile/@familyName,'details:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'code:%')
>  or jcr:contains(@rep:principalName,'code:*')
>  or jcr:contains(@cq:first-name,'code:*')
>  or jcr:contains(@cq:last-name,'code:*')
>  or jcr:contains(profile/@givenName,'code:*')
>  or jcr:contains(profile/@familyName,'code:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), '123456%')
>  or jcr:contains(@rep:principalName,'123456*')
>  or jcr:contains(@cq:first-name,'123456*')
>  or jcr:contains(@cq:last-name,'123456*')
>  or jcr:contains(profile/@givenName,'123456*')
>  or jcr:contains(profile/@familyName,'123456*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'further%')
>  or jcr:contains(@rep:principalName,'further*')
>  or jcr:contains(@cq:first-name,'further*')
>  or jcr:contains(@cq:last-name,'further*')
>  or jcr:contains(profile/@givenName,'further*')
>  or jcr:contains(profile/@familyName,'further*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'dial%')
>  or jcr:contains(@rep:principalName,'dial*')
>  or jcr:contains(@cq:first-name,'dial*')
>  or jcr:contains(@cq:last-name,'dial*')
>  or jcr:contains(profile/@givenName,'dial*')
>  or jcr:contains(profile/@familyName,'dial*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'in%')
>  or jcr:contains(@rep:principalName,'in*')
>  or jcr:contains(@cq:first-name,'in*')
>  or jcr:contains(@cq:last-name,'in*')
>  or jcr:contains(profile/@givenName,'in*')
>  or jcr:contains(profile/@familyName,'in*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'numbers:%')
>  or jcr:contains(@rep:principalName,'numbers:*')
>  or jcr:contains(@cq:first-name,'numbers:*')
>  or jcr:contains(@cq:last-name,'numbers:*')
>  or jcr:contains(profile/@givenName,'numbers:*')
>  or jcr:contains(profile/@familyName,'numbers:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'http://wikipedia.org%')
>  or jcr:contains(@rep:principalName,'http://wikipedia.org*')
>  or jcr:contains(@cq:first-name,'http://wikipedia.org*')
>  or jcr:contains(@cq:last-name,'http://wikipedia.org*')
>  or jcr:contains(profile/@givenName,'http://wikipedia.org*')
>  or jcr:contains(profile/@familyName,'http://wikipedia.org*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'number(s):%')
>  or jcr:contains(@rep:principalName,'number(s):*')
>  or jcr:contains(@cq:first-name,'number(s):*')
>  or jcr:contains(@cq:last-name,'number(s):*')
>  or jcr:contains(profile/@givenName,'number(s):*')
>  or jcr:contains(profile/@familyName,'number(s):*') )
>  and (  jcr:like(fn:lower-cas

[jira] [Updated] (OAK-6359) Change behavior for very complex queries

2017-06-19 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller updated OAK-6359:

Component/s: query

> Change behavior for very complex queries
> 
>
> Key: OAK-6359
> URL: https://issues.apache.org/jira/browse/OAK-6359
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Thomas Mueller
>Priority: Critical
> Fix For: 1.8
>
>
> Very complex queries can cause take a long time to parse, and can possibly 
> cause OOME. It would be good if processing queries automatically stops after 
> some number of loops, after some time, or some amount of memory was used. And 
> then throw an exception "query too complex".
> Example query:
> {noformat}
> //element(*,rep:Authorizable)[ ( (  
>  jcr:like(fn:lower-case(fn:name()), 'audio%')
>  or jcr:contains(@rep:principalName,'audio*')
>  or jcr:contains(@cq:first-name,'audio*')
>  or jcr:contains(@cq:last-name,'audio*')
>  or jcr:contains(profile/@givenName,'audio*')
>  or jcr:contains(profile/@familyName,'audio*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'details:%')
>  or jcr:contains(@rep:principalName,'details:*')
>  or jcr:contains(@cq:first-name,'details:*')
>  or jcr:contains(@cq:last-name,'details:*')
>  or jcr:contains(profile/@givenName,'details:*')
>  or jcr:contains(profile/@familyName,'details:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'code:%')
>  or jcr:contains(@rep:principalName,'code:*')
>  or jcr:contains(@cq:first-name,'code:*')
>  or jcr:contains(@cq:last-name,'code:*')
>  or jcr:contains(profile/@givenName,'code:*')
>  or jcr:contains(profile/@familyName,'code:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), '123456%')
>  or jcr:contains(@rep:principalName,'123456*')
>  or jcr:contains(@cq:first-name,'123456*')
>  or jcr:contains(@cq:last-name,'123456*')
>  or jcr:contains(profile/@givenName,'123456*')
>  or jcr:contains(profile/@familyName,'123456*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'further%')
>  or jcr:contains(@rep:principalName,'further*')
>  or jcr:contains(@cq:first-name,'further*')
>  or jcr:contains(@cq:last-name,'further*')
>  or jcr:contains(profile/@givenName,'further*')
>  or jcr:contains(profile/@familyName,'further*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'dial%')
>  or jcr:contains(@rep:principalName,'dial*')
>  or jcr:contains(@cq:first-name,'dial*')
>  or jcr:contains(@cq:last-name,'dial*')
>  or jcr:contains(profile/@givenName,'dial*')
>  or jcr:contains(profile/@familyName,'dial*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'in%')
>  or jcr:contains(@rep:principalName,'in*')
>  or jcr:contains(@cq:first-name,'in*')
>  or jcr:contains(@cq:last-name,'in*')
>  or jcr:contains(profile/@givenName,'in*')
>  or jcr:contains(profile/@familyName,'in*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'numbers:%')
>  or jcr:contains(@rep:principalName,'numbers:*')
>  or jcr:contains(@cq:first-name,'numbers:*')
>  or jcr:contains(@cq:last-name,'numbers:*')
>  or jcr:contains(profile/@givenName,'numbers:*')
>  or jcr:contains(profile/@familyName,'numbers:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'http://wikipedia.org%')
>  or jcr:contains(@rep:principalName,'http://wikipedia.org*')
>  or jcr:contains(@cq:first-name,'http://wikipedia.org*')
>  or jcr:contains(@cq:last-name,'http://wikipedia.org*')
>  or jcr:contains(profile/@givenName,'http://wikipedia.org*')
>  or jcr:contains(profile/@familyName,'http://wikipedia.org*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'number(s):%')
>  or jcr:contains(@rep:principalName,'number(s):*')
>  or jcr:contains(@cq:first-name,'number(s):*')
>  or jcr:contains(@cq:last-name,'number(s):*')
>  or jcr:contains(profile/@givenName,'number(s):*')
>  or jcr:contains(profile/@familyName,'number(s):*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 

[jira] [Created] (OAK-6367) Build Jackrabbit Oak #426 failed

2017-06-19 Thread Hudson (JIRA)
Hudson created OAK-6367:
---

 Summary: Build Jackrabbit Oak #426 failed
 Key: OAK-6367
 URL: https://issues.apache.org/jira/browse/OAK-6367
 Project: Jackrabbit Oak
  Issue Type: Bug
  Components: continuous integration
Reporter: Hudson


Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/

The build Jackrabbit Oak #426 has failed.
First failed run: [Jackrabbit Oak 
#426|https://builds.apache.org/job/Jackrabbit%20Oak/426/] [console 
log|https://builds.apache.org/job/Jackrabbit%20Oak/426/console]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6248) Enable use of pre extracted text cache

2017-06-19 Thread Chetan Mehrotra (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Mehrotra resolved OAK-6248.
--
   Resolution: Fixed
Fix Version/s: 1.7.2

Done with 1799173. One can now specify the pre extracted text directory via  
`--pre-extracted-text-dir` option

> Enable use of pre extracted text cache
> --
>
> Key: OAK-6248
> URL: https://issues.apache.org/jira/browse/OAK-6248
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8, 1.7.2
>
>
> The indexing logic should be able to use Pre Extracted text support [1]
> [1] https://jackrabbit.apache.org/oak/docs/query/lucene.html#text-extraction



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-6367) Build Jackrabbit Oak #426 failed

2017-06-19 Thread Marcel Reutegger (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcel Reutegger updated OAK-6367:
--
Description: 
Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/

The build Jackrabbit Oak #426 has failed.
First failed run: [Jackrabbit Oak 
#426|https://builds.apache.org/job/Jackrabbit%20Oak/426/] [console 
log|https://builds.apache.org/job/Jackrabbit%20Oak/426/console]

{noformat}
[INFO] Running org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.788 s 
<<< FAILURE! - in org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest
[ERROR] 
validateMigration(org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest)  
Time elapsed: 3.788 s  <<< ERROR!
javax.jcr.RepositoryException: Failed to copy content
Caused by: java.lang.IllegalStateException: Branch with failed reset
Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: OakOak0100: 
Branch reset failed
Caused by: org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: 
Empty branch cannot be reset
{noformat}

  was:
Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/

The build Jackrabbit Oak #426 has failed.
First failed run: [Jackrabbit Oak 
#426|https://builds.apache.org/job/Jackrabbit%20Oak/426/] [console 
log|https://builds.apache.org/job/Jackrabbit%20Oak/426/console]


> Build Jackrabbit Oak #426 failed
> 
>
> Key: OAK-6367
> URL: https://issues.apache.org/jira/browse/OAK-6367
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: continuous integration
>Reporter: Hudson
>
> Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/
> The build Jackrabbit Oak #426 has failed.
> First failed run: [Jackrabbit Oak 
> #426|https://builds.apache.org/job/Jackrabbit%20Oak/426/] [console 
> log|https://builds.apache.org/job/Jackrabbit%20Oak/426/console]
> {noformat}
> [INFO] Running org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.788 
> s <<< FAILURE! - in org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest
> [ERROR] 
> validateMigration(org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest)  
> Time elapsed: 3.788 s  <<< ERROR!
> javax.jcr.RepositoryException: Failed to copy content
> Caused by: java.lang.IllegalStateException: Branch with failed reset
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: OakOak0100: 
> Branch reset failed
> Caused by: org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: 
> Empty branch cannot be reset
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-3679) Rollback to timestamp

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig resolved OAK-3679.

   Resolution: Later
Fix Version/s: (was: 1.8)

As of OAK-4095 the {{journal.log}} file contains timestamps of the respective 
revisions. This is sufficient to roll back manually easily enough for people 
who know what they are doing. Once we have a strong case for simplifying this 
through tooling we can take it up again. 

> Rollback to timestamp
> -
>
> Key: OAK-3679
> URL: https://issues.apache.org/jira/browse/OAK-3679
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: core, documentmk, segment-tar
>Reporter: Thomas Mueller
>  Labels: operations, tooling
>
> We should have a feature to roll back to a certain point in time. The use 
> case are: 
> * undo a failed, large operation (for example upgrade, migration, installing 
> a package),
> * on a copy of the repository, switch to an old state for reading old content
> * recover from a corruption (for example corruption due to incorrect 
> "discovery" state, such as concurrent async index updates).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6286) Use BlobStore in read only mode by default unless read-write mode enabled

2017-06-19 Thread Chetan Mehrotra (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Mehrotra resolved OAK-6286.
--
Resolution: Fixed

> Use BlobStore in read only mode by default unless read-write mode enabled
> -
>
> Key: OAK-6286
> URL: https://issues.apache.org/jira/browse/OAK-6286
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.2
>
>
> Currently by default the NodeStore implementations are configured in read 
> only mode (unless "--read-write" option is enabled). This ensures that any 
> command does not make any change in NodeStore
> Similar support should be added for BlobStore



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6367) Build Jackrabbit Oak #426 failed

2017-06-19 Thread Marcel Reutegger (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcel Reutegger resolved OAK-6367.
---
Resolution: Duplicate

> Build Jackrabbit Oak #426 failed
> 
>
> Key: OAK-6367
> URL: https://issues.apache.org/jira/browse/OAK-6367
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: continuous integration
>Reporter: Hudson
>
> Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/
> The build Jackrabbit Oak #426 has failed.
> First failed run: [Jackrabbit Oak 
> #426|https://builds.apache.org/job/Jackrabbit%20Oak/426/] [console 
> log|https://builds.apache.org/job/Jackrabbit%20Oak/426/console]
> {noformat}
> [INFO] Running org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.788 
> s <<< FAILURE! - in org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest
> [ERROR] 
> validateMigration(org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest)  
> Time elapsed: 3.788 s  <<< ERROR!
> javax.jcr.RepositoryException: Failed to copy content
> Caused by: java.lang.IllegalStateException: Branch with failed reset
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: OakOak0100: 
> Branch reset failed
> Caused by: org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: 
> Empty branch cannot be reset
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (OAK-6367) Build Jackrabbit Oak #426 failed

2017-06-19 Thread Marcel Reutegger (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcel Reutegger closed OAK-6367.
-

> Build Jackrabbit Oak #426 failed
> 
>
> Key: OAK-6367
> URL: https://issues.apache.org/jira/browse/OAK-6367
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: continuous integration
>Reporter: Hudson
>
> Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/
> The build Jackrabbit Oak #426 has failed.
> First failed run: [Jackrabbit Oak 
> #426|https://builds.apache.org/job/Jackrabbit%20Oak/426/] [console 
> log|https://builds.apache.org/job/Jackrabbit%20Oak/426/console]
> {noformat}
> [INFO] Running org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.788 
> s <<< FAILURE! - in org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest
> [ERROR] 
> validateMigration(org.apache.jackrabbit.oak.upgrade.cli.JdbcToSegmentTest)  
> Time elapsed: 3.788 s  <<< ERROR!
> javax.jcr.RepositoryException: Failed to copy content
> Caused by: java.lang.IllegalStateException: Branch with failed reset
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: OakOak0100: 
> Branch reset failed
> Caused by: org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: 
> Empty branch cannot be reset
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5635) Revisit FileStoreStats mbean stats format

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-5635:
---
Fix Version/s: 1.7.6

> Revisit FileStoreStats mbean stats format
> -
>
> Key: OAK-5635
> URL: https://issues.apache.org/jira/browse/OAK-5635
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segment-tar
>Reporter: Alex Deparvu
>  Labels: monitoring
> Fix For: 1.8, 1.7.6
>
>
> This is a bigger refactoring item to revisit the format of the exposed data, 
> moving towards having it in a more machine consumable friendly format.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-5635) Revisit FileStoreStats mbean stats format

2017-06-19 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053757#comment-16053757
 ] 

Michael Dürig commented on OAK-5635:


For now let's go through all JMX endpoints and duplicate those that are not 
easily machine consumable accordingly. 

> Revisit FileStoreStats mbean stats format
> -
>
> Key: OAK-5635
> URL: https://issues.apache.org/jira/browse/OAK-5635
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segment-tar
>Reporter: Alex Deparvu
>  Labels: monitoring
> Fix For: 1.8, 1.7.6
>
>
> This is a bigger refactoring item to revisit the format of the exposed data, 
> moving towards having it in a more machine consumable friendly format.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-5278) Improved compaction estimator

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig resolved OAK-5278.

   Resolution: Later
Fix Version/s: (was: 1.8)

Resolving as later as the idea might be good. However as long as the current 
approach is sufficient we don't need to further complicate things. 

> Improved compaction estimator
> -
>
> Key: OAK-5278
> URL: https://issues.apache.org/jira/browse/OAK-5278
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: segment-tar
>Reporter: Michael Dürig
>Priority: Minor
>  Labels: gc, operations
>
> OAK-4293 introduced a new approach for estimating whether we actually want to 
> run or skip a gc cycle. That approach is purely based on the absolute growth 
> of the repository's on disk footprint. 
> I think this can be further refined as with the {{GCJournal}} we can 
> effectively extrapolate the amount of garbage at a given point in time given 
> the history of previous gc cycles. E.g. let {{S_n}} be the size of the 
> repository and {{G_n}} the percentage of garbage right before the {{n}}-th gc 
> cycle. We can then linearly extrapolate the garbage {{G_n+1}} for the 
> {{n+1}}-the gc cycle along the repository sizes:
> {code}
> G_n+1 = G_n * (S_k+1 - S_k)/(S_k - S_k-1)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-5885) segment-tar should have a tarmkrecovery command

2017-06-19 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053773#comment-16053773
 ] 

Michael Dürig commented on OAK-5885:


The problem with the previous tooling was that it couldn't guarantee that the 
returned segment ids where in reverse chronological order. They would be in 
most cases but when written by different {{SegmentBufferWritters}} concurrently 
this wasn't guaranteed. 

To fix this we need to keep a (soft) link from each head super root records to 
its predecessor. This could either be done by introducing a new record type for 
super root records or by adding a property to the super root recording with 
that information. The former approach would require us to bump the segment 
format version while staying backwards compatible with the previous version. 
The latter approach would not require a segment version bump but rely on 
heuristics to detect super roots (by checking the presence of the right child 
nodes and properties). Given the minor priority of this issue and the fact that 
the previous version of this tooling also had the limitation re. heuristics I 
would opt. for the latter approach. 

> segment-tar should have a tarmkrecovery command
> ---
>
> Key: OAK-5885
> URL: https://issues.apache.org/jira/browse/OAK-5885
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: run, segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
>  Labels: tooling
> Fix For: 1.8, 1.7.6
>
>
> {{oak-segment}} had a {{tarmkrecovery}} command responsible with listing 
> candidates for head journal entries. We should re-enable this also for 
> {{oak-segment-tar}}.
> /cc [~mduerig] [~frm]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5885) segment-tar should have a tarmkrecovery command

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-5885:
---
Fix Version/s: (was: 1.7.3)
   1.7.6

> segment-tar should have a tarmkrecovery command
> ---
>
> Key: OAK-5885
> URL: https://issues.apache.org/jira/browse/OAK-5885
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: run, segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
>  Labels: tooling
> Fix For: 1.8, 1.7.6
>
>
> {{oak-segment}} had a {{tarmkrecovery}} command responsible with listing 
> candidates for head journal entries. We should re-enable this also for 
> {{oak-segment-tar}}.
> /cc [~mduerig] [~frm]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-6337) Decide major version bump of o.a.j.o.api.jmx

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-6337:
--
Fix Version/s: (was: 1.7.2)
   1.7.3
   1.8

> Decide major version bump of o.a.j.o.api.jmx
> 
>
> Key: OAK-6337
> URL: https://issues.apache.org/jira/browse/OAK-6337
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: api
>Reporter: Vikas Saurabh
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.3
>
>
> With some mixup while resolving OAK-2808, 
> [r1797739|https://svn.apache.org/r1797739] made into 1.7.1 release while 
> [r1797778|https://svn.apache.org/1797778].
> This led to OAK-6335 once 1.7.1 was released. To fix OAK-6335, I bumped 
> o.a.j.o.api.jmx from 4.6.0 to 5.0.0.
> This bump in major version might not be desirable in general (as was being 
> discussed on oak-dev recently 
> [here|https://lists.apache.org/thread.html/598a58727f3652f28879825ecc501b466586054a0fbd9c27b56b6338@%3Coak-commits.jackrabbit.apache.org%3E]
>  \[0].
> Towards that, if we decide to change baseline plugin's config to use last 
> stable branch's releases as base then we can avoid this major bump.
> [~chetanm] suggested offline that maybe we should "temporarily" add that 
> removed method back with deprecated flag + revert major version bump. That 
> should give us some breathing space to decide about how do we want to go 
> ahead with this.
> \[0]: 
> https://lists.apache.org/thread.html/598a58727f3652f28879825ecc501b466586054a0fbd9c27b56b6338@%3Coak-commits.jackrabbit.apache.org%3E



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5918) Document enhancements in DocumentNodeStore in 1.6

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5918:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Document enhancements in DocumentNodeStore in 1.6
> -
>
> Key: OAK-5918
> URL: https://issues.apache.org/jira/browse/OAK-5918
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: doc
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.3
>
>
> This task is meant to collect and refer work done in 1.6 release which needs 
> to be documented in Oak docs. Specially those enhancements which impact 
> system administration or new features which need be to enabled as per 
> requirements should be documented
> Issues in documentmk, mongomk, rdbmk 
> [jql|https://issues.apache.org/jira/issues/?jql=project%20%3D%20OAK%20AND%20fixVersion%20%3D%201.6.0%20and%20component%20in%20(documentmk%2C%20mongomk%2C%20rdbmk)%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC%2C%20created%20ASC&startIndex=200]
> * OAK-1312 - Bundle nodes into a document (/)
> * OAK-4180 - Use another NodeStore as a local cache for a remote Document 
> store (/)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5877) Oak upgrade usage note refers to oak-run

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5877:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Oak upgrade usage note refers to oak-run
> 
>
> Key: OAK-5877
> URL: https://issues.apache.org/jira/browse/OAK-5877
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: upgrade
>Reporter: Michael Dürig
>Priority: Minor
>  Labels: production, tooling, usability
> Fix For: 1.8, 1.7.3
>
>
> Running {{java -jar oak-upgrade*.jar}} prints 
> {noformat}
> Usage: java -jar oak-run-*-jr2.jar upgrade [options] jcr2_source [destination]
>(to upgrade a JCR 2 repository)
>java -jar oak-run-*-jr2.jar upgrade [options] source destination
>(to migrate an Oak repository)
> {noformat}
> Which incorrectly refers to {{oak-run upgrade}}. The latter will send me back 
> to {{oak-run}}: "This command was moved to the oak-upgrade module". 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5976) Document enhancements in 1.6 release

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5976:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Document enhancements in 1.6 release
> 
>
> Key: OAK-5976
> URL: https://issues.apache.org/jira/browse/OAK-5976
> Project: Jackrabbit Oak
>  Issue Type: Documentation
>  Components: doc
>Reporter: Chetan Mehrotra
> Fix For: 1.8, 1.7.3
>
>
> We should document all the enhancement that have been done in Oak 1.6 release 
> and refer to them in main doc
> * 1.6.0 Release Notes - 
> https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.6.0/RELEASE-NOTES.txt



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5965) Support path exclusion in secondary nodestore

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5965:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Support path exclusion in secondary nodestore
> -
>
> Key: OAK-5965
> URL: https://issues.apache.org/jira/browse/OAK-5965
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>  Labels: secondary-nodestore
> Fix For: 1.8, 1.7.3
>
>
> Secondary NodeStore feature (OAK-4180) for now currently supports path 
> inclusion. It would be useful to have support for path exclusion also.
> Using this a user can can include all content  under / but exclude 
> /oak:index/uuid/:index entries.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5455) Specify versions for maven plugins used in build for ensuring stable builds

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5455:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Specify versions for maven plugins used in build for ensuring stable builds
> ---
>
> Key: OAK-5455
> URL: https://issues.apache.org/jira/browse/OAK-5455
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: parent
>Reporter: Chetan Mehrotra
>Assignee: Julian Reschke
> Fix For: 1.8, 1.7.3
>
> Attachments: enforce.diff, OAK-5455-v1.patch
>
>
> Running a check for plugin versions used in oak build 
> {noformat}
> mvn versions:display-plugin-updates
> {noformat}
> leads to following warning
> {noformat}
> [INFO] The following plugin updates are available:
> [INFO]   org.apache.felix:maven-scr-plugin .. 1.16.0 -> 1.21.0
> [INFO] 
> [WARNING] The following plugins do not have their version specified:
> [WARNING]   maven-compiler-plugin .. 2.0.2
> [WARNING]   maven-deploy-plugin . (from super-pom) 2.4
> [WARNING]   maven-failsafe-plugin . 2.12.4
> [WARNING]   maven-jar-plugin . 2.1
> [WARNING]   maven-javadoc-plugin . 2.0
> [WARNING]   maven-release-plugin . (from super-pom) 2.0-beta-4
> [WARNING]   maven-resources-plugin ... 2.2
> [WARNING]   maven-surefire-plugin .. 2.4.2
> [INFO] 
> [WARNING] Project does not define minimum Maven version, default is: 2.0
> [INFO] Plugins require minimum Maven version of: 3.0.5
> [INFO] Note: the super-pom from Maven 3.3.9 defines some of the plugin
> [INFO]   versions and may be influencing the plugins required minimum 
> Maven
> [INFO]   version.
> [INFO] 
> [ERROR] Project does not define required minimum version of Maven.
> [ERROR] Update the pom.xml to contain
> [ERROR] 
> [ERROR]   3.0.5
> [ERROR] 
> {noformat}
> As a fix we should
> # Specify version for all maven plugin in use
> # Specify minimum version of maven to be used (version used in CI is 3.2.1)
> # Configure enforcer plugin to ensure that in future no plugin is used 
> without specifying the version [1]
> [1] http://maven.apache.org/enforcer/enforcer-rules/requirePluginVersions.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5740) deliver overflow change even without new commit

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5740:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> deliver overflow change even without new commit
> ---
>
> Key: OAK-5740
> URL: https://issues.apache.org/jira/browse/OAK-5740
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: jcr
>Affects Versions: 1.6.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Minor
> Fix For: 1.8, 1.7.3
>
> Attachments: OAK-5740.StuffAtop-v2-patch.patch, 
> OAK-5740.testcase.patch, OAK-5740.testcase.patch, OAK-5740.v2.patch
>
>
> As [reported|http://markmail.org/message/2qxle24f6zu2vpms] by [~catholicon] 
> on oak-dev the observation queue only delivers the so-called _overflow 
> entry/change_ only when new commits are 'coming in'. We might want to 
> consider fixing this, even though arguably this is a very rare case (since 
> typically the observation queue is configured to be very large)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5975) Document enhancements in Observation in 1.6

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5975:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Document enhancements in Observation in 1.6
> ---
>
> Key: OAK-5975
> URL: https://issues.apache.org/jira/browse/OAK-5975
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: doc
>Reporter: Chetan Mehrotra
> Fix For: 1.8, 1.7.3
>
>
> This task is meant to collect and refer work done in 1.6 release which needs 
> to be documented in Oak docs wrt JCR Observation area
> * OAK-4796 - filter events before adding to ChangeProcessor's queue
> * OAK-5020 - Improved support for node removals
> * OAK-5021 - Improve observation of files



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5963) Disable S3 proactive caching by default

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5963:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Disable S3 proactive caching by default
> ---
>
> Key: OAK-5963
> URL: https://issues.apache.org/jira/browse/OAK-5963
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
>  Labels: candidate_oak_1_2, candidate_oak_1_4, candidate_oak_1_6
> Fix For: 1.8, 1.7.3
>
>
> The older JR2 CachingDataStore which is extended by S3DataStore enables 
> proactive caching by default which leads to many binaries being downloaded 
> twice. This should be disabled by default in Oak.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5602) avoid missing journal entries

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5602:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> avoid missing journal entries
> -
>
> Key: OAK-5602
> URL: https://issues.apache.org/jira/browse/OAK-5602
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.6.0
>Reporter: Stefan Egli
> Fix For: 1.8, 1.7.3
>
>
> As a follow up of OAK-5601: that one is about a situation where 
> backgroundRead falls way behind and journal GC cleans up journal entries 
> before it was able to read it. We should generally look into improving this 
> situation, eg by having journal GC not do the clean up as long as there are 
> instances that haven't read the entry. This could perhaps be achieved by 
> periodically having each instance inform the rest about what journal entries 
> it has processed (or simpler: how far back it would be -find- fine for 
> journal entries to be GC'ed)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5896) fix typo in Not condition handling

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5896:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> fix typo in Not condition handling
> --
>
> Key: OAK-5896
> URL: https://issues.apache.org/jira/browse/OAK-5896
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: query
>Affects Versions: 1.6.1
>Reporter: Dave Brosius
>Assignee: Dave Brosius
>Priority: Trivial
> Fix For: 1.8, 1.7.3
>
> Attachments: 5896.txt
>
>
> code stutters the same condition twice, looks like a typo. patch attached.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5884) Evaluate utility of RepositoryGrowthTest benchmark

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5884:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Evaluate utility of RepositoryGrowthTest benchmark
> --
>
> Key: OAK-5884
> URL: https://issues.apache.org/jira/browse/OAK-5884
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: run
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Fix For: 1.8, 1.7.3
>
>
> {{RepositoryGrowthTest}} is a benchmark which makes use of the deprecated 
> {{SegmentFixture}}. Since OAK-5834 removes the old {{oak-segment}} module and 
> the code associated with it, {{RepositoryGrowthTest}} was also removed. If 
> there's value in it, we can adapt it to work with the new 
> {{SegmentTarFixture}}.
> /cc [~chetanm]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-6265) Remove Mounts.defaultMount methods

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-6265:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Remove Mounts.defaultMount methods
> --
>
> Key: OAK-6265
> URL: https://issues.apache.org/jira/browse/OAK-6265
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: composite
>Reporter: Robert Munteanu
>Assignee: Robert Munteanu
> Fix For: 1.8, 1.7.3
>
> Attachments: 0001-OAK-6265-Remove-Mounts.defaultMount-methods.patch, 
> 0002-OAK-6265-Remove-Mounts.defaultMount-methods.patch
>
>
> These methods break the abstraction of constructing a composite setup
> and should not be used. Since the exported package version of 
> {{org.apache.jackrabbit.oak.spi.mount}} is set to 3.0.0 for 1.8-SNAPSHOT and 
> it was 2.2.0 in 1.6.1 it's safe to remove these methods if we do it before a 
> release from the 1.8 stream is cut.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-6252) re-introduce ServerCommand

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-6252:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> re-introduce ServerCommand
> --
>
> Key: OAK-6252
> URL: https://issues.apache.org/jira/browse/OAK-6252
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: run
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Fix For: 1.8, 1.7.3
>
> Attachments: OAK-6252-1.diff
>
>
> With 
> https://github.com/apache/jackrabbit-oak/commit/22067b7ffefa85b6673a919b30071c9b646d9298
>  the {{ServerCommand}} has been removed.
> Re-introduce it.
> /cc [~frm]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5763) Improve LockException message on org.apache.jackrabbit.oak.jcr.lock.LockManagerImpl unlock

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5763:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Improve LockException message on 
> org.apache.jackrabbit.oak.jcr.lock.LockManagerImpl unlock
> --
>
> Key: OAK-5763
> URL: https://issues.apache.org/jira/browse/OAK-5763
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: jcr
>Affects Versions: 1.3.3
>Reporter: Borja Clemente
> Fix For: 1.8, 1.7.3
>
>
> The error message on LockManagerImpl unlock method could also include 
> information about the user attempting to unlock the node as well as the 
> actual lock owner.
> Current exception is:
> {code}
> LockException("Not an owner of the lock " + path)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-6117) Enable lucene indexing via oak-run

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-6117:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Enable lucene indexing via oak-run
> --
>
> Key: OAK-6117
> URL: https://issues.apache.org/jira/browse/OAK-6117
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: lucene, run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.3
>
>
> This is first step of OAK-6081 where we need to enable performing Lucene 
> indexing via oak-run. The indexing approach would be very close to what we 
> already have with minimal changes done.
> Once this is done it should be possible to connect oak-run against an 
> existing setup and let it reindexing specific set of indexes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5998) Clarify and complete missing stuff in current Oak documentation

2017-06-19 Thread Davide Giannella (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-5998:
--
Fix Version/s: (was: 1.7.2)
   1.7.3

> Clarify and complete missing stuff in current Oak documentation
> ---
>
> Key: OAK-5998
> URL: https://issues.apache.org/jira/browse/OAK-5998
> Project: Jackrabbit Oak
>  Issue Type: Documentation
>  Components: doc
>Reporter: Chetan Mehrotra
> Fix For: 1.8, 1.7.3
>
>
> Current Oak documentation [1] is missing details around certain aspects of 
> Oak which makes it difficult for a new person to get up and running and more 
> important operate Oak in productions easily. Purpose of this task is list out 
> topic which must be documented to enable easier usage of Oak
> * Getting Started - Getting Oak with all features properly is tricky and not 
> easily possible with our [current getting 
> started|https://jackrabbit.apache.org/oak/docs/construct.html] specially for 
> prod setup. 
> ** Possibly refer or move doc of current examples  here
> ** For a new user not aware of JCR but aware of document storage a brief 
> overview on the JCR and how its the api for Oak which is to be used
> * Maintenance and Operations - 
> ** Oak being MVCC storage requires certain maintenance task like RevisionGC 
> and BlobGC to be run periodically.  
> ** Then related MBean should be documented. Note some part is covered in 
> [Segment 
> docs|https://jackrabbit.apache.org/oak/docs/nodestore/segment/overview.html#garbage-collection]
>  but we need to complete it for Document and also provide an overview
> ** System requirements in terms of RAM due to memory mapped usage, cloned 
> index files and persistent cache
> * How clustering works in Oak - [Clustering 
> doc|https://jackrabbit.apache.org/oak/docs/clustering.html] should provide 
> details on 
> ** how it works
> ** importance of background read and background write
> ** Effect of eventual consistency in cluster setup 
> ** Sticky session requirement (as per usecase)
> * Observation
> ** How it works
> ** How external and local events are generated
> ** Queue behaviour and overflow
> * Clarify requirement from host application wrt 
> ** scheduling singleton jobs used in Oak and how they should be executed
> ** Scheduling maintenance operations via MBeans
> * Document various OSGi config - May be generate a doc for all OSGi config in 
> Oak via some tooling
> Note list above is tentative and would be edited to determine important 
> topics in coming days and then specific sub task can be created to complete 
> it. Once any list item is completed add (/) against the list entry
> [1] https://jackrabbit.apache.org/oak/docs/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6360) Failed to retrieve previously indexed checkpoint in composite node store

2017-06-19 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053881#comment-16053881
 ] 

Tomek Rękawek commented on OAK-6360:


Added extra diagnostic info in [r1799187|https://svn.apache.org/r1799187].

> Failed to retrieve previously indexed checkpoint in composite node store
> 
>
> Key: OAK-6360
> URL: https://issues.apache.org/jira/browse/OAK-6360
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: composite
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.8
>
>
> In some cases following exception is logged:
> {noformat}
> 15.06.2017 11:51:13.871 *WARN* [sling-oak-1-Registered Service.722] 
> org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate [fulltext-async] 
> Failed to retrieve previously indexed checkpoint 
> 94afbccc-67da-41c2-bb31-45cc9f50a663; re-running the initial index update
> 15.06.2017 11:51:13.872 *INFO* [async-index-update-fulltext-async] 
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Found a new index node 
> [lucene]. Reindexing is requested
> 15.06.2017 11:51:13.877 *INFO* [async-index-update-fulltext-async] 
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing will be 
> performed for following indexes: [/oak:index/lucene]
> {noformat}
> We should investigate this. Let's start with logging more info (list of 
> checkpoints available on all stores and their metadata) if the checkpoint 
> can't be retrieved.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (OAK-6360) Failed to retrieve previously indexed checkpoint in composite node store

2017-06-19 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053881#comment-16053881
 ] 

Tomek Rękawek edited comment on OAK-6360 at 6/19/17 12:25 PM:
--

Added extra diagnostic info in [r1799187|https://svn.apache.org/r1799187], 
[r1799191|https://svn.apache.org/r1799191].


was (Author: tomek.rekawek):
Added extra diagnostic info in [r1799187|https://svn.apache.org/r1799187].

> Failed to retrieve previously indexed checkpoint in composite node store
> 
>
> Key: OAK-6360
> URL: https://issues.apache.org/jira/browse/OAK-6360
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: composite
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.8
>
>
> In some cases following exception is logged:
> {noformat}
> 15.06.2017 11:51:13.871 *WARN* [sling-oak-1-Registered Service.722] 
> org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate [fulltext-async] 
> Failed to retrieve previously indexed checkpoint 
> 94afbccc-67da-41c2-bb31-45cc9f50a663; re-running the initial index update
> 15.06.2017 11:51:13.872 *INFO* [async-index-update-fulltext-async] 
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Found a new index node 
> [lucene]. Reindexing is requested
> 15.06.2017 11:51:13.877 *INFO* [async-index-update-fulltext-async] 
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing will be 
> performed for following indexes: [/oak:index/lucene]
> {noformat}
> We should investigate this. Let's start with logging more info (list of 
> checkpoints available on all stores and their metadata) if the checkpoint 
> can't be retrieved.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5655) TarMK: Analyse locality of reference

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-5655:
---
Fix Version/s: (was: 1.7.3)
   1.7.6
   1.8

> TarMK: Analyse locality of reference 
> -
>
> Key: OAK-5655
> URL: https://issues.apache.org/jira/browse/OAK-5655
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: segment-tar
>Reporter: Michael Dürig
>  Labels: scalability
> Fix For: 1.8, 1.7.6
>
> Attachments: segment-per-path-compacted-nocache.png, 
> segment-per-path-compacted-nostringcache.png, segment-per-path-compacted.png, 
> segment-per-path.png
>
>
> We need to better understand the locality aspects of content stored in TarMK: 
> * How is related content spread over segments?
> * What content do we consider related? 
> * How does locality of related content develop over time when changes are 
> applied?
> * What changes do we consider typical?
> * What is the impact of compaction on locality? 
> * What is the impact of the deduplication caches on locality (during normal 
> operation and during compaction)?
> * How good are checkpoints deduplicated? Can we monitor this online?
> * ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-5103) Backup is not incremental i.e. previous tar files are duplicated

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-5103:
---
Priority: Minor  (was: Major)

> Backup is not incremental i.e. previous tar files are duplicated
> 
>
> Key: OAK-5103
> URL: https://issues.apache.org/jira/browse/OAK-5103
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segment-tar
>Affects Versions: 1.5.12
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
>  Labels: operations, production, tooling
> Fix For: 1.8
>
>
> Performing two backups via {{RepositoryManagementMBean.startBackup}}, 
> directory size increases not only with the delta, but also again with the 
> size of existing tar files. This lead me to the conclusion that backup is not 
> incremental.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-4866) Design and implement a proper backup and restore API

2017-06-19 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-4866:
---
Priority: Minor  (was: Major)

> Design and implement a proper backup and restore API
> 
>
> Key: OAK-4866
> URL: https://issues.apache.org/jira/browse/OAK-4866
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segment-tar
>Reporter: Francesco Mari
>Priority: Minor
>  Labels: operations, production
> Fix For: 1.8
>
>
> The current backup and restore API in {{org.apache.jackrabbit.oak.backup}} 
> refers to classes and interfaces that should remain private to the 
> oak-segment-tar bundle. This, in fact, is the reason why that package was not 
> exported as part of the effort for OAK-4843. The current backup and restore 
> API should be redesigned to be used from the outside of oak-segment-tar 
> without exporting implementation details.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (OAK-6368) Builder options not picked up after MongoDB is set

2017-06-19 Thread Marcel Reutegger (JIRA)
Marcel Reutegger created OAK-6368:
-

 Summary: Builder options not picked up after MongoDB is set
 Key: OAK-6368
 URL: https://issues.apache.org/jira/browse/OAK-6368
 Project: Jackrabbit Oak
  Issue Type: Bug
  Components: mongomk
Reporter: Marcel Reutegger
Assignee: Marcel Reutegger
Priority: Minor
 Fix For: 1.8


The MongoDocumentStore does not pick up options set on the builder after 
{{Builder.setMongoDB()}} is called.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6368) Builder options not picked up after MongoDB is set

2017-06-19 Thread Marcel Reutegger (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053912#comment-16053912
 ] 

Marcel Reutegger commented on OAK-6368:
---

Added an ignored test: http://svn.apache.org/r1799199

> Builder options not picked up after MongoDB is set
> --
>
> Key: OAK-6368
> URL: https://issues.apache.org/jira/browse/OAK-6368
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: mongomk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Fix For: 1.8
>
>
> The MongoDocumentStore does not pick up options set on the builder after 
> {{Builder.setMongoDB()}} is called.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6356) Allow CompositePermissionProvider to OR entries

2017-06-19 Thread Alex Deparvu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053953#comment-16053953
 ] 

Alex Deparvu commented on OAK-6356:
---

[~anchela] started work on this branch [0] please take a look!

* I have added OR variants to existing tests that cover the composite package.
* Not sure how to enable the OR version via config, would appreciate a hand here
* found 2 interesting items in the {{CompositeTreePermission}}, but those are 
not relevant to this patch, at least I don't think they are: {{canReadAll}} 
always returns false and second is {{canReadProperties}} does not seem to use a 
composite aggregation of existing treePermissions.
* had to locally bump a few OSGi export versions, not exactly sure why 
(probably unrelated to the patch)


[0] https://github.com/apache/jackrabbit-oak/compare/trunk...stillalex:oak-6356

> Allow CompositePermissionProvider to OR entries
> ---
>
> Key: OAK-6356
> URL: https://issues.apache.org/jira/browse/OAK-6356
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
>
> Currently the {{CompositePermissionProvider}} ANDs the entries and if any of 
> those denies a check, all the chain will fail early. I'd like to extend this 
> mechanism to 'OR' items if needed.
> A first application of this ORing could be the multiplexed permission store 
> where the default store could deny a check but a mount could allow it, so it 
> could be seen as valid.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6368) Builder options not picked up after MongoDB is set

2017-06-19 Thread Marcel Reutegger (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcel Reutegger resolved OAK-6368.
---
   Resolution: Fixed
Fix Version/s: 1.7.3

Fixed in trunk: http://svn.apache.org/r1799210

> Builder options not picked up after MongoDB is set
> --
>
> Key: OAK-6368
> URL: https://issues.apache.org/jira/browse/OAK-6368
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: mongomk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Fix For: 1.8, 1.7.3
>
>
> The MongoDocumentStore does not pick up options set on the builder after 
> {{Builder.setMongoDB()}} is called.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (OAK-6369) Reduce cache segments for revisions sweep

2017-06-19 Thread Marcel Reutegger (JIRA)
Marcel Reutegger created OAK-6369:
-

 Summary: Reduce cache segments for revisions sweep
 Key: OAK-6369
 URL: https://issues.apache.org/jira/browse/OAK-6369
 Project: Jackrabbit Oak
  Issue Type: Improvement
  Components: run
Reporter: Marcel Reutegger
Assignee: Marcel Reutegger
Priority: Minor
 Fix For: 1.8


The revisions sweep command in oak-run is single threaded and the number of 
segments in a cache can therefore be reduced. This allows to cache bigger 
entries, which otherwise would get evicted immediately.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (OAK-6369) Reduce cache segments for revisions sweep

2017-06-19 Thread Marcel Reutegger (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcel Reutegger resolved OAK-6369.
---
   Resolution: Fixed
Fix Version/s: 1.7.3

Done in trunk: http://svn.apache.org/r1799211

> Reduce cache segments for revisions sweep
> -
>
> Key: OAK-6369
> URL: https://issues.apache.org/jira/browse/OAK-6369
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: run
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Fix For: 1.8, 1.7.3
>
>
> The revisions sweep command in oak-run is single threaded and the number of 
> segments in a cache can therefore be reduced. This allows to cache bigger 
> entries, which otherwise would get evicted immediately.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (OAK-6359) Change behavior for very complex queries

2017-06-19 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller reassigned OAK-6359:
---

Assignee: Thomas Mueller

> Change behavior for very complex queries
> 
>
> Key: OAK-6359
> URL: https://issues.apache.org/jira/browse/OAK-6359
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Critical
> Fix For: 1.8, 1.7.3
>
>
> Very complex queries can cause take a long time to parse, and can possibly 
> cause OOME. It would be good if processing queries automatically stops after 
> some number of loops, after some time, or some amount of memory was used. And 
> then throw an exception "query too complex".
> Example query:
> {noformat}
> //element(*,rep:Authorizable)[ ( (  
>  jcr:like(fn:lower-case(fn:name()), 'audio%')
>  or jcr:contains(@rep:principalName,'audio*')
>  or jcr:contains(@cq:first-name,'audio*')
>  or jcr:contains(@cq:last-name,'audio*')
>  or jcr:contains(profile/@givenName,'audio*')
>  or jcr:contains(profile/@familyName,'audio*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'details:%')
>  or jcr:contains(@rep:principalName,'details:*')
>  or jcr:contains(@cq:first-name,'details:*')
>  or jcr:contains(@cq:last-name,'details:*')
>  or jcr:contains(profile/@givenName,'details:*')
>  or jcr:contains(profile/@familyName,'details:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'code:%')
>  or jcr:contains(@rep:principalName,'code:*')
>  or jcr:contains(@cq:first-name,'code:*')
>  or jcr:contains(@cq:last-name,'code:*')
>  or jcr:contains(profile/@givenName,'code:*')
>  or jcr:contains(profile/@familyName,'code:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), '123456%')
>  or jcr:contains(@rep:principalName,'123456*')
>  or jcr:contains(@cq:first-name,'123456*')
>  or jcr:contains(@cq:last-name,'123456*')
>  or jcr:contains(profile/@givenName,'123456*')
>  or jcr:contains(profile/@familyName,'123456*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'further%')
>  or jcr:contains(@rep:principalName,'further*')
>  or jcr:contains(@cq:first-name,'further*')
>  or jcr:contains(@cq:last-name,'further*')
>  or jcr:contains(profile/@givenName,'further*')
>  or jcr:contains(profile/@familyName,'further*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'dial%')
>  or jcr:contains(@rep:principalName,'dial*')
>  or jcr:contains(@cq:first-name,'dial*')
>  or jcr:contains(@cq:last-name,'dial*')
>  or jcr:contains(profile/@givenName,'dial*')
>  or jcr:contains(profile/@familyName,'dial*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'in%')
>  or jcr:contains(@rep:principalName,'in*')
>  or jcr:contains(@cq:first-name,'in*')
>  or jcr:contains(@cq:last-name,'in*')
>  or jcr:contains(profile/@givenName,'in*')
>  or jcr:contains(profile/@familyName,'in*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'numbers:%')
>  or jcr:contains(@rep:principalName,'numbers:*')
>  or jcr:contains(@cq:first-name,'numbers:*')
>  or jcr:contains(@cq:last-name,'numbers:*')
>  or jcr:contains(profile/@givenName,'numbers:*')
>  or jcr:contains(profile/@familyName,'numbers:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'http://wikipedia.org%')
>  or jcr:contains(@rep:principalName,'http://wikipedia.org*')
>  or jcr:contains(@cq:first-name,'http://wikipedia.org*')
>  or jcr:contains(@cq:last-name,'http://wikipedia.org*')
>  or jcr:contains(profile/@givenName,'http://wikipedia.org*')
>  or jcr:contains(profile/@familyName,'http://wikipedia.org*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'number(s):%')
>  or jcr:contains(@rep:principalName,'number(s):*')
>  or jcr:contains(@cq:first-name,'number(s):*')
>  or jcr:contains(@cq:last-name,'number(s):*')
>  or jcr:contains(profile/@givenName,'number(s):*')
>  or jcr:contains(profile/@familyName,'num

[jira] [Updated] (OAK-6359) Change behavior for very complex queries

2017-06-19 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller updated OAK-6359:

Fix Version/s: 1.7.3

> Change behavior for very complex queries
> 
>
> Key: OAK-6359
> URL: https://issues.apache.org/jira/browse/OAK-6359
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Critical
> Fix For: 1.8, 1.7.3
>
>
> Very complex queries can cause take a long time to parse, and can possibly 
> cause OOME. It would be good if processing queries automatically stops after 
> some number of loops, after some time, or some amount of memory was used. And 
> then throw an exception "query too complex".
> Example query:
> {noformat}
> //element(*,rep:Authorizable)[ ( (  
>  jcr:like(fn:lower-case(fn:name()), 'audio%')
>  or jcr:contains(@rep:principalName,'audio*')
>  or jcr:contains(@cq:first-name,'audio*')
>  or jcr:contains(@cq:last-name,'audio*')
>  or jcr:contains(profile/@givenName,'audio*')
>  or jcr:contains(profile/@familyName,'audio*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'details:%')
>  or jcr:contains(@rep:principalName,'details:*')
>  or jcr:contains(@cq:first-name,'details:*')
>  or jcr:contains(@cq:last-name,'details:*')
>  or jcr:contains(profile/@givenName,'details:*')
>  or jcr:contains(profile/@familyName,'details:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'code:%')
>  or jcr:contains(@rep:principalName,'code:*')
>  or jcr:contains(@cq:first-name,'code:*')
>  or jcr:contains(@cq:last-name,'code:*')
>  or jcr:contains(profile/@givenName,'code:*')
>  or jcr:contains(profile/@familyName,'code:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), '123456%')
>  or jcr:contains(@rep:principalName,'123456*')
>  or jcr:contains(@cq:first-name,'123456*')
>  or jcr:contains(@cq:last-name,'123456*')
>  or jcr:contains(profile/@givenName,'123456*')
>  or jcr:contains(profile/@familyName,'123456*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'further%')
>  or jcr:contains(@rep:principalName,'further*')
>  or jcr:contains(@cq:first-name,'further*')
>  or jcr:contains(@cq:last-name,'further*')
>  or jcr:contains(profile/@givenName,'further*')
>  or jcr:contains(profile/@familyName,'further*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'dial%')
>  or jcr:contains(@rep:principalName,'dial*')
>  or jcr:contains(@cq:first-name,'dial*')
>  or jcr:contains(@cq:last-name,'dial*')
>  or jcr:contains(profile/@givenName,'dial*')
>  or jcr:contains(profile/@familyName,'dial*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'in%')
>  or jcr:contains(@rep:principalName,'in*')
>  or jcr:contains(@cq:first-name,'in*')
>  or jcr:contains(@cq:last-name,'in*')
>  or jcr:contains(profile/@givenName,'in*')
>  or jcr:contains(profile/@familyName,'in*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'numbers:%')
>  or jcr:contains(@rep:principalName,'numbers:*')
>  or jcr:contains(@cq:first-name,'numbers:*')
>  or jcr:contains(@cq:last-name,'numbers:*')
>  or jcr:contains(profile/@givenName,'numbers:*')
>  or jcr:contains(profile/@familyName,'numbers:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'http://wikipedia.org%')
>  or jcr:contains(@rep:principalName,'http://wikipedia.org*')
>  or jcr:contains(@cq:first-name,'http://wikipedia.org*')
>  or jcr:contains(@cq:last-name,'http://wikipedia.org*')
>  or jcr:contains(profile/@givenName,'http://wikipedia.org*')
>  or jcr:contains(profile/@familyName,'http://wikipedia.org*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'number(s):%')
>  or jcr:contains(@rep:principalName,'number(s):*')
>  or jcr:contains(@cq:first-name,'number(s):*')
>  or jcr:contains(@cq:last-name,'number(s):*')
>  or jcr:contains(profile/@givenName,'number(s):*')
>  or jcr:contains(profile/@familyName,'number(s):*') 

[jira] [Commented] (OAK-6360) Failed to retrieve previously indexed checkpoint in composite node store

2017-06-19 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054098#comment-16054098
 ] 

Tomek Rękawek commented on OAK-6360:


Modified the behaviour of the release() method in 
[r1799213|https://svn.apache.org/r1799213].

Up until now, the retrieve() and release() methods used following method to 
resolve the name of the partial checkpoint:

* the property {{composite.checkpoint.mount.MOUNTNAME}} from the global 
checkpoint info contains the partial checkpoint name,
* the name is the same both for global and partial checkpoint (this is probably 
redundant),
* the checkpoint with the matching {{name}} property is being used.

We shouldn't use the last method for the release() and this is the change in 
this commit.

> Failed to retrieve previously indexed checkpoint in composite node store
> 
>
> Key: OAK-6360
> URL: https://issues.apache.org/jira/browse/OAK-6360
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: composite
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.8
>
>
> In some cases following exception is logged:
> {noformat}
> 15.06.2017 11:51:13.871 *WARN* [sling-oak-1-Registered Service.722] 
> org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate [fulltext-async] 
> Failed to retrieve previously indexed checkpoint 
> 94afbccc-67da-41c2-bb31-45cc9f50a663; re-running the initial index update
> 15.06.2017 11:51:13.872 *INFO* [async-index-update-fulltext-async] 
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Found a new index node 
> [lucene]. Reindexing is requested
> 15.06.2017 11:51:13.877 *INFO* [async-index-update-fulltext-async] 
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing will be 
> performed for following indexes: [/oak:index/lucene]
> {noformat}
> We should investigate this. Let's start with logging more info (list of 
> checkpoints available on all stores and their metadata) if the checkpoint 
> can't be retrieved.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6062) Test failure: CopyBinariesTest.validateMigration

2017-06-19 Thread Marcel Reutegger (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054106#comment-16054106
 ] 

Marcel Reutegger commented on OAK-6062:
---

Was able to reproduce this once while repeatedly running CopyBinariesTest until 
it fails.

> Test failure: CopyBinariesTest.validateMigration
> 
>
> Key: OAK-6062
> URL: https://issues.apache.org/jira/browse/OAK-6062
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: continuous integration, documentmk
>Reporter: Hudson
>Assignee: Marcel Reutegger
>  Labels: CI, flaky-test, jenkins
> Fix For: 1.8
>
>
> Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/
> The build Jackrabbit Oak #146 has failed.
> First failed run: [Jackrabbit Oak 
> #146|https://builds.apache.org/job/Jackrabbit%20Oak/146/] [console 
> log|https://builds.apache.org/job/Jackrabbit%20Oak/146/console]
> The test failure is:
> {noformat}
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest
> validateMigration[Copy references, no blobstores defined, document -> 
> segment-tar](org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest)  
> Time elapsed: 2.534 sec  <<< ERROR!
> javax.jcr.RepositoryException: Failed to copy content
>   at 
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:183)
> Caused by: java.lang.IllegalStateException: Branch with failed reset
>   at 
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:183)
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: OakOak0100: 
> Branch reset failed
>   at 
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:183)
> Caused by: org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: 
> Empty branch cannot be reset
>   at 
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:183)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (OAK-6062) Test failure: CopyBinariesTest.validateMigration

2017-06-19 Thread Marcel Reutegger (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcel Reutegger reassigned OAK-6062:
-

Assignee: Marcel Reutegger

> Test failure: CopyBinariesTest.validateMigration
> 
>
> Key: OAK-6062
> URL: https://issues.apache.org/jira/browse/OAK-6062
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: continuous integration, documentmk
>Reporter: Hudson
>Assignee: Marcel Reutegger
>  Labels: CI, flaky-test, jenkins
> Fix For: 1.8
>
>
> Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/
> The build Jackrabbit Oak #146 has failed.
> First failed run: [Jackrabbit Oak 
> #146|https://builds.apache.org/job/Jackrabbit%20Oak/146/] [console 
> log|https://builds.apache.org/job/Jackrabbit%20Oak/146/console]
> The test failure is:
> {noformat}
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest
> validateMigration[Copy references, no blobstores defined, document -> 
> segment-tar](org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest)  
> Time elapsed: 2.534 sec  <<< ERROR!
> javax.jcr.RepositoryException: Failed to copy content
>   at 
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:183)
> Caused by: java.lang.IllegalStateException: Branch with failed reset
>   at 
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:183)
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: OakOak0100: 
> Branch reset failed
>   at 
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:183)
> Caused by: org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: 
> Empty branch cannot be reset
>   at 
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:183)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (OAK-6370) Improve documentation for text pre-extraction

2017-06-19 Thread Chetan Mehrotra (JIRA)
Chetan Mehrotra created OAK-6370:


 Summary: Improve documentation for text pre-extraction
 Key: OAK-6370
 URL: https://issues.apache.org/jira/browse/OAK-6370
 Project: Jackrabbit Oak
  Issue Type: Documentation
  Components: lucene, run
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
 Fix For: 1.8


The docs for pre-extraction does not make things very clear. This should be 
improved



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6370) Improve documentation for text pre-extraction

2017-06-19 Thread Chetan Mehrotra (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054124#comment-16054124
 ] 

Chetan Mehrotra commented on OAK-6370:
--

new docs at 
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-doc/src/site/markdown/query/pre-extract-text.md

> Improve documentation for text pre-extraction
> -
>
> Key: OAK-6370
> URL: https://issues.apache.org/jira/browse/OAK-6370
> Project: Jackrabbit Oak
>  Issue Type: Documentation
>  Components: lucene, run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> The docs for pre-extraction does not make things very clear. This should be 
> improved



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6370) Improve documentation for text pre-extraction

2017-06-19 Thread David Gonzalez (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054133#comment-16054133
 ] 

David Gonzalez commented on OAK-6370:
-

think that clarifies the mechanics of the tool…. but i think clarifications on 
its expected uses cases are warranted?

1) Im moving many NEW files into Oak; this tool presumes that the files are 
already in Oak; so what is the safe way(s) to get them in without incurring the 
cost of text extraction? 
1a) How is 1 handled in the context of a new new Oak repo that is not under use 
(ie. you have more leeway configuring/disabling features)
1b) How is 1 handled if you're migrating large sets into a repo being used (ie. 
you cant turn off text extraction wholesale because someone might be normally 
uploading something and expect it to be indexed right away) 

2) Does this extract text from *everything* when you run it? Can you limit it 
to parts of the JCR? Or by new/modified content?

3) Should this be run on the same machine that is running the Oak repo or on a 
different machine w/ the paths mounted?

4) Would this need to be run on every publish instance? Or could you run this 
once (ex. the CSV extraction) and re-purpose that to save time? (Is the time 
saved meaningful? Or is it preferred to run this process once per oak instance?)

> Improve documentation for text pre-extraction
> -
>
> Key: OAK-6370
> URL: https://issues.apache.org/jira/browse/OAK-6370
> Project: Jackrabbit Oak
>  Issue Type: Documentation
>  Components: lucene, run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> The docs for pre-extraction does not make things very clear. This should be 
> improved



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6359) Change behavior for very complex queries

2017-06-19 Thread Thomas Mueller (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054143#comment-16054143
 ] 

Thomas Mueller commented on OAK-6359:
-

http://svn.apache.org/r1799219 (trunk)

Limit the XPath -> SQL-2 conversion to at most create 1000 union statements. 
This is to ensure parsing a query doesn't run out of memory. When going over 
this limit, the query is converted as is, that means, no "union" statement is 
made (instead, the SQL-2 query uses "or" conditions just like the XPath query). 
That means possibly no index can be used. The behavior of such cases is already 
limited (with traversal limit for example). It still needs to be tested if the 
SQL-2 query doesn't result in a memory problem, due to possible conversion to 
union there.

> Change behavior for very complex queries
> 
>
> Key: OAK-6359
> URL: https://issues.apache.org/jira/browse/OAK-6359
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Critical
> Fix For: 1.8, 1.7.3
>
>
> Very complex queries can cause take a long time to parse, and can possibly 
> cause OOME. It would be good if processing queries automatically stops after 
> some number of loops, after some time, or some amount of memory was used. And 
> then throw an exception "query too complex".
> Example query:
> {noformat}
> //element(*,rep:Authorizable)[ ( (  
>  jcr:like(fn:lower-case(fn:name()), 'audio%')
>  or jcr:contains(@rep:principalName,'audio*')
>  or jcr:contains(@cq:first-name,'audio*')
>  or jcr:contains(@cq:last-name,'audio*')
>  or jcr:contains(profile/@givenName,'audio*')
>  or jcr:contains(profile/@familyName,'audio*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'details:%')
>  or jcr:contains(@rep:principalName,'details:*')
>  or jcr:contains(@cq:first-name,'details:*')
>  or jcr:contains(@cq:last-name,'details:*')
>  or jcr:contains(profile/@givenName,'details:*')
>  or jcr:contains(profile/@familyName,'details:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'code:%')
>  or jcr:contains(@rep:principalName,'code:*')
>  or jcr:contains(@cq:first-name,'code:*')
>  or jcr:contains(@cq:last-name,'code:*')
>  or jcr:contains(profile/@givenName,'code:*')
>  or jcr:contains(profile/@familyName,'code:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), '123456%')
>  or jcr:contains(@rep:principalName,'123456*')
>  or jcr:contains(@cq:first-name,'123456*')
>  or jcr:contains(@cq:last-name,'123456*')
>  or jcr:contains(profile/@givenName,'123456*')
>  or jcr:contains(profile/@familyName,'123456*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'further%')
>  or jcr:contains(@rep:principalName,'further*')
>  or jcr:contains(@cq:first-name,'further*')
>  or jcr:contains(@cq:last-name,'further*')
>  or jcr:contains(profile/@givenName,'further*')
>  or jcr:contains(profile/@familyName,'further*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'dial%')
>  or jcr:contains(@rep:principalName,'dial*')
>  or jcr:contains(@cq:first-name,'dial*')
>  or jcr:contains(@cq:last-name,'dial*')
>  or jcr:contains(profile/@givenName,'dial*')
>  or jcr:contains(profile/@familyName,'dial*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'in%')
>  or jcr:contains(@rep:principalName,'in*')
>  or jcr:contains(@cq:first-name,'in*')
>  or jcr:contains(@cq:last-name,'in*')
>  or jcr:contains(profile/@givenName,'in*')
>  or jcr:contains(profile/@familyName,'in*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'numbers:%')
>  or jcr:contains(@rep:principalName,'numbers:*')
>  or jcr:contains(@cq:first-name,'numbers:*')
>  or jcr:contains(@cq:last-name,'numbers:*')
>  or jcr:contains(profile/@givenName,'numbers:*')
>  or jcr:contains(profile/@familyName,'numbers:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'http://wikipedia.org%')
>  or jcr:contains(@rep:principalName,'http://wikipedia.org*')
>  or jcr:contains(@cq:first-name,'http://wikipedia.org*')
>  or jcr:contains(@cq:last-name,'http://wikipedia.org*')
>  or jcr:contains(profile/@givenName,'http://wikipedia.org*')
>  or jcr:contains(profile/@familyName,'http://wikipedia.org

[jira] [Updated] (OAK-6359) Change behavior for very complex queries

2017-06-19 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller updated OAK-6359:

Labels: candidate_oak_1_2 candidate_oak_1_4 candidate_oak_1_6  (was: )

> Change behavior for very complex queries
> 
>
> Key: OAK-6359
> URL: https://issues.apache.org/jira/browse/OAK-6359
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Critical
>  Labels: candidate_oak_1_2, candidate_oak_1_4, candidate_oak_1_6
> Fix For: 1.8, 1.7.3
>
>
> Very complex queries can cause take a long time to parse, and can possibly 
> cause OOME. It would be good if processing queries automatically stops after 
> some number of loops, after some time, or some amount of memory was used. And 
> then throw an exception "query too complex".
> Example query:
> {noformat}
> //element(*,rep:Authorizable)[ ( (  
>  jcr:like(fn:lower-case(fn:name()), 'audio%')
>  or jcr:contains(@rep:principalName,'audio*')
>  or jcr:contains(@cq:first-name,'audio*')
>  or jcr:contains(@cq:last-name,'audio*')
>  or jcr:contains(profile/@givenName,'audio*')
>  or jcr:contains(profile/@familyName,'audio*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'details:%')
>  or jcr:contains(@rep:principalName,'details:*')
>  or jcr:contains(@cq:first-name,'details:*')
>  or jcr:contains(@cq:last-name,'details:*')
>  or jcr:contains(profile/@givenName,'details:*')
>  or jcr:contains(profile/@familyName,'details:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'code:%')
>  or jcr:contains(@rep:principalName,'code:*')
>  or jcr:contains(@cq:first-name,'code:*')
>  or jcr:contains(@cq:last-name,'code:*')
>  or jcr:contains(profile/@givenName,'code:*')
>  or jcr:contains(profile/@familyName,'code:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), '123456%')
>  or jcr:contains(@rep:principalName,'123456*')
>  or jcr:contains(@cq:first-name,'123456*')
>  or jcr:contains(@cq:last-name,'123456*')
>  or jcr:contains(profile/@givenName,'123456*')
>  or jcr:contains(profile/@familyName,'123456*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'further%')
>  or jcr:contains(@rep:principalName,'further*')
>  or jcr:contains(@cq:first-name,'further*')
>  or jcr:contains(@cq:last-name,'further*')
>  or jcr:contains(profile/@givenName,'further*')
>  or jcr:contains(profile/@familyName,'further*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'dial%')
>  or jcr:contains(@rep:principalName,'dial*')
>  or jcr:contains(@cq:first-name,'dial*')
>  or jcr:contains(@cq:last-name,'dial*')
>  or jcr:contains(profile/@givenName,'dial*')
>  or jcr:contains(profile/@familyName,'dial*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'in%')
>  or jcr:contains(@rep:principalName,'in*')
>  or jcr:contains(@cq:first-name,'in*')
>  or jcr:contains(@cq:last-name,'in*')
>  or jcr:contains(profile/@givenName,'in*')
>  or jcr:contains(profile/@familyName,'in*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'numbers:%')
>  or jcr:contains(@rep:principalName,'numbers:*')
>  or jcr:contains(@cq:first-name,'numbers:*')
>  or jcr:contains(@cq:last-name,'numbers:*')
>  or jcr:contains(profile/@givenName,'numbers:*')
>  or jcr:contains(profile/@familyName,'numbers:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'http://wikipedia.org%')
>  or jcr:contains(@rep:principalName,'http://wikipedia.org*')
>  or jcr:contains(@cq:first-name,'http://wikipedia.org*')
>  or jcr:contains(@cq:last-name,'http://wikipedia.org*')
>  or jcr:contains(profile/@givenName,'http://wikipedia.org*')
>  or jcr:contains(profile/@familyName,'http://wikipedia.org*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'number(s):%')
>  or jcr:contains(@rep:principalName,'number(s):*')
>  or jcr:contains(@cq:first-name,'number(s):*')
>  or jcr:contains(@cq:las

[jira] [Updated] (OAK-6356) Allow CompositePermissionProvider to OR entries

2017-06-19 Thread Alex Deparvu (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Deparvu updated OAK-6356:
--
Fix Version/s: 1.7.3

> Allow CompositePermissionProvider to OR entries
> ---
>
> Key: OAK-6356
> URL: https://issues.apache.org/jira/browse/OAK-6356
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core, security
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
> Fix For: 1.7.3
>
>
> Currently the {{CompositePermissionProvider}} ANDs the entries and if any of 
> those denies a check, all the chain will fail early. I'd like to extend this 
> mechanism to 'OR' items if needed.
> A first application of this ORing could be the multiplexed permission store 
> where the default store could deny a check but a mount could allow it, so it 
> could be seen as valid.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OAK-3777) Multiplexing support in default PermissionStore implementation

2017-06-19 Thread Alex Deparvu (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Deparvu updated OAK-3777:
--
Fix Version/s: 1.8

> Multiplexing support in default PermissionStore implementation
> --
>
> Key: OAK-3777
> URL: https://issues.apache.org/jira/browse/OAK-3777
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: core
>Reporter: Chetan Mehrotra
>Assignee: Alex Deparvu
> Fix For: 1.8
>
> Attachments: OAK-3777-as-composite.patch
>
>
> Similar to other parts we need to prototype support for multiplexing in 
> default permission store



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6359) Change behavior for very complex queries

2017-06-19 Thread Thomas Mueller (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054246#comment-16054246
 ] 

Thomas Mueller commented on OAK-6359:
-

As far as I see, the bottleneck is now in the SQL-2 part:

{noformat}
at 
org.apache.jackrabbit.oak.query.ast.AndImpl.addToUnionList(AndImpl.java:240)
...
at 
org.apache.jackrabbit.oak.query.ast.AndImpl.addToUnionList(AndImpl.java:236)
at 
org.apache.jackrabbit.oak.query.ast.AndImpl.convertToUnion(AndImpl.java:313)
...
at 
org.apache.jackrabbit.oak.query.ast.AndImpl.convertToUnion(AndImpl.java:293)
at 
org.apache.jackrabbit.oak.query.ast.AndImpl.convertToUnion(AndImpl.java:293)
at 
org.apache.jackrabbit.oak.query.QueryImpl.buildAlternativeQuery(QueryImpl.java:1234)
at 
org.apache.jackrabbit.oak.query.QueryEngineImpl.parseQuery(QueryEngineImpl.java:203)
at 
org.apache.jackrabbit.oak.query.QueryEngineImpl.executeQuery(QueryEngineImpl.java:254)
at 
org.apache.jackrabbit.oak.jcr.query.QueryManagerImpl.executeQuery(QueryManagerImpl.java:136)
{noformat}

> Change behavior for very complex queries
> 
>
> Key: OAK-6359
> URL: https://issues.apache.org/jira/browse/OAK-6359
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Critical
>  Labels: candidate_oak_1_2, candidate_oak_1_4, candidate_oak_1_6
> Fix For: 1.8, 1.7.3
>
>
> Very complex queries can cause take a long time to parse, and can possibly 
> cause OOME. It would be good if processing queries automatically stops after 
> some number of loops, after some time, or some amount of memory was used. And 
> then throw an exception "query too complex".
> Example query:
> {noformat}
> //element(*,rep:Authorizable)[ ( (  
>  jcr:like(fn:lower-case(fn:name()), 'audio%')
>  or jcr:contains(@rep:principalName,'audio*')
>  or jcr:contains(@cq:first-name,'audio*')
>  or jcr:contains(@cq:last-name,'audio*')
>  or jcr:contains(profile/@givenName,'audio*')
>  or jcr:contains(profile/@familyName,'audio*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'details:%')
>  or jcr:contains(@rep:principalName,'details:*')
>  or jcr:contains(@cq:first-name,'details:*')
>  or jcr:contains(@cq:last-name,'details:*')
>  or jcr:contains(profile/@givenName,'details:*')
>  or jcr:contains(profile/@familyName,'details:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'conference%')
>  or jcr:contains(@rep:principalName,'conference*')
>  or jcr:contains(@cq:first-name,'conference*')
>  or jcr:contains(@cq:last-name,'conference*')
>  or jcr:contains(profile/@givenName,'conference*')
>  or jcr:contains(profile/@familyName,'conference*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'code:%')
>  or jcr:contains(@rep:principalName,'code:*')
>  or jcr:contains(@cq:first-name,'code:*')
>  or jcr:contains(@cq:last-name,'code:*')
>  or jcr:contains(profile/@givenName,'code:*')
>  or jcr:contains(profile/@familyName,'code:*') )
>  and (  jcr:like(fn:lower-case(fn:name()), '123456%')
>  or jcr:contains(@rep:principalName,'123456*')
>  or jcr:contains(@cq:first-name,'123456*')
>  or jcr:contains(@cq:last-name,'123456*')
>  or jcr:contains(profile/@givenName,'123456*')
>  or jcr:contains(profile/@familyName,'123456*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'further%')
>  or jcr:contains(@rep:principalName,'further*')
>  or jcr:contains(@cq:first-name,'further*')
>  or jcr:contains(@cq:last-name,'further*')
>  or jcr:contains(profile/@givenName,'further*')
>  or jcr:contains(profile/@familyName,'further*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'dial%')
>  or jcr:contains(@rep:principalName,'dial*')
>  or jcr:contains(@cq:first-name,'dial*')
>  or jcr:contains(@cq:last-name,'dial*')
>  or jcr:contains(profile/@givenName,'dial*')
>  or jcr:contains(profile/@familyName,'dial*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'in%')
>  or jcr:contains(@rep:principalName,'in*')
>  or jcr:contains(@cq:first-name,'in*')
>  or jcr:contains(@cq:last-name,'in*')
>  or jcr:contains(profile/@givenName,'in*')
>  or jcr:contains(profile/@familyName,'in*') )
>  and (  jcr:like(fn:lower-case(fn:name()), 'numbers:%')
>  or jcr:contains(@rep:principalName,'numbers:*')
>  or jcr:contains(@cq:first-name,'numbers:*')
>  or jcr:contains(@cq:last-name,'numbers:*')
>  or jcr:contains(profile/@givenName,'numbers:*')
>  or jcr

[jira] [Commented] (OAK-3777) Multiplexing support in default PermissionStore implementation

2017-06-19 Thread Alex Deparvu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054353#comment-16054353
 ] 

Alex Deparvu commented on OAK-3777:
---

I've rebased my current work on top of OAK-6356. The basics of multiplex are 
covered by the {{/MultiplexingProviderTest}} test while the random one 
{{MutiplexingProviderRandomTestIT}} should cover more complex scenarios [0].

[~anchela] does it make sense to break away the perf tests to a dedicated 
issue? I can create one and start collecting numbers as a followup.
I'm not sure if there's anything else we needed to cover here?

[0] https://github.com/stillalex/jackrabbit-oak/compare/oak-6356...oak-3777

> Multiplexing support in default PermissionStore implementation
> --
>
> Key: OAK-3777
> URL: https://issues.apache.org/jira/browse/OAK-3777
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: core
>Reporter: Chetan Mehrotra
>Assignee: Alex Deparvu
> Fix For: 1.8
>
> Attachments: OAK-3777-as-composite.patch
>
>
> Similar to other parts we need to prototype support for multiplexing in 
> default permission store



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6360) Failed to retrieve previously indexed checkpoint in composite node store

2017-06-19 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054649#comment-16054649
 ] 

Tomek Rękawek commented on OAK-6360:


Added debug info for the release() and checkpoint() methods in 
[r1799271|https://svn.apache.org/r1799271].

> Failed to retrieve previously indexed checkpoint in composite node store
> 
>
> Key: OAK-6360
> URL: https://issues.apache.org/jira/browse/OAK-6360
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: composite
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.8
>
>
> In some cases following exception is logged:
> {noformat}
> 15.06.2017 11:51:13.871 *WARN* [sling-oak-1-Registered Service.722] 
> org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate [fulltext-async] 
> Failed to retrieve previously indexed checkpoint 
> 94afbccc-67da-41c2-bb31-45cc9f50a663; re-running the initial index update
> 15.06.2017 11:51:13.872 *INFO* [async-index-update-fulltext-async] 
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Found a new index node 
> [lucene]. Reindexing is requested
> 15.06.2017 11:51:13.877 *INFO* [async-index-update-fulltext-async] 
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing will be 
> performed for following indexes: [/oak:index/lucene]
> {noformat}
> We should investigate this. Let's start with logging more info (list of 
> checkpoints available on all stores and their metadata) if the checkpoint 
> can't be retrieved.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6370) Improve documentation for text pre-extraction

2017-06-19 Thread Chetan Mehrotra (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16055169#comment-16055169
 ] 

Chetan Mehrotra commented on OAK-6370:
--

bq.  Im moving many NEW files into Oak; this tool presumes that the files are 
already in Oak; so what is the safe way(s) to get them in without incurring the 
cost of text extraction? 

Pre extraction support is mostly meant to reduce reindexing time. For speeding 
up incremental indexing see OAK-2787. Another option here would be to move the 
text extraction as part of asset processing. This depends on application built 
on top of Oak. For example if there is an asset management system which does 
some post processing upon asset binary addition then it can also do text 
extraction and create a node having plain text data which gets indexed quickly.

bq. Does this extract text from everything when you run it? Can you limit it to 
parts of the JCR? Or by new/modified content?

It takes a "path" argument

bq. Should this be run on the same machine that is running the Oak repo or on a 
different machine w/ the paths mounted?

Need not be. For first phase i.e. csv generation it needs access to NodeStore. 
So if its a SegmentNodeStore based setup then it needs to be run from that 
machine (or its clone). For text extraction phase it needs to just access the 
BlobStore

bq.  Would this need to be run on every publish instance? Or could you run this 
once (ex. the CSV extraction) and re-purpose that to save time? (Is the time 
saved meaningful? Or is it preferred to run this process once per oak instance?)

It works at blobId level which are stable. So run it once per cluster. If the 
publish instance have more or less same data then you need to run it only for 
one of them



> Improve documentation for text pre-extraction
> -
>
> Key: OAK-6370
> URL: https://issues.apache.org/jira/browse/OAK-6370
> Project: Jackrabbit Oak
>  Issue Type: Documentation
>  Components: lucene, run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> The docs for pre-extraction does not make things very clear. This should be 
> improved



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OAK-6062) Test failure: CopyBinariesTest.validateMigration

2017-06-19 Thread Marcel Reutegger (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16055206#comment-16055206
 ] 

Marcel Reutegger commented on OAK-6062:
---

And once again with debugging enabled. The actual exception that triggered the 
branch reset in my case was:

{noformat}
java.lang.IllegalStateException: This builder does not exist: NAME
at 
com.google.common.base.Preconditions.checkState(Preconditions.java:150)
at 
org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:506)
at 
org.apache.jackrabbit.oak.plugins.document.AbstractDocumentNodeBuilder.setChildNode(AbstractDocumentNodeBuilder.java:59)
at 
org.apache.jackrabbit.oak.plugins.document.AbstractDocumentNodeBuilder.setChildNode(AbstractDocumentNodeBuilder.java:62)
at 
org.apache.jackrabbit.oak.plugins.document.AbstractDocumentNodeBuilder.setChildNode(AbstractDocumentNodeBuilder.java:62)
at 
org.apache.jackrabbit.oak.plugins.nodetype.TypeRegistration.mergeSubtree(TypeRegistration.java:288)
at 
org.apache.jackrabbit.oak.plugins.nodetype.TypeRegistration.mergeSupertype(TypeRegistration.java:269)
at 
org.apache.jackrabbit.oak.plugins.nodetype.TypeRegistration.mergeSupertypes(TypeRegistration.java:214)
at 
org.apache.jackrabbit.oak.plugins.nodetype.TypeRegistration.apply(TypeRegistration.java:144)
at 
org.apache.jackrabbit.oak.plugins.nodetype.TypeEditorProvider.getRootEditor(TypeEditorProvider.java:80)
at 
org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade$2.getRootEditor(RepositoryUpgrade.java:611)
at 
org.apache.jackrabbit.oak.spi.commit.CompositeEditorProvider.getRootEditor(CompositeEditorProvider.java:80)
at 
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:53)
at 
org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade$LoggingCompositeHook.processCommit(RepositoryUpgrade.java:1054)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch$Persisted$1.call(DocumentNodeStoreBranch.java:608)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch$Persisted$1.call(DocumentNodeStoreBranch.java:604)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.withCurrentBranch(DocumentNodeStoreBranch.java:317)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.access$700(DocumentNodeStoreBranch.java:58)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch$Persisted.merge(DocumentNodeStoreBranch.java:604)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.merge0(DocumentNodeStoreBranch.java:189)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.merge(DocumentNodeStoreBranch.java:123)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentRootBuilder.merge(DocumentRootBuilder.java:164)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.merge(DocumentNodeStore.java:1812)
at 
org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.migrateWithoutCheckpoints(RepositorySidegrade.java:448)
at 
org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copyState(RepositorySidegrade.java:341)
at 
org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:302)
at 
org.apache.jackrabbit.oak.upgrade.cli.AbstractOak2OakTest.initContent(AbstractOak2OakTest.java:139)
at 
org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:187)
{noformat}

> Test failure: CopyBinariesTest.validateMigration
> 
>
> Key: OAK-6062
> URL: https://issues.apache.org/jira/browse/OAK-6062
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: continuous integration, documentmk
>Reporter: Hudson
>Assignee: Marcel Reutegger
>  Labels: CI, flaky-test, jenkins
> Fix For: 1.8
>
>
> Jenkins CI failure: https://builds.apache.org/view/J/job/Jackrabbit%20Oak/
> The build Jackrabbit Oak #146 has failed.
> First failed run: [Jackrabbit Oak 
> #146|https://builds.apache.org/job/Jackrabbit%20Oak/146/] [console 
> log|https://builds.apache.org/job/Jackrabbit%20Oak/146/console]
> The test failure is:
> {noformat}
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest
> validateMigration[Copy references, no blobstores defined, document -> 
> segment-tar](org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest)  
> Time elapsed: 2.534 sec  <<< ERROR!
> javax.jcr.RepositoryException: Failed to copy content
>   at 
> org.apache.jackrabbit.oak.upgrade.cli.blob.CopyBinariesTest.prepare(CopyBinariesTest.java:183)
> Caused by: java.lang.IllegalStateException: Branch with failed re