[jira] [Resolved] (OAK-7262) LockBasedScheduler#getHeadNodeState poor performance due to lock contention in commitTimeHistogram implementation
[
https://issues.apache.org/jira/browse/OAK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu resolved OAK-7262.
--
Resolution: Fixed
> LockBasedScheduler#getHeadNodeState poor performance due to lock contention
> in commitTimeHistogram implementation
> -
>
> Key: OAK-7262
> URL: https://issues.apache.org/jira/browse/OAK-7262
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> With OAK-4732, we slightly prioritised reads, allowing callers of
> {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time
> on the commit semaphore, in order to fetch the latest head. The amount of
> time to wait for was computed as the median of the latest 1000 commits, but
> the implementation was flexible enough to replace the 0.5 quantile with a
> different value.
> The initial implementation used {{SynchronizedDescriptiveStatistics}} from
> {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed
> by a {{SlindingWindowReservoir}} from the dropwizard metric library
> ({{io.dropwizard.metrics}}).
> One of the problems observed with the current implementation is that, under
> heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
> decreases due to lock contention in {{commitTimeHistogram}} implementation.
> Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
> copies an array of 1000 elements at each invocation of
> {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
> each element in the array.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7262) LockBasedScheduler#getHeadNodeState poor performance due to lock contention in commitTimeHistogram implementation
[
https://issues.apache.org/jira/browse/OAK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363569#comment-16363569
]
Andrei Dulceanu commented on OAK-7262:
--
Fixed at r1824200.
> LockBasedScheduler#getHeadNodeState poor performance due to lock contention
> in commitTimeHistogram implementation
> -
>
> Key: OAK-7262
> URL: https://issues.apache.org/jira/browse/OAK-7262
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> With OAK-4732, we slightly prioritised reads, allowing callers of
> {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time
> on the commit semaphore, in order to fetch the latest head. The amount of
> time to wait for was computed as the median of the latest 1000 commits, but
> the implementation was flexible enough to replace the 0.5 quantile with a
> different value.
> The initial implementation used {{SynchronizedDescriptiveStatistics}} from
> {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed
> by a {{SlindingWindowReservoir}} from the dropwizard metric library
> ({{io.dropwizard.metrics}}).
> One of the problems observed with the current implementation is that, under
> heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
> decreases due to lock contention in {{commitTimeHistogram}} implementation.
> Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
> copies an array of 1000 elements at each invocation of
> {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
> each element in the array.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7258) Build Jackrabbit Oak #1240 failed
[ https://issues.apache.org/jira/browse/OAK-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363559#comment-16363559 ] Hudson commented on OAK-7258: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #1246|https://builds.apache.org/job/Jackrabbit%20Oak/1246/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1246/console] > Build Jackrabbit Oak #1240 failed > - > > Key: OAK-7258 > URL: https://issues.apache.org/jira/browse/OAK-7258 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1240 has failed. > First failed run: [Jackrabbit Oak > #1240|https://builds.apache.org/job/Jackrabbit%20Oak/1240/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1240/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7266) Standalone example system console fails to render
[
https://issues.apache.org/jira/browse/OAK-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-7266:
-
Labels: candidate_oak_1_8 (was: )
> Standalone example system console fails to render
> -
>
> Key: OAK-7266
> URL: https://issues.apache.org/jira/browse/OAK-7266
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples
>Affects Versions: 1.8.0
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Major
> Labels: candidate_oak_1_8
> Fix For: 1.9.0, 1.10
>
>
> Accessing http://localhost:8080/osgi/system/console/bundles with
> oak-standalone fails to render with following exception
> {noformat}
> 2018-02-14 10:36:40.415 WARN 23249 --- [qtp349420578-14]
> org.eclipse.jetty.server.HttpChannel : /osgi/system/console/bundles
> java.lang.NoSuchMethodError:
> org.json.JSONObject.valueToString(Ljava/lang/Object;)Ljava/lang/String;
> at org.json.JSONWriter.value(JSONWriter.java:316)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.writeJSON(BundlesServlet.java:612)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.writeJSON(BundlesServlet.java:558)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.renderContent(BundlesServlet.java:532)
> at
> org.apache.felix.webconsole.AbstractWebConsolePlugin.doGet(AbstractWebConsolePlugin.java:194)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.doGet(BundlesServlet.java:317)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-7266) Standalone example system console fails to render
[
https://issues.apache.org/jira/browse/OAK-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-7266.
--
Resolution: Fixed
Done with 1824198
> Standalone example system console fails to render
> -
>
> Key: OAK-7266
> URL: https://issues.apache.org/jira/browse/OAK-7266
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples
>Affects Versions: 1.8.0
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Major
> Labels: candidate_oak_1_8
> Fix For: 1.9.0, 1.10
>
>
> Accessing http://localhost:8080/osgi/system/console/bundles with
> oak-standalone fails to render with following exception
> {noformat}
> 2018-02-14 10:36:40.415 WARN 23249 --- [qtp349420578-14]
> org.eclipse.jetty.server.HttpChannel : /osgi/system/console/bundles
> java.lang.NoSuchMethodError:
> org.json.JSONObject.valueToString(Ljava/lang/Object;)Ljava/lang/String;
> at org.json.JSONWriter.value(JSONWriter.java:316)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.writeJSON(BundlesServlet.java:612)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.writeJSON(BundlesServlet.java:558)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.renderContent(BundlesServlet.java:532)
> at
> org.apache.felix.webconsole.AbstractWebConsolePlugin.doGet(AbstractWebConsolePlugin.java:194)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.doGet(BundlesServlet.java:317)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7266) Standalone example system console fails to render
[
https://issues.apache.org/jira/browse/OAK-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363518#comment-16363518
]
Chetan Mehrotra commented on OAK-7266:
--
This happens due an old oak-json dependency being pulled by tika parsers
> Standalone example system console fails to render
> -
>
> Key: OAK-7266
> URL: https://issues.apache.org/jira/browse/OAK-7266
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples
>Affects Versions: 1.8.0
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> Accessing http://localhost:8080/osgi/system/console/bundles with
> oak-standalone fails to render with following exception
> {noformat}
> 2018-02-14 10:36:40.415 WARN 23249 --- [qtp349420578-14]
> org.eclipse.jetty.server.HttpChannel : /osgi/system/console/bundles
> java.lang.NoSuchMethodError:
> org.json.JSONObject.valueToString(Ljava/lang/Object;)Ljava/lang/String;
> at org.json.JSONWriter.value(JSONWriter.java:316)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.writeJSON(BundlesServlet.java:612)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.writeJSON(BundlesServlet.java:558)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.renderContent(BundlesServlet.java:532)
> at
> org.apache.felix.webconsole.AbstractWebConsolePlugin.doGet(AbstractWebConsolePlugin.java:194)
> at
> org.apache.felix.webconsole.internal.core.BundlesServlet.doGet(BundlesServlet.java:317)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7266) Standalone example system console fails to render
Chetan Mehrotra created OAK-7266:
Summary: Standalone example system console fails to render
Key: OAK-7266
URL: https://issues.apache.org/jira/browse/OAK-7266
Project: Jackrabbit Oak
Issue Type: Bug
Components: examples
Affects Versions: 1.8.0
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Fix For: 1.9.0, 1.10
Accessing http://localhost:8080/osgi/system/console/bundles with oak-standalone
fails to render with following exception
{noformat}
2018-02-14 10:36:40.415 WARN 23249 --- [qtp349420578-14]
org.eclipse.jetty.server.HttpChannel : /osgi/system/console/bundles
java.lang.NoSuchMethodError:
org.json.JSONObject.valueToString(Ljava/lang/Object;)Ljava/lang/String;
at org.json.JSONWriter.value(JSONWriter.java:316)
at
org.apache.felix.webconsole.internal.core.BundlesServlet.writeJSON(BundlesServlet.java:612)
at
org.apache.felix.webconsole.internal.core.BundlesServlet.writeJSON(BundlesServlet.java:558)
at
org.apache.felix.webconsole.internal.core.BundlesServlet.renderContent(BundlesServlet.java:532)
at
org.apache.felix.webconsole.AbstractWebConsolePlugin.doGet(AbstractWebConsolePlugin.java:194)
at
org.apache.felix.webconsole.internal.core.BundlesServlet.doGet(BundlesServlet.java:317)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7265) Standalone example application fails to start
[
https://issues.apache.org/jira/browse/OAK-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-7265:
-
Labels: candidate_oak_1_8 (was: )
> Standalone example application fails to start
> -
>
> Key: OAK-7265
> URL: https://issues.apache.org/jira/browse/OAK-7265
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples, pojosr
>Affects Versions: 1.8.0
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Major
> Labels: candidate_oak_1_8
> Fix For: 1.9.0, 1.10
>
>
> On users dl it was reported that standalone example application fails to
> start with following exception
> {noformat}
> java.lang.NoSuchMethodException:
> org.springframework.boot.loader.jar.JarEntry.getUrl()
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getUrlMethod(SpringBootSupport.java:135)
> at
> org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getEntry(SpringBootSupport.java:109)
> at org.apache.felix.connect.PojoSRBundle.getEntry(PojoSRBundle.java:471)
> at
> org.apache.felix.webconsole.plugins.scriptconsole.internal.ScriptEngineManager.bundleChanged(ScriptEngineManager.java:117)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:821)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:771)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.run(EventDispatcher.java:993)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:52)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:94)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-7265) Standalone example application fails to start
[
https://issues.apache.org/jira/browse/OAK-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-7265.
--
Resolution: Fixed
Fixed with 1824196
> Standalone example application fails to start
> -
>
> Key: OAK-7265
> URL: https://issues.apache.org/jira/browse/OAK-7265
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples, pojosr
>Affects Versions: 1.8.0
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Major
> Fix For: 1.10
>
>
> On users dl it was reported that standalone example application fails to
> start with following exception
> {noformat}
> java.lang.NoSuchMethodException:
> org.springframework.boot.loader.jar.JarEntry.getUrl()
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getUrlMethod(SpringBootSupport.java:135)
> at
> org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getEntry(SpringBootSupport.java:109)
> at org.apache.felix.connect.PojoSRBundle.getEntry(PojoSRBundle.java:471)
> at
> org.apache.felix.webconsole.plugins.scriptconsole.internal.ScriptEngineManager.bundleChanged(ScriptEngineManager.java:117)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:821)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:771)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.run(EventDispatcher.java:993)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:52)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:94)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7265) Standalone example application fails to start
[
https://issues.apache.org/jira/browse/OAK-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-7265:
-
Fix Version/s: 1.9.0
> Standalone example application fails to start
> -
>
> Key: OAK-7265
> URL: https://issues.apache.org/jira/browse/OAK-7265
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples, pojosr
>Affects Versions: 1.8.0
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> On users dl it was reported that standalone example application fails to
> start with following exception
> {noformat}
> java.lang.NoSuchMethodException:
> org.springframework.boot.loader.jar.JarEntry.getUrl()
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getUrlMethod(SpringBootSupport.java:135)
> at
> org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getEntry(SpringBootSupport.java:109)
> at org.apache.felix.connect.PojoSRBundle.getEntry(PojoSRBundle.java:471)
> at
> org.apache.felix.webconsole.plugins.scriptconsole.internal.ScriptEngineManager.bundleChanged(ScriptEngineManager.java:117)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:821)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:771)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.run(EventDispatcher.java:993)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:52)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:94)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7265) Standalone example application fails to start
Chetan Mehrotra created OAK-7265:
Summary: Standalone example application fails to start
Key: OAK-7265
URL: https://issues.apache.org/jira/browse/OAK-7265
Project: Jackrabbit Oak
Issue Type: Bug
Components: examples, pojosr
Affects Versions: 1.8.0
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Fix For: 1.10
On users dl it was reported that standalone example application fails to start
with following exception
{noformat}
java.lang.NoSuchMethodException:
org.springframework.boot.loader.jar.JarEntry.getUrl()
at java.lang.Class.getMethod(Class.java:1786)
at
org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getUrlMethod(SpringBootSupport.java:135)
at
org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getEntry(SpringBootSupport.java:109)
at org.apache.felix.connect.PojoSRBundle.getEntry(PojoSRBundle.java:471)
at
org.apache.felix.webconsole.plugins.scriptconsole.internal.ScriptEngineManager.bundleChanged(ScriptEngineManager.java:117)
at
org.apache.felix.connect.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:821)
at
org.apache.felix.connect.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:771)
at
org.apache.felix.connect.felix.framework.util.EventDispatcher.run(EventDispatcher.java:993)
at
org.apache.felix.connect.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:52)
at
org.apache.felix.connect.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:94)
at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7265) Standalone example application fails to start
[
https://issues.apache.org/jira/browse/OAK-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363495#comment-16363495
]
Chetan Mehrotra commented on OAK-7265:
--
This happens because the
[getUrl|https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot-tools/spring-boot-loader/src/main/java/org/springframework/boot/loader/jar/JarEntry.java#L73]
method is made package scoped
> Standalone example application fails to start
> -
>
> Key: OAK-7265
> URL: https://issues.apache.org/jira/browse/OAK-7265
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples, pojosr
>Affects Versions: 1.8.0
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Major
> Fix For: 1.10
>
>
> On users dl it was reported that standalone example application fails to
> start with following exception
> {noformat}
> java.lang.NoSuchMethodException:
> org.springframework.boot.loader.jar.JarEntry.getUrl()
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getUrlMethod(SpringBootSupport.java:135)
> at
> org.apache.jackrabbit.oak.run.osgi.SpringBootSupport$SpringBootJarRevision.getEntry(SpringBootSupport.java:109)
> at org.apache.felix.connect.PojoSRBundle.getEntry(PojoSRBundle.java:471)
> at
> org.apache.felix.webconsole.plugins.scriptconsole.internal.ScriptEngineManager.bundleChanged(ScriptEngineManager.java:117)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:821)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:771)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.run(EventDispatcher.java:993)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:52)
> at
> org.apache.felix.connect.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:94)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7258) Build Jackrabbit Oak #1240 failed
[ https://issues.apache.org/jira/browse/OAK-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362713#comment-16362713 ] Hudson commented on OAK-7258: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #1245|https://builds.apache.org/job/Jackrabbit%20Oak/1245/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1245/console] > Build Jackrabbit Oak #1240 failed > - > > Key: OAK-7258 > URL: https://issues.apache.org/jira/browse/OAK-7258 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1240 has failed. > First failed run: [Jackrabbit Oak > #1240|https://builds.apache.org/job/Jackrabbit%20Oak/1240/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1240/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7262) LockBasedScheduler#getHeadNodeState poor performance due to lock contention in commitTimeHistogram implementation
[
https://issues.apache.org/jira/browse/OAK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362617#comment-16362617
]
Francesco Mari commented on OAK-7262:
-
[~dulceanu], the benchmarks results don't show critical fluctuations in the
performance. I agree to go forward with the patch and move to real-world
testing.
> LockBasedScheduler#getHeadNodeState poor performance due to lock contention
> in commitTimeHistogram implementation
> -
>
> Key: OAK-7262
> URL: https://issues.apache.org/jira/browse/OAK-7262
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> With OAK-4732, we slightly prioritised reads, allowing callers of
> {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time
> on the commit semaphore, in order to fetch the latest head. The amount of
> time to wait for was computed as the median of the latest 1000 commits, but
> the implementation was flexible enough to replace the 0.5 quantile with a
> different value.
> The initial implementation used {{SynchronizedDescriptiveStatistics}} from
> {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed
> by a {{SlindingWindowReservoir}} from the dropwizard metric library
> ({{io.dropwizard.metrics}}).
> One of the problems observed with the current implementation is that, under
> heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
> decreases due to lock contention in {{commitTimeHistogram}} implementation.
> Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
> copies an array of 1000 elements at each invocation of
> {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
> each element in the array.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7262) LockBasedScheduler#getHeadNodeState poor performance due to lock contention in commitTimeHistogram implementation
[
https://issues.apache.org/jira/browse/OAK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362564#comment-16362564
]
Andrei Dulceanu commented on OAK-7262:
--
Here are the first results for patched vs. trunk (on my local machine):
{noformat}
# BasicWriteTest C min 10% 50% 90% max
N
Patched1 19 21 22 25 67
2569
Trunk 1 19 21 22 26 63
2501
# ConcurrentReadTest C min 10% 50% 90% max
N
Patched1 54 112 153 217 314
379
Trunk 1 50 115 140 165 249
427
# ConcurrentWriteTest C min 10% 50% 90% max
N
Patched1 81 84 89 104 222
645
Trunk 1 83 85 90 110 267
627
# ConcurrentReadWriteTest C min 10% 50% 90% max
N
Patched1 52 136 188 276 359
307
Trunk 1 49 127 170 242 372
340
# ConcurrentWriteReadTest C min 10% 50% 90% max
N
Patched1 6 8 17 76 226
1871
Trunk 1 6 8 16 66 250
2057
{noformat}
[~frm], it appears that there's a slight performance drop for some tests (i.e.,
{{BasicWriteTest}}, {{ConcurrentReadTest}}). One thing to be noted is that
usually there's a lot of variance when running the benchmarks locally.
Moreover, as noted in previous experiments, these benchmarks might not be
really accurate at predicting how the system performs under heavy-load with
multiple concurrent writers ({{SegmentCompactionIT}} is the only example which
comes to mind here).
My proposal is to move forward with this patch. We can easily revert it later,
provided real-world usage proves it was a wrong solution.
> LockBasedScheduler#getHeadNodeState poor performance due to lock contention
> in commitTimeHistogram implementation
> -
>
> Key: OAK-7262
> URL: https://issues.apache.org/jira/browse/OAK-7262
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> With OAK-4732, we slightly prioritised reads, allowing callers of
> {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time
> on the commit semaphore, in order to fetch the latest head. The amount of
> time to wait for was computed as the median of the latest 1000 commits, but
> the implementation was flexible enough to replace the 0.5 quantile with a
> different value.
> The initial implementation used {{SynchronizedDescriptiveStatistics}} from
> {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed
> by a {{SlindingWindowReservoir}} from the dropwizard metric library
> ({{io.dropwizard.metrics}}).
> One of the problems observed with the current implementation is that, under
> heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
> decreases due to lock contention in {{commitTimeHistogram}} implementation.
> Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
> copies an array of 1000 elements at each invocation of
> {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
> each element in the array.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7258) Build Jackrabbit Oak #1240 failed
[ https://issues.apache.org/jira/browse/OAK-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362550#comment-16362550 ] Hudson commented on OAK-7258: - Build is still failing. Failed run: [Jackrabbit Oak #1244|https://builds.apache.org/job/Jackrabbit%20Oak/1244/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1244/console] > Build Jackrabbit Oak #1240 failed > - > > Key: OAK-7258 > URL: https://issues.apache.org/jira/browse/OAK-7258 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1240 has failed. > First failed run: [Jackrabbit Oak > #1240|https://builds.apache.org/job/Jackrabbit%20Oak/1240/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1240/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7264) Missing OSGi Provide-Capability for classed
[ https://issues.apache.org/jira/browse/OAK-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362508#comment-16362508 ] Chetan Mehrotra commented on OAK-7264: -- Some of these service are registered programatically via bundleContext upon component activation hence Bnd is not able to detect them. I have not been able to find a way where I can hint Bnd to include such capabilities. Similar problem was seen with SLING-6474 > Missing OSGi Provide-Capability for classed > --- > > Key: OAK-7264 > URL: https://issues.apache.org/jira/browse/OAK-7264 > Project: Jackrabbit Oak > Issue Type: Bug > Components: composite, core >Affects Versions: 1.8.2 >Reporter: Oleg Cohen >Priority: Minor > > Deploying Oak 1.8.2 into Karaf OSGi container v4.1.4 in addition to needing > oak-store-composite bundle I had to add a custom bundle with dummy > Provide-Capability for the following services: > > > osgi.service;objectClass="com.codahale.metrics.MetricRegistry", > > osgi.service;objectClass="org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean", > > osgi.service;objectClass="org.apache.jackrabbit.oak.api.jmx.QueryEngineSettingsMBean", > > osgi.service;objectClass="org.apache.jackrabbit.oak.spi.state.NodeStore", > > osgi.service;objectClass="org.apache.jackrabbit.oak.spi.state.NodeStoreProvider", > > osgi.service;objectClass="org.apache.jackrabbit.oak.stats.StatisticsProvider", > > osgi.service;objectClass="org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore", > > osgi.service;objectClass="java.util.concurrent.Executor", > > osgi.service;objectClass="org.apache.jackrabbit.oak.spi.blob.BlobStore", > > osgi.service;objectClass="org.apache.jackrabbit.oak.spi.mount.MountInfoProvider" > > > > The suspicion is that the OSGi bundles providing these services don't have > the respective Provide-Capability generated in the MANIFEST. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OAK-7264) Missing OSGi Provide-Capability for classed
Oleg Cohen created OAK-7264: --- Summary: Missing OSGi Provide-Capability for classed Key: OAK-7264 URL: https://issues.apache.org/jira/browse/OAK-7264 Project: Jackrabbit Oak Issue Type: Bug Components: composite, core Affects Versions: 1.8.2 Reporter: Oleg Cohen Deploying Oak 1.8.2 into Karaf OSGi container v4.1.4 in addition to needing oak-store-composite bundle I had to add a custom bundle with dummy Provide-Capability for the following services: osgi.service;objectClass="com.codahale.metrics.MetricRegistry", osgi.service;objectClass="org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean", osgi.service;objectClass="org.apache.jackrabbit.oak.api.jmx.QueryEngineSettingsMBean", osgi.service;objectClass="org.apache.jackrabbit.oak.spi.state.NodeStore", osgi.service;objectClass="org.apache.jackrabbit.oak.spi.state.NodeStoreProvider", osgi.service;objectClass="org.apache.jackrabbit.oak.stats.StatisticsProvider", osgi.service;objectClass="org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore", osgi.service;objectClass="java.util.concurrent.Executor", osgi.service;objectClass="org.apache.jackrabbit.oak.spi.blob.BlobStore", osgi.service;objectClass="org.apache.jackrabbit.oak.spi.mount.MountInfoProvider" The suspicion is that the OSGi bundles providing these services don't have the respective Provide-Capability generated in the MANIFEST. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7263) oak-lucene should not depend on oak-store-document
[
https://issues.apache.org/jira/browse/OAK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362499#comment-16362499
]
Chetan Mehrotra edited comment on OAK-7263 at 2/13/18 3:34 PM:
---
This dependency is required untill we tackle OAK-6513. Probably we can work on
to make it optional
was (Author: chetanm):
This dependency is required untill we tackle OAK-6513
> oak-lucene should not depend on oak-store-document
> --
>
> Key: OAK-7263
> URL: https://issues.apache.org/jira/browse/OAK-7263
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Robert Munteanu
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> {{oak-lucene}} has a hard dependency on {{oak-store-document}} and that looks
> wrong to me.
> {noformat}[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile
> (default-compile) on project oak-lucene: Compilation failure: Compilation
> failure:
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[31,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[37,46]
> cannot find symbol
> [ERROR] symbol: class JournalProperty
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[33,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[34,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[38,47]
> cannot find symbol
> [ERROR] symbol: class JournalPropertyBuilder
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[106,12]
> cannot find symbol
> [ERROR] symbol: class JournalProperty
> [ERROR] location: class
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyBuilder
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexProviderService.java:[55,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[29,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[33,31]
> cannot find symbol
> [ERROR] symbol: class JournalProperty
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[22,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[23,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[25,54]
> cannot find symbol
> [ERROR] symbol: class JournalPropertyService
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[33,12]
> cannot find symbol
> [ERROR] symbol: class JournalPropertyBuilder
> [ERROR] location: class
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyService
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[50,5]
> method does not override or implement a method from a supertype
> [ERROR]
>
[jira] [Commented] (OAK-7263) oak-lucene should not depend on oak-store-document
[
https://issues.apache.org/jira/browse/OAK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362499#comment-16362499
]
Chetan Mehrotra commented on OAK-7263:
--
This dependency is required untill we tackle OAK-6513
> oak-lucene should not depend on oak-store-document
> --
>
> Key: OAK-7263
> URL: https://issues.apache.org/jira/browse/OAK-7263
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Robert Munteanu
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> {{oak-lucene}} has a hard dependency on {{oak-store-document}} and that looks
> wrong to me.
> {noformat}[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile
> (default-compile) on project oak-lucene: Compilation failure: Compilation
> failure:
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[31,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[37,46]
> cannot find symbol
> [ERROR] symbol: class JournalProperty
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[33,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[34,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[38,47]
> cannot find symbol
> [ERROR] symbol: class JournalPropertyBuilder
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[106,12]
> cannot find symbol
> [ERROR] symbol: class JournalProperty
> [ERROR] location: class
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyBuilder
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexProviderService.java:[55,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[29,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[33,31]
> cannot find symbol
> [ERROR] symbol: class JournalProperty
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[22,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[23,54]
> package org.apache.jackrabbit.oak.plugins.document.spi does not exist
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[25,54]
> cannot find symbol
> [ERROR] symbol: class JournalPropertyService
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[33,12]
> cannot find symbol
> [ERROR] symbol: class JournalPropertyBuilder
> [ERROR] location: class
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyService
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[50,5]
> method does not override or implement a method from a supertype
> [ERROR]
> /home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[61,5]
> method does not override or implement
[jira] [Updated] (OAK-7263) oak-lucene should not depend on oak-store-document
[
https://issues.apache.org/jira/browse/OAK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-7263:
Description:
{{oak-lucene}} has a hard dependency on {{oak-store-document}} and that looks
wrong to me.
{noformat}[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile)
on project oak-lucene: Compilation failure: Compilation failure:
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[31,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[37,46]
cannot find symbol
[ERROR] symbol: class JournalProperty
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[33,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[34,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[38,47]
cannot find symbol
[ERROR] symbol: class JournalPropertyBuilder
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[106,12]
cannot find symbol
[ERROR] symbol: class JournalProperty
[ERROR] location: class
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyBuilder
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexProviderService.java:[55,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[29,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[33,31]
cannot find symbol
[ERROR] symbol: class JournalProperty
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[22,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[23,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[25,54]
cannot find symbol
[ERROR] symbol: class JournalPropertyService
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[33,12]
cannot find symbol
[ERROR] symbol: class JournalPropertyBuilder
[ERROR] location: class
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyService
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[50,5]
method does not override or implement a method from a supertype
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[61,5]
method does not override or implement a method from a supertype
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[78,5]
method does not override or implement a method from a supertype
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[105,5]
method does not override or implement a method from a supertype
[ERROR]
[jira] [Updated] (OAK-7263) oak-lucene should not depend on oak-store-document
[
https://issues.apache.org/jira/browse/OAK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-7263:
Environment: (was: {{oak-lucene}} has a hard dependency on
{{oak-store-document}} and that looks wrong to me.
{noformat}[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile)
on project oak-lucene: Compilation failure: Compilation failure:
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[31,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[37,46]
cannot find symbol
[ERROR] symbol: class JournalProperty
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[33,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[34,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[38,47]
cannot find symbol
[ERROR] symbol: class JournalPropertyBuilder
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[106,12]
cannot find symbol
[ERROR] symbol: class JournalProperty
[ERROR] location: class
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyBuilder
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexProviderService.java:[55,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[29,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[33,31]
cannot find symbol
[ERROR] symbol: class JournalProperty
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[22,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[23,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[25,54]
cannot find symbol
[ERROR] symbol: class JournalPropertyService
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[33,12]
cannot find symbol
[ERROR] symbol: class JournalPropertyBuilder
[ERROR] location: class
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyService
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[50,5]
method does not override or implement a method from a supertype
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[61,5]
method does not override or implement a method from a supertype
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[78,5]
method does not override or implement a method from a supertype
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[105,5]
method does not override or implement a method from a supertype
[ERROR]
[jira] [Created] (OAK-7263) oak-lucene should not depend on oak-store-document
Robert Munteanu created OAK-7263:
Summary: oak-lucene should not depend on oak-store-document
Key: OAK-7263
URL: https://issues.apache.org/jira/browse/OAK-7263
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Environment: {{oak-lucene}} has a hard dependency on
{{oak-store-document}} and that looks wrong to me.
{noformat}[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile)
on project oak-lucene: Compilation failure: Compilation failure:
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[31,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneDocumentHolder.java:[37,46]
cannot find symbol
[ERROR] symbol: class JournalProperty
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[33,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[34,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[38,47]
cannot find symbol
[ERROR] symbol: class JournalPropertyBuilder
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[106,12]
cannot find symbol
[ERROR] symbol: class JournalProperty
[ERROR] location: class
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyBuilder
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexProviderService.java:[55,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[29,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/IndexedPaths.java:[33,31]
cannot find symbol
[ERROR] symbol: class JournalProperty
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[22,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[23,54]
package org.apache.jackrabbit.oak.plugins.document.spi does not exist
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[25,54]
cannot find symbol
[ERROR] symbol: class JournalPropertyService
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyService.java:[33,12]
cannot find symbol
[ERROR] symbol: class JournalPropertyBuilder
[ERROR] location: class
org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.LuceneJournalPropertyService
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[50,5]
method does not override or implement a method from a supertype
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[61,5]
method does not override or implement a method from a supertype
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[78,5]
method does not override or implement a method from a supertype
[ERROR]
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/hybrid/LuceneJournalPropertyBuilder.java:[105,5]
method
[jira] [Commented] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362422#comment-16362422
]
Robert Munteanu commented on OAK-7203:
--
[~stillalex] - I still think we should not make the dependency optional:
{quote}Thinking about this a bit more, I think the current setup is correct. We
should not allow a 'default' service to be registered or default values to be
used when it's possible that the value will change. The components must only
use the 'right' MountInfoProvider from the beginning, otherwise we open the
door to inconsistent behaviour.{quote}
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Assignee: Alex Deparvu
>Priority: Major
> Fix For: 1.9.0
>
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu updated OAK-7203:
--
Fix Version/s: 1.9.0
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Assignee: Alex Deparvu
>Priority: Major
> Fix For: 1.9.0
>
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu reassigned OAK-7203:
-
Assignee: Alex Deparvu
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Assignee: Alex Deparvu
>Priority: Major
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7258) Build Jackrabbit Oak #1240 failed
[ https://issues.apache.org/jira/browse/OAK-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362394#comment-16362394 ] Hudson commented on OAK-7258: - Build is still failing. Failed run: [Jackrabbit Oak #1243|https://builds.apache.org/job/Jackrabbit%20Oak/1243/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1243/console] > Build Jackrabbit Oak #1240 failed > - > > Key: OAK-7258 > URL: https://issues.apache.org/jira/browse/OAK-7258 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1240 has failed. > First failed run: [Jackrabbit Oak > #1240|https://builds.apache.org/job/Jackrabbit%20Oak/1240/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1240/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7262) LockBasedScheduler#getHeadNodeState poor performance due to lock contention in commitTimeHistogram implementation
[
https://issues.apache.org/jira/browse/OAK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362368#comment-16362368
]
Francesco Mari commented on OAK-7262:
-
[~dulceanu], I think it's a reasonable approach. Looking forward to the
benchmark results.
> LockBasedScheduler#getHeadNodeState poor performance due to lock contention
> in commitTimeHistogram implementation
> -
>
> Key: OAK-7262
> URL: https://issues.apache.org/jira/browse/OAK-7262
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> With OAK-4732, we slightly prioritised reads, allowing callers of
> {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time
> on the commit semaphore, in order to fetch the latest head. The amount of
> time to wait for was computed as the median of the latest 1000 commits, but
> the implementation was flexible enough to replace the 0.5 quantile with a
> different value.
> The initial implementation used {{SynchronizedDescriptiveStatistics}} from
> {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed
> by a {{SlindingWindowReservoir}} from the dropwizard metric library
> ({{io.dropwizard.metrics}}).
> One of the problems observed with the current implementation is that, under
> heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
> decreases due to lock contention in {{commitTimeHistogram}} implementation.
> Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
> copies an array of 1000 elements at each invocation of
> {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
> each element in the array.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362358#comment-16362358
]
Alex Deparvu commented on OAK-7203:
---
I took another look at the patch and I'm ok with applying it (didn't run full
IT tests on it yet but plan to do so next).
[~rombert] do you have any comments re the patch as it looks now?
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Priority: Major
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7262) LockBasedScheduler#getHeadNodeState poor performance due to lock contention in commitTimeHistogram implementation
[
https://issues.apache.org/jira/browse/OAK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362338#comment-16362338
]
Andrei Dulceanu commented on OAK-7262:
--
[~frm], looking at possible {{Reservoir}} implementations needed for backing up
the {{Histogram}} used for bookkeeping the commit time, I came across of
{{UniformReservoir}}, which provides a random sampling reservoir, "which offers
a 99.9% confidence level with a 5% margin of error assuming a normal
distribution" (according to the JavaDoc). I think this is a reasonable choice,
since it uses under the hood an {{AtomicLongArray}} (no locks acquired for
important operations such as #size, #getSnapshot, etc.).
My idea is to replace the implementation and start measuring the effect by
running some of our benchmarks (patched vs trunk) to asses whether there's a
performance gain or not.
WDYT?
> LockBasedScheduler#getHeadNodeState poor performance due to lock contention
> in commitTimeHistogram implementation
> -
>
> Key: OAK-7262
> URL: https://issues.apache.org/jira/browse/OAK-7262
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> With OAK-4732, we slightly prioritised reads, allowing callers of
> {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time
> on the commit semaphore, in order to fetch the latest head. The amount of
> time to wait for was computed as the median of the latest 1000 commits, but
> the implementation was flexible enough to replace the 0.5 quantile with a
> different value.
> The initial implementation used {{SynchronizedDescriptiveStatistics}} from
> {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed
> by a {{SlindingWindowReservoir}} from the dropwizard metric library
> ({{io.dropwizard.metrics}}).
> One of the problems observed with the current implementation is that, under
> heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
> decreases due to lock contention in {{commitTimeHistogram}} implementation.
> Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
> copies an array of 1000 elements at each invocation of
> {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
> each element in the array.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362337#comment-16362337
]
Alex Deparvu commented on OAK-7203:
---
bq. any chance to get this fixed for 1.8
this might seem a bit harsh, but even if we fix this for 1.10 I would not
backport it to 1.8 unless there's a severe bug affecting setups. Having to pull
in one extra bundle is low pain if you look at the size of a regular deployment
these days.
having said this, I would like to make the dependency optional and not move any
classes to oak-core, but I must admit I'm a bit lost in the OSGi specifics, and
not sure what the best practice is anymore. I was especially confused by the
bind/unbind methods which at one point were used (if present).
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Priority: Major
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7262) LockBasedScheduler#getHeadNodeState poor performance due to lock contention in commitTimeHistogram implementation
[
https://issues.apache.org/jira/browse/OAK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu updated OAK-7262:
-
Description:
With OAK-4732, we slightly prioritised reads, allowing callers of
{{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time on
the commit semaphore, in order to fetch the latest head. The amount of time to
wait for was computed as the median of the latest 1000 commits, but the
implementation was flexible enough to replace the 0.5 quantile with a different
value.
The initial implementation used {{SynchronizedDescriptiveStatistics}} from
{{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed by
a {{SlindingWindowReservoir}} from the dropwizard metric library
({{io.dropwizard.metrics}}).
One of the problems observed with the current implementation is that, under
heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
decreases due to lock contention in {{commitTimeHistogram}} implementation.
Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
copies an array of 1000 elements at each invocation of
{{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
each element in the array.
was:
With OAK-4732, we slightly prioritised reads, allowing callers of
{{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time on
the commit semaphore, in order to fetch the latest head. The amount of time to
wait for was computed as the median of the latest 1000 commits, but the
implementation was flexible enough to replace the 0.5 quantile with a different
value.
The initial implementation used {{SynchronizedDescriptiveStatistics}} from
{{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} from the
dropwizard metric library (\{{io.dropwizard.metrics}}).
One of the problems observed with the current implementation is that, under
heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
decreases due to lock contention in {{commitTimeHistogram}} implementation.
Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
copies an array of 1000 elements at each invocation of
{{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
each element in the array.
> LockBasedScheduler#getHeadNodeState poor performance due to lock contention
> in commitTimeHistogram implementation
> -
>
> Key: OAK-7262
> URL: https://issues.apache.org/jira/browse/OAK-7262
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> With OAK-4732, we slightly prioritised reads, allowing callers of
> {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time
> on the commit semaphore, in order to fetch the latest head. The amount of
> time to wait for was computed as the median of the latest 1000 commits, but
> the implementation was flexible enough to replace the 0.5 quantile with a
> different value.
> The initial implementation used {{SynchronizedDescriptiveStatistics}} from
> {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed
> by a {{SlindingWindowReservoir}} from the dropwizard metric library
> ({{io.dropwizard.metrics}}).
> One of the problems observed with the current implementation is that, under
> heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
> decreases due to lock contention in {{commitTimeHistogram}} implementation.
> Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
> copies an array of 1000 elements at each invocation of
> {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
> each element in the array.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7262) LockBasedScheduler#getHeadNodeState poor performance due to lock contention in commitTimeHistogram implementation
Andrei Dulceanu created OAK-7262:
Summary: LockBasedScheduler#getHeadNodeState poor performance due
to lock contention in commitTimeHistogram implementation
Key: OAK-7262
URL: https://issues.apache.org/jira/browse/OAK-7262
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segment-tar
Reporter: Andrei Dulceanu
Assignee: Andrei Dulceanu
Fix For: 1.9.0, 1.10
With OAK-4732, we slightly prioritised reads, allowing callers of
{{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time on
the commit semaphore, in order to fetch the latest head. The amount of time to
wait for was computed as the median of the latest 1000 commits, but the
implementation was flexible enough to replace the 0.5 quantile with a different
value.
The initial implementation used {{SynchronizedDescriptiveStatistics}} from
{{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} from the
dropwizard metric library (\{{io.dropwizard.metrics}}).
One of the problems observed with the current implementation is that, under
heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
decreases due to lock contention in {{commitTimeHistogram}} implementation.
Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
copies an array of 1000 elements at each invocation of
{{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
each element in the array.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7258) Build Jackrabbit Oak #1240 failed
[ https://issues.apache.org/jira/browse/OAK-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362245#comment-16362245 ] Hudson commented on OAK-7258: - Build is still failing. Failed run: [Jackrabbit Oak #1242|https://builds.apache.org/job/Jackrabbit%20Oak/1242/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1242/console] > Build Jackrabbit Oak #1240 failed > - > > Key: OAK-7258 > URL: https://issues.apache.org/jira/browse/OAK-7258 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1240 has failed. > First failed run: [Jackrabbit Oak > #1240|https://builds.apache.org/job/Jackrabbit%20Oak/1240/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1240/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7255) Upgrade jackson dependencies to version 2.9.4
[ https://issues.apache.org/jira/browse/OAK-7255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361243#comment-16361243 ] Julian Reschke edited comment on OAK-7255 at 2/13/18 12:34 PM: --- We probably have them in more - I didn't check yet. EDIT: Oak 1.6 seems to be on a version older than the reported vulnerable range. was (Author: reschke): We probably have them in more - I didn't check yet. EDIT: Oak 1.6 seems to be a version older than the reported vulnerable range. > Upgrade jackson dependencies to version 2.9.4 > - > > Key: OAK-7255 > URL: https://issues.apache.org/jira/browse/OAK-7255 > Project: Jackrabbit Oak > Issue Type: Task > Components: oak-http, solr >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > Labels: candidate_oak_1_8 > Fix For: 1.9.0, 1.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-6921) Support pluggable segment storage
[
https://issues.apache.org/jira/browse/OAK-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362159#comment-16362159
]
Tomek Rękawek commented on OAK-6921:
Fixed for trunk in [r1824115|https://svn.apache.org/r1824115].
> Support pluggable segment storage
> -
>
> Key: OAK-6921
> URL: https://issues.apache.org/jira/browse/OAK-6921
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Major
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-6921.patch, current-state.png, new-interfaces.png
>
>
> h3. Rationale
> segment-tar, as names suggest, stores the segments in a bunch of tar
> archives, inside the {{segmentstore}} directory on the local file system. For
> some cases, especially in the cloud deployments, it may be interesting to
> store the segments outside the local FS - the remote storage such as Amazon
> S3, Azure Blob Storage or HDFS may be cheaper than a mounted disk, more
> scalable, easier for the provisioning, etc.
> h3. Storing segment in tar files
> !current-state.png!
> There are 3 classes responsible for handling tar files in the segment-tar:
> TarFiles, TarWriter and TarReader. The TarFiles manages the {{segmentstore}}
> directory, scans it for the .tars and for each one creates a TarReader. It
> also creates a single TarWriter object, used to write (and also read) the
> most recent tar file.
> The TarWriter appends segments to the latest tar file and also serializes the
> auxiliary indexes: segment index, binary references index and the segment
> graph. It also takes of synchronization, as we're dealing with a mutable data
> structure - tar file opened in the append mode.
> The TarReader not only reads the segments from the tar file, but is also
> responsible for the revision GC (mark & sweep methods) and recovering data
> from files which hasn't been closed cleanly (eg. have no index).
> h3. New abstraction layer - SegmentArchiveManager
> !new-interfaces.png!
> In order to store segments not in the tar files, but somewhere else, it'd be
> possible to create own implementation of the TarFiles, TarWriter and
> TarReader. However, such implementation would duplicate a lot of code, not
> strictly related to the persistence - mark(), sweep(), synchronization, etc.
> Rather than that, the attached patch presents a different approach: a new
> layer of abstraction is injected into TarFiles, TarWriter and TarReader - it
> only takes care of the segments persistence and knows nothing about the
> synchronization, GC, etc. - leaving it to the upper layer.
> The new abstraction layer is modelled using 3 new classes:
> SegmentArchiveManager, SegmentArchiveReader and SegmentArchiveWriter. They
> are strictly related to the existing Tar* classes and used by them.
> SegmentArchiveManager provides a bunch of file system-style methods, like
> open(), create(), delete(), exists(), etc. The open() and create() returns
> instances of the SAReader and SAWriter.
> SegmentArchiveReader, despite from reading segments, can also load and parse
> the index, graph and binary references. The logic responsible for parsing
> these structures has been already extracted, so it doesn't need to be
> duplicated in the SAReader implementations. Also, SAReader needs to be aware
> about the index, since it contains the segment offsets.
> The SAWriter class allows to write and read the segments and also store the
> indexes. It isn't thread safe - it assumes that the synchronization is
> already done on the higher layers.
> In the patch, I've moved the tar implementation to the new classes:
> SegmentTarManager, SegmentTarReader and SegmentTarWriter.
> h3. Other files
> Apart from the segments, the {{segmentstore}} directory also contains
> following files:
> * repo.lock
> * journal.log
> * gc.log
> * manifest
> All these files are supported by the new SegmentNodeStorePersistence. Usually
> there's a simple interface (RepositoryLock, JournalLogFile, etc.) for
> handling the files.
> h3. TODO
> * The names and package locations for all the affected classes are subjects
> to change - after applying the patch the TarFiles doesn't deal with the .tar
> files anymore, similarly the TarReader and TarWriter delegates the low-level
> file access duties to the SegmentArchiveReader and Writer. I didn't want to
> change the names yet, to make it easier to understand and rebase the patch
> with the trunk changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-6921) Support pluggable segment storage
[
https://issues.apache.org/jira/browse/OAK-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek resolved OAK-6921.
Resolution: Fixed
> Support pluggable segment storage
> -
>
> Key: OAK-6921
> URL: https://issues.apache.org/jira/browse/OAK-6921
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Major
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-6921.patch, current-state.png, new-interfaces.png
>
>
> h3. Rationale
> segment-tar, as names suggest, stores the segments in a bunch of tar
> archives, inside the {{segmentstore}} directory on the local file system. For
> some cases, especially in the cloud deployments, it may be interesting to
> store the segments outside the local FS - the remote storage such as Amazon
> S3, Azure Blob Storage or HDFS may be cheaper than a mounted disk, more
> scalable, easier for the provisioning, etc.
> h3. Storing segment in tar files
> !current-state.png!
> There are 3 classes responsible for handling tar files in the segment-tar:
> TarFiles, TarWriter and TarReader. The TarFiles manages the {{segmentstore}}
> directory, scans it for the .tars and for each one creates a TarReader. It
> also creates a single TarWriter object, used to write (and also read) the
> most recent tar file.
> The TarWriter appends segments to the latest tar file and also serializes the
> auxiliary indexes: segment index, binary references index and the segment
> graph. It also takes of synchronization, as we're dealing with a mutable data
> structure - tar file opened in the append mode.
> The TarReader not only reads the segments from the tar file, but is also
> responsible for the revision GC (mark & sweep methods) and recovering data
> from files which hasn't been closed cleanly (eg. have no index).
> h3. New abstraction layer - SegmentArchiveManager
> !new-interfaces.png!
> In order to store segments not in the tar files, but somewhere else, it'd be
> possible to create own implementation of the TarFiles, TarWriter and
> TarReader. However, such implementation would duplicate a lot of code, not
> strictly related to the persistence - mark(), sweep(), synchronization, etc.
> Rather than that, the attached patch presents a different approach: a new
> layer of abstraction is injected into TarFiles, TarWriter and TarReader - it
> only takes care of the segments persistence and knows nothing about the
> synchronization, GC, etc. - leaving it to the upper layer.
> The new abstraction layer is modelled using 3 new classes:
> SegmentArchiveManager, SegmentArchiveReader and SegmentArchiveWriter. They
> are strictly related to the existing Tar* classes and used by them.
> SegmentArchiveManager provides a bunch of file system-style methods, like
> open(), create(), delete(), exists(), etc. The open() and create() returns
> instances of the SAReader and SAWriter.
> SegmentArchiveReader, despite from reading segments, can also load and parse
> the index, graph and binary references. The logic responsible for parsing
> these structures has been already extracted, so it doesn't need to be
> duplicated in the SAReader implementations. Also, SAReader needs to be aware
> about the index, since it contains the segment offsets.
> The SAWriter class allows to write and read the segments and also store the
> indexes. It isn't thread safe - it assumes that the synchronization is
> already done on the higher layers.
> In the patch, I've moved the tar implementation to the new classes:
> SegmentTarManager, SegmentTarReader and SegmentTarWriter.
> h3. Other files
> Apart from the segments, the {{segmentstore}} directory also contains
> following files:
> * repo.lock
> * journal.log
> * gc.log
> * manifest
> All these files are supported by the new SegmentNodeStorePersistence. Usually
> there's a simple interface (RepositoryLock, JournalLogFile, etc.) for
> handling the files.
> h3. TODO
> * The names and package locations for all the affected classes are subjects
> to change - after applying the patch the TarFiles doesn't deal with the .tar
> files anymore, similarly the TarReader and TarWriter delegates the low-level
> file access duties to the SegmentArchiveReader and Writer. I didn't want to
> change the names yet, to make it easier to understand and rebase the patch
> with the trunk changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OAK-6921) Support pluggable segment storage
[
https://issues.apache.org/jira/browse/OAK-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek reassigned OAK-6921:
--
Assignee: Tomek Rękawek
> Support pluggable segment storage
> -
>
> Key: OAK-6921
> URL: https://issues.apache.org/jira/browse/OAK-6921
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Major
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-6921.patch, current-state.png, new-interfaces.png
>
>
> h3. Rationale
> segment-tar, as names suggest, stores the segments in a bunch of tar
> archives, inside the {{segmentstore}} directory on the local file system. For
> some cases, especially in the cloud deployments, it may be interesting to
> store the segments outside the local FS - the remote storage such as Amazon
> S3, Azure Blob Storage or HDFS may be cheaper than a mounted disk, more
> scalable, easier for the provisioning, etc.
> h3. Storing segment in tar files
> !current-state.png!
> There are 3 classes responsible for handling tar files in the segment-tar:
> TarFiles, TarWriter and TarReader. The TarFiles manages the {{segmentstore}}
> directory, scans it for the .tars and for each one creates a TarReader. It
> also creates a single TarWriter object, used to write (and also read) the
> most recent tar file.
> The TarWriter appends segments to the latest tar file and also serializes the
> auxiliary indexes: segment index, binary references index and the segment
> graph. It also takes of synchronization, as we're dealing with a mutable data
> structure - tar file opened in the append mode.
> The TarReader not only reads the segments from the tar file, but is also
> responsible for the revision GC (mark & sweep methods) and recovering data
> from files which hasn't been closed cleanly (eg. have no index).
> h3. New abstraction layer - SegmentArchiveManager
> !new-interfaces.png!
> In order to store segments not in the tar files, but somewhere else, it'd be
> possible to create own implementation of the TarFiles, TarWriter and
> TarReader. However, such implementation would duplicate a lot of code, not
> strictly related to the persistence - mark(), sweep(), synchronization, etc.
> Rather than that, the attached patch presents a different approach: a new
> layer of abstraction is injected into TarFiles, TarWriter and TarReader - it
> only takes care of the segments persistence and knows nothing about the
> synchronization, GC, etc. - leaving it to the upper layer.
> The new abstraction layer is modelled using 3 new classes:
> SegmentArchiveManager, SegmentArchiveReader and SegmentArchiveWriter. They
> are strictly related to the existing Tar* classes and used by them.
> SegmentArchiveManager provides a bunch of file system-style methods, like
> open(), create(), delete(), exists(), etc. The open() and create() returns
> instances of the SAReader and SAWriter.
> SegmentArchiveReader, despite from reading segments, can also load and parse
> the index, graph and binary references. The logic responsible for parsing
> these structures has been already extracted, so it doesn't need to be
> duplicated in the SAReader implementations. Also, SAReader needs to be aware
> about the index, since it contains the segment offsets.
> The SAWriter class allows to write and read the segments and also store the
> indexes. It isn't thread safe - it assumes that the synchronization is
> already done on the higher layers.
> In the patch, I've moved the tar implementation to the new classes:
> SegmentTarManager, SegmentTarReader and SegmentTarWriter.
> h3. Other files
> Apart from the segments, the {{segmentstore}} directory also contains
> following files:
> * repo.lock
> * journal.log
> * gc.log
> * manifest
> All these files are supported by the new SegmentNodeStorePersistence. Usually
> there's a simple interface (RepositoryLock, JournalLogFile, etc.) for
> handling the files.
> h3. TODO
> * The names and package locations for all the affected classes are subjects
> to change - after applying the patch the TarFiles doesn't deal with the .tar
> files anymore, similarly the TarReader and TarWriter delegates the low-level
> file access duties to the SegmentArchiveReader and Writer. I didn't want to
> change the names yet, to make it easier to understand and rebase the patch
> with the trunk changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (OAK-5506) reject item names with unpaired surrogates early
[
https://issues.apache.org/jira/browse/OAK-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360873#comment-16360873
]
Julian Reschke edited comment on OAK-5506 at 2/13/18 10:39 AM:
---
FWIW, I just noticed that the various (R)DBs vary in their behavior:
- H2DB and Derby roundtrip any string
- PostgreSQL rejects the invalid string early
- DB2 and Oracle fail the same way as segment store (they persist the
replacement character)
- MySQL and SQLServer fail the same way as DB2 and Oracle, but here it's the
RDBDocumentStore's fault, because the ID column is binary, and we transform to
byte sequences ourselves
See OAK-7261 and related tickets.
was (Author: reschke):
FWIW, I just noticed that the various (R)DBs vary in their behavior:
- H2DB and Derby roundtrip any string
- PostgreSQL rejects the invalid string early
- DB2 and Oracle fail the same way as segment store (they persist the
replacement character)
- MySQL and SQLServer fail the same way as DB2 and Oracle, but here it's the
RDBDocumentStore's fault, because the ID column is binary, and we transform to
byte sequences ourselves
...will file individual tickets...
> reject item names with unpaired surrogates early
>
>
> Key: OAK-5506
> URL: https://issues.apache.org/jira/browse/OAK-5506
> Project: Jackrabbit Oak
> Issue Type: Wish
> Components: core, jcr, segment-tar
>Affects Versions: 1.5.18
>Reporter: Julian Reschke
>Priority: Minor
> Fix For: 1.10
>
> Attachments: OAK-5506-01.patch, OAK-5506-02.patch, OAK-5506-4.diff,
> OAK-5506-bench.diff, OAK-5506-jcr-level.diff, OAK-5506-name-conversion.diff,
> OAK-5506-segment.diff, OAK-5506-segment2.diff, OAK-5506-segment3.diff,
> OAK-5506.diff, ValidNamesTest.java
>
>
> Apparently, the following node name is accepted:
>{{"foo\ud800"}}
> but a subsequent {{getPath()}} call fails:
> {noformat}
> javax.jcr.InvalidItemStateException: This item [/test_node/foo?] does not
> exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.operation.ItemOperation.checkPreconditions(ItemOperation.java:34)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.getPath(ItemImpl.java:140)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getPath(NodeImpl.java:106)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.nameTest(ValidNamesTest.java:271)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.testUnpairedSurrogate(ValidNamesTest.java:259)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source){noformat}
> (test case follows)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7261) RDBDocumentStore: inconsistent behaviour for invalid Strings as document ID
Julian Reschke created OAK-7261: --- Summary: RDBDocumentStore: inconsistent behaviour for invalid Strings as document ID Key: OAK-7261 URL: https://issues.apache.org/jira/browse/OAK-7261 Project: Jackrabbit Oak Issue Type: Bug Components: rdbmk Reporter: Julian Reschke Assignee: Julian Reschke - H2DB and Derby roundtrip any string - PostgreSQL rejects the invalid string early - DB2 and Oracle fail the same way as segment store (they persist the replacement character) (see OAK-5506) - MySQL and SQLServer fail the same way as DB2 and Oracle, but here it's the RDBDocumentStore's fault, because the ID column is binary, and we transform to byte sequences ourselves -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-6922) Azure support for the segment-tar
[
https://issues.apache.org/jira/browse/OAK-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337848#comment-16337848
]
Tomek Rękawek edited comment on OAK-6922 at 2/13/18 10:36 AM:
--
The azure implementation:
https://github.com/trekawek/jackrabbit-oak/tree/segment-tar-trunk/azure
was (Author: tomek.rekawek):
The azure implementation:
https://github.com/trekawek/jackrabbit-oak/tree/segment-tar/azure
> Azure support for the segment-tar
> -
>
> Key: OAK-6922
> URL: https://issues.apache.org/jira/browse/OAK-6922
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar
>Reporter: Tomek Rękawek
>Priority: Major
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-6922.patch
>
>
> An Azure Blob Storage implementation of the segment storage, based on the
> OAK-6921 work.
> h3. Segment files layout
> Thew new implementation doesn't use tar files. They are replaced with
> directories, storing segments, named after their UUIDs. This approach has
> following advantages:
> * no need to call seek(), which may be expensive on a remote file system.
> Rather than that we can read the whole file (=segment) at once.
> * it's possible to send multiple segments at once, asynchronously, which
> reduces the performance overhead (see below).
> The file structure is as follows:
> {noformat}
> [~]$ az storage blob list -c oak --output table
> Name Blob Type
> Blob TierLengthContent Type Last Modified
> ---
> --- -
> oak/data0a.tar/.ca1326d1-edf4-4d53-aef0-0f14a6d05b63 BlockBlob
> 192 application/octet-stream 2018-01-31T10:59:14+00:00
> oak/data0a.tar/0001.c6e03426-db9d-4315-a20a-12559e6aee54 BlockBlob
> 262144application/octet-stream 2018-01-31T10:59:14+00:00
> oak/data0a.tar/0002.b3784e27-6d16-4f80-afc1-6f3703f6bdb9 BlockBlob
> 262144application/octet-stream 2018-01-31T10:59:14+00:00
> oak/data0a.tar/0003.5d2f9588-0c92-4547-abf7-0263ee7c37bb BlockBlob
> 259216application/octet-stream 2018-01-31T10:59:14+00:00
> ...
> oak/data0a.tar/006e.7b8cf63d-849a-4120-aa7c-47c3dde25e48 BlockBlob
> 4368 application/octet-stream 2018-01-31T12:01:09+00:00
> oak/data0a.tar/006f.93799ae9-288e-4b32-afc2-bbc676fad7e5 BlockBlob
> 3792 application/octet-stream 2018-01-31T12:01:14+00:00
> oak/data0a.tar/0070.8b2d5ff2-6a74-4ac3-a3cc-cc439367c2aa BlockBlob
> 3680 application/octet-stream 2018-01-31T12:01:14+00:00
> oak/data0a.tar/0071.2a1c49f0-ce33-4777-a042-8aa8a704d202 BlockBlob
> 7760 application/octet-stream 2018-01-31T12:10:54+00:00
> oak/journal.log.001 AppendBlob
> 1010 application/octet-stream 2018-01-31T12:10:54+00:00
> oak/manifest BlockBlob
> 46application/octet-stream 2018-01-31T10:59:14+00:00
> oak/repo.lock BlockBlob
> application/octet-stream 2018-01-31T10:59:14+00:00
> {noformat}
> For the segment files, each name is prefixed with the index number. This
> allows to maintain an order, as in the tar archive. This order is normally
> stored in the index files as well, but if it's missing, the recovery process
> needs it.
> Each file contains the raw segment data, with no padding/headers. Apart from
> the segment files, there are 3 special files: binary references (.brf),
> segment graph (.gph) and segment index (.idx).
> h3. Asynchronous writes
> Normally, all the TarWriter writes are synchronous, appending the segments to
> the tar file. In case of Azure Blob Storage each write involves a network
> latency. That's why the SegmentWriteQueue was introduced. The segments are
> added to the blocking dequeue, which is served by a number of the consumer
> threads, writing the segments to the cloud. There's also a map UUID->Segment,
> which allows to return the segments in case they are requested by the
> readSegment() method before they are actually persisted. Segments are removed
> from the map only after a successful write operation.
> The flush() method blocks accepting the new segments and returns after all
> waiting segments are written. The close() method waits until the current
> operations are finished and stops all threads.
> The asynchronous mode can be disabled by setting the number of threads to 0.
> h5. Queue
[jira] [Commented] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362077#comment-16362077
]
Robert Munteanu commented on OAK-7203:
--
I've done some thinking about this and I'm not 100% sure we should move it, so
I'll discuss on oak-dev.
Pros:
- simpler deployment, fewer bundles to manage
Cons:
- splitting the logic from {{oak-store-composite}} as we should only move one
implementation class
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Priority: Major
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362018#comment-16362018
]
Oliver Lietz commented on OAK-7203:
---
[~rombert], [~stillalex] any chance to get this fixed for 1.8? It's quite
strange to depend on an experimental module when running on OSGi.
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Priority: Major
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
